ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.61 by root, Sun Oct 22 13:52:11 2006 UTC vs.
Revision 1.66 by root, Mon Oct 23 22:54:27 2006 UTC

… …
51 51
52This module implements asynchronous I/O using whatever means your 52This module implements asynchronous I/O using whatever means your
53operating system supports. 53operating system supports.
54 54
55Currently, a number of threads are started that execute your read/writes 55Currently, a number of threads are started that execute your read/writes
56and signal their completion. You don't need thread support in your libc or 56and signal their completion. You don't need thread support in perl, and
57perl, and the threads created by this module will not be visible to the 57the threads created by this module will not be visible to perl. In the
58pthreads library. In the future, this module might make use of the native 58future, this module might make use of the native aio functions available
59aio functions available on many operating systems. However, they are often 59on many operating systems. However, they are often not well-supported
60not well-supported (Linux doesn't allow them on normal files currently, 60(Linux doesn't allow them on normal files currently, for example),
61for example), and they would only support aio_read and aio_write, so the 61and they would only support aio_read and aio_write, so the remaining
62remaining functionality would have to be implemented using threads anyway. 62functionality would have to be implemented using threads anyway.
63 63
64Although the module will work with in the presence of other threads, it is 64Although the module will work with in the presence of other threads, it is
65currently not reentrant, so use appropriate locking yourself, always call 65currently not reentrant, so use appropriate locking yourself, always call
66C<poll_cb> from within the same thread, or never call C<poll_cb> (or other 66C<poll_cb> from within the same thread, or never call C<poll_cb> (or other
67C<aio_> functions) recursively. 67C<aio_> functions) recursively.
… …
79 our $VERSION = '2.0'; 79 our $VERSION = '2.0';
80 80
81 our @EXPORT = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat 81 our @EXPORT = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat
82 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink 82 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink
83 aio_fsync aio_fdatasync aio_readahead aio_rename aio_link aio_move 83 aio_fsync aio_fdatasync aio_readahead aio_rename aio_link aio_move
84 aio_group); 84 aio_group aio_nop);
85 our @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel max_outstanding nreqs); 85 our @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel max_outstanding nreqs);
86 86
87 @IO::AIO::GRP::ISA = 'IO::AIO::REQ'; 87 @IO::AIO::GRP::ISA = 'IO::AIO::REQ';
88 88
89 require XSLoader; 89 require XSLoader;
… …
533 add $grp 533 add $grp
534 (aio_stat ...), 534 (aio_stat ...),
535 (aio_stat ...), 535 (aio_stat ...),
536 ...; 536 ...;
537 537
538=item aio_nop $callback->()
539
540This is a special request - it does nothing in itself and is only used for
541side effects, such as when you want to add a dummy request to a group so
542that finishing the requests in the group depends on executing the given
543code.
544
545While this request does nothing, it still goes through the execution
546phase and still requires a worker thread. Thus, the callback will not
547be executed immediately but only after other requests in the queue have
548entered their execution phase. This can be used to measure request
549latency.
550
538=item IO::AIO::aio_sleep $fractional_seconds, $callback->() *NOT EXPORTED* 551=item IO::AIO::aio_sleep $fractional_seconds, $callback->() *NOT EXPORTED*
539 552
540Mainly used for debugging and benchmarking, this aio request puts one of 553Mainly used for debugging and benchmarking, this aio request puts one of
541the request workers to sleep for the given time. 554the request workers to sleep for the given time.
542 555
… …
560callback) and B<done> (request has reached the end of its lifetime and 573callback) and B<done> (request has reached the end of its lifetime and
561holds no resources anymore). 574holds no resources anymore).
562 575
563=over 4 576=over 4
564 577
565=item $req->cancel 578=item cancel $req
566 579
567Cancels the request, if possible. Has the effect of skipping execution 580Cancels the request, if possible. Has the effect of skipping execution
568when entering the B<execute> state and skipping calling the callback when 581when entering the B<execute> state and skipping calling the callback when
569entering the the B<result> state, but will leave the request otherwise 582entering the the B<result> state, but will leave the request otherwise
570untouched. That means that requests that currently execute will not be 583untouched. That means that requests that currently execute will not be
571stopped and resources held by the request will not be freed prematurely. 584stopped and resources held by the request will not be freed prematurely.
572 585
586=item cb $req $callback->(...)
587
588Replace (or simply set) the callback registered to the request.
589
573=back 590=back
574 591
575=head2 IO::AIO::GRP CLASS 592=head2 IO::AIO::GRP CLASS
576 593
577This class is a subclass of L<IO::AIO::REQ>, so all its methods apply to 594This class is a subclass of L<IO::AIO::REQ>, so all its methods apply to
… …
603 }; 620 };
604 621
605This makes it very easy to create composite requests (see the source of 622This makes it very easy to create composite requests (see the source of
606C<aio_move> for an application) that work and feel like simple requests. 623C<aio_move> for an application) that work and feel like simple requests.
607 624
625=over 4
626
608The IO::AIO::GRP objects will be cleaned up during calls to 627=item * The IO::AIO::GRP objects will be cleaned up during calls to
609C<IO::AIO::poll_cb>, just like any other request. 628C<IO::AIO::poll_cb>, just like any other request.
610 629
611They can be canceled like any other request. Canceling will cancel not 630=item * They can be canceled like any other request. Canceling will cancel not
612only the request itself, but also all requests it contains. 631only the request itself, but also all requests it contains.
613 632
614They can also can also be added to other IO::AIO::GRP objects. 633=item * They can also can also be added to other IO::AIO::GRP objects.
615 634
616You must not add requests to a group from within the group callback (or 635=item * You must not add requests to a group from within the group callback (or
617any later time). 636any later time).
637
638=item * This does not harmonise well with C<max_outstanding>, so best do
639not combine C<aio_group> with it. Groups and feeders are recommended for
640this kind of concurrency-limiting.
641
642=back
618 643
619Their lifetime, simplified, looks like this: when they are empty, they 644Their lifetime, simplified, looks like this: when they are empty, they
620will finish very quickly. If they contain only requests that are in the 645will finish very quickly. If they contain only requests that are in the
621C<done> state, they will also finish. Otherwise they will continue to 646C<done> state, they will also finish. Otherwise they will continue to
622exist. 647exist.
… …
626group. And only when all those requests have finished will the the group 651group. And only when all those requests have finished will the the group
627itself finish. 652itself finish.
628 653
629=over 4 654=over 4
630 655
656=item add $grp ...
657
631=item $grp->add (...) 658=item $grp->add (...)
632
633=item add $grp ...
634 659
635Add one or more requests to the group. Any type of L<IO::AIO::REQ> can 660Add one or more requests to the group. Any type of L<IO::AIO::REQ> can
636be added, including other groups, as long as you do not create circular 661be added, including other groups, as long as you do not create circular
637dependencies. 662dependencies.
638 663
… …
641=item $grp->result (...) 666=item $grp->result (...)
642 667
643Set the result value(s) that will be passed to the group callback when all 668Set the result value(s) that will be passed to the group callback when all
644subrequests have finished. By default, no argument will be passed. 669subrequests have finished. By default, no argument will be passed.
645 670
646=item $grp->set_feeder ($callback->($grp)) 671=item feed $grp $callback->($grp)
647 672
648[VERY EXPERIMENTAL] 673[VERY EXPERIMENTAL]
649 674
650Sets a feeder/generator on this group: every group can have an attached 675Sets a feeder/generator on this group: every group can have an attached
651generator that generates requests if idle. The idea behind this is that, 676generator that generates requests if idle. The idea behind this is that,
… …
654example, C<aio_scandir> might generate hundreds of thousands C<aio_stat> 679example, C<aio_scandir> might generate hundreds of thousands C<aio_stat>
655requests, delaying any later requests for a long time. 680requests, delaying any later requests for a long time.
656 681
657To avoid this, and allow incremental generation of requests, you can 682To avoid this, and allow incremental generation of requests, you can
658instead a group and set a feeder on it that generates those requests. The 683instead a group and set a feeder on it that generates those requests. The
659feeder will be called whenever there are few enough (see C<feeder_limit>, 684feed callback will be called whenever there are few enough (see C<feed_limit>,
660below) requests active in the group itself and is expected to queue more 685below) requests active in the group itself and is expected to queue more
661requests. 686requests.
662 687
663The feeder can queue as many requests as it likes (i.e. C<add> does not 688The feed can queue as many requests as it likes (i.e. C<add> does not
664impose any limits). 689impose any limits).
665 690
666If the feeder does not queue more requests when called, it will be 691If the feed does not queue more requests when called, it will be
667automatically removed from the group. 692automatically removed from the group.
668 693
669If the feeder limit is C<0>, it will be set to C<2> automatically. 694If the feed limit is C<0>, it will be set to C<2> automatically.
670 695
671Example: 696Example:
672 697
673 # stat all files in @files, but only ever use four aio requests concurrently: 698 # stat all files in @files, but only ever use four aio requests concurrently:
674 699
675 my $grp = aio_group sub { print "finished\n" }; 700 my $grp = aio_group sub { print "finished\n" };
676 $grp->feeder_limit (4); 701 feed_limit $grp 4;
677 $grp->set_feeder (sub { 702 feed $grp sub {
678 my $file = pop @files 703 my $file = pop @files
679 or return; 704 or return;
680 705
681 add $grp aio_stat $file, sub { ... }; 706 add $grp aio_stat $file, sub { ... };
682 }); 707 };
683 708
684=item $grp->feeder_limit ($num) 709=item feed_limit $grp $num
685 710
686Sets the feeder limit for the group: The feeder will be called whenever 711Sets the feeder limit for the group: The feeder will be called whenever
687the group contains less than this many requests. 712the group contains less than this many requests.
688 713
689Setting the limit to C<0> will pause the feeding process. 714Setting the limit to C<0> will pause the feeding process.
… …
784 809
785Under normal circumstances you don't need to call this function. 810Under normal circumstances you don't need to call this function.
786 811
787=item $oldnreqs = IO::AIO::max_outstanding $nreqs 812=item $oldnreqs = IO::AIO::max_outstanding $nreqs
788 813
814[DEPRECATED]
815
789Sets the maximum number of outstanding requests to C<$nreqs>. If you 816Sets the maximum number of outstanding requests to C<$nreqs>. If you
790try to queue up more than this number of requests, the caller will block until 817try to queue up more than this number of requests, the caller will block until
791some requests have been handled. 818some requests have been handled.
792 819
793The default is very large, so normally there is no practical limit. If you 820The default is very large, so normally there is no practical limit. If you
794queue up many requests in a loop it often improves speed if you set 821queue up many requests in a loop it often improves speed if you set
795this to a relatively low number, such as C<100>. 822this to a relatively low number, such as C<100>.
823
824This function does not work well together with C<aio_group>'s, and their
825feeder interface is better suited to limiting concurrency, so do not use
826this function.
796 827
797Under normal circumstances you don't need to call this function. 828Under normal circumstances you don't need to call this function.
798 829
799=back 830=back
800 831

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines