ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.61 by root, Sun Oct 22 13:52:11 2006 UTC vs.
Revision 1.67 by root, Tue Oct 24 02:25:16 2006 UTC

… …
51 51
52This module implements asynchronous I/O using whatever means your 52This module implements asynchronous I/O using whatever means your
53operating system supports. 53operating system supports.
54 54
55Currently, a number of threads are started that execute your read/writes 55Currently, a number of threads are started that execute your read/writes
56and signal their completion. You don't need thread support in your libc or 56and signal their completion. You don't need thread support in perl, and
57perl, and the threads created by this module will not be visible to the 57the threads created by this module will not be visible to perl. In the
58pthreads library. In the future, this module might make use of the native 58future, this module might make use of the native aio functions available
59aio functions available on many operating systems. However, they are often 59on many operating systems. However, they are often not well-supported
60not well-supported (Linux doesn't allow them on normal files currently, 60(Linux doesn't allow them on normal files currently, for example),
61for example), and they would only support aio_read and aio_write, so the 61and they would only support aio_read and aio_write, so the remaining
62remaining functionality would have to be implemented using threads anyway. 62functionality would have to be implemented using threads anyway.
63 63
64Although the module will work with in the presence of other threads, it is 64Although the module will work with in the presence of other threads, it is
65currently not reentrant, so use appropriate locking yourself, always call 65currently not reentrant, so use appropriate locking yourself, always call
66C<poll_cb> from within the same thread, or never call C<poll_cb> (or other 66C<poll_cb> from within the same thread, or never call C<poll_cb> (or other
67C<aio_> functions) recursively. 67C<aio_> functions) recursively.
… …
76use base 'Exporter'; 76use base 'Exporter';
77 77
78BEGIN { 78BEGIN {
79 our $VERSION = '2.0'; 79 our $VERSION = '2.0';
80 80
81 our @EXPORT = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat 81 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat
82 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink 82 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink
83 aio_fsync aio_fdatasync aio_readahead aio_rename aio_link aio_move 83 aio_fsync aio_fdatasync aio_readahead aio_rename aio_link aio_move
84 aio_group); 84 aio_group aio_nop);
85 our @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel max_outstanding nreqs); 85 our @EXPORT = (@AIO_REQ, qw(aioreq_pri));
86 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
87 min_parallel max_parallel max_outstanding nreqs);
86 88
87 @IO::AIO::GRP::ISA = 'IO::AIO::REQ'; 89 @IO::AIO::GRP::ISA = 'IO::AIO::REQ';
88 90
89 require XSLoader; 91 require XSLoader;
90 XSLoader::load ("IO::AIO", $VERSION); 92 XSLoader::load ("IO::AIO", $VERSION);
… …
533 add $grp 535 add $grp
534 (aio_stat ...), 536 (aio_stat ...),
535 (aio_stat ...), 537 (aio_stat ...),
536 ...; 538 ...;
537 539
540=item aio_nop $callback->()
541
542This is a special request - it does nothing in itself and is only used for
543side effects, such as when you want to add a dummy request to a group so
544that finishing the requests in the group depends on executing the given
545code.
546
547While this request does nothing, it still goes through the execution
548phase and still requires a worker thread. Thus, the callback will not
549be executed immediately but only after other requests in the queue have
550entered their execution phase. This can be used to measure request
551latency.
552
538=item IO::AIO::aio_sleep $fractional_seconds, $callback->() *NOT EXPORTED* 553=item IO::AIO::aio_sleep $fractional_seconds, $callback->() *NOT EXPORTED*
539 554
540Mainly used for debugging and benchmarking, this aio request puts one of 555Mainly used for debugging and benchmarking, this aio request puts one of
541the request workers to sleep for the given time. 556the request workers to sleep for the given time.
542 557
… …
560callback) and B<done> (request has reached the end of its lifetime and 575callback) and B<done> (request has reached the end of its lifetime and
561holds no resources anymore). 576holds no resources anymore).
562 577
563=over 4 578=over 4
564 579
565=item $req->cancel 580=item cancel $req
566 581
567Cancels the request, if possible. Has the effect of skipping execution 582Cancels the request, if possible. Has the effect of skipping execution
568when entering the B<execute> state and skipping calling the callback when 583when entering the B<execute> state and skipping calling the callback when
569entering the the B<result> state, but will leave the request otherwise 584entering the the B<result> state, but will leave the request otherwise
570untouched. That means that requests that currently execute will not be 585untouched. That means that requests that currently execute will not be
571stopped and resources held by the request will not be freed prematurely. 586stopped and resources held by the request will not be freed prematurely.
572 587
588=item cb $req $callback->(...)
589
590Replace (or simply set) the callback registered to the request.
591
573=back 592=back
574 593
575=head2 IO::AIO::GRP CLASS 594=head2 IO::AIO::GRP CLASS
576 595
577This class is a subclass of L<IO::AIO::REQ>, so all its methods apply to 596This class is a subclass of L<IO::AIO::REQ>, so all its methods apply to
… …
603 }; 622 };
604 623
605This makes it very easy to create composite requests (see the source of 624This makes it very easy to create composite requests (see the source of
606C<aio_move> for an application) that work and feel like simple requests. 625C<aio_move> for an application) that work and feel like simple requests.
607 626
627=over 4
628
608The IO::AIO::GRP objects will be cleaned up during calls to 629=item * The IO::AIO::GRP objects will be cleaned up during calls to
609C<IO::AIO::poll_cb>, just like any other request. 630C<IO::AIO::poll_cb>, just like any other request.
610 631
611They can be canceled like any other request. Canceling will cancel not 632=item * They can be canceled like any other request. Canceling will cancel not
612only the request itself, but also all requests it contains. 633only the request itself, but also all requests it contains.
613 634
614They can also can also be added to other IO::AIO::GRP objects. 635=item * They can also can also be added to other IO::AIO::GRP objects.
615 636
616You must not add requests to a group from within the group callback (or 637=item * You must not add requests to a group from within the group callback (or
617any later time). 638any later time).
639
640=item * This does not harmonise well with C<max_outstanding>, so best do
641not combine C<aio_group> with it. Groups and feeders are recommended for
642this kind of concurrency-limiting.
643
644=back
618 645
619Their lifetime, simplified, looks like this: when they are empty, they 646Their lifetime, simplified, looks like this: when they are empty, they
620will finish very quickly. If they contain only requests that are in the 647will finish very quickly. If they contain only requests that are in the
621C<done> state, they will also finish. Otherwise they will continue to 648C<done> state, they will also finish. Otherwise they will continue to
622exist. 649exist.
… …
626group. And only when all those requests have finished will the the group 653group. And only when all those requests have finished will the the group
627itself finish. 654itself finish.
628 655
629=over 4 656=over 4
630 657
658=item add $grp ...
659
631=item $grp->add (...) 660=item $grp->add (...)
632
633=item add $grp ...
634 661
635Add one or more requests to the group. Any type of L<IO::AIO::REQ> can 662Add one or more requests to the group. Any type of L<IO::AIO::REQ> can
636be added, including other groups, as long as you do not create circular 663be added, including other groups, as long as you do not create circular
637dependencies. 664dependencies.
638 665
… …
641=item $grp->result (...) 668=item $grp->result (...)
642 669
643Set the result value(s) that will be passed to the group callback when all 670Set the result value(s) that will be passed to the group callback when all
644subrequests have finished. By default, no argument will be passed. 671subrequests have finished. By default, no argument will be passed.
645 672
646=item $grp->set_feeder ($callback->($grp)) 673=item feed $grp $callback->($grp)
647 674
648[VERY EXPERIMENTAL] 675[VERY EXPERIMENTAL]
649 676
650Sets a feeder/generator on this group: every group can have an attached 677Sets a feeder/generator on this group: every group can have an attached
651generator that generates requests if idle. The idea behind this is that, 678generator that generates requests if idle. The idea behind this is that,
… …
654example, C<aio_scandir> might generate hundreds of thousands C<aio_stat> 681example, C<aio_scandir> might generate hundreds of thousands C<aio_stat>
655requests, delaying any later requests for a long time. 682requests, delaying any later requests for a long time.
656 683
657To avoid this, and allow incremental generation of requests, you can 684To avoid this, and allow incremental generation of requests, you can
658instead a group and set a feeder on it that generates those requests. The 685instead a group and set a feeder on it that generates those requests. The
659feeder will be called whenever there are few enough (see C<feeder_limit>, 686feed callback will be called whenever there are few enough (see C<feed_limit>,
660below) requests active in the group itself and is expected to queue more 687below) requests active in the group itself and is expected to queue more
661requests. 688requests.
662 689
663The feeder can queue as many requests as it likes (i.e. C<add> does not 690The feed can queue as many requests as it likes (i.e. C<add> does not
664impose any limits). 691impose any limits).
665 692
666If the feeder does not queue more requests when called, it will be 693If the feed does not queue more requests when called, it will be
667automatically removed from the group. 694automatically removed from the group.
668 695
669If the feeder limit is C<0>, it will be set to C<2> automatically. 696If the feed limit is C<0>, it will be set to C<2> automatically.
670 697
671Example: 698Example:
672 699
673 # stat all files in @files, but only ever use four aio requests concurrently: 700 # stat all files in @files, but only ever use four aio requests concurrently:
674 701
675 my $grp = aio_group sub { print "finished\n" }; 702 my $grp = aio_group sub { print "finished\n" };
676 $grp->feeder_limit (4); 703 feed_limit $grp 4;
677 $grp->set_feeder (sub { 704 feed $grp sub {
678 my $file = pop @files 705 my $file = pop @files
679 or return; 706 or return;
680 707
681 add $grp aio_stat $file, sub { ... }; 708 add $grp aio_stat $file, sub { ... };
682 }); 709 };
683 710
684=item $grp->feeder_limit ($num) 711=item feed_limit $grp $num
685 712
686Sets the feeder limit for the group: The feeder will be called whenever 713Sets the feeder limit for the group: The feeder will be called whenever
687the group contains less than this many requests. 714the group contains less than this many requests.
688 715
689Setting the limit to C<0> will pause the feeding process. 716Setting the limit to C<0> will pause the feeding process.
… …
784 811
785Under normal circumstances you don't need to call this function. 812Under normal circumstances you don't need to call this function.
786 813
787=item $oldnreqs = IO::AIO::max_outstanding $nreqs 814=item $oldnreqs = IO::AIO::max_outstanding $nreqs
788 815
816[DEPRECATED]
817
789Sets the maximum number of outstanding requests to C<$nreqs>. If you 818Sets the maximum number of outstanding requests to C<$nreqs>. If you
790try to queue up more than this number of requests, the caller will block until 819try to queue up more than this number of requests, the caller will block until
791some requests have been handled. 820some requests have been handled.
792 821
793The default is very large, so normally there is no practical limit. If you 822The default is very large, so normally there is no practical limit. If you
794queue up many requests in a loop it often improves speed if you set 823queue up many requests in a loop it often improves speed if you set
795this to a relatively low number, such as C<100>. 824this to a relatively low number, such as C<100>.
825
826This function does not work well together with C<aio_group>'s, and their
827feeder interface is better suited to limiting concurrency, so do not use
828this function.
796 829
797Under normal circumstances you don't need to call this function. 830Under normal circumstances you don't need to call this function.
798 831
799=back 832=back
800 833

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines