ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.59 by root, Sun Oct 22 10:33:26 2006 UTC vs.
Revision 1.79 by root, Thu Oct 26 14:35:34 2006 UTC

18 }; 18 };
19 19
20 # version 2+ has request and group objects 20 # version 2+ has request and group objects
21 use IO::AIO 2; 21 use IO::AIO 2;
22 22
23 aioreq_pri 4; # give next request a very high priority
23 my $req = aio_unlink "/tmp/file", sub { }; 24 my $req = aio_unlink "/tmp/file", sub { };
24 $req->cancel; # cancel request if still in queue 25 $req->cancel; # cancel request if still in queue
25 26
26 my $grp = aio_group sub { print "all stats done\n" }; 27 my $grp = aio_group sub { print "all stats done\n" };
27 add $grp aio_stat "..." for ...; 28 add $grp aio_stat "..." for ...;
50=head1 DESCRIPTION 51=head1 DESCRIPTION
51 52
52This module implements asynchronous I/O using whatever means your 53This module implements asynchronous I/O using whatever means your
53operating system supports. 54operating system supports.
54 55
55Currently, a number of threads are started that execute your read/writes 56In this version, a number of threads are started that execute your
56and signal their completion. You don't need thread support in your libc or 57requests and signal their completion. You don't need thread support
57perl, and the threads created by this module will not be visible to the 58in perl, and the threads created by this module will not be visible
58pthreads library. In the future, this module might make use of the native 59to perl. In the future, this module might make use of the native aio
59aio functions available on many operating systems. However, they are often 60functions available on many operating systems. However, they are often
60not well-supported (Linux doesn't allow them on normal files currently, 61not well-supported or restricted (Linux doesn't allow them on normal
61for example), and they would only support aio_read and aio_write, so the 62files currently, for example), and they would only support aio_read and
62remaining functionality would have to be implemented using threads anyway. 63aio_write, so the remaining functionality would have to be implemented
64using threads anyway.
63 65
64Although the module will work with in the presence of other threads, it is 66Although the module will work with in the presence of other (Perl-)
65currently not reentrant, so use appropriate locking yourself, always call 67threads, it is currently not reentrant in any way, so use appropriate
66C<poll_cb> from within the same thread, or never call C<poll_cb> (or other 68locking yourself, always call C<poll_cb> from within the same thread, or
67C<aio_> functions) recursively. 69never call C<poll_cb> (or other C<aio_> functions) recursively.
70
71=head1 REQUEST ANATOMY AND LIFETIME
72
73Every C<aio_*> function creates a request. which is a C data structure not
74directly visible to Perl.
75
76If called in non-void context, every request function returns a Perl
77object representing the request. In void context, nothing is returned,
78which saves a bit of memory.
79
80The perl object is a fairly standard ref-to-hash object. The hash contents
81are not used by IO::AIO so you are free to store anything you like in it.
82
83During their existance, aio requests travel through the following states,
84in order:
85
86=over 4
87
88=item ready
89
90Immediately after a request is created it is put into the ready state,
91waiting for a thread to execute it.
92
93=item execute
94
95A thread has accepted the request for processing and is currently
96executing it (e.g. blocking in read).
97
98=item pending
99
100The request has been executed and is waiting for result processing.
101
102While request submission and execution is fully asynchronous, result
103processing is not and relies on the perl interpreter calling C<poll_cb>
104(or another function with the same effect).
105
106=item result
107
108The request results are processed synchronously by C<poll_cb>.
109
110The C<poll_cb> function will process all outstanding aio requests by
111calling their callbacks, freeing memory associated with them and managing
112any groups they are contained in.
113
114=item done
115
116Request has reached the end of its lifetime and holds no resources anymore
117(except possibly for the Perl object, but its connection to the actual
118aio request is severed and calling its methods will either do nothing or
119result in a runtime error).
68 120
69=cut 121=cut
70 122
71package IO::AIO; 123package IO::AIO;
72 124
76use base 'Exporter'; 128use base 'Exporter';
77 129
78BEGIN { 130BEGIN {
79 our $VERSION = '2.0'; 131 our $VERSION = '2.0';
80 132
81 our @EXPORT = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat 133 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat
82 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink 134 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink
83 aio_fsync aio_fdatasync aio_readahead aio_rename aio_link aio_move 135 aio_fsync aio_fdatasync aio_readahead aio_rename aio_link aio_move
84 aio_group); 136 aio_group aio_nop);
85 our @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel max_outstanding nreqs); 137 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice));
138 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
139 min_parallel max_parallel nreqs);
86 140
87 @IO::AIO::GRP::ISA = 'IO::AIO::REQ'; 141 @IO::AIO::GRP::ISA = 'IO::AIO::REQ';
88 142
89 require XSLoader; 143 require XSLoader;
90 XSLoader::load ("IO::AIO", $VERSION); 144 XSLoader::load ("IO::AIO", $VERSION);
121environment, d) use Glib::filename_from_unicode on unicode filenames or e) 175environment, d) use Glib::filename_from_unicode on unicode filenames or e)
122use something else. 176use something else.
123 177
124=over 4 178=over 4
125 179
180=item aioreq_pri $pri
181
182Sets the priority for the next aio request. The default priority
183is C<0>, the minimum and maximum priorities are C<-4> and C<4>,
184respectively. Requests with higher priority will be serviced first.
185
186The priority will be reset to C<0> after each call to one of the C<aio_>
187functions.
188
189Example: open a file with low priority, then read something from it with
190higher priority so the read request is serviced before other low priority
191open requests (potentially spamming the cache):
192
193 aioreq_pri -3;
194 aio_open ..., sub {
195 return unless $_[0];
196
197 aioreq_pri -2;
198 aio_read $_[0], ..., sub {
199 ...
200 };
201 };
202
203=item aioreq_nice $pri_adjust
204
205Similar to C<aioreq_pri>, but subtracts the given value from the current
206priority, so effects are cumulative.
207
126=item aio_open $pathname, $flags, $mode, $callback->($fh) 208=item aio_open $pathname, $flags, $mode, $callback->($fh)
127 209
128Asynchronously open or create a file and call the callback with a newly 210Asynchronously open or create a file and call the callback with a newly
129created filehandle for the file. 211created filehandle for the file.
130 212
181 $_[0] > 0 or die "read error: $!"; 263 $_[0] > 0 or die "read error: $!";
182 print "read $_[0] bytes: <$buffer>\n"; 264 print "read $_[0] bytes: <$buffer>\n";
183 }; 265 };
184 266
185=item aio_move $srcpath, $dstpath, $callback->($status) 267=item aio_move $srcpath, $dstpath, $callback->($status)
186
187[EXPERIMENTAL due to internal aio_group use]
188 268
189Try to move the I<file> (directories not supported as either source or 269Try to move the I<file> (directories not supported as either source or
190destination) from C<$srcpath> to C<$dstpath> and call the callback with 270destination) from C<$srcpath> to C<$dstpath> and call the callback with
191the C<0> (error) or C<-1> ok. 271the C<0> (error) or C<-1> ok.
192 272
345The callback a single argument which is either C<undef> or an array-ref 425The callback a single argument which is either C<undef> or an array-ref
346with the filenames. 426with the filenames.
347 427
348=item aio_scandir $path, $maxreq, $callback->($dirs, $nondirs) 428=item aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
349 429
350[EXPERIMENTAL due to internal aio_group use]
351
352Scans a directory (similar to C<aio_readdir>) but additionally tries to 430Scans a directory (similar to C<aio_readdir>) but additionally tries to
353separate the entries of directory C<$path> into two sets of names, ones 431efficiently separate the entries of directory C<$path> into two sets of
354you can recurse into (directories or links to them), and ones you cannot 432names, directories you can recurse into (directories), and ones you cannot
355recurse into (everything else). 433recurse into (everything else, including symlinks to directories).
356 434
357C<aio_scandir> is a composite request that consists of many sub 435C<aio_scandir> is a composite request that creates of many sub requests_
358requests. C<$maxreq> specifies the maximum number of outstanding aio 436C<$maxreq> specifies the maximum number of outstanding aio requests that
359requests that this function generates. If it is C<< <= 0 >>, then a 437this function generates. If it is C<< <= 0 >>, then a suitable default
360suitable default will be chosen (currently 8). 438will be chosen (currently 6).
361 439
362On error, the callback is called without arguments, otherwise it receives 440On error, the callback is called without arguments, otherwise it receives
363two array-refs with path-relative entry names. 441two array-refs with path-relative entry names.
364 442
365Example: 443Example:
404sub aio_scandir($$$) { 482sub aio_scandir($$$) {
405 my ($path, $maxreq, $cb) = @_; 483 my ($path, $maxreq, $cb) = @_;
406 484
407 my $grp = aio_group $cb; 485 my $grp = aio_group $cb;
408 486
409 $maxreq = 8 if $maxreq <= 0; 487 $maxreq = 6 if $maxreq <= 0;
410 488
411 # stat once 489 # stat once
412 add $grp aio_stat $path, sub { 490 add $grp aio_stat $path, sub {
413 return $grp->result () if $_[0]; 491 return $grp->result () if $_[0];
414 my $now = time; 492 my $now = time;
442 map [$_, sprintf "%s%04d", (/.\./ ? "1" : "0"), length], 520 map [$_, sprintf "%s%04d", (/.\./ ? "1" : "0"), length],
443 @$entries]; 521 @$entries];
444 522
445 my (@dirs, @nondirs); 523 my (@dirs, @nondirs);
446 524
447 my ($statcb, $schedcb); 525 my $statgrp = add $grp aio_group sub {
448 my $nreq = 0; 526 $grp->result (\@dirs, \@nondirs);
527 };
449 528
450 $schedcb = sub { 529 limit $statgrp $maxreq;
451 if (@$entries) { 530 feed $statgrp sub {
452 if ($nreq < $maxreq) { 531 return unless @$entries;
453 my $ent = pop @$entries; 532 my $entry = pop @$entries;
533
534 add $statgrp aio_stat "$path/$entry/.", sub {
535 if ($_[0] < 0) {
536 push @nondirs, $entry;
537 } else {
538 # need to check for real directory
539 add $statgrp aio_lstat "$path/$entry", sub {
540 if (-d _) {
541 push @dirs, $entry;
542
543 unless (--$ndirs) {
544 push @nondirs, @$entries;
545 feed $statgrp;
546 }
547 } else {
548 push @nondirs, $entry;
549 }
454 $nreq++; 550 }
455 add $grp aio_stat "$path/$ent/.", sub { $statcb->($_[0], $ent) };
456 } 551 }
457 } elsif (!$nreq) {
458 # finished
459 undef $statcb;
460 undef $schedcb;
461 $grp->result (\@dirs, \@nondirs) if $cb;
462 undef $cb;
463 } 552 };
464 }; 553 };
465 $statcb = sub {
466 my ($status, $entry) = @_;
467
468 if ($status < 0) {
469 $nreq--;
470 push @nondirs, $entry;
471 &$schedcb;
472 } else {
473 # need to check for real directory
474 add $grp aio_lstat "$path/$entry", sub {
475 $nreq--;
476
477 if (-d _) {
478 push @dirs, $entry;
479
480 if (!--$ndirs) {
481 push @nondirs, @$entries;
482 $entries = [];
483 }
484 } else {
485 push @nondirs, $entry;
486 }
487
488 &$schedcb;
489 }
490 }
491 };
492
493 &$schedcb while @$entries && $nreq < $maxreq;
494 }; 554 };
495 }; 555 };
496 }; 556 };
497 557
498 $grp 558 $grp
511If this call isn't available because your OS lacks it or it couldn't be 571If this call isn't available because your OS lacks it or it couldn't be
512detected, it will be emulated by calling C<fsync> instead. 572detected, it will be emulated by calling C<fsync> instead.
513 573
514=item aio_group $callback->(...) 574=item aio_group $callback->(...)
515 575
516[EXPERIMENTAL]
517
518This is a very special aio request: Instead of doing something, it is a 576This is a very special aio request: Instead of doing something, it is a
519container for other aio requests, which is useful if you want to bundle 577container for other aio requests, which is useful if you want to bundle
520many requests into a single, composite, request. 578many requests into a single, composite, request with a definite callback
579and the ability to cancel the whole request with its subrequests.
521 580
522Returns an object of class L<IO::AIO::GRP>. See its documentation below 581Returns an object of class L<IO::AIO::GRP>. See its documentation below
523for more info. 582for more info.
524 583
525Example: 584Example:
531 add $grp 590 add $grp
532 (aio_stat ...), 591 (aio_stat ...),
533 (aio_stat ...), 592 (aio_stat ...),
534 ...; 593 ...;
535 594
595=item aio_nop $callback->()
596
597This is a special request - it does nothing in itself and is only used for
598side effects, such as when you want to add a dummy request to a group so
599that finishing the requests in the group depends on executing the given
600code.
601
602While this request does nothing, it still goes through the execution
603phase and still requires a worker thread. Thus, the callback will not
604be executed immediately but only after other requests in the queue have
605entered their execution phase. This can be used to measure request
606latency.
607
536=item IO::AIO::aio_sleep $fractional_seconds, $callback->() *NOT EXPORTED* 608=item IO::AIO::aio_busy $fractional_seconds, $callback->() *NOT EXPORTED*
537 609
538Mainly used for debugging and benchmarking, this aio request puts one of 610Mainly used for debugging and benchmarking, this aio request puts one of
539the request workers to sleep for the given time. 611the request workers to sleep for the given time.
540 612
541While it is theoretically handy to have simple I/O scheduling requests 613While it is theoretically handy to have simple I/O scheduling requests
542like sleep and file handle readable/writable, the overhead this creates 614like sleep and file handle readable/writable, the overhead this creates is
543is immense, so do not use this function except to put your application 615immense (it blocks a thread for a long time) so do not use this function
544under artificial I/O pressure. 616except to put your application under artificial I/O pressure.
545 617
546=back 618=back
547 619
548=head2 IO::AIO::REQ CLASS 620=head2 IO::AIO::REQ CLASS
549 621
550All non-aggregate C<aio_*> functions return an object of this class when 622All non-aggregate C<aio_*> functions return an object of this class when
551called in non-void context. 623called in non-void context.
552 624
553A request always moves through the following five states in its lifetime,
554in order: B<ready> (request has been created, but has not been executed
555yet), B<execute> (request is currently being executed), B<pending>
556(request has been executed but callback has not been called yet),
557B<result> (results are being processed synchronously, includes calling the
558callback) and B<done> (request has reached the end of its lifetime and
559holds no resources anymore).
560
561=over 4 625=over 4
562 626
563=item $req->cancel 627=item cancel $req
564 628
565Cancels the request, if possible. Has the effect of skipping execution 629Cancels the request, if possible. Has the effect of skipping execution
566when entering the B<execute> state and skipping calling the callback when 630when entering the B<execute> state and skipping calling the callback when
567entering the the B<result> state, but will leave the request otherwise 631entering the the B<result> state, but will leave the request otherwise
568untouched. That means that requests that currently execute will not be 632untouched. That means that requests that currently execute will not be
569stopped and resources held by the request will not be freed prematurely. 633stopped and resources held by the request will not be freed prematurely.
570 634
635=item cb $req $callback->(...)
636
637Replace (or simply set) the callback registered to the request.
638
571=back 639=back
572 640
573=head2 IO::AIO::GRP CLASS 641=head2 IO::AIO::GRP CLASS
574 642
575This class is a subclass of L<IO::AIO::REQ>, so all its methods apply to 643This class is a subclass of L<IO::AIO::REQ>, so all its methods apply to
601 }; 669 };
602 670
603This makes it very easy to create composite requests (see the source of 671This makes it very easy to create composite requests (see the source of
604C<aio_move> for an application) that work and feel like simple requests. 672C<aio_move> for an application) that work and feel like simple requests.
605 673
674=over 4
675
606The IO::AIO::GRP objects will be cleaned up during calls to 676=item * The IO::AIO::GRP objects will be cleaned up during calls to
607C<IO::AIO::poll_cb>, just like any other request. 677C<IO::AIO::poll_cb>, just like any other request.
608 678
609They can be canceled like any other request. Canceling will cancel not 679=item * They can be canceled like any other request. Canceling will cancel not
610only the request itself, but also all requests it contains. 680only the request itself, but also all requests it contains.
611 681
612They can also can also be added to other IO::AIO::GRP objects. 682=item * They can also can also be added to other IO::AIO::GRP objects.
683
684=item * You must not add requests to a group from within the group callback (or
685any later time).
686
687=back
613 688
614Their lifetime, simplified, looks like this: when they are empty, they 689Their lifetime, simplified, looks like this: when they are empty, they
615will finish very quickly. If they contain only requests that are in the 690will finish very quickly. If they contain only requests that are in the
616C<done> state, they will also finish. Otherwise they will continue to 691C<done> state, they will also finish. Otherwise they will continue to
617exist. 692exist.
621group. And only when all those requests have finished will the the group 696group. And only when all those requests have finished will the the group
622itself finish. 697itself finish.
623 698
624=over 4 699=over 4
625 700
701=item add $grp ...
702
626=item $grp->add (...) 703=item $grp->add (...)
627
628=item add $grp ...
629 704
630Add one or more requests to the group. Any type of L<IO::AIO::REQ> can 705Add one or more requests to the group. Any type of L<IO::AIO::REQ> can
631be added, including other groups, as long as you do not create circular 706be added, including other groups, as long as you do not create circular
632dependencies. 707dependencies.
633 708
634Returns all its arguments. 709Returns all its arguments.
635 710
711=item $grp->cancel_subs
712
713Cancel all subrequests and clears any feeder, but not the group request
714itself. Useful when you queued a lot of events but got a result early.
715
636=item $grp->result (...) 716=item $grp->result (...)
637 717
638Set the result value(s) that will be passed to the group callback when all 718Set the result value(s) that will be passed to the group callback when all
639subrequests have finished. By default, no argument will be passed. 719subrequests have finished. By default, no argument will be passed.
720
721=item feed $grp $callback->($grp)
722
723Sets a feeder/generator on this group: every group can have an attached
724generator that generates requests if idle. The idea behind this is that,
725although you could just queue as many requests as you want in a group,
726this might starve other requests for a potentially long time. For
727example, C<aio_scandir> might generate hundreds of thousands C<aio_stat>
728requests, delaying any later requests for a long time.
729
730To avoid this, and allow incremental generation of requests, you can
731instead a group and set a feeder on it that generates those requests. The
732feed callback will be called whenever there are few enough (see C<limit>,
733below) requests active in the group itself and is expected to queue more
734requests.
735
736The feed callback can queue as many requests as it likes (i.e. C<add> does
737not impose any limits).
738
739If the feed does not queue more requests when called, it will be
740automatically removed from the group.
741
742If the feed limit is C<0>, it will be set to C<2> automatically.
743
744Example:
745
746 # stat all files in @files, but only ever use four aio requests concurrently:
747
748 my $grp = aio_group sub { print "finished\n" };
749 limit $grp 4;
750 feed $grp sub {
751 my $file = pop @files
752 or return;
753
754 add $grp aio_stat $file, sub { ... };
755 };
756
757=item limit $grp $num
758
759Sets the feeder limit for the group: The feeder will be called whenever
760the group contains less than this many requests.
761
762Setting the limit to C<0> will pause the feeding process.
640 763
641=back 764=back
642 765
643=head2 SUPPORT FUNCTIONS 766=head2 SUPPORT FUNCTIONS
644 767
657 780
658Process all outstanding events on the result pipe. You have to call this 781Process all outstanding events on the result pipe. You have to call this
659regularly. Returns the number of events processed. Returns immediately 782regularly. Returns the number of events processed. Returns immediately
660when no events are outstanding. 783when no events are outstanding.
661 784
785If not all requests were processed for whatever reason, the filehandle
786will still be ready when C<poll_cb> returns.
787
662Example: Install an Event watcher that automatically calls 788Example: Install an Event watcher that automatically calls
663IO::AIO::poll_cb with high priority: 789IO::AIO::poll_cb with high priority:
664 790
665 Event->io (fd => IO::AIO::poll_fileno, 791 Event->io (fd => IO::AIO::poll_fileno,
666 poll => 'r', async => 1, 792 poll => 'r', async => 1,
667 cb => \&IO::AIO::poll_cb); 793 cb => \&IO::AIO::poll_cb);
668 794
795=item IO::AIO::poll_some $max_requests
796
797Similar to C<poll_cb>, but only processes up to C<$max_requests> requests
798at a time.
799
800Useful if you want to ensure some level of interactiveness when perl is
801not fast enough to process all requests in time.
802
803Example: Install an Event watcher that automatically calls
804IO::AIO::poll_some with low priority, to ensure that other parts of the
805program get the CPU sometimes even under high AIO load.
806
807 Event->io (fd => IO::AIO::poll_fileno,
808 poll => 'r', nice => 1,
809 cb => sub { IO::AIO::poll_some 256 });
810
669=item IO::AIO::poll_wait 811=item IO::AIO::poll_wait
670 812
671Wait till the result filehandle becomes ready for reading (simply does a 813Wait till the result filehandle becomes ready for reading (simply does a
672C<select> on the filehandle. This is useful if you want to synchronously wait 814C<select> on the filehandle. This is useful if you want to synchronously wait
673for some requests to finish). 815for some requests to finish).
702 IO::AIO::poll_wait, IO::AIO::poll_cb 844 IO::AIO::poll_wait, IO::AIO::poll_cb
703 if IO::AIO::nreqs; 845 if IO::AIO::nreqs;
704 846
705=item IO::AIO::min_parallel $nthreads 847=item IO::AIO::min_parallel $nthreads
706 848
707Set the minimum number of AIO threads to C<$nthreads>. The current default 849Set the minimum number of AIO threads to C<$nthreads>. The current
708is C<4>, which means four asynchronous operations can be done at one time 850default is C<8>, which means eight asynchronous operations can execute
709(the number of outstanding operations, however, is unlimited). 851concurrently at any one time (the number of outstanding requests,
852however, is unlimited).
710 853
711IO::AIO starts threads only on demand, when an AIO request is queued and 854IO::AIO starts threads only on demand, when an AIO request is queued and
712no free thread exists. 855no free thread exists.
713 856
714It is recommended to keep the number of threads low, as some Linux 857It is recommended to keep the number of threads relatively low, as some
715kernel versions will scale negatively with the number of threads (higher 858Linux kernel versions will scale negatively with the number of threads
716parallelity => MUCH higher latency). With current Linux 2.6 versions, 4-32 859(higher parallelity => MUCH higher latency). With current Linux 2.6
717threads should be fine. 860versions, 4-32 threads should be fine.
718 861
719Under most circumstances you don't need to call this function, as the 862Under most circumstances you don't need to call this function, as the
720module selects a default that is suitable for low to moderate load. 863module selects a default that is suitable for low to moderate load.
721 864
722=item IO::AIO::max_parallel $nthreads 865=item IO::AIO::max_parallel $nthreads
731This module automatically runs C<max_parallel 0> at program end, to ensure 874This module automatically runs C<max_parallel 0> at program end, to ensure
732that all threads are killed and that there are no outstanding requests. 875that all threads are killed and that there are no outstanding requests.
733 876
734Under normal circumstances you don't need to call this function. 877Under normal circumstances you don't need to call this function.
735 878
736=item $oldnreqs = IO::AIO::max_outstanding $nreqs 879=item $oldmaxreqs = IO::AIO::max_outstanding $maxreqs
880
881This is a very bad function to use in interactive programs because it
882blocks, and a bad way to reduce concurrency because it is inexact: Better
883use an C<aio_group> together with a feed callback.
737 884
738Sets the maximum number of outstanding requests to C<$nreqs>. If you 885Sets the maximum number of outstanding requests to C<$nreqs>. If you
739try to queue up more than this number of requests, the caller will block until 886to queue up more than this number of requests, the next call to the
740some requests have been handled. 887C<poll_cb> (and C<poll_some> and other functions calling C<poll_cb>)
888function will block until the limit is no longer exceeded.
741 889
742The default is very large, so normally there is no practical limit. If you 890The default value is very large, so there is no practical limit on the
743queue up many requests in a loop it often improves speed if you set 891number of outstanding requests.
744this to a relatively low number, such as C<100>.
745 892
746Under normal circumstances you don't need to call this function. 893You can still queue as many requests as you want. Therefore,
894C<max_oustsanding> is mainly useful in simple scripts (with low values) or
895as a stop gap to shield against fatal memory overflow (with large values).
747 896
748=back 897=back
749 898
750=cut 899=cut
751 900
763 or return undef; 912 or return undef;
764 913
765 *$sym 914 *$sym
766} 915}
767 916
768min_parallel 4; 917min_parallel 8;
769 918
770END { 919END {
771 max_parallel 0; 920 max_parallel 0;
772} 921}
773 922
778This module should do "the right thing" when the process using it forks: 927This module should do "the right thing" when the process using it forks:
779 928
780Before the fork, IO::AIO enters a quiescent state where no requests 929Before the fork, IO::AIO enters a quiescent state where no requests
781can be added in other threads and no results will be processed. After 930can be added in other threads and no results will be processed. After
782the fork the parent simply leaves the quiescent state and continues 931the fork the parent simply leaves the quiescent state and continues
783request/result processing, while the child clears the request/result 932request/result processing, while the child frees the request/result queue
784queue (so the requests started before the fork will only be handled in 933(so that the requests started before the fork will only be handled in the
785the parent). Threads will be started on demand until the limit ste in the 934parent). Threads will be started on demand until the limit set in the
786parent process has been reached again. 935parent process has been reached again.
787 936
788In short: the parent will, after a short pause, continue as if fork had 937In short: the parent will, after a short pause, continue as if fork had
789not been called, while the child will act as if IO::AIO has not been used 938not been called, while the child will act as if IO::AIO has not been used
790yet. 939yet.
791 940
941=head2 MEMORY USAGE
942
943Per-request usage:
944
945Each aio request uses - depending on your architecture - around 100-200
946bytes of memory. In addition, stat requests need a stat buffer (possibly
947a few hundred bytes), readdir requires a result buffer and so on. Perl
948scalars and other data passed into aio requests will also be locked and
949will consume memory till the request has entered the done state.
950
951This is now awfully much, so queuing lots of requests is not usually a
952problem.
953
954Per-thread usage:
955
956In the execution phase, some aio requests require more memory for
957temporary buffers, and each thread requires a stack and other data
958structures (usually around 16k-128k, depending on the OS).
959
960=head1 KNOWN BUGS
961
962Known bugs will be fixed in the next release.
963
792=head1 SEE ALSO 964=head1 SEE ALSO
793 965
794L<Coro>, L<Linux::AIO> (obsolete). 966L<Coro::AIO>.
795 967
796=head1 AUTHOR 968=head1 AUTHOR
797 969
798 Marc Lehmann <schmorp@schmorp.de> 970 Marc Lehmann <schmorp@schmorp.de>
799 http://home.schmorp.de/ 971 http://home.schmorp.de/

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines