ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.78 by root, Thu Oct 26 12:38:04 2006 UTC vs.
Revision 1.81 by root, Fri Oct 27 19:17:23 2006 UTC

134 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink 134 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink
135 aio_fsync aio_fdatasync aio_readahead aio_rename aio_link aio_move 135 aio_fsync aio_fdatasync aio_readahead aio_rename aio_link aio_move
136 aio_group aio_nop); 136 aio_group aio_nop);
137 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice)); 137 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice));
138 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush 138 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
139 min_parallel max_parallel nreqs); 139 min_parallel max_parallel nreqs nready npending);
140 140
141 @IO::AIO::GRP::ISA = 'IO::AIO::REQ'; 141 @IO::AIO::GRP::ISA = 'IO::AIO::REQ';
142 142
143 require XSLoader; 143 require XSLoader;
144 XSLoader::load ("IO::AIO", $VERSION); 144 XSLoader::load ("IO::AIO", $VERSION);
175environment, d) use Glib::filename_from_unicode on unicode filenames or e) 175environment, d) use Glib::filename_from_unicode on unicode filenames or e)
176use something else. 176use something else.
177 177
178=over 4 178=over 4
179 179
180=item aioreq_pri $pri 180=item $prev_pri = aioreq_pri [$pri]
181 181
182Sets the priority for the next aio request. The default priority 182Returns the priority value that would be used for the next request and, if
183C<$pri> is given, sets the priority for the next aio request.
184
183is C<0>, the minimum and maximum priorities are C<-4> and C<4>, 185The default priority is C<0>, the minimum and maximum priorities are C<-4>
184respectively. Requests with higher priority will be serviced first. 186and C<4>, respectively. Requests with higher priority will be serviced
187first.
185 188
186The priority will be reset to C<0> after each call to one of the C<aio_> 189The priority will be reset to C<0> after each call to one of the C<aio_*>
187functions. 190functions.
188 191
189Example: open a file with low priority, then read something from it with 192Example: open a file with low priority, then read something from it with
190higher priority so the read request is serviced before other low priority 193higher priority so the read request is serviced before other low priority
191open requests (potentially spamming the cache): 194open requests (potentially spamming the cache):
283=cut 286=cut
284 287
285sub aio_move($$$) { 288sub aio_move($$$) {
286 my ($src, $dst, $cb) = @_; 289 my ($src, $dst, $cb) = @_;
287 290
291 my $pri = aioreq_pri;
288 my $grp = aio_group $cb; 292 my $grp = aio_group $cb;
289 293
294 aioreq_pri $pri;
290 add $grp aio_rename $src, $dst, sub { 295 add $grp aio_rename $src, $dst, sub {
291 if ($_[0] && $! == EXDEV) { 296 if ($_[0] && $! == EXDEV) {
297 aioreq_pri $pri;
292 add $grp aio_open $src, O_RDONLY, 0, sub { 298 add $grp aio_open $src, O_RDONLY, 0, sub {
293 if (my $src_fh = $_[0]) { 299 if (my $src_fh = $_[0]) {
294 my @stat = stat $src_fh; 300 my @stat = stat $src_fh;
295 301
302 aioreq_pri $pri;
296 add $grp aio_open $dst, O_WRONLY, 0200, sub { 303 add $grp aio_open $dst, O_WRONLY, 0200, sub {
297 if (my $dst_fh = $_[0]) { 304 if (my $dst_fh = $_[0]) {
305 aioreq_pri $pri;
298 add $grp aio_sendfile $dst_fh, $src_fh, 0, $stat[7], sub { 306 add $grp aio_sendfile $dst_fh, $src_fh, 0, $stat[7], sub {
299 close $src_fh; 307 close $src_fh;
300 308
301 if ($_[0] == $stat[7]) { 309 if ($_[0] == $stat[7]) {
302 utime $stat[8], $stat[9], $dst; 310 utime $stat[8], $stat[9], $dst;
303 chmod $stat[2] & 07777, $dst_fh; 311 chmod $stat[2] & 07777, $dst_fh;
304 chown $stat[4], $stat[5], $dst_fh; 312 chown $stat[4], $stat[5], $dst_fh;
305 close $dst_fh; 313 close $dst_fh;
306 314
315 aioreq_pri $pri;
307 add $grp aio_unlink $src, sub { 316 add $grp aio_unlink $src, sub {
308 $grp->result ($_[0]); 317 $grp->result ($_[0]);
309 }; 318 };
310 } else { 319 } else {
311 my $errno = $!; 320 my $errno = $!;
321 aioreq_pri $pri;
312 add $grp aio_unlink $dst, sub { 322 add $grp aio_unlink $dst, sub {
313 $! = $errno; 323 $! = $errno;
314 $grp->result (-1); 324 $grp->result (-1);
315 }; 325 };
316 } 326 }
433recurse into (everything else, including symlinks to directories). 443recurse into (everything else, including symlinks to directories).
434 444
435C<aio_scandir> is a composite request that creates of many sub requests_ 445C<aio_scandir> is a composite request that creates of many sub requests_
436C<$maxreq> specifies the maximum number of outstanding aio requests that 446C<$maxreq> specifies the maximum number of outstanding aio requests that
437this function generates. If it is C<< <= 0 >>, then a suitable default 447this function generates. If it is C<< <= 0 >>, then a suitable default
438will be chosen (currently 6). 448will be chosen (currently 4).
439 449
440On error, the callback is called without arguments, otherwise it receives 450On error, the callback is called without arguments, otherwise it receives
441two array-refs with path-relative entry names. 451two array-refs with path-relative entry names.
442 452
443Example: 453Example:
480=cut 490=cut
481 491
482sub aio_scandir($$$) { 492sub aio_scandir($$$) {
483 my ($path, $maxreq, $cb) = @_; 493 my ($path, $maxreq, $cb) = @_;
484 494
495 my $pri = aioreq_pri;
496
485 my $grp = aio_group $cb; 497 my $grp = aio_group $cb;
486 498
487 $maxreq = 6 if $maxreq <= 0; 499 $maxreq = 4 if $maxreq <= 0;
488 500
489 # stat once 501 # stat once
502 aioreq_pri $pri;
490 add $grp aio_stat $path, sub { 503 add $grp aio_stat $path, sub {
491 return $grp->result () if $_[0]; 504 return $grp->result () if $_[0];
492 my $now = time; 505 my $now = time;
493 my $hash1 = join ":", (stat _)[0,1,3,7,9]; 506 my $hash1 = join ":", (stat _)[0,1,3,7,9];
494 507
495 # read the directory entries 508 # read the directory entries
509 aioreq_pri $pri;
496 add $grp aio_readdir $path, sub { 510 add $grp aio_readdir $path, sub {
497 my $entries = shift 511 my $entries = shift
498 or return $grp->result (); 512 or return $grp->result ();
499 513
500 # stat the dir another time 514 # stat the dir another time
515 aioreq_pri $pri;
501 add $grp aio_stat $path, sub { 516 add $grp aio_stat $path, sub {
502 my $hash2 = join ":", (stat _)[0,1,3,7,9]; 517 my $hash2 = join ":", (stat _)[0,1,3,7,9];
503 518
504 my $ndirs; 519 my $ndirs;
505 520
529 limit $statgrp $maxreq; 544 limit $statgrp $maxreq;
530 feed $statgrp sub { 545 feed $statgrp sub {
531 return unless @$entries; 546 return unless @$entries;
532 my $entry = pop @$entries; 547 my $entry = pop @$entries;
533 548
549 aioreq_pri $pri;
534 add $statgrp aio_stat "$path/$entry/.", sub { 550 add $statgrp aio_stat "$path/$entry/.", sub {
535 if ($_[0] < 0) { 551 if ($_[0] < 0) {
536 push @nondirs, $entry; 552 push @nondirs, $entry;
537 } else { 553 } else {
538 # need to check for real directory 554 # need to check for real directory
555 aioreq_pri $pri;
539 add $statgrp aio_lstat "$path/$entry", sub { 556 add $statgrp aio_lstat "$path/$entry", sub {
540 if (-d _) { 557 if (-d _) {
541 push @dirs, $entry; 558 push @dirs, $entry;
542 559
543 unless (--$ndirs) { 560 unless (--$ndirs) {
714itself. Useful when you queued a lot of events but got a result early. 731itself. Useful when you queued a lot of events but got a result early.
715 732
716=item $grp->result (...) 733=item $grp->result (...)
717 734
718Set the result value(s) that will be passed to the group callback when all 735Set the result value(s) that will be passed to the group callback when all
719subrequests have finished. By default, no argument will be passed. 736subrequests have finished and set thre groups errno to the current value
737of errno (just like calling C<errno> without an error number). By default,
738no argument will be passed and errno is zero.
739
740=item $grp->errno ([$errno])
741
742Sets the group errno value to C<$errno>, or the current value of errno
743when the argument is missing.
744
745Every aio request has an associated errno value that is restored when
746the callback is invoked. This method lets you change this value from its
747default (0).
748
749Calling C<result> will also set errno, so make sure you either set C<$!>
750before the call to C<result>, or call c<errno> after it.
720 751
721=item feed $grp $callback->($grp) 752=item feed $grp $callback->($grp)
722 753
723Sets a feeder/generator on this group: every group can have an attached 754Sets a feeder/generator on this group: every group can have an attached
724generator that generates requests if idle. The idea behind this is that, 755generator that generates requests if idle. The idea behind this is that,
816 847
817See C<nreqs> for an example. 848See C<nreqs> for an example.
818 849
819=item IO::AIO::nreqs 850=item IO::AIO::nreqs
820 851
821Returns the number of requests currently outstanding (i.e. for which their 852Returns the number of requests currently in the ready, execute or pending
822callback has not been invoked yet). 853states (i.e. for which their callback has not been invoked yet).
823 854
824Example: wait till there are no outstanding requests anymore: 855Example: wait till there are no outstanding requests anymore:
825 856
826 IO::AIO::poll_wait, IO::AIO::poll_cb 857 IO::AIO::poll_wait, IO::AIO::poll_cb
827 while IO::AIO::nreqs; 858 while IO::AIO::nreqs;
859
860=item IO::AIO::nready
861
862Returns the number of requests currently in the ready state (not yet
863executed).
864
865=item IO::AIO::npending
866
867Returns the number of requests currently in the pending state (executed,
868but not yet processed by poll_cb).
828 869
829=item IO::AIO::flush 870=item IO::AIO::flush
830 871
831Wait till all outstanding AIO requests have been handled. 872Wait till all outstanding AIO requests have been handled.
832 873
874This module automatically runs C<max_parallel 0> at program end, to ensure 915This module automatically runs C<max_parallel 0> at program end, to ensure
875that all threads are killed and that there are no outstanding requests. 916that all threads are killed and that there are no outstanding requests.
876 917
877Under normal circumstances you don't need to call this function. 918Under normal circumstances you don't need to call this function.
878 919
879=item $oldnreqs = IO::AIO::max_outstanding $nreqs 920=item $oldmaxreqs = IO::AIO::max_outstanding $maxreqs
880 921
881[REMOVED] 922This is a very bad function to use in interactive programs because it
923blocks, and a bad way to reduce concurrency because it is inexact: Better
924use an C<aio_group> together with a feed callback.
882 925
883Pre-2.x versions used max_outstanding for a crude request queue length limit.
884
885In 2.x+ you are advised to use a group and a feeder to limit
886concurrency. The max_outstanding feature ran very unstable (endless
887recursions causing segfaults, bad interaction with groups etc.) and was
888removed.
889
890I am deeply sorry, but I am still on the hunt for a good limiting interface.
891
892Original description was as follows:
893
894Sets the maximum number of outstanding requests to C<$nreqs>. If you try 926Sets the maximum number of outstanding requests to C<$nreqs>. If you
895to queue up more than this number of requests, the caller will block until 927to queue up more than this number of requests, the next call to the
896some requests have been handled. 928C<poll_cb> (and C<poll_some> and other functions calling C<poll_cb>)
929function will block until the limit is no longer exceeded.
930
931The default value is very large, so there is no practical limit on the
932number of outstanding requests.
933
934You can still queue as many requests as you want. Therefore,
935C<max_oustsanding> is mainly useful in simple scripts (with low values) or
936as a stop gap to shield against fatal memory overflow (with large values).
897 937
898=back 938=back
899 939
900=cut 940=cut
901 941
914 954
915 *$sym 955 *$sym
916} 956}
917 957
918min_parallel 8; 958min_parallel 8;
919
920END {
921 max_parallel 0;
922}
923 959
9241; 9601;
925 961
926=head2 FORK BEHAVIOUR 962=head2 FORK BEHAVIOUR
927 963

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines