[ViewVC] Diff of: cvs/IO-AIO/AIO.pm

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.72 by root, Tue Oct 24 14:25:53 2006 UTC vs.
Revision 1.77 by root, Wed Oct 25 17:57:30 2006 UTC

                      aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink
                      aio_fsync aio_fdatasync aio_readahead aio_rename aio_link aio_move
                      aio_group aio_nop);
    our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice));
    our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
-                       min_parallel max_parallel max_outstanding nreqs);
+                       min_parallel max_parallel nreqs);
    @IO::AIO::GRP::ISA = 'IO::AIO::REQ';
    require XSLoader;
    XSLoader::load ("IO::AIO", $VERSION);
 with the filenames.
 =item aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
 Scans a directory (similar to C<aio_readdir>) but additionally tries to
-separate the entries of directory C<$path> into two sets of names, ones
+efficiently separate the entries of directory C<$path> into two sets of
-you can recurse into (directories or links to them), and ones you cannot
+names, directories you can recurse into (directories), and ones you cannot
-recurse into (everything else).
+recurse into (everything else, including symlinks to directories).
 C<aio_scandir> is a composite request that creates of many sub requests_
 C<$maxreq> specifies the maximum number of outstanding aio requests that
 this function generates. If it is C<< <= 0 >>, then a suitable default
 will be chosen (currently 6).
                               map [$_, sprintf "%s%04d", (/.\./ ? "1" : "0"), length],
                                  @$entries];
             my (@dirs, @nondirs);
-            my ($statcb, $schedcb);
-            my $nreq = 0;
-            my $statgrp = add $grp aio_group;
+            my $statgrp = add $grp aio_group sub {
+               $grp->result (\@dirs, \@nondirs);
+            };
-            $schedcb = sub {
+            limit $statgrp $maxreq;
-               if (@$entries) {
+            feed $statgrp sub {
-                  if ($nreq < $maxreq) {
+               return unless @$entries;
-                     my $ent = pop @$entries;
+               my $entry = pop @$entries;
+               add $statgrp aio_stat "$path/$entry/.", sub {
+                  if ($_[0] < 0) {
+                     push @nondirs, $entry;
+                  } else {
+                     # need to check for real directory
+                     add $statgrp aio_lstat "$path/$entry", sub {
+                        if (-d _) {
+                           push @dirs, $entry;
+                           unless (--$ndirs) {
+                              push @nondirs, @$entries;
+                              feed $statgrp;
+                           }
+                        } else {
+                           push @nondirs, $entry;
+                        }
-                     $nreq++;
+                     }
-                     add $statgrp aio_stat "$path/$ent/.", sub { $statcb->($_[0], $ent) };
                   }
-               } elsif (!$nreq) {
-                  # finished
-                  $statgrp->cancel;
-                  undef $statcb;
-                  undef $schedcb;
-                  $grp->result (\@dirs, \@nondirs);
-               }
+               };
             };
-            $statcb = sub {
-               my ($status, $entry) = @_;
-               if ($status < 0) {
-                  $nreq--;
-                  push @nondirs, $entry;
-                  &$schedcb;
-               } else {
-                  # need to check for real directory
-                  add $grp aio_lstat "$path/$entry", sub {
-                     $nreq--;
-                     if (-d _) {
-                        push @dirs, $entry;
-                        if (!--$ndirs) {
-                           push @nondirs, @$entries;
-                           $entries = [];
-                        }
-                     } else {
-                        push @nondirs, $entry;
-                     }
-                     &$schedcb;
-                  }
-               }
-            };
-            &$schedcb while @$entries && $nreq < $maxreq;
          };
       };
    };
    $grp
 If this call isn't available because your OS lacks it or it couldn't be
 detected, it will be emulated by calling C<fsync> instead.
 =item aio_group $callback->(...)
-[EXPERIMENTAL]
 This is a very special aio request: Instead of doing something, it is a
 container for other aio requests, which is useful if you want to bundle
 many requests into a single, composite, request with a definite callback
 and the ability to cancel the whole request with its subrequests.
 =item * They can also can also be added to other IO::AIO::GRP objects.
 =item * You must not add requests to a group from within the group callback (or
 any later time).
-=item * This does not harmonise well with C<max_outstanding>, so best do
-not combine C<aio_group> with it. Groups and feeders are recommended for
-this kind of concurrency-limiting.
 =back
 Their lifetime, simplified, looks like this: when they are empty, they
 will finish very quickly. If they contain only requests that are in the
 C<done> state, they will also finish. Otherwise they will continue to
 be added, including other groups, as long as you do not create circular
 dependencies.
 Returns all its arguments.
+=item $grp->cancel_subs
+Cancel all subrequests and clears any feeder, but not the group request
+itself. Useful when you queued a lot of events but got a result early.
 =item $grp->result (...)
 Set the result value(s) that will be passed to the group callback when all
 subrequests have finished. By default, no argument will be passed.
 =item feed $grp $callback->($grp)
-[VERY EXPERIMENTAL]
 Sets a feeder/generator on this group: every group can have an attached
 generator that generates requests if idle. The idea behind this is that,
 although you could just queue as many requests as you want in a group,
 this might starve other requests for a potentially long time.  For
 Under normal circumstances you don't need to call this function.
 =item $oldnreqs = IO::AIO::max_outstanding $nreqs
-[DEPRECATED]
+[REMOVED]
+Pre-2.x versions used max_outstanding for a crude request queue length limit.
+In 2.x+ you are advised to use a group and a feeder to limit
+concurrency. The max_outstanding feature ran very unstable (endless
+recursions causing segfaults, bad interaction with groups etc.) and was
+removed.
+I am deeply sorry, but I am still on the hunt for a good limiting interface.
+Original description was as follows:
-Sets the maximum number of outstanding requests to C<$nreqs>. If you
+Sets the maximum number of outstanding requests to C<$nreqs>. If you try
-try to queue up more than this number of requests, the caller will block until
+to queue up more than this number of requests, the caller will block until
 some requests have been handled.
-The default is very large, so normally there is no practical limit. If you
-queue up many requests in a loop it often improves speed if you set
-this to a relatively low number, such as C<100>.
-This function does not work well together with C<aio_group>'s, and their
-feeder interface is better suited to limiting concurrency, so do not use
-this function.
-Under normal circumstances you don't need to call this function.
 =back
 =cut
 request/result processing, while the child frees the request/result queue
 (so that the requests started before the fork will only be handled in the
 parent). Threads will be started on demand until the limit set in the
 parent process has been reached again.
-Temporary memory that was allocated for request processing is not
-reclaimed in the child, however. While this is possible in some cases, it
-is almost impossible in others (threads are evil you know), so you will
-have to live with it. This is around 64k buffer (for sendfile, readahead
-emulation) + the size of the directory being scanned (readdir).
 In short: the parent will, after a short pause, continue as if fork had
 not been called, while the child will act as if IO::AIO has not been used
 yet.
 =head2 MEMORY USAGE
 temporary buffers, and each thread requires a stack and other data
 structures (usually around 16k-128k, depending on the OS).
 =head1 KNOWN BUGS
-See FORK BEHAVIOUR, above.
+Known bugs will be fixed in the next release.
 =head1 SEE ALSO
 L<Coro::AIO>.

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing IO-AIO/AIO.pm (file contents): Revision 1.72 by root, Tue Oct 24 14:25:53 2006 UTC vs. Revision 1.77 by root, Wed Oct 25 17:57:30 2006 UTC

Diff Legend

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.72 by root, Tue Oct 24 14:25:53 2006 UTC vs.
Revision 1.77 by root, Wed Oct 25 17:57:30 2006 UTC