[ViewVC] Diff of: cvs/IO-AIO/AIO.pm

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.142 by root, Wed Oct 22 18:15:36 2008 UTC vs.
Revision 1.155 by root, Sun Jun 14 20:36:59 2009 UTC

 use strict 'vars';
 use base 'Exporter';
 BEGIN {
-   our $VERSION = '3.16';
+   our $VERSION = '3.23';
    our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close
-                     aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir
+                     aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
                      aio_scandir aio_symlink aio_readlink aio_sync aio_fsync
                      aio_fdatasync aio_sync_file_range aio_pathsync aio_readahead
                      aio_rename aio_link aio_move aio_copy aio_group
                      aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
                      aio_chmod aio_utime aio_truncate);
    our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice));
    our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
                        min_parallel max_parallel max_idle
                        nreqs nready npending nthreads
                        max_poll_time max_poll_reqs);
+   push @AIO_REQ, qw(aio_busy); # not exported
    @IO::AIO::GRP::ISA = 'IO::AIO::REQ';
    require XSLoader;
    XSLoader::load ("IO::AIO", $VERSION);
 =item aio_read  $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
 =item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
-Reads or writes C<$length> bytes from the specified C<$fh> and C<$offset>
+Reads or writes C<$length> bytes from or to the specified C<$fh> and
-into the scalar given by C<$data> and offset C<$dataoffset> and calls the
+C<$offset> into the scalar given by C<$data> and offset C<$dataoffset>
-callback without the actual number of bytes read (or -1 on error, just
+and calls the callback without the actual number of bytes read (or -1 on
-like the syscall).
+error, just like the syscall).
+C<aio_read> will, like C<sysread>, shrink or grow the C<$data> scalar to
+offset plus the actual number of bytes read.
 If C<$offset> is undefined, then the current file descriptor offset will
 be used (and updated), otherwise the file descriptor offset will not be
 changed by these calls.
-If C<$length> is undefined in C<aio_write>, use the remaining length of C<$data>.
+If C<$length> is undefined in C<aio_write>, use the remaining length of
+C<$data>.
 If C<$dataoffset> is less than zero, it will be counted from the end of
 C<$data>.
 The C<$data> scalar I<MUST NOT> be modified in any way while the request
 Unlike the POSIX call of the same name, C<aio_readdir> reads an entire
 directory (i.e. opendir + readdir + closedir). The entries will not be
 sorted, and will B<NOT> include the C<.> and C<..> entries.
-The callback a single argument which is either C<undef> or an array-ref
+The callback is passed a single argument which is either C<undef> or an
-with the filenames.
+array-ref with the filenames.
+=item aio_readdirx $pathname, $flags, $callback->($entries, $flags)
+Quite similar to C<aio_readdir>, but the C<$flags> argument allows to tune
+behaviour and output format. In case of an error, C<$entries> will be
+C<undef>.
+The flags are a combination of the following constants, ORed together (the
+flags will also be passed to the callback, possibly modified):
+=over 4
+=item IO::AIO::READDIR_DENTS
+When this flag is off, then the callback gets an arrayref with of names
+only (as with C<aio_readdir>), otherwise it gets an arrayref with
+C<[$name, $type, $inode]> arrayrefs, each describing a single directory
+entry in more detail.
+C<$name> is the name of the entry.
+C<$type> is one of the C<IO::AIO::DT_xxx> constants:
+C<IO::AIO::DT_UNKNOWN>, C<IO::AIO::DT_FIFO>, C<IO::AIO::DT_CHR>, C<IO::AIO::DT_DIR>,
+C<IO::AIO::DT_BLK>, C<IO::AIO::DT_REG>, C<IO::AIO::DT_LNK>, C<IO::AIO::DT_SOCK>,
+C<IO::AIO::DT_WHT>.
+C<IO::AIO::DT_UNKNOWN> means just that: readdir does not know. If you need to
+know, you have to run stat yourself. Also, for speed reasons, the C<$type>
+scalars are read-only: you can not modify them.
+C<$inode> is the inode number (which might not be exact on systems with 64
+bit inode numbers and 32 bit perls). This field has unspecified content on
+systems that do not deliver the inode information.
+=item IO::AIO::READDIR_DIRS_FIRST
+When this flag is set, then the names will be returned in an order where
+likely directories come first. This is useful when you need to quickly
+find directories, or you want to find all directories while avoiding to
+stat() each entry.
+If the system returns type information in readdir, then this is used
+to find directories directly.  Otherwise, likely directories are files
+beginning with ".", or otherwise files with no dots, of which files with
+short names are tried first.
+=item IO::AIO::READDIR_STAT_ORDER
+When this flag is set, then the names will be returned in an order
+suitable for stat()'ing each one. That is, when you plan to stat()
+all files in the given directory, then the returned order will likely
+be fastest.
+If both this flag and C<IO::AIO::READDIR_DIRS_FIRST> are specified, then
+the likely dirs come first, resulting in a less optimal stat order.
+=item IO::AIO::READDIR_FOUND_UNKNOWN
+This flag should not be set when calling C<aio_readdirx>. Instead, it
+is being set by C<aio_readdirx>, when any of the C<$type>'s found were
+C<IO::AIO::DT_UNKNOWN>. The absense of this flag therefore indicates that all
+C<$type>'s are known, which can be used to speed up some algorithms.
+=back
 =item aio_load $path, $data, $callback->($status)
 This is a composite request that tries to fully load the given file into
    my $grp = aio_group $cb;
    aioreq_pri $pri;
    add $grp aio_open $src, O_RDONLY, 0, sub {
       if (my $src_fh = $_[0]) {
-         my @stat = stat $src_fh;
+         my @stat = stat $src_fh; # hmm, might bock over nfs?
          aioreq_pri $pri;
          add $grp aio_open $dst, O_CREAT | O_WRONLY | O_TRUNC, 0200, sub {
             if (my $dst_fh = $_[0]) {
                aioreq_pri $pri;
                add $grp aio_sendfile $dst_fh, $src_fh, 0, $stat[7], sub {
                   if ($_[0] == $stat[7]) {
                      $grp->result (0);
                      close $src_fh;
-                     # those should not normally block. should. should.
+                     my $ch = sub {
-                     utime $stat[8], $stat[9], $dst;
+                        aioreq_pri $pri;
-                     chmod $stat[2] & 07777, $dst_fh;
+                        add $grp aio_chmod $dst_fh, $stat[2] & 07777, sub {
-                     chown $stat[4], $stat[5], $dst_fh;
+                           aioreq_pri $pri;
+                           add $grp aio_chown $dst_fh, $stat[4], $stat[5], sub {
+                              aioreq_pri $pri;
+                              add $grp aio_close $dst_fh;
+                           }
+                        };
+                     };
                      aioreq_pri $pri;
-                     add $grp aio_close $dst_fh;
+                     add $grp aio_utime $dst_fh, $stat[8], $stat[9], sub {
+                        if ($_[0] < 0 && $! == ENOSYS) {
+                           aioreq_pri $pri;
+                           add $grp aio_utime $dst, $stat[8], $stat[9], $ch;
+                        } else {
+                           $ch->();
+                        }
+                     };
                   } else {
                      $grp->result (-1);
                      close $src_fh;
                      close $dst_fh;
 Implementation notes.
 The C<aio_readdir> cannot be avoided, but C<stat()>'ing every entry can.
+If readdir returns file type information, then this is used directly to
+find directories.
-After reading the directory, the modification time, size etc. of the
+Otherwise, after reading the directory, the modification time, size etc.
-directory before and after the readdir is checked, and if they match (and
+of the directory before and after the readdir is checked, and if they
-isn't the current time), the link count will be used to decide how many
+match (and isn't the current time), the link count will be used to decide
-entries are directories (if >= 2). Otherwise, no knowledge of the number
+how many entries are directories (if >= 2). Otherwise, no knowledge of the
-of subdirectories will be assumed.
+number of subdirectories will be assumed.
-Then entries will be sorted into likely directories (everything without
+Then entries will be sorted into likely directories a non-initial dot
-a non-initial dot currently) and likely non-directories (everything
+currently) and likely non-directories (see C<aio_readdirx>). Then every
-else). Then every entry plus an appended C</.> will be C<stat>'ed,
+entry plus an appended C</.> will be C<stat>'ed, likely directories first,
-likely directories first. If that succeeds, it assumes that the entry
+in order of their inode numbers. If that succeeds, it assumes that the
-is a directory or a symlink to directory (which will be checked
+entry is a directory or a symlink to directory (which will be checked
 seperately). This is often faster than stat'ing the entry itself because
 filesystems might detect the type of the entry without reading the inode
-data (e.g. ext2fs filetype feature).
+data (e.g. ext2fs filetype feature), even on systems that cannot return
+the filetype information on readdir.
 If the known number of directories (link count - 2) has been reached, the
 rest of the entries is assumed to be non-directories.
 This only works with certainty on POSIX (= UNIX) filesystems, which
       my $now = time;
       my $hash1 = join ":", (stat _)[0,1,3,7,9];
       # read the directory entries
       aioreq_pri $pri;
-      add $grp aio_readdir $path, sub {
+      add $grp aio_readdirx $path, READDIR_DIRS_FIRST, sub {
          my $entries = shift
             or return $grp->result ();
          # stat the dir another time
          aioreq_pri $pri;
             # take the slow route if anything looks fishy
             if ($hash1 ne $hash2 or (stat _)[9] == $now) {
                $ndirs = -1;
             } else {
                # if nlink == 2, we are finished
-               # on non-posix-fs's, we rely on nlink < 2
+               # for non-posix-fs's, we rely on nlink < 2
                $ndirs = (stat _)[3] - 2
                   or return $grp->result ([], $entries);
             }
-            # sort into likely dirs and likely nondirs
-            # dirs == files without ".", short entries first
-            $entries = [map $_->[0],
-                           sort { $b->[1] cmp $a->[1] }
-                              map [$_, sprintf "%s%04d", (/.\./ ? "1" : "0"), length],
-                                 @$entries];
             my (@dirs, @nondirs);
             my $statgrp = add $grp aio_group sub {
                $grp->result (\@dirs, \@nondirs);
             };
             limit $statgrp $maxreq;
             feed $statgrp sub {
                return unless @$entries;
-               my $entry = pop @$entries;
+               my $entry = shift @$entries;
                aioreq_pri $pri;
                add $statgrp aio_stat "$path/$entry/.", sub {
                   if ($_[0] < 0) {
                      push @nondirs, $entry;
 =item cancel $req
 Cancels the request, if possible. Has the effect of skipping execution
 when entering the B<execute> state and skipping calling the callback when
 entering the the B<result> state, but will leave the request otherwise
-untouched. That means that requests that currently execute will not be
+untouched (with the exception of readdir). That means that requests that
-stopped and resources held by the request will not be freed prematurely.
+currently execute will not be stopped and resources held by the request
+will not be freed prematurely.
 =item cb $req $callback->(...)
 Replace (or simply set) the callback registered to the request.

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing IO-AIO/AIO.pm (file contents): Revision 1.142 by root, Wed Oct 22 18:15:36 2008 UTC vs. Revision 1.155 by root, Sun Jun 14 20:36:59 2009 UTC

Diff Legend

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.142 by root, Wed Oct 22 18:15:36 2008 UTC vs.
Revision 1.155 by root, Sun Jun 14 20:36:59 2009 UTC