[ViewVC] Diff of: cvs/IO-AIO/AIO.pm

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.191 by root, Thu May 19 23:57:12 2011 UTC vs.
Revision 1.201 by root, Tue Jul 5 09:24:11 2011 UTC

 use common::sense;
 use base 'Exporter';
 BEGIN {
-   our $VERSION = '3.8';
+   our $VERSION = '3.93';
    our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close
                      aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
-                     aio_scandir aio_symlink aio_readlink aio_sync aio_fsync
+                     aio_scandir aio_symlink aio_readlink aio_realpath aio_sync aio_fsync
                      aio_fdatasync aio_sync_file_range aio_pathsync aio_readahead
                      aio_rename aio_link aio_move aio_copy aio_group
                      aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
                      aio_chmod aio_utime aio_truncate
                      aio_msync aio_mtouch aio_mlock aio_mlockall
    aio_unlink $pathname, $callback->($status)
    aio_mknod $path, $mode, $dev, $callback->($status)
    aio_link $srcpath, $dstpath, $callback->($status)
    aio_symlink $srcpath, $dstpath, $callback->($status)
    aio_readlink $path, $callback->($link)
+   aio_realpath $path, $callback->($link)
    aio_rename $srcpath, $dstpath, $callback->($status)
    aio_mkdir $pathname, $mode, $callback->($status)
    aio_rmdir $pathname, $callback->($status)
    aio_readdir $pathname, $callback->($entries)
    aio_readdirx $pathname, $flags, $callback->($entries, $flags)
       } else {
          die "open failed: $!\n";
       }
    };
+In addition to all the common open modes/flags (C<O_RDONLY>, C<O_WRONLY>,
+C<O_RDWR>, C<O_CREAT>, C<O_TRUNC>, C<O_EXCL> and C<O_APPEND>), the
+following POSIX and non-POSIX constants are available (missing ones on
+your system are, as usual, C<0>):
+C<O_ASYNC>, C<O_DIRECT>, C<O_NOATIME>, C<O_CLOEXEC>, C<O_NOCTTY>, C<O_NOFOLLOW>,
+C<O_NONBLOCK>, C<O_EXEC>, C<O_SEARCH>, C<O_DIRECTORY>, C<O_DSYNC>,
+C<O_RSYNC>, C<O_SYNC> and C<O_TTY_INIT>.
 =item aio_close $fh, $callback->($status)
 Asynchronously close a file and call the callback with the result
 code.
 Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
 reading at byte offset C<$in_offset>, and starts writing at the current
 file offset of C<$out_fh>. Because of that, it is not safe to issue more
 than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each
-other.
+other. The same C<$in_fh> works fine though, as this function does not
+move or use the file offset of C<$in_fh>.
 Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than
-are written, and there is no way to find out how many bytes have been read
+are written, and there is no way to find out how many more bytes have been
-from C<aio_sendfile> alone, as C<aio_sendfile> only provides the number of
+read from C<aio_sendfile> alone, as C<aio_sendfile> only provides the
-bytes written to C<$out_fh>. Only if the result value equals C<$length>
+number of bytes written to C<$out_fh>. Only if the result value equals
-one can assume that C<$length> bytes have been read.
+C<$length> one can assume that C<$length> bytes have been read.
 Unlike with other C<aio_> functions, it makes a lot of sense to use
 C<aio_sendfile> on non-blocking sockets, as long as one end (typically
 the C<$in_fh>) is a file - the file I/O will then be asynchronous, while
-the socket I/O will be non-blocking. Note, however, that you can run into
+the socket I/O will be non-blocking. Note, however, that you can run
-a trap where C<aio_sendfile> reads some data with readahead, then fails
+into a trap where C<aio_sendfile> reads some data with readahead, then
-to write all data, and when the socket is ready the next time, the data
+fails to write all data, and when the socket is ready the next time, the
-in the cache is already lost, forcing C<aio_sendfile> to again hit the
+data in the cache is already lost, forcing C<aio_sendfile> to again hit
-disk. Explicit C<aio_read> + C<aio_write> let's you control resource usage
+the disk. Explicit C<aio_read> + C<aio_write> let's you better control
-much better.
+resource usage.
-This call tries to make use of a native C<sendfile> syscall to provide
+This call tries to make use of a native C<sendfile>-like syscall to
-zero-copy operation. For this to work, C<$out_fh> should refer to a
+provide zero-copy operation. For this to work, C<$out_fh> should refer to
-socket, and C<$in_fh> should refer to an mmap'able file.
+a socket, and C<$in_fh> should refer to an mmap'able file.
 If a native sendfile cannot be found or it fails with C<ENOSYS>,
-C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or C<ENOTSOCK>,
+C<EINVAL>, C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or
-it will be emulated, so you can call C<aio_sendfile> on any type of
+C<ENOTSOCK>, it will be emulated, so you can call C<aio_sendfile> on any
-filehandle regardless of the limitations of the operating system.
+type of filehandle regardless of the limitations of the operating system.
+As native sendfile syscalls (as practically any non-POSIX interface hacked
+together in a hurry to improve benchmark numbers) tend to be rather buggy
+on many systems, this implementation tries to work around some known bugs
+in Linux and FreeBSD kernels (probably others, too), but that might fail,
+so you really really should check the return value of C<aio_sendfile> -
+fewre bytes than expected might have been transferred.
 =item aio_readahead $fh,$offset,$length, $callback->($retval)
 C<aio_readahead> populates the page cache with data from a file so that
 Asynchronously read the symlink specified by C<$path> and pass it to
 the callback. If an error occurs, nothing or undef gets passed to the
 callback.
+=item aio_realpath $path, $callback->($path)
+Asynchronously make the path absolute and resolve any symlinks in
+C<$path>. The resulting path only consists of directories.
+This request can be used to get the absolute path of the current working
+directory by passing it a path of F<.> (a single dot).
 =item aio_rename $srcpath, $dstpath, $callback->($status)
 Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as
 rename(2) and call the callback with the result code.
 systems that do not deliver the inode information.
 =item IO::AIO::READDIR_DIRS_FIRST
 When this flag is set, then the names will be returned in an order where
-likely directories come first. This is useful when you need to quickly
+likely directories come first, in optimal stat order. This is useful when
-find directories, or you want to find all directories while avoiding to
+you need to quickly find directories, or you want to find all directories
-stat() each entry.
+while avoiding to stat() each entry.
 If the system returns type information in readdir, then this is used
-to find directories directly.  Otherwise, likely directories are files
+to find directories directly. Otherwise, likely directories are names
-beginning with ".", or otherwise files with no dots, of which files with
+beginning with ".", or otherwise names with no dots, of which names with
 short names are tried first.
 =item IO::AIO::READDIR_STAT_ORDER
 When this flag is set, then the names will be returned in an order
       if ($_[0] && $! == EXDEV) {
          aioreq_pri $pri;
          add $grp aio_copy $src, $dst, sub {
             $grp->result ($_[0]);
-            if (!$_[0]) {
+            unless ($_[0]) {
                aioreq_pri $pri;
                add $grp aio_unlink $src;
             }
          };
       } else {
 If not all requests were processed for whatever reason, the filehandle
 will still be ready when C<poll_cb> returns, so normally you don't have to
 do anything special to have it called later.
+Apart from calling C<IO::AIO::poll_cb> when the event filehandle becomes
+ready, it can be beneficial to call this function from loops which submit
+a lot of requests, to make sure the results get processed when they become
+available and not just when the loop is finished and the event loop takes
+over again. This function returns very fast when there are no outstanding
+requests.
 Example: Install an Event watcher that automatically calls
 IO::AIO::poll_cb with high priority (more examples can be found in the
 SYNOPSIS section, at the top of this document):
    Event->io (fd => IO::AIO::poll_fileno,
 Sets the minimum idle timeout (default 10) after which worker threads are
 allowed to exit. SEe C<IO::AIO::max_idle>.
 =item IO::AIO::max_outstanding $maxreqs
+Sets the maximum number of outstanding requests to C<$nreqs>. If
+you do queue up more than this number of requests, the next call to
+C<IO::AIO::poll_cb> (and other functions calling C<poll_cb>, such as
+C<IO::AIO::flush> or C<IO::AIO::poll>) will block until the limit is no
+longer exceeded.
+In other words, this setting does not enforce a queue limit, but can be
+used to make poll functions block if the limit is exceeded.
 This is a very bad function to use in interactive programs because it
 blocks, and a bad way to reduce concurrency because it is inexact: Better
 use an C<aio_group> together with a feed callback.
-Sets the maximum number of outstanding requests to C<$nreqs>. If you
+It's main use is in scripts without an event loop - when you want to stat
-do queue up more than this number of requests, the next call to the
+a lot of files, you can write somehting like this:
-C<poll_cb> (and C<poll_some> and other functions calling C<poll_cb>)
-function will block until the limit is no longer exceeded.
-The default value is very large, so there is no practical limit on the
+   IO::AIO::max_outstanding 32;
-number of outstanding requests.
-You can still queue as many requests as you want. Therefore,
+   for my $path (...) {
-C<max_outstanding> is mainly useful in simple scripts (with low values) or
+      aio_stat $path , ...;
-as a stop gap to shield against fatal memory overflow (with large values).
+      IO::AIO::poll_cb;
+   }
+   IO::AIO::flush;
+The call to C<poll_cb> inside the loop will normally return instantly, but
+as soon as more thna C<32> reqeusts are in-flight, it will block until
+some requests have been handled. This keeps the loop from pushing a large
+number of C<aio_stat> requests onto the queue.
+The default value for C<max_outstanding> is very large, so there is no
+practical limit on the number of outstanding requests.
 =back
 =head3 STATISTICAL INFORMATION
  Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
                              \&IO::AIO::poll_cb);
 =head2 FORK BEHAVIOUR
-This module should do "the right thing" when the process using it forks:
+Usage of pthreads in a program changes the semantics of fork
+considerably. Specifically, only async-safe functions can be called after
+fork. Perl doesn't know about this, so in general, you cannot call fork
+with defined behaviour in perl. IO::AIO uses pthreads, so this applies,
+but many other extensions and (for inexplicable reasons) perl itself often
+is linked against pthreads, so this limitation applies.
-Before the fork, IO::AIO enters a quiescent state where no requests
+Some operating systems have extensions that allow safe use of fork, and
-can be added in other threads and no results will be processed. After
+this module should do "the right thing" on those, and tries on others. At
-the fork the parent simply leaves the quiescent state and continues
+the time of this writing (2011) only GNU/Linux supports these extensions
-request/result processing, while the child frees the request/result queue
+to POSIX.
-(so that the requests started before the fork will only be handled in the
-parent). Threads will be started on demand until the limit set in the
-parent process has been reached again.
-In short: the parent will, after a short pause, continue as if fork had
-not been called, while the child will act as if IO::AIO has not been used
-yet.
 =head2 MEMORY USAGE
 Per-request usage:

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing IO-AIO/AIO.pm (file contents): Revision 1.191 by root, Thu May 19 23:57:12 2011 UTC vs. Revision 1.201 by root, Tue Jul 5 09:24:11 2011 UTC

Diff Legend

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.191 by root, Thu May 19 23:57:12 2011 UTC vs.
Revision 1.201 by root, Tue Jul 5 09:24:11 2011 UTC