ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.193 by root, Thu May 26 04:15:37 2011 UTC vs.
Revision 1.202 by root, Tue Jul 5 14:02:15 2011 UTC

… …
168use common::sense; 168use common::sense;
169 169
170use base 'Exporter'; 170use base 'Exporter';
171 171
172BEGIN { 172BEGIN {
173 our $VERSION = '3.8'; 173 our $VERSION = '3.93';
174 174
175 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close 175 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close
176 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx 176 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
177 aio_scandir aio_symlink aio_readlink aio_sync aio_fsync 177 aio_scandir aio_symlink aio_readlink aio_realpath aio_sync aio_fsync
178 aio_fdatasync aio_sync_file_range aio_pathsync aio_readahead 178 aio_fdatasync aio_sync_file_range aio_pathsync aio_readahead
179 aio_rename aio_link aio_move aio_copy aio_group 179 aio_rename aio_link aio_move aio_copy aio_group
180 aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown 180 aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
181 aio_chmod aio_utime aio_truncate 181 aio_chmod aio_utime aio_truncate
182 aio_msync aio_mtouch aio_mlock aio_mlockall 182 aio_msync aio_mtouch aio_mlock aio_mlockall
… …
222 aio_unlink $pathname, $callback->($status) 222 aio_unlink $pathname, $callback->($status)
223 aio_mknod $path, $mode, $dev, $callback->($status) 223 aio_mknod $path, $mode, $dev, $callback->($status)
224 aio_link $srcpath, $dstpath, $callback->($status) 224 aio_link $srcpath, $dstpath, $callback->($status)
225 aio_symlink $srcpath, $dstpath, $callback->($status) 225 aio_symlink $srcpath, $dstpath, $callback->($status)
226 aio_readlink $path, $callback->($link) 226 aio_readlink $path, $callback->($link)
227 aio_realpath $path, $callback->($link)
227 aio_rename $srcpath, $dstpath, $callback->($status) 228 aio_rename $srcpath, $dstpath, $callback->($status)
228 aio_mkdir $pathname, $mode, $callback->($status) 229 aio_mkdir $pathname, $mode, $callback->($status)
229 aio_rmdir $pathname, $callback->($status) 230 aio_rmdir $pathname, $callback->($status)
230 aio_readdir $pathname, $callback->($entries) 231 aio_readdir $pathname, $callback->($entries)
231 aio_readdirx $pathname, $flags, $callback->($entries, $flags) 232 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
… …
368 } else { 369 } else {
369 die "open failed: $!\n"; 370 die "open failed: $!\n";
370 } 371 }
371 }; 372 };
372 373
374In addition to all the common open modes/flags (C<O_RDONLY>, C<O_WRONLY>,
375C<O_RDWR>, C<O_CREAT>, C<O_TRUNC>, C<O_EXCL> and C<O_APPEND>), the
376following POSIX and non-POSIX constants are available (missing ones on
377your system are, as usual, C<0>):
378
379C<O_ASYNC>, C<O_DIRECT>, C<O_NOATIME>, C<O_CLOEXEC>, C<O_NOCTTY>, C<O_NOFOLLOW>,
380C<O_NONBLOCK>, C<O_EXEC>, C<O_SEARCH>, C<O_DIRECTORY>, C<O_DSYNC>,
381C<O_RSYNC>, C<O_SYNC> and C<O_TTY_INIT>.
382
373 383
374=item aio_close $fh, $callback->($status) 384=item aio_close $fh, $callback->($status)
375 385
376Asynchronously close a file and call the callback with the result 386Asynchronously close a file and call the callback with the result
377code. 387code.
… …
427 437
428Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts 438Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
429reading at byte offset C<$in_offset>, and starts writing at the current 439reading at byte offset C<$in_offset>, and starts writing at the current
430file offset of C<$out_fh>. Because of that, it is not safe to issue more 440file offset of C<$out_fh>. Because of that, it is not safe to issue more
431than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each 441than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each
432other. 442other. The same C<$in_fh> works fine though, as this function does not
443move or use the file offset of C<$in_fh>.
433 444
434Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than 445Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than
435are written, and there is no way to find out how many bytes have been read 446are written, and there is no way to find out how many more bytes have been
436from C<aio_sendfile> alone, as C<aio_sendfile> only provides the number of 447read from C<aio_sendfile> alone, as C<aio_sendfile> only provides the
437bytes written to C<$out_fh>. Only if the result value equals C<$length> 448number of bytes written to C<$out_fh>. Only if the result value equals
438one can assume that C<$length> bytes have been read. 449C<$length> one can assume that C<$length> bytes have been read.
439 450
440Unlike with other C<aio_> functions, it makes a lot of sense to use 451Unlike with other C<aio_> functions, it makes a lot of sense to use
441C<aio_sendfile> on non-blocking sockets, as long as one end (typically 452C<aio_sendfile> on non-blocking sockets, as long as one end (typically
442the C<$in_fh>) is a file - the file I/O will then be asynchronous, while 453the C<$in_fh>) is a file - the file I/O will then be asynchronous, while
443the socket I/O will be non-blocking. Note, however, that you can run into 454the socket I/O will be non-blocking. Note, however, that you can run
444a trap where C<aio_sendfile> reads some data with readahead, then fails 455into a trap where C<aio_sendfile> reads some data with readahead, then
445to write all data, and when the socket is ready the next time, the data 456fails to write all data, and when the socket is ready the next time, the
446in the cache is already lost, forcing C<aio_sendfile> to again hit the 457data in the cache is already lost, forcing C<aio_sendfile> to again hit
447disk. Explicit C<aio_read> + C<aio_write> let's you control resource usage 458the disk. Explicit C<aio_read> + C<aio_write> let's you better control
448much better. 459resource usage.
449 460
450This call tries to make use of a native C<sendfile> syscall to provide 461This call tries to make use of a native C<sendfile>-like syscall to
451zero-copy operation. For this to work, C<$out_fh> should refer to a 462provide zero-copy operation. For this to work, C<$out_fh> should refer to
452socket, and C<$in_fh> should refer to an mmap'able file. 463a socket, and C<$in_fh> should refer to an mmap'able file.
453 464
454If a native sendfile cannot be found or it fails with C<ENOSYS>, 465If a native sendfile cannot be found or it fails with C<ENOSYS>,
455C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or C<ENOTSOCK>, 466C<EINVAL>, C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or
456it will be emulated, so you can call C<aio_sendfile> on any type of 467C<ENOTSOCK>, it will be emulated, so you can call C<aio_sendfile> on any
457filehandle regardless of the limitations of the operating system. 468type of filehandle regardless of the limitations of the operating system.
469
470As native sendfile syscalls (as practically any non-POSIX interface hacked
471together in a hurry to improve benchmark numbers) tend to be rather buggy
472on many systems, this implementation tries to work around some known bugs
473in Linux and FreeBSD kernels (probably others, too), but that might fail,
474so you really really should check the return value of C<aio_sendfile> -
475fewre bytes than expected might have been transferred.
458 476
459 477
460=item aio_readahead $fh,$offset,$length, $callback->($retval) 478=item aio_readahead $fh,$offset,$length, $callback->($retval)
461 479
462C<aio_readahead> populates the page cache with data from a file so that 480C<aio_readahead> populates the page cache with data from a file so that
… …
626Asynchronously read the symlink specified by C<$path> and pass it to 644Asynchronously read the symlink specified by C<$path> and pass it to
627the callback. If an error occurs, nothing or undef gets passed to the 645the callback. If an error occurs, nothing or undef gets passed to the
628callback. 646callback.
629 647
630 648
649=item aio_realpath $path, $callback->($path)
650
651Asynchronously make the path absolute and resolve any symlinks in
652C<$path>. The resulting path only consists of directories (Same as
653L<Cwd::realpath>).
654
655This request can be used to get the absolute path of the current working
656directory by passing it a path of F<.> (a single dot).
657
658
631=item aio_rename $srcpath, $dstpath, $callback->($status) 659=item aio_rename $srcpath, $dstpath, $callback->($status)
632 660
633Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as 661Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as
634rename(2) and call the callback with the result code. 662rename(2) and call the callback with the result code.
635 663
… …
853 if ($_[0] && $! == EXDEV) { 881 if ($_[0] && $! == EXDEV) {
854 aioreq_pri $pri; 882 aioreq_pri $pri;
855 add $grp aio_copy $src, $dst, sub { 883 add $grp aio_copy $src, $dst, sub {
856 $grp->result ($_[0]); 884 $grp->result ($_[0]);
857 885
858 if (!$_[0]) { 886 unless ($_[0]) {
859 aioreq_pri $pri; 887 aioreq_pri $pri;
860 add $grp aio_unlink $src; 888 add $grp aio_unlink $src;
861 } 889 }
862 }; 890 };
863 } else { 891 } else {
… …
1555Sets the minimum idle timeout (default 10) after which worker threads are 1583Sets the minimum idle timeout (default 10) after which worker threads are
1556allowed to exit. SEe C<IO::AIO::max_idle>. 1584allowed to exit. SEe C<IO::AIO::max_idle>.
1557 1585
1558=item IO::AIO::max_outstanding $maxreqs 1586=item IO::AIO::max_outstanding $maxreqs
1559 1587
1588Sets the maximum number of outstanding requests to C<$nreqs>. If
1589you do queue up more than this number of requests, the next call to
1590C<IO::AIO::poll_cb> (and other functions calling C<poll_cb>, such as
1591C<IO::AIO::flush> or C<IO::AIO::poll>) will block until the limit is no
1592longer exceeded.
1593
1594In other words, this setting does not enforce a queue limit, but can be
1595used to make poll functions block if the limit is exceeded.
1596
1560This is a very bad function to use in interactive programs because it 1597This is a very bad function to use in interactive programs because it
1561blocks, and a bad way to reduce concurrency because it is inexact: Better 1598blocks, and a bad way to reduce concurrency because it is inexact: Better
1562use an C<aio_group> together with a feed callback. 1599use an C<aio_group> together with a feed callback.
1563 1600
1564Sets the maximum number of outstanding requests to C<$nreqs>. If you 1601It's main use is in scripts without an event loop - when you want to stat
1565do queue up more than this number of requests, the next call to the 1602a lot of files, you can write somehting like this:
1566C<poll_cb> (and C<poll_some> and other functions calling C<poll_cb>)
1567function will block until the limit is no longer exceeded.
1568 1603
1569The default value is very large, so there is no practical limit on the 1604 IO::AIO::max_outstanding 32;
1570number of outstanding requests.
1571 1605
1572You can still queue as many requests as you want. Therefore, 1606 for my $path (...) {
1573C<max_outstanding> is mainly useful in simple scripts (with low values) or 1607 aio_stat $path , ...;
1574as a stop gap to shield against fatal memory overflow (with large values). 1608 IO::AIO::poll_cb;
1609 }
1610
1611 IO::AIO::flush;
1612
1613The call to C<poll_cb> inside the loop will normally return instantly, but
1614as soon as more thna C<32> reqeusts are in-flight, it will block until
1615some requests have been handled. This keeps the loop from pushing a large
1616number of C<aio_stat> requests onto the queue.
1617
1618The default value for C<max_outstanding> is very large, so there is no
1619practical limit on the number of outstanding requests.
1575 1620
1576=back 1621=back
1577 1622
1578=head3 STATISTICAL INFORMATION 1623=head3 STATISTICAL INFORMATION
1579 1624
… …
1755 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno => 1800 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
1756 \&IO::AIO::poll_cb); 1801 \&IO::AIO::poll_cb);
1757 1802
1758=head2 FORK BEHAVIOUR 1803=head2 FORK BEHAVIOUR
1759 1804
1760This module should do "the right thing" when the process using it forks: 1805Usage of pthreads in a program changes the semantics of fork
1806considerably. Specifically, only async-safe functions can be called after
1807fork. Perl doesn't know about this, so in general, you cannot call fork
1808with defined behaviour in perl. IO::AIO uses pthreads, so this applies,
1809but many other extensions and (for inexplicable reasons) perl itself often
1810is linked against pthreads, so this limitation applies.
1761 1811
1762Before the fork, IO::AIO enters a quiescent state where no requests 1812Some operating systems have extensions that allow safe use of fork, and
1763can be added in other threads and no results will be processed. After 1813this module should do "the right thing" on those, and tries on others. At
1764the fork the parent simply leaves the quiescent state and continues 1814the time of this writing (2011) only GNU/Linux supports these extensions
1765request/result processing, while the child frees the request/result queue 1815to POSIX.
1766(so that the requests started before the fork will only be handled in the
1767parent). Threads will be started on demand until the limit set in the
1768parent process has been reached again.
1769
1770In short: the parent will, after a short pause, continue as if fork had
1771not been called, while the child will act as if IO::AIO has not been used
1772yet.
1773 1816
1774=head2 MEMORY USAGE 1817=head2 MEMORY USAGE
1775 1818
1776Per-request usage: 1819Per-request usage:
1777 1820

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines