ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.194 by root, Fri May 27 00:44:49 2011 UTC vs.
Revision 1.199 by root, Wed Jun 29 12:46:36 2011 UTC

… …
168use common::sense; 168use common::sense;
169 169
170use base 'Exporter'; 170use base 'Exporter';
171 171
172BEGIN { 172BEGIN {
173 our $VERSION = '3.9'; 173 our $VERSION = '3.92';
174 174
175 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close 175 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close
176 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx 176 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
177 aio_scandir aio_symlink aio_readlink aio_sync aio_fsync 177 aio_scandir aio_symlink aio_readlink aio_sync aio_fsync
178 aio_fdatasync aio_sync_file_range aio_pathsync aio_readahead 178 aio_fdatasync aio_sync_file_range aio_pathsync aio_readahead
… …
436 436
437Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts 437Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
438reading at byte offset C<$in_offset>, and starts writing at the current 438reading at byte offset C<$in_offset>, and starts writing at the current
439file offset of C<$out_fh>. Because of that, it is not safe to issue more 439file offset of C<$out_fh>. Because of that, it is not safe to issue more
440than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each 440than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each
441other. 441other. The same C<$in_fh> works fine though, as this function does not
442move or use the file offset of C<$in_fh>.
442 443
443Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than 444Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than
444are written, and there is no way to find out how many bytes have been read 445are written, and there is no way to find out how many more bytes have been
445from C<aio_sendfile> alone, as C<aio_sendfile> only provides the number of 446read from C<aio_sendfile> alone, as C<aio_sendfile> only provides the
446bytes written to C<$out_fh>. Only if the result value equals C<$length> 447number of bytes written to C<$out_fh>. Only if the result value equals
447one can assume that C<$length> bytes have been read. 448C<$length> one can assume that C<$length> bytes have been read.
448 449
449Unlike with other C<aio_> functions, it makes a lot of sense to use 450Unlike with other C<aio_> functions, it makes a lot of sense to use
450C<aio_sendfile> on non-blocking sockets, as long as one end (typically 451C<aio_sendfile> on non-blocking sockets, as long as one end (typically
451the C<$in_fh>) is a file - the file I/O will then be asynchronous, while 452the C<$in_fh>) is a file - the file I/O will then be asynchronous, while
452the socket I/O will be non-blocking. Note, however, that you can run into 453the socket I/O will be non-blocking. Note, however, that you can run
453a trap where C<aio_sendfile> reads some data with readahead, then fails 454into a trap where C<aio_sendfile> reads some data with readahead, then
454to write all data, and when the socket is ready the next time, the data 455fails to write all data, and when the socket is ready the next time, the
455in the cache is already lost, forcing C<aio_sendfile> to again hit the 456data in the cache is already lost, forcing C<aio_sendfile> to again hit
456disk. Explicit C<aio_read> + C<aio_write> let's you control resource usage 457the disk. Explicit C<aio_read> + C<aio_write> let's you better control
457much better. 458resource usage.
458 459
459This call tries to make use of a native C<sendfile> syscall to provide 460This call tries to make use of a native C<sendfile>-like syscall to
460zero-copy operation. For this to work, C<$out_fh> should refer to a 461provide zero-copy operation. For this to work, C<$out_fh> should refer to
461socket, and C<$in_fh> should refer to an mmap'able file. 462a socket, and C<$in_fh> should refer to an mmap'able file.
462 463
463If a native sendfile cannot be found or it fails with C<ENOSYS>, 464If a native sendfile cannot be found or it fails with C<ENOSYS>,
464C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or C<ENOTSOCK>, 465C<EINVAL>, C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or
465it will be emulated, so you can call C<aio_sendfile> on any type of 466C<ENOTSOCK>, it will be emulated, so you can call C<aio_sendfile> on any
466filehandle regardless of the limitations of the operating system. 467type of filehandle regardless of the limitations of the operating system.
468
469As native sendfile syscalls (as practically any non-POSIX interface hacked
470together in a hurry to improve benchmark numbers) tend to be rather buggy
471on many systems, this implementation tries to work around some known bugs
472in Linux and FreeBSD kernels (probably others, too), but that might fail,
473so you really really should check the return value of C<aio_sendfile> -
474fewre bytes than expected might have been transferred.
467 475
468 476
469=item aio_readahead $fh,$offset,$length, $callback->($retval) 477=item aio_readahead $fh,$offset,$length, $callback->($retval)
470 478
471C<aio_readahead> populates the page cache with data from a file so that 479C<aio_readahead> populates the page cache with data from a file so that
… …
862 if ($_[0] && $! == EXDEV) { 870 if ($_[0] && $! == EXDEV) {
863 aioreq_pri $pri; 871 aioreq_pri $pri;
864 add $grp aio_copy $src, $dst, sub { 872 add $grp aio_copy $src, $dst, sub {
865 $grp->result ($_[0]); 873 $grp->result ($_[0]);
866 874
867 if (!$_[0]) { 875 unless ($_[0]) {
868 aioreq_pri $pri; 876 aioreq_pri $pri;
869 add $grp aio_unlink $src; 877 add $grp aio_unlink $src;
870 } 878 }
871 }; 879 };
872 } else { 880 } else {
… …
1564Sets the minimum idle timeout (default 10) after which worker threads are 1572Sets the minimum idle timeout (default 10) after which worker threads are
1565allowed to exit. SEe C<IO::AIO::max_idle>. 1573allowed to exit. SEe C<IO::AIO::max_idle>.
1566 1574
1567=item IO::AIO::max_outstanding $maxreqs 1575=item IO::AIO::max_outstanding $maxreqs
1568 1576
1577Sets the maximum number of outstanding requests to C<$nreqs>. If
1578you do queue up more than this number of requests, the next call to
1579C<IO::AIO::poll_cb> (and other functions calling C<poll_cb>, such as
1580C<IO::AIO::flush> or C<IO::AIO::poll>) will block until the limit is no
1581longer exceeded.
1582
1583In other words, this setting does not enforce a queue limit, but can be
1584used to make poll functions block if the limit is exceeded.
1585
1569This is a very bad function to use in interactive programs because it 1586This is a very bad function to use in interactive programs because it
1570blocks, and a bad way to reduce concurrency because it is inexact: Better 1587blocks, and a bad way to reduce concurrency because it is inexact: Better
1571use an C<aio_group> together with a feed callback. 1588use an C<aio_group> together with a feed callback.
1572 1589
1573Sets the maximum number of outstanding requests to C<$nreqs>. If you 1590It's main use is in scripts without an event loop - when you want to stat
1574do queue up more than this number of requests, the next call to the 1591a lot of files, you can write somehting like this:
1575C<poll_cb> (and C<poll_some> and other functions calling C<poll_cb>)
1576function will block until the limit is no longer exceeded.
1577 1592
1578The default value is very large, so there is no practical limit on the 1593 IO::AIO::max_outstanding 32;
1579number of outstanding requests.
1580 1594
1581You can still queue as many requests as you want. Therefore, 1595 for my $path (...) {
1582C<max_outstanding> is mainly useful in simple scripts (with low values) or 1596 aio_stat $path , ...;
1583as a stop gap to shield against fatal memory overflow (with large values). 1597 IO::AIO::poll_cb;
1598 }
1599
1600 IO::AIO::flush;
1601
1602The call to C<poll_cb> inside the loop will normally return instantly, but
1603as soon as more thna C<32> reqeusts are in-flight, it will block until
1604some requests have been handled. This keeps the loop from pushing a large
1605number of C<aio_stat> requests onto the queue.
1606
1607The default value for C<max_outstanding> is very large, so there is no
1608practical limit on the number of outstanding requests.
1584 1609
1585=back 1610=back
1586 1611
1587=head3 STATISTICAL INFORMATION 1612=head3 STATISTICAL INFORMATION
1588 1613
… …
1764 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno => 1789 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
1765 \&IO::AIO::poll_cb); 1790 \&IO::AIO::poll_cb);
1766 1791
1767=head2 FORK BEHAVIOUR 1792=head2 FORK BEHAVIOUR
1768 1793
1769This module should do "the right thing" when the process using it forks: 1794Usage of pthreads in a program changes the semantics of fork
1795considerably. Specifically, only async-safe functions can be called after
1796fork. Perl doesn't know about this, so in general, you cannot call fork
1797with defined behaviour in perl. IO::AIO uses pthreads, so this applies,
1798but many other extensions and (for inexplicable reasons) perl itself often
1799is linked against pthreads, so this limitation applies.
1770 1800
1771Before the fork, IO::AIO enters a quiescent state where no requests 1801Some operating systems have extensions that allow safe use of fork, and
1772can be added in other threads and no results will be processed. After 1802this module should do "the right thing" on those, and tries on others. At
1773the fork the parent simply leaves the quiescent state and continues 1803the time of this writing (2011) only GNU/Linux supports these extensions
1774request/result processing, while the child frees the request/result queue 1804to POSIX.
1775(so that the requests started before the fork will only be handled in the
1776parent). Threads will be started on demand until the limit set in the
1777parent process has been reached again.
1778
1779In short: the parent will, after a short pause, continue as if fork had
1780not been called, while the child will act as if IO::AIO has not been used
1781yet.
1782 1805
1783=head2 MEMORY USAGE 1806=head2 MEMORY USAGE
1784 1807
1785Per-request usage: 1808Per-request usage:
1786 1809

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines