ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.194 by root, Fri May 27 00:44:49 2011 UTC vs.
Revision 1.204 by root, Mon Jul 18 02:02:26 2011 UTC

… …
168use common::sense; 168use common::sense;
169 169
170use base 'Exporter'; 170use base 'Exporter';
171 171
172BEGIN { 172BEGIN {
173 our $VERSION = '3.9'; 173 our $VERSION = '3.93';
174 174
175 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close 175 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close
176 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx 176 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
177 aio_scandir aio_symlink aio_readlink aio_sync aio_fsync 177 aio_scandir aio_symlink aio_readlink aio_realpath aio_sync aio_fsync
178 aio_fdatasync aio_sync_file_range aio_pathsync aio_readahead 178 aio_fdatasync aio_sync_file_range aio_fallocate
179 aio_pathsync aio_readahead
179 aio_rename aio_link aio_move aio_copy aio_group 180 aio_rename aio_link aio_move aio_copy aio_group
180 aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown 181 aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
181 aio_chmod aio_utime aio_truncate 182 aio_chmod aio_utime aio_truncate
182 aio_msync aio_mtouch aio_mlock aio_mlockall 183 aio_msync aio_mtouch aio_mlock aio_mlockall
183 aio_statvfs); 184 aio_statvfs);
… …
222 aio_unlink $pathname, $callback->($status) 223 aio_unlink $pathname, $callback->($status)
223 aio_mknod $path, $mode, $dev, $callback->($status) 224 aio_mknod $path, $mode, $dev, $callback->($status)
224 aio_link $srcpath, $dstpath, $callback->($status) 225 aio_link $srcpath, $dstpath, $callback->($status)
225 aio_symlink $srcpath, $dstpath, $callback->($status) 226 aio_symlink $srcpath, $dstpath, $callback->($status)
226 aio_readlink $path, $callback->($link) 227 aio_readlink $path, $callback->($link)
228 aio_realpath $path, $callback->($link)
227 aio_rename $srcpath, $dstpath, $callback->($status) 229 aio_rename $srcpath, $dstpath, $callback->($status)
228 aio_mkdir $pathname, $mode, $callback->($status) 230 aio_mkdir $pathname, $mode, $callback->($status)
229 aio_rmdir $pathname, $callback->($status) 231 aio_rmdir $pathname, $callback->($status)
230 aio_readdir $pathname, $callback->($entries) 232 aio_readdir $pathname, $callback->($entries)
231 aio_readdirx $pathname, $flags, $callback->($entries, $flags) 233 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
… …
436 438
437Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts 439Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
438reading at byte offset C<$in_offset>, and starts writing at the current 440reading at byte offset C<$in_offset>, and starts writing at the current
439file offset of C<$out_fh>. Because of that, it is not safe to issue more 441file offset of C<$out_fh>. Because of that, it is not safe to issue more
440than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each 442than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each
441other. 443other. The same C<$in_fh> works fine though, as this function does not
444move or use the file offset of C<$in_fh>.
442 445
443Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than 446Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than
444are written, and there is no way to find out how many bytes have been read 447are written, and there is no way to find out how many more bytes have been
445from C<aio_sendfile> alone, as C<aio_sendfile> only provides the number of 448read from C<aio_sendfile> alone, as C<aio_sendfile> only provides the
446bytes written to C<$out_fh>. Only if the result value equals C<$length> 449number of bytes written to C<$out_fh>. Only if the result value equals
447one can assume that C<$length> bytes have been read. 450C<$length> one can assume that C<$length> bytes have been read.
448 451
449Unlike with other C<aio_> functions, it makes a lot of sense to use 452Unlike with other C<aio_> functions, it makes a lot of sense to use
450C<aio_sendfile> on non-blocking sockets, as long as one end (typically 453C<aio_sendfile> on non-blocking sockets, as long as one end (typically
451the C<$in_fh>) is a file - the file I/O will then be asynchronous, while 454the C<$in_fh>) is a file - the file I/O will then be asynchronous, while
452the socket I/O will be non-blocking. Note, however, that you can run into 455the socket I/O will be non-blocking. Note, however, that you can run
453a trap where C<aio_sendfile> reads some data with readahead, then fails 456into a trap where C<aio_sendfile> reads some data with readahead, then
454to write all data, and when the socket is ready the next time, the data 457fails to write all data, and when the socket is ready the next time, the
455in the cache is already lost, forcing C<aio_sendfile> to again hit the 458data in the cache is already lost, forcing C<aio_sendfile> to again hit
456disk. Explicit C<aio_read> + C<aio_write> let's you control resource usage 459the disk. Explicit C<aio_read> + C<aio_write> let's you better control
457much better. 460resource usage.
458 461
459This call tries to make use of a native C<sendfile> syscall to provide 462This call tries to make use of a native C<sendfile>-like syscall to
460zero-copy operation. For this to work, C<$out_fh> should refer to a 463provide zero-copy operation. For this to work, C<$out_fh> should refer to
461socket, and C<$in_fh> should refer to an mmap'able file. 464a socket, and C<$in_fh> should refer to an mmap'able file.
462 465
463If a native sendfile cannot be found or it fails with C<ENOSYS>, 466If a native sendfile cannot be found or it fails with C<ENOSYS>,
464C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or C<ENOTSOCK>, 467C<EINVAL>, C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or
465it will be emulated, so you can call C<aio_sendfile> on any type of 468C<ENOTSOCK>, it will be emulated, so you can call C<aio_sendfile> on any
466filehandle regardless of the limitations of the operating system. 469type of filehandle regardless of the limitations of the operating system.
470
471As native sendfile syscalls (as practically any non-POSIX interface hacked
472together in a hurry to improve benchmark numbers) tend to be rather buggy
473on many systems, this implementation tries to work around some known bugs
474in Linux and FreeBSD kernels (probably others, too), but that might fail,
475so you really really should check the return value of C<aio_sendfile> -
476fewre bytes than expected might have been transferred.
467 477
468 478
469=item aio_readahead $fh,$offset,$length, $callback->($retval) 479=item aio_readahead $fh,$offset,$length, $callback->($retval)
470 480
471C<aio_readahead> populates the page cache with data from a file so that 481C<aio_readahead> populates the page cache with data from a file so that
… …
635Asynchronously read the symlink specified by C<$path> and pass it to 645Asynchronously read the symlink specified by C<$path> and pass it to
636the callback. If an error occurs, nothing or undef gets passed to the 646the callback. If an error occurs, nothing or undef gets passed to the
637callback. 647callback.
638 648
639 649
650=item aio_realpath $path, $callback->($path)
651
652Asynchronously make the path absolute and resolve any symlinks in
653C<$path>. The resulting path only consists of directories (Same as
654L<Cwd::realpath>).
655
656This request can be used to get the absolute path of the current working
657directory by passing it a path of F<.> (a single dot).
658
659
640=item aio_rename $srcpath, $dstpath, $callback->($status) 660=item aio_rename $srcpath, $dstpath, $callback->($status)
641 661
642Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as 662Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as
643rename(2) and call the callback with the result code. 663rename(2) and call the callback with the result code.
644 664
… …
862 if ($_[0] && $! == EXDEV) { 882 if ($_[0] && $! == EXDEV) {
863 aioreq_pri $pri; 883 aioreq_pri $pri;
864 add $grp aio_copy $src, $dst, sub { 884 add $grp aio_copy $src, $dst, sub {
865 $grp->result ($_[0]); 885 $grp->result ($_[0]);
866 886
867 if (!$_[0]) { 887 unless ($_[0]) {
868 aioreq_pri $pri; 888 aioreq_pri $pri;
869 add $grp aio_unlink $src; 889 add $grp aio_unlink $src;
870 } 890 }
871 }; 891 };
872 } else { 892 } else {
… …
1564Sets the minimum idle timeout (default 10) after which worker threads are 1584Sets the minimum idle timeout (default 10) after which worker threads are
1565allowed to exit. SEe C<IO::AIO::max_idle>. 1585allowed to exit. SEe C<IO::AIO::max_idle>.
1566 1586
1567=item IO::AIO::max_outstanding $maxreqs 1587=item IO::AIO::max_outstanding $maxreqs
1568 1588
1589Sets the maximum number of outstanding requests to C<$nreqs>. If
1590you do queue up more than this number of requests, the next call to
1591C<IO::AIO::poll_cb> (and other functions calling C<poll_cb>, such as
1592C<IO::AIO::flush> or C<IO::AIO::poll>) will block until the limit is no
1593longer exceeded.
1594
1595In other words, this setting does not enforce a queue limit, but can be
1596used to make poll functions block if the limit is exceeded.
1597
1569This is a very bad function to use in interactive programs because it 1598This is a very bad function to use in interactive programs because it
1570blocks, and a bad way to reduce concurrency because it is inexact: Better 1599blocks, and a bad way to reduce concurrency because it is inexact: Better
1571use an C<aio_group> together with a feed callback. 1600use an C<aio_group> together with a feed callback.
1572 1601
1573Sets the maximum number of outstanding requests to C<$nreqs>. If you 1602It's main use is in scripts without an event loop - when you want to stat
1574do queue up more than this number of requests, the next call to the 1603a lot of files, you can write somehting like this:
1575C<poll_cb> (and C<poll_some> and other functions calling C<poll_cb>)
1576function will block until the limit is no longer exceeded.
1577 1604
1578The default value is very large, so there is no practical limit on the 1605 IO::AIO::max_outstanding 32;
1579number of outstanding requests.
1580 1606
1581You can still queue as many requests as you want. Therefore, 1607 for my $path (...) {
1582C<max_outstanding> is mainly useful in simple scripts (with low values) or 1608 aio_stat $path , ...;
1583as a stop gap to shield against fatal memory overflow (with large values). 1609 IO::AIO::poll_cb;
1610 }
1611
1612 IO::AIO::flush;
1613
1614The call to C<poll_cb> inside the loop will normally return instantly, but
1615as soon as more thna C<32> reqeusts are in-flight, it will block until
1616some requests have been handled. This keeps the loop from pushing a large
1617number of C<aio_stat> requests onto the queue.
1618
1619The default value for C<max_outstanding> is very large, so there is no
1620practical limit on the number of outstanding requests.
1584 1621
1585=back 1622=back
1586 1623
1587=head3 STATISTICAL INFORMATION 1624=head3 STATISTICAL INFORMATION
1588 1625
… …
1764 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno => 1801 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
1765 \&IO::AIO::poll_cb); 1802 \&IO::AIO::poll_cb);
1766 1803
1767=head2 FORK BEHAVIOUR 1804=head2 FORK BEHAVIOUR
1768 1805
1769This module should do "the right thing" when the process using it forks: 1806Usage of pthreads in a program changes the semantics of fork
1807considerably. Specifically, only async-safe functions can be called after
1808fork. Perl doesn't know about this, so in general, you cannot call fork
1809with defined behaviour in perl if pthreads are involved. IO::AIO uses
1810pthreads, so this applies, but many other extensions and (for inexplicable
1811reasons) perl itself often is linked against pthreads, so this limitation
1812applies to quite a lot of perls.
1770 1813
1771Before the fork, IO::AIO enters a quiescent state where no requests 1814This module no longer tries to fight your OS, or POSIX. That means IO::AIO
1772can be added in other threads and no results will be processed. After 1815only works in the process that loaded it. Forking is fully supported, but
1773the fork the parent simply leaves the quiescent state and continues 1816using IO::AIO in the child is not.
1774request/result processing, while the child frees the request/result queue
1775(so that the requests started before the fork will only be handled in the
1776parent). Threads will be started on demand until the limit set in the
1777parent process has been reached again.
1778 1817
1779In short: the parent will, after a short pause, continue as if fork had 1818You might get around by not I<using> IO::AIO before (or after)
1780not been called, while the child will act as if IO::AIO has not been used 1819forking. You could also try to call the L<IO::AIO::reinit> function in the
1781yet. 1820child:
1821
1822=over 4
1823
1824=item IO::AIO::reinit
1825
1826Abondons all current requests and I/O threads and simply reinitialises all
1827data structures. This is not an operation suppported by any standards, but
1828happens to work on GNU/Linux and some newer BSD systems.
1829
1830The only reasonable use for this function is to call it after forking, if
1831C<IO::AIO> was used in the parent. Calling it while IO::AIO is active in
1832the process will result in undefined behaviour. Calling it at any time
1833will also result in any undefined (by POSIX) behaviour.
1834
1835=back
1782 1836
1783=head2 MEMORY USAGE 1837=head2 MEMORY USAGE
1784 1838
1785Per-request usage: 1839Per-request usage:
1786 1840

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines