ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.106 by root, Fri Jun 1 05:51:21 2007 UTC vs.
Revision 1.115 by root, Mon Sep 24 18:14:00 2007 UTC

62etc.), but can also be used to easily do operations in parallel that are 62etc.), but can also be used to easily do operations in parallel that are
63normally done sequentially, e.g. stat'ing many files, which is much faster 63normally done sequentially, e.g. stat'ing many files, which is much faster
64on a RAID volume or over NFS when you do a number of stat operations 64on a RAID volume or over NFS when you do a number of stat operations
65concurrently. 65concurrently.
66 66
67While most of this works on all types of file descriptors (for example 67While most of this works on all types of file descriptors (for
68sockets), using these functions on file descriptors that support 68example sockets), using these functions on file descriptors that
69nonblocking operation (again, sockets, pipes etc.) is very inefficient or 69support nonblocking operation (again, sockets, pipes etc.) is very
70might not work (aio_read fails on sockets/pipes/fifos). Use an event loop 70inefficient. Use an event loop for that (such as the L<Event|Event>
71for that (such as the L<Event|Event> module): IO::AIO will naturally fit 71module): IO::AIO will naturally fit into such an event loop itself.
72into such an event loop itself.
73 72
74In this version, a number of threads are started that execute your 73In this version, a number of threads are started that execute your
75requests and signal their completion. You don't need thread support 74requests and signal their completion. You don't need thread support
76in perl, and the threads created by this module will not be visible 75in perl, and the threads created by this module will not be visible
77to perl. In the future, this module might make use of the native aio 76to perl. In the future, this module might make use of the native aio
79not well-supported or restricted (GNU/Linux doesn't allow them on normal 78not well-supported or restricted (GNU/Linux doesn't allow them on normal
80files currently, for example), and they would only support aio_read and 79files currently, for example), and they would only support aio_read and
81aio_write, so the remaining functionality would have to be implemented 80aio_write, so the remaining functionality would have to be implemented
82using threads anyway. 81using threads anyway.
83 82
84Although the module will work with in the presence of other (Perl-) 83Although the module will work in the presence of other (Perl-) threads,
85threads, it is currently not reentrant in any way, so use appropriate 84it is currently not reentrant in any way, so use appropriate locking
86locking yourself, always call C<poll_cb> from within the same thread, or 85yourself, always call C<poll_cb> from within the same thread, or never
87never call C<poll_cb> (or other C<aio_> functions) recursively. 86call C<poll_cb> (or other C<aio_> functions) recursively.
88 87
89=head2 EXAMPLE 88=head2 EXAMPLE
90 89
91This is a simple example that uses the Event module and loads 90This is a simple example that uses the Event module and loads
92F</etc/passwd> asynchronously: 91F</etc/passwd> asynchronously:
190use strict 'vars'; 189use strict 'vars';
191 190
192use base 'Exporter'; 191use base 'Exporter';
193 192
194BEGIN { 193BEGIN {
195 our $VERSION = '2.4'; 194 our $VERSION = '2.41';
196 195
197 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat 196 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat
198 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink 197 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink
199 aio_readlink aio_fsync aio_fdatasync aio_readahead aio_rename aio_link 198 aio_readlink aio_fsync aio_fdatasync aio_readahead aio_rename aio_link
200 aio_move aio_copy aio_group aio_nop aio_mknod aio_load aio_rmtree aio_mkdir 199 aio_move aio_copy aio_group aio_nop aio_mknod aio_load aio_rmtree aio_mkdir
201 aio_chown aio_chmod aio_utime); 200 aio_chown aio_chmod aio_utime aio_truncate);
202 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice aio_block)); 201 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice aio_block));
203 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush 202 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
204 min_parallel max_parallel max_idle 203 min_parallel max_parallel max_idle
205 nreqs nready npending nthreads 204 nreqs nready npending nthreads
206 max_poll_time max_poll_reqs); 205 max_poll_time max_poll_reqs);
325 324
326=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 325=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
327 326
328=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 327=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
329 328
330Reads or writes C<length> bytes from the specified C<fh> and C<offset> 329Reads or writes C<$length> bytes from the specified C<$fh> and C<$offset>
331into the scalar given by C<data> and offset C<dataoffset> and calls the 330into the scalar given by C<$data> and offset C<$dataoffset> and calls the
332callback without the actual number of bytes read (or -1 on error, just 331callback without the actual number of bytes read (or -1 on error, just
333like the syscall). 332like the syscall).
334 333
334If C<$offset> is undefined, then the current file descriptor offset will
335be used (and updated), otherwise the file descriptor offset will not be
336changed by these calls.
337
338If C<$length> is undefined in C<aio_write>, use the remaining length of C<$data>.
339
340If C<$dataoffset> is less than zero, it will be counted from the end of
341C<$data>.
342
335The C<$data> scalar I<MUST NOT> be modified in any way while the request 343The C<$data> scalar I<MUST NOT> be modified in any way while the request
336is outstanding. Modifying it can result in segfaults or WW3 (if the 344is outstanding. Modifying it can result in segfaults or World War III (if
337necessary/optional hardware is installed). 345the necessary/optional hardware is installed).
338 346
339Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at 347Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at
340offset C<0> within the scalar: 348offset C<0> within the scalar:
341 349
342 aio_read $fh, 7, 15, $buffer, 0, sub { 350 aio_read $fh, 7, 15, $buffer, 0, sub {
417utime(2). If called on a file descriptor, uses futimes(2) if available, 425utime(2). If called on a file descriptor, uses futimes(2) if available,
418otherwise returns ENOSYS, so this is not portable. 426otherwise returns ENOSYS, so this is not portable.
419 427
420Examples: 428Examples:
421 429
422 # set atime and mtime to current time: 430 # set atime and mtime to current time (basically touch(1)):
423 aio_utime "path", undef, undef; 431 aio_utime "path", undef, undef;
424 # set atime to current time and mtime to beginning of the epoch: 432 # set atime to current time and mtime to beginning of the epoch:
425 aio_utime "path", time, undef; # undef==0 433 aio_utime "path", time, undef; # undef==0
426 434
427 435
434 442
435 # same as "chown root path" in the shell: 443 # same as "chown root path" in the shell:
436 aio_chown "path", 0, -1; 444 aio_chown "path", 0, -1;
437 # same as above: 445 # same as above:
438 aio_chown "path", 0, undef; 446 aio_chown "path", 0, undef;
447
448
449=item aio_truncate $fh_or_path, $offset, $callback->($status)
450
451Works like truncate(2) or ftruncate(2).
439 452
440 453
441=item aio_chmod $fh_or_path, $mode, $callback->($status) 454=item aio_chmod $fh_or_path, $mode, $callback->($status)
442 455
443Works like perl's C<chmod> function. 456Works like perl's C<chmod> function.
1194This is a very bad function to use in interactive programs because it 1207This is a very bad function to use in interactive programs because it
1195blocks, and a bad way to reduce concurrency because it is inexact: Better 1208blocks, and a bad way to reduce concurrency because it is inexact: Better
1196use an C<aio_group> together with a feed callback. 1209use an C<aio_group> together with a feed callback.
1197 1210
1198Sets the maximum number of outstanding requests to C<$nreqs>. If you 1211Sets the maximum number of outstanding requests to C<$nreqs>. If you
1199to queue up more than this number of requests, the next call to the 1212do queue up more than this number of requests, the next call to the
1200C<poll_cb> (and C<poll_some> and other functions calling C<poll_cb>) 1213C<poll_cb> (and C<poll_some> and other functions calling C<poll_cb>)
1201function will block until the limit is no longer exceeded. 1214function will block until the limit is no longer exceeded.
1202 1215
1203The default value is very large, so there is no practical limit on the 1216The default value is very large, so there is no practical limit on the
1204number of outstanding requests. 1217number of outstanding requests.
1283bytes of memory. In addition, stat requests need a stat buffer (possibly 1296bytes of memory. In addition, stat requests need a stat buffer (possibly
1284a few hundred bytes), readdir requires a result buffer and so on. Perl 1297a few hundred bytes), readdir requires a result buffer and so on. Perl
1285scalars and other data passed into aio requests will also be locked and 1298scalars and other data passed into aio requests will also be locked and
1286will consume memory till the request has entered the done state. 1299will consume memory till the request has entered the done state.
1287 1300
1288This is now awfully much, so queuing lots of requests is not usually a 1301This is not awfully much, so queuing lots of requests is not usually a
1289problem. 1302problem.
1290 1303
1291Per-thread usage: 1304Per-thread usage:
1292 1305
1293In the execution phase, some aio requests require more memory for 1306In the execution phase, some aio requests require more memory for

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines