ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.105 by root, Sun Mar 25 00:20:27 2007 UTC vs.
Revision 1.109 by root, Sun Jun 3 09:44:17 2007 UTC

62etc.), but can also be used to easily do operations in parallel that are 62etc.), but can also be used to easily do operations in parallel that are
63normally done sequentially, e.g. stat'ing many files, which is much faster 63normally done sequentially, e.g. stat'ing many files, which is much faster
64on a RAID volume or over NFS when you do a number of stat operations 64on a RAID volume or over NFS when you do a number of stat operations
65concurrently. 65concurrently.
66 66
67While most of this works on all types of file descriptors (for example 67While most of this works on all types of file descriptors (for
68sockets), using these functions on file descriptors that support 68example sockets), using these functions on file descriptors that
69nonblocking operation (again, sockets, pipes etc.) is very inefficient or 69support nonblocking operation (again, sockets, pipes etc.) is very
70might not work (aio_read fails on sockets/pipes/fifos). Use an event loop 70inefficient. Use an event loop for that (such as the L<Event|Event>
71for that (such as the L<Event|Event> module): IO::AIO will naturally fit 71module): IO::AIO will naturally fit into such an event loop itself.
72into such an event loop itself.
73 72
74In this version, a number of threads are started that execute your 73In this version, a number of threads are started that execute your
75requests and signal their completion. You don't need thread support 74requests and signal their completion. You don't need thread support
76in perl, and the threads created by this module will not be visible 75in perl, and the threads created by this module will not be visible
77to perl. In the future, this module might make use of the native aio 76to perl. In the future, this module might make use of the native aio
79not well-supported or restricted (GNU/Linux doesn't allow them on normal 78not well-supported or restricted (GNU/Linux doesn't allow them on normal
80files currently, for example), and they would only support aio_read and 79files currently, for example), and they would only support aio_read and
81aio_write, so the remaining functionality would have to be implemented 80aio_write, so the remaining functionality would have to be implemented
82using threads anyway. 81using threads anyway.
83 82
84Although the module will work with in the presence of other (Perl-) 83Although the module will work in the presence of other (Perl-) threads,
85threads, it is currently not reentrant in any way, so use appropriate 84it is currently not reentrant in any way, so use appropriate locking
86locking yourself, always call C<poll_cb> from within the same thread, or 85yourself, always call C<poll_cb> from within the same thread, or never
87never call C<poll_cb> (or other C<aio_> functions) recursively. 86call C<poll_cb> (or other C<aio_> functions) recursively.
88 87
89=head2 EXAMPLE 88=head2 EXAMPLE
90 89
91This is a simple example that uses the Event module and loads 90This is a simple example that uses the Event module and loads
92F</etc/passwd> asynchronously: 91F</etc/passwd> asynchronously:
190use strict 'vars'; 189use strict 'vars';
191 190
192use base 'Exporter'; 191use base 'Exporter';
193 192
194BEGIN { 193BEGIN {
195 our $VERSION = '2.33'; 194 our $VERSION = '2.4';
196 195
197 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat 196 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat
198 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink 197 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink
199 aio_readlink aio_fsync aio_fdatasync aio_readahead aio_rename aio_link 198 aio_readlink aio_fsync aio_fdatasync aio_readahead aio_rename aio_link
200 aio_move aio_copy aio_group aio_nop aio_mknod aio_load aio_rmtree aio_mkdir); 199 aio_move aio_copy aio_group aio_nop aio_mknod aio_load aio_rmtree aio_mkdir
200 aio_chown aio_chmod aio_utime);
201 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice aio_block)); 201 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice aio_block));
202 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush 202 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
203 min_parallel max_parallel max_idle 203 min_parallel max_parallel max_idle
204 nreqs nready npending nthreads 204 nreqs nready npending nthreads
205 max_poll_time max_poll_reqs); 205 max_poll_time max_poll_reqs);
271 aio_read $_[0], ..., sub { 271 aio_read $_[0], ..., sub {
272 ... 272 ...
273 }; 273 };
274 }; 274 };
275 275
276
276=item aioreq_nice $pri_adjust 277=item aioreq_nice $pri_adjust
277 278
278Similar to C<aioreq_pri>, but subtracts the given value from the current 279Similar to C<aioreq_pri>, but subtracts the given value from the current
279priority, so the effect is cumulative. 280priority, so the effect is cumulative.
281
280 282
281=item aio_open $pathname, $flags, $mode, $callback->($fh) 283=item aio_open $pathname, $flags, $mode, $callback->($fh)
282 284
283Asynchronously open or create a file and call the callback with a newly 285Asynchronously open or create a file and call the callback with a newly
284created filehandle for the file. 286created filehandle for the file.
305 } else { 307 } else {
306 die "open failed: $!\n"; 308 die "open failed: $!\n";
307 } 309 }
308 }; 310 };
309 311
312
310=item aio_close $fh, $callback->($status) 313=item aio_close $fh, $callback->($status)
311 314
312Asynchronously close a file and call the callback with the result 315Asynchronously close a file and call the callback with the result
313code. I<WARNING:> although accepted, you should not pass in a perl 316code. I<WARNING:> although accepted, you should not pass in a perl
314filehandle here, as perl will likely close the file descriptor another 317filehandle here, as perl will likely close the file descriptor another
316C<close> or just let filehandles go out of scope. 319C<close> or just let filehandles go out of scope.
317 320
318This is supposed to be a bug in the API, so that might change. It's 321This is supposed to be a bug in the API, so that might change. It's
319therefore best to avoid this function. 322therefore best to avoid this function.
320 323
324
321=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 325=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
322 326
323=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 327=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
324 328
325Reads or writes C<length> bytes from the specified C<fh> and C<offset> 329Reads or writes C<$length> bytes from the specified C<$fh> and C<$offset>
326into the scalar given by C<data> and offset C<dataoffset> and calls the 330into the scalar given by C<$data> and offset C<$dataoffset> and calls the
327callback without the actual number of bytes read (or -1 on error, just 331callback without the actual number of bytes read (or -1 on error, just
328like the syscall). 332like the syscall).
329 333
334If C<$offset> is undefined, then the current file offset will be used (and
335updated), otherwise the file offset will not be changed by these calls.
336
337If C<$length> is undefined in C<aio_write>, use the remaining length of C<$data>.
338
339If C<$dataoffset> is less than zero, it will be counted from the end of
340C<$data>.
341
330The C<$data> scalar I<MUST NOT> be modified in any way while the request 342The C<$data> scalar I<MUST NOT> be modified in any way while the request
331is outstanding. Modifying it can result in segfaults or WW3 (if the 343is outstanding. Modifying it can result in segfaults or World War III (if
332necessary/optional hardware is installed). 344the necessary/optional hardware is installed).
333 345
334Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at 346Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at
335offset C<0> within the scalar: 347offset C<0> within the scalar:
336 348
337 aio_read $fh, 7, 15, $buffer, 0, sub { 349 aio_read $fh, 7, 15, $buffer, 0, sub {
338 $_[0] > 0 or die "read error: $!"; 350 $_[0] > 0 or die "read error: $!";
339 print "read $_[0] bytes: <$buffer>\n"; 351 print "read $_[0] bytes: <$buffer>\n";
340 }; 352 };
353
341 354
342=item aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval) 355=item aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
343 356
344Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts 357Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
345reading at byte offset C<$in_offset>, and starts writing at the current 358reading at byte offset C<$in_offset>, and starts writing at the current
359C<$in_fh> than are written, and there is no way to find out how many 372C<$in_fh> than are written, and there is no way to find out how many
360bytes have been read from C<aio_sendfile> alone, as C<aio_sendfile> only 373bytes have been read from C<aio_sendfile> alone, as C<aio_sendfile> only
361provides the number of bytes written to C<$out_fh>. Only if the result 374provides the number of bytes written to C<$out_fh>. Only if the result
362value equals C<$length> one can assume that C<$length> bytes have been 375value equals C<$length> one can assume that C<$length> bytes have been
363read. 376read.
377
364 378
365=item aio_readahead $fh,$offset,$length, $callback->($retval) 379=item aio_readahead $fh,$offset,$length, $callback->($retval)
366 380
367C<aio_readahead> populates the page cache with data from a file so that 381C<aio_readahead> populates the page cache with data from a file so that
368subsequent reads from that file will not block on disk I/O. The C<$offset> 382subsequent reads from that file will not block on disk I/O. The C<$offset>
374file. The current file offset of the file is left unchanged. 388file. The current file offset of the file is left unchanged.
375 389
376If that syscall doesn't exist (likely if your OS isn't Linux) it will be 390If that syscall doesn't exist (likely if your OS isn't Linux) it will be
377emulated by simply reading the data, which would have a similar effect. 391emulated by simply reading the data, which would have a similar effect.
378 392
393
379=item aio_stat $fh_or_path, $callback->($status) 394=item aio_stat $fh_or_path, $callback->($status)
380 395
381=item aio_lstat $fh, $callback->($status) 396=item aio_lstat $fh, $callback->($status)
382 397
383Works like perl's C<stat> or C<lstat> in void context. The callback will 398Works like perl's C<stat> or C<lstat> in void context. The callback will
396 aio_stat "/etc/passwd", sub { 411 aio_stat "/etc/passwd", sub {
397 $_[0] and die "stat failed: $!"; 412 $_[0] and die "stat failed: $!";
398 print "size is ", -s _, "\n"; 413 print "size is ", -s _, "\n";
399 }; 414 };
400 415
416
417=item aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
418
419Works like perl's C<utime> function (including the special case of $atime
420and $mtime being undef). Fractional times are supported if the underlying
421syscalls support them.
422
423When called with a pathname, uses utimes(2) if available, otherwise
424utime(2). If called on a file descriptor, uses futimes(2) if available,
425otherwise returns ENOSYS, so this is not portable.
426
427Examples:
428
429 # set atime and mtime to current time (basically touch(1)):
430 aio_utime "path", undef, undef;
431 # set atime to current time and mtime to beginning of the epoch:
432 aio_utime "path", time, undef; # undef==0
433
434
435=item aio_chown $fh_or_path, $uid, $gid, $callback->($status)
436
437Works like perl's C<chown> function, except that C<undef> for either $uid
438or $gid is being interpreted as "do not change" (but -1 can also be used).
439
440Examples:
441
442 # same as "chown root path" in the shell:
443 aio_chown "path", 0, -1;
444 # same as above:
445 aio_chown "path", 0, undef;
446
447
448=item aio_chmod $fh_or_path, $mode, $callback->($status)
449
450Works like perl's C<chmod> function.
451
452
401=item aio_unlink $pathname, $callback->($status) 453=item aio_unlink $pathname, $callback->($status)
402 454
403Asynchronously unlink (delete) a file and call the callback with the 455Asynchronously unlink (delete) a file and call the callback with the
404result code. 456result code.
405 457
458
406=item aio_mknod $path, $mode, $dev, $callback->($status) 459=item aio_mknod $path, $mode, $dev, $callback->($status)
407 460
408[EXPERIMENTAL] 461[EXPERIMENTAL]
409 462
410Asynchronously create a device node (or fifo). See mknod(2). 463Asynchronously create a device node (or fifo). See mknod(2).
411 464
412The only (POSIX-) portable way of calling this function is: 465The only (POSIX-) portable way of calling this function is:
413 466
414 aio_mknod $path, IO::AIO::S_IFIFO | $mode, 0, sub { ... 467 aio_mknod $path, IO::AIO::S_IFIFO | $mode, 0, sub { ...
468
415 469
416=item aio_link $srcpath, $dstpath, $callback->($status) 470=item aio_link $srcpath, $dstpath, $callback->($status)
417 471
418Asynchronously create a new link to the existing object at C<$srcpath> at 472Asynchronously create a new link to the existing object at C<$srcpath> at
419the path C<$dstpath> and call the callback with the result code. 473the path C<$dstpath> and call the callback with the result code.
420 474
475
421=item aio_symlink $srcpath, $dstpath, $callback->($status) 476=item aio_symlink $srcpath, $dstpath, $callback->($status)
422 477
423Asynchronously create a new symbolic link to the existing object at C<$srcpath> at 478Asynchronously create a new symbolic link to the existing object at C<$srcpath> at
424the path C<$dstpath> and call the callback with the result code. 479the path C<$dstpath> and call the callback with the result code.
480
425 481
426=item aio_readlink $path, $callback->($link) 482=item aio_readlink $path, $callback->($link)
427 483
428Asynchronously read the symlink specified by C<$path> and pass it to 484Asynchronously read the symlink specified by C<$path> and pass it to
429the callback. If an error occurs, nothing or undef gets passed to the 485the callback. If an error occurs, nothing or undef gets passed to the
430callback. 486callback.
431 487
488
432=item aio_rename $srcpath, $dstpath, $callback->($status) 489=item aio_rename $srcpath, $dstpath, $callback->($status)
433 490
434Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as 491Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as
435rename(2) and call the callback with the result code. 492rename(2) and call the callback with the result code.
493
436 494
437=item aio_mkdir $pathname, $mode, $callback->($status) 495=item aio_mkdir $pathname, $mode, $callback->($status)
438 496
439Asynchronously mkdir (create) a directory and call the callback with 497Asynchronously mkdir (create) a directory and call the callback with
440the result code. C<$mode> will be modified by the umask at the time the 498the result code. C<$mode> will be modified by the umask at the time the
441request is executed, so do not change your umask. 499request is executed, so do not change your umask.
442 500
501
443=item aio_rmdir $pathname, $callback->($status) 502=item aio_rmdir $pathname, $callback->($status)
444 503
445Asynchronously rmdir (delete) a directory and call the callback with the 504Asynchronously rmdir (delete) a directory and call the callback with the
446result code. 505result code.
506
447 507
448=item aio_readdir $pathname, $callback->($entries) 508=item aio_readdir $pathname, $callback->($entries)
449 509
450Unlike the POSIX call of the same name, C<aio_readdir> reads an entire 510Unlike the POSIX call of the same name, C<aio_readdir> reads an entire
451directory (i.e. opendir + readdir + closedir). The entries will not be 511directory (i.e. opendir + readdir + closedir). The entries will not be
452sorted, and will B<NOT> include the C<.> and C<..> entries. 512sorted, and will B<NOT> include the C<.> and C<..> entries.
453 513
454The callback a single argument which is either C<undef> or an array-ref 514The callback a single argument which is either C<undef> or an array-ref
455with the filenames. 515with the filenames.
516
456 517
457=item aio_load $path, $data, $callback->($status) 518=item aio_load $path, $data, $callback->($status)
458 519
459This is a composite request that tries to fully load the given file into 520This is a composite request that tries to fully load the given file into
460memory. Status is the same as with aio_read. 521memory. Status is the same as with aio_read.

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines