ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.105 by root, Sun Mar 25 00:20:27 2007 UTC vs.
Revision 1.108 by root, Fri Jun 1 13:25:50 2007 UTC

62etc.), but can also be used to easily do operations in parallel that are 62etc.), but can also be used to easily do operations in parallel that are
63normally done sequentially, e.g. stat'ing many files, which is much faster 63normally done sequentially, e.g. stat'ing many files, which is much faster
64on a RAID volume or over NFS when you do a number of stat operations 64on a RAID volume or over NFS when you do a number of stat operations
65concurrently. 65concurrently.
66 66
67While most of this works on all types of file descriptors (for example 67While most of this works on all types of file descriptors (for
68sockets), using these functions on file descriptors that support 68example sockets), using these functions on file descriptors that
69nonblocking operation (again, sockets, pipes etc.) is very inefficient or 69support nonblocking operation (again, sockets, pipes etc.) is very
70might not work (aio_read fails on sockets/pipes/fifos). Use an event loop 70inefficient. Use an event loop for that (such as the L<Event|Event>
71for that (such as the L<Event|Event> module): IO::AIO will naturally fit 71module): IO::AIO will naturally fit into such an event loop itself.
72into such an event loop itself.
73 72
74In this version, a number of threads are started that execute your 73In this version, a number of threads are started that execute your
75requests and signal their completion. You don't need thread support 74requests and signal their completion. You don't need thread support
76in perl, and the threads created by this module will not be visible 75in perl, and the threads created by this module will not be visible
77to perl. In the future, this module might make use of the native aio 76to perl. In the future, this module might make use of the native aio
79not well-supported or restricted (GNU/Linux doesn't allow them on normal 78not well-supported or restricted (GNU/Linux doesn't allow them on normal
80files currently, for example), and they would only support aio_read and 79files currently, for example), and they would only support aio_read and
81aio_write, so the remaining functionality would have to be implemented 80aio_write, so the remaining functionality would have to be implemented
82using threads anyway. 81using threads anyway.
83 82
84Although the module will work with in the presence of other (Perl-) 83Although the module will work in the presence of other (Perl-) threads,
85threads, it is currently not reentrant in any way, so use appropriate 84it is currently not reentrant in any way, so use appropriate locking
86locking yourself, always call C<poll_cb> from within the same thread, or 85yourself, always call C<poll_cb> from within the same thread, or never
87never call C<poll_cb> (or other C<aio_> functions) recursively. 86call C<poll_cb> (or other C<aio_> functions) recursively.
88 87
89=head2 EXAMPLE 88=head2 EXAMPLE
90 89
91This is a simple example that uses the Event module and loads 90This is a simple example that uses the Event module and loads
92F</etc/passwd> asynchronously: 91F</etc/passwd> asynchronously:
190use strict 'vars'; 189use strict 'vars';
191 190
192use base 'Exporter'; 191use base 'Exporter';
193 192
194BEGIN { 193BEGIN {
195 our $VERSION = '2.33'; 194 our $VERSION = '2.4';
196 195
197 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat 196 our @AIO_REQ = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat
198 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink 197 aio_lstat aio_unlink aio_rmdir aio_readdir aio_scandir aio_symlink
199 aio_readlink aio_fsync aio_fdatasync aio_readahead aio_rename aio_link 198 aio_readlink aio_fsync aio_fdatasync aio_readahead aio_rename aio_link
200 aio_move aio_copy aio_group aio_nop aio_mknod aio_load aio_rmtree aio_mkdir); 199 aio_move aio_copy aio_group aio_nop aio_mknod aio_load aio_rmtree aio_mkdir
200 aio_chown aio_chmod aio_utime);
201 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice aio_block)); 201 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice aio_block));
202 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush 202 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
203 min_parallel max_parallel max_idle 203 min_parallel max_parallel max_idle
204 nreqs nready npending nthreads 204 nreqs nready npending nthreads
205 max_poll_time max_poll_reqs); 205 max_poll_time max_poll_reqs);
271 aio_read $_[0], ..., sub { 271 aio_read $_[0], ..., sub {
272 ... 272 ...
273 }; 273 };
274 }; 274 };
275 275
276
276=item aioreq_nice $pri_adjust 277=item aioreq_nice $pri_adjust
277 278
278Similar to C<aioreq_pri>, but subtracts the given value from the current 279Similar to C<aioreq_pri>, but subtracts the given value from the current
279priority, so the effect is cumulative. 280priority, so the effect is cumulative.
281
280 282
281=item aio_open $pathname, $flags, $mode, $callback->($fh) 283=item aio_open $pathname, $flags, $mode, $callback->($fh)
282 284
283Asynchronously open or create a file and call the callback with a newly 285Asynchronously open or create a file and call the callback with a newly
284created filehandle for the file. 286created filehandle for the file.
305 } else { 307 } else {
306 die "open failed: $!\n"; 308 die "open failed: $!\n";
307 } 309 }
308 }; 310 };
309 311
312
310=item aio_close $fh, $callback->($status) 313=item aio_close $fh, $callback->($status)
311 314
312Asynchronously close a file and call the callback with the result 315Asynchronously close a file and call the callback with the result
313code. I<WARNING:> although accepted, you should not pass in a perl 316code. I<WARNING:> although accepted, you should not pass in a perl
314filehandle here, as perl will likely close the file descriptor another 317filehandle here, as perl will likely close the file descriptor another
316C<close> or just let filehandles go out of scope. 319C<close> or just let filehandles go out of scope.
317 320
318This is supposed to be a bug in the API, so that might change. It's 321This is supposed to be a bug in the API, so that might change. It's
319therefore best to avoid this function. 322therefore best to avoid this function.
320 323
324
321=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 325=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
322 326
323=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 327=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
324 328
325Reads or writes C<length> bytes from the specified C<fh> and C<offset> 329Reads or writes C<length> bytes from the specified C<fh> and C<offset>
326into the scalar given by C<data> and offset C<dataoffset> and calls the 330into the scalar given by C<data> and offset C<dataoffset> and calls the
327callback without the actual number of bytes read (or -1 on error, just 331callback without the actual number of bytes read (or -1 on error, just
328like the syscall). 332like the syscall). If C<offset> is undefined, then the current file offset
333will be used (and updated), otherwise the file offset will not be changed
334by these calls.
329 335
330The C<$data> scalar I<MUST NOT> be modified in any way while the request 336The C<$data> scalar I<MUST NOT> be modified in any way while the request
331is outstanding. Modifying it can result in segfaults or WW3 (if the 337is outstanding. Modifying it can result in segfaults or World War III (if
332necessary/optional hardware is installed). 338the necessary/optional hardware is installed).
333 339
334Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at 340Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at
335offset C<0> within the scalar: 341offset C<0> within the scalar:
336 342
337 aio_read $fh, 7, 15, $buffer, 0, sub { 343 aio_read $fh, 7, 15, $buffer, 0, sub {
338 $_[0] > 0 or die "read error: $!"; 344 $_[0] > 0 or die "read error: $!";
339 print "read $_[0] bytes: <$buffer>\n"; 345 print "read $_[0] bytes: <$buffer>\n";
340 }; 346 };
347
341 348
342=item aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval) 349=item aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
343 350
344Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts 351Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
345reading at byte offset C<$in_offset>, and starts writing at the current 352reading at byte offset C<$in_offset>, and starts writing at the current
359C<$in_fh> than are written, and there is no way to find out how many 366C<$in_fh> than are written, and there is no way to find out how many
360bytes have been read from C<aio_sendfile> alone, as C<aio_sendfile> only 367bytes have been read from C<aio_sendfile> alone, as C<aio_sendfile> only
361provides the number of bytes written to C<$out_fh>. Only if the result 368provides the number of bytes written to C<$out_fh>. Only if the result
362value equals C<$length> one can assume that C<$length> bytes have been 369value equals C<$length> one can assume that C<$length> bytes have been
363read. 370read.
371
364 372
365=item aio_readahead $fh,$offset,$length, $callback->($retval) 373=item aio_readahead $fh,$offset,$length, $callback->($retval)
366 374
367C<aio_readahead> populates the page cache with data from a file so that 375C<aio_readahead> populates the page cache with data from a file so that
368subsequent reads from that file will not block on disk I/O. The C<$offset> 376subsequent reads from that file will not block on disk I/O. The C<$offset>
374file. The current file offset of the file is left unchanged. 382file. The current file offset of the file is left unchanged.
375 383
376If that syscall doesn't exist (likely if your OS isn't Linux) it will be 384If that syscall doesn't exist (likely if your OS isn't Linux) it will be
377emulated by simply reading the data, which would have a similar effect. 385emulated by simply reading the data, which would have a similar effect.
378 386
387
379=item aio_stat $fh_or_path, $callback->($status) 388=item aio_stat $fh_or_path, $callback->($status)
380 389
381=item aio_lstat $fh, $callback->($status) 390=item aio_lstat $fh, $callback->($status)
382 391
383Works like perl's C<stat> or C<lstat> in void context. The callback will 392Works like perl's C<stat> or C<lstat> in void context. The callback will
396 aio_stat "/etc/passwd", sub { 405 aio_stat "/etc/passwd", sub {
397 $_[0] and die "stat failed: $!"; 406 $_[0] and die "stat failed: $!";
398 print "size is ", -s _, "\n"; 407 print "size is ", -s _, "\n";
399 }; 408 };
400 409
410
411=item aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
412
413Works like perl's C<utime> function (including the special case of $atime
414and $mtime being undef). Fractional times are supported if the underlying
415syscalls support them.
416
417When called with a pathname, uses utimes(2) if available, otherwise
418utime(2). If called on a file descriptor, uses futimes(2) if available,
419otherwise returns ENOSYS, so this is not portable.
420
421Examples:
422
423 # set atime and mtime to current time (basically touch(1)):
424 aio_utime "path", undef, undef;
425 # set atime to current time and mtime to beginning of the epoch:
426 aio_utime "path", time, undef; # undef==0
427
428
429=item aio_chown $fh_or_path, $uid, $gid, $callback->($status)
430
431Works like perl's C<chown> function, except that C<undef> for either $uid
432or $gid is being interpreted as "do not change" (but -1 can also be used).
433
434Examples:
435
436 # same as "chown root path" in the shell:
437 aio_chown "path", 0, -1;
438 # same as above:
439 aio_chown "path", 0, undef;
440
441
442=item aio_chmod $fh_or_path, $mode, $callback->($status)
443
444Works like perl's C<chmod> function.
445
446
401=item aio_unlink $pathname, $callback->($status) 447=item aio_unlink $pathname, $callback->($status)
402 448
403Asynchronously unlink (delete) a file and call the callback with the 449Asynchronously unlink (delete) a file and call the callback with the
404result code. 450result code.
405 451
452
406=item aio_mknod $path, $mode, $dev, $callback->($status) 453=item aio_mknod $path, $mode, $dev, $callback->($status)
407 454
408[EXPERIMENTAL] 455[EXPERIMENTAL]
409 456
410Asynchronously create a device node (or fifo). See mknod(2). 457Asynchronously create a device node (or fifo). See mknod(2).
411 458
412The only (POSIX-) portable way of calling this function is: 459The only (POSIX-) portable way of calling this function is:
413 460
414 aio_mknod $path, IO::AIO::S_IFIFO | $mode, 0, sub { ... 461 aio_mknod $path, IO::AIO::S_IFIFO | $mode, 0, sub { ...
462
415 463
416=item aio_link $srcpath, $dstpath, $callback->($status) 464=item aio_link $srcpath, $dstpath, $callback->($status)
417 465
418Asynchronously create a new link to the existing object at C<$srcpath> at 466Asynchronously create a new link to the existing object at C<$srcpath> at
419the path C<$dstpath> and call the callback with the result code. 467the path C<$dstpath> and call the callback with the result code.
420 468
469
421=item aio_symlink $srcpath, $dstpath, $callback->($status) 470=item aio_symlink $srcpath, $dstpath, $callback->($status)
422 471
423Asynchronously create a new symbolic link to the existing object at C<$srcpath> at 472Asynchronously create a new symbolic link to the existing object at C<$srcpath> at
424the path C<$dstpath> and call the callback with the result code. 473the path C<$dstpath> and call the callback with the result code.
474
425 475
426=item aio_readlink $path, $callback->($link) 476=item aio_readlink $path, $callback->($link)
427 477
428Asynchronously read the symlink specified by C<$path> and pass it to 478Asynchronously read the symlink specified by C<$path> and pass it to
429the callback. If an error occurs, nothing or undef gets passed to the 479the callback. If an error occurs, nothing or undef gets passed to the
430callback. 480callback.
431 481
482
432=item aio_rename $srcpath, $dstpath, $callback->($status) 483=item aio_rename $srcpath, $dstpath, $callback->($status)
433 484
434Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as 485Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as
435rename(2) and call the callback with the result code. 486rename(2) and call the callback with the result code.
487
436 488
437=item aio_mkdir $pathname, $mode, $callback->($status) 489=item aio_mkdir $pathname, $mode, $callback->($status)
438 490
439Asynchronously mkdir (create) a directory and call the callback with 491Asynchronously mkdir (create) a directory and call the callback with
440the result code. C<$mode> will be modified by the umask at the time the 492the result code. C<$mode> will be modified by the umask at the time the
441request is executed, so do not change your umask. 493request is executed, so do not change your umask.
442 494
495
443=item aio_rmdir $pathname, $callback->($status) 496=item aio_rmdir $pathname, $callback->($status)
444 497
445Asynchronously rmdir (delete) a directory and call the callback with the 498Asynchronously rmdir (delete) a directory and call the callback with the
446result code. 499result code.
500
447 501
448=item aio_readdir $pathname, $callback->($entries) 502=item aio_readdir $pathname, $callback->($entries)
449 503
450Unlike the POSIX call of the same name, C<aio_readdir> reads an entire 504Unlike the POSIX call of the same name, C<aio_readdir> reads an entire
451directory (i.e. opendir + readdir + closedir). The entries will not be 505directory (i.e. opendir + readdir + closedir). The entries will not be
452sorted, and will B<NOT> include the C<.> and C<..> entries. 506sorted, and will B<NOT> include the C<.> and C<..> entries.
453 507
454The callback a single argument which is either C<undef> or an array-ref 508The callback a single argument which is either C<undef> or an array-ref
455with the filenames. 509with the filenames.
510
456 511
457=item aio_load $path, $data, $callback->($status) 512=item aio_load $path, $data, $callback->($status)
458 513
459This is a composite request that tries to fully load the given file into 514This is a composite request that tries to fully load the given file into
460memory. Status is the same as with aio_read. 515memory. Status is the same as with aio_read.

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines