ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/README
(Generate patch)

Comparing IO-AIO/README (file contents):
Revision 1.27 by root, Sat Oct 6 14:05:37 2007 UTC vs.
Revision 1.35 by root, Tue Apr 21 20:06:05 2009 UTC

24 $req->cancel; # cancel request if still in queue 24 $req->cancel; # cancel request if still in queue
25 25
26 my $grp = aio_group sub { print "all stats done\n" }; 26 my $grp = aio_group sub { print "all stats done\n" };
27 add $grp aio_stat "..." for ...; 27 add $grp aio_stat "..." for ...;
28 28
29 # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
30 use AnyEvent::AIO;
31
29 # AnyEvent integration 32 # EV integration
30 open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!"; 33 my $w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
31 my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });
32 34
33 # Event integration 35 # Event integration
34 Event->io (fd => IO::AIO::poll_fileno, 36 Event->io (fd => IO::AIO::poll_fileno,
35 poll => 'r', 37 poll => 'r',
36 cb => \&IO::AIO::poll_cb); 38 cb => \&IO::AIO::poll_cb);
173 All the "aio_*" calls are more or less thin wrappers around the syscall 175 All the "aio_*" calls are more or less thin wrappers around the syscall
174 with the same name (sans "aio_"). The arguments are similar or 176 with the same name (sans "aio_"). The arguments are similar or
175 identical, and they all accept an additional (and optional) $callback 177 identical, and they all accept an additional (and optional) $callback
176 argument which must be a code reference. This code reference will get 178 argument which must be a code reference. This code reference will get
177 called with the syscall return code (e.g. most syscalls return -1 on 179 called with the syscall return code (e.g. most syscalls return -1 on
178 error, unlike perl, which usually delivers "false") as it's sole 180 error, unlike perl, which usually delivers "false") as its sole argument
179 argument when the given syscall has been executed asynchronously. 181 after the given syscall has been executed asynchronously.
180 182
181 All functions expecting a filehandle keep a copy of the filehandle 183 All functions expecting a filehandle keep a copy of the filehandle
182 internally until the request has finished. 184 internally until the request has finished.
183 185
184 All functions return request objects of type IO::AIO::REQ that allow 186 All functions return request objects of type IO::AIO::REQ that allow
197 the user environment, d) use Glib::filename_from_unicode on unicode 199 the user environment, d) use Glib::filename_from_unicode on unicode
198 filenames or e) use something else to ensure your scalar has the correct 200 filenames or e) use something else to ensure your scalar has the correct
199 contents. 201 contents.
200 202
201 This works, btw. independent of the internal UTF-8 bit, which IO::AIO 203 This works, btw. independent of the internal UTF-8 bit, which IO::AIO
202 handles correctly wether it is set or not. 204 handles correctly whether it is set or not.
203 205
204 $prev_pri = aioreq_pri [$pri] 206 $prev_pri = aioreq_pri [$pri]
205 Returns the priority value that would be used for the next request 207 Returns the priority value that would be used for the next request
206 and, if $pri is given, sets the priority for the next aio request. 208 and, if $pri is given, sets the priority for the next aio request.
207 209
262 Asynchronously close a file and call the callback with the result 264 Asynchronously close a file and call the callback with the result
263 code. 265 code.
264 266
265 Unfortunately, you can't do this to perl. Perl *insists* very 267 Unfortunately, you can't do this to perl. Perl *insists* very
266 strongly on closing the file descriptor associated with the 268 strongly on closing the file descriptor associated with the
267 filehandle itself. Here is what aio_close will try: 269 filehandle itself.
268 270
269 1. dup()licate the fd 271 Therefore, "aio_close" will not close the filehandle - instead it
270 2. asynchronously close() the duplicated fd 272 will use dup2 to overwrite the file descriptor with the write-end of
271 3. dup()licate the fd once more 273 a pipe (the pipe fd will be created on demand and will be cached).
272 4. let perl close() the filehandle
273 5. asynchronously close the duplicated fd
274 274
275 The idea is that the first close() flushes stuff to disk that 275 Or in other words: the file descriptor will be closed, but it will
276 closing an fd will flush, so when perl closes the fd, nothing much 276 not be free for reuse until the perl filehandle is closed.
277 will need to be flushed. The second async. close() will then flush
278 stuff to disk that closing the last fd to the file will flush.
279
280 Just FYI, SuSv3 has this to say on close:
281
282 All outstanding record locks owned by the process on the file
283 associated with the file descriptor shall be removed.
284
285 If fildes refers to a socket, close() shall cause the socket to be
286 destroyed. ... close() shall block for up to the current linger
287 interval until all data is transmitted.
288 [this actually sounds like a specification bug, but who knows]
289
290 And at least Linux additionally actually flushes stuff on every
291 close, even when the file itself is still open.
292
293 Sounds enourmously inefficient and complicated? Yes... please show
294 me how to nuke perl's fd out of existence...
295 277
296 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 278 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
297 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 279 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
298 Reads or writes $length bytes from the specified $fh and $offset 280 Reads or writes $length bytes from or to the specified $fh and
299 into the scalar given by $data and offset $dataoffset and calls the 281 $offset into the scalar given by $data and offset $dataoffset and
300 callback without the actual number of bytes read (or -1 on error, 282 calls the callback without the actual number of bytes read (or -1 on
301 just like the syscall). 283 error, just like the syscall).
284
285 "aio_read" will, like "sysread", shrink or grow the $data scalar to
286 offset plus the actual number of bytes read.
302 287
303 If $offset is undefined, then the current file descriptor offset 288 If $offset is undefined, then the current file descriptor offset
304 will be used (and updated), otherwise the file descriptor offset 289 will be used (and updated), otherwise the file descriptor offset
305 will not be changed by these calls. 290 will not be changed by these calls.
306 291
469 aio_copy $srcpath, $dstpath, $callback->($status) 454 aio_copy $srcpath, $dstpath, $callback->($status)
470 Try to copy the *file* (directories not supported as either source 455 Try to copy the *file* (directories not supported as either source
471 or destination) from $srcpath to $dstpath and call the callback with 456 or destination) from $srcpath to $dstpath and call the callback with
472 the 0 (error) or -1 ok. 457 the 0 (error) or -1 ok.
473 458
474 This is a composite request that it creates the destination file 459 This is a composite request that creates the destination file with
475 with mode 0200 and copies the contents of the source file into it 460 mode 0200 and copies the contents of the source file into it using
476 using "aio_sendfile", followed by restoring atime, mtime, access 461 "aio_sendfile", followed by restoring atime, mtime, access mode and
477 mode and uid/gid, in that order. 462 uid/gid, in that order.
478 463
479 If an error occurs, the partial destination file will be unlinked, 464 If an error occurs, the partial destination file will be unlinked,
480 if possible, except when setting atime, mtime, access mode and 465 if possible, except when setting atime, mtime, access mode and
481 uid/gid, where errors are being ignored. 466 uid/gid, where errors are being ignored.
482 467
483 aio_move $srcpath, $dstpath, $callback->($status) 468 aio_move $srcpath, $dstpath, $callback->($status)
484 Try to move the *file* (directories not supported as either source 469 Try to move the *file* (directories not supported as either source
485 or destination) from $srcpath to $dstpath and call the callback with 470 or destination) from $srcpath to $dstpath and call the callback with
486 the 0 (error) or -1 ok. 471 the 0 (error) or -1 ok.
487 472
488 This is a composite request that tries to rename(2) the file first. 473 This is a composite request that tries to rename(2) the file first;
489 If rename files with "EXDEV", it copies the file with "aio_copy" 474 if rename fails with "EXDEV", it copies the file with "aio_copy"
490 and, if that is successful, unlinking the $srcpath. 475 and, if that is successful, unlinks the $srcpath.
491 476
492 aio_scandir $path, $maxreq, $callback->($dirs, $nondirs) 477 aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
493 Scans a directory (similar to "aio_readdir") but additionally tries 478 Scans a directory (similar to "aio_readdir") but additionally tries
494 to efficiently separate the entries of directory $path into two sets 479 to efficiently separate the entries of directory $path into two sets
495 of names, directories you can recurse into (directories), and ones 480 of names, directories you can recurse into (directories), and ones
546 Delete a directory tree starting (and including) $path, return the 531 Delete a directory tree starting (and including) $path, return the
547 status of the final "rmdir" only. This is a composite request that 532 status of the final "rmdir" only. This is a composite request that
548 uses "aio_scandir" to recurse into and rmdir directories, and unlink 533 uses "aio_scandir" to recurse into and rmdir directories, and unlink
549 everything else. 534 everything else.
550 535
536 aio_sync $callback->($status)
537 Asynchronously call sync and call the callback when finished.
538
551 aio_fsync $fh, $callback->($status) 539 aio_fsync $fh, $callback->($status)
552 Asynchronously call fsync on the given filehandle and call the 540 Asynchronously call fsync on the given filehandle and call the
553 callback with the fsync result code. 541 callback with the fsync result code.
554 542
555 aio_fdatasync $fh, $callback->($status) 543 aio_fdatasync $fh, $callback->($status)
556 Asynchronously call fdatasync on the given filehandle and call the 544 Asynchronously call fdatasync on the given filehandle and call the
557 callback with the fdatasync result code. 545 callback with the fdatasync result code.
558 546
559 If this call isn't available because your OS lacks it or it couldn't 547 If this call isn't available because your OS lacks it or it couldn't
560 be detected, it will be emulated by calling "fsync" instead. 548 be detected, it will be emulated by calling "fsync" instead.
549
550 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
551 Sync the data portion of the file specified by $offset and $length
552 to disk (but NOT the metadata), by calling the Linux-specific
553 sync_file_range call. If sync_file_range is not available or it
554 returns ENOSYS, then fdatasync or fsync is being substituted.
555
556 $flags can be a combination of
557 "IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE",
558 "IO::AIO::SYNC_FILE_RANGE_WRITE" and
559 "IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER": refer to the sync_file_range
560 manpage for details.
561
562 aio_pathsync $path, $callback->($status)
563 This request tries to open, fsync and close the given path. This is
564 a composite request intended to sync directories after directory
565 operations (E.g. rename). This might not work on all operating
566 systems or have any specific effect, but usually it makes sure that
567 directory changes get written to disc. It works for anything that
568 can be opened for read-only, not just directories.
569
570 Passes 0 when everything went ok, and -1 on error.
561 571
562 aio_group $callback->(...) 572 aio_group $callback->(...)
563 This is a very special aio request: Instead of doing something, it 573 This is a very special aio request: Instead of doing something, it
564 is a container for other aio requests, which is useful if you want 574 is a container for other aio requests, which is useful if you want
565 to bundle many requests into a single, composite, request with a 575 to bundle many requests into a single, composite, request with a
647 }; 657 };
648 658
649 This makes it very easy to create composite requests (see the source of 659 This makes it very easy to create composite requests (see the source of
650 "aio_move" for an application) that work and feel like simple requests. 660 "aio_move" for an application) that work and feel like simple requests.
651 661
652 * The IO::AIO::GRP objects will be cleaned up during calls to 662 * The IO::AIO::GRP objects will be cleaned up during calls to
653 "IO::AIO::poll_cb", just like any other request. 663 "IO::AIO::poll_cb", just like any other request.
664
654 * They can be canceled like any other request. Canceling will cancel not 665 * They can be canceled like any other request. Canceling will cancel
655 only the request itself, but also all requests it contains. 666 not only the request itself, but also all requests it contains.
667
656 * They can also can also be added to other IO::AIO::GRP objects. 668 * They can also can also be added to other IO::AIO::GRP objects.
669
657 * You must not add requests to a group from within the group callback 670 * You must not add requests to a group from within the group callback
658 (or any later time). 671 (or any later time).
659 672
660 Their lifetime, simplified, looks like this: when they are empty, they 673 Their lifetime, simplified, looks like this: when they are empty, they
661 will finish very quickly. If they contain only requests that are in the 674 will finish very quickly. If they contain only requests that are in the
662 "done" state, they will also finish. Otherwise they will continue to 675 "done" state, they will also finish. Otherwise they will continue to
663 exist. 676 exist.
664 677
665 That means after creating a group you have some time to add requests. 678 That means after creating a group you have some time to add requests
666 And in the callbacks of those requests, you can add further requests to 679 (precisely before the callback has been invoked, which is only done
667 the group. And only when all those requests have finished will the the 680 within the "poll_cb"). And in the callbacks of those requests, you can
668 group itself finish. 681 add further requests to the group. And only when all those requests have
682 finished will the the group itself finish.
669 683
670 add $grp ... 684 add $grp ...
671 $grp->add (...) 685 $grp->add (...)
672 Add one or more requests to the group. Any type of IO::AIO::REQ can 686 Add one or more requests to the group. Any type of IO::AIO::REQ can
673 be added, including other groups, as long as you do not create 687 be added, including other groups, as long as you do not create
680 request itself. Useful when you queued a lot of events but got a 694 request itself. Useful when you queued a lot of events but got a
681 result early. 695 result early.
682 696
683 $grp->result (...) 697 $grp->result (...)
684 Set the result value(s) that will be passed to the group callback 698 Set the result value(s) that will be passed to the group callback
685 when all subrequests have finished and set thre groups errno to the 699 when all subrequests have finished and set the groups errno to the
686 current value of errno (just like calling "errno" without an error 700 current value of errno (just like calling "errno" without an error
687 number). By default, no argument will be passed and errno is zero. 701 number). By default, no argument will be passed and errno is zero.
688 702
689 $grp->errno ([$errno]) 703 $grp->errno ([$errno])
690 Sets the group errno value to $errno, or the current value of errno 704 Sets the group errno value to $errno, or the current value of errno
716 does not impose any limits). 730 does not impose any limits).
717 731
718 If the feed does not queue more requests when called, it will be 732 If the feed does not queue more requests when called, it will be
719 automatically removed from the group. 733 automatically removed from the group.
720 734
721 If the feed limit is 0, it will be set to 2 automatically. 735 If the feed limit is 0 when this method is called, it will be set to
736 2 automatically.
722 737
723 Example: 738 Example:
724 739
725 # stat all files in @files, but only ever use four aio requests concurrently: 740 # stat all files in @files, but only ever use four aio requests concurrently:
726 741
736 limit $grp $num 751 limit $grp $num
737 Sets the feeder limit for the group: The feeder will be called 752 Sets the feeder limit for the group: The feeder will be called
738 whenever the group contains less than this many requests. 753 whenever the group contains less than this many requests.
739 754
740 Setting the limit to 0 will pause the feeding process. 755 Setting the limit to 0 will pause the feeding process.
756
757 The default value for the limit is 0, but note that setting a feeder
758 automatically bumps it up to 2.
741 759
742 SUPPORT FUNCTIONS 760 SUPPORT FUNCTIONS
743 EVENT PROCESSING AND EVENT LOOP INTEGRATION 761 EVENT PROCESSING AND EVENT LOOP INTEGRATION
744 $fileno = IO::AIO::poll_fileno 762 $fileno = IO::AIO::poll_fileno
745 Return the *request result pipe file descriptor*. This filehandle 763 Return the *request result pipe file descriptor*. This filehandle
749 767
750 See "poll_cb" for an example. 768 See "poll_cb" for an example.
751 769
752 IO::AIO::poll_cb 770 IO::AIO::poll_cb
753 Process some outstanding events on the result pipe. You have to call 771 Process some outstanding events on the result pipe. You have to call
754 this regularly. Returns the number of events processed. Returns 772 this regularly. Returns 0 if all events could be processed, or -1 if
755 immediately when no events are outstanding. The amount of events 773 it returned earlier for whatever reason. Returns immediately when no
774 events are outstanding. The amount of events processed depends on
756 processed depends on the settings of "IO::AIO::max_poll_req" and 775 the settings of "IO::AIO::max_poll_req" and
757 "IO::AIO::max_poll_time". 776 "IO::AIO::max_poll_time".
758 777
759 If not all requests were processed for whatever reason, the 778 If not all requests were processed for whatever reason, the
760 filehandle will still be ready when "poll_cb" returns. 779 filehandle will still be ready when "poll_cb" returns, so normally
780 you don't have to do anything special to have it called later.
761 781
762 Example: Install an Event watcher that automatically calls 782 Example: Install an Event watcher that automatically calls
763 IO::AIO::poll_cb with high priority: 783 IO::AIO::poll_cb with high priority:
764 784
765 Event->io (fd => IO::AIO::poll_fileno, 785 Event->io (fd => IO::AIO::poll_fileno,
871 891
872 The default is probably ok in most situations, especially if thread 892 The default is probably ok in most situations, especially if thread
873 creation is fast. If thread creation is very slow on your system you 893 creation is fast. If thread creation is very slow on your system you
874 might want to use larger values. 894 might want to use larger values.
875 895
876 $oldmaxreqs = IO::AIO::max_outstanding $maxreqs 896 IO::AIO::max_outstanding $maxreqs
877 This is a very bad function to use in interactive programs because 897 This is a very bad function to use in interactive programs because
878 it blocks, and a bad way to reduce concurrency because it is 898 it blocks, and a bad way to reduce concurrency because it is
879 inexact: Better use an "aio_group" together with a feed callback. 899 inexact: Better use an "aio_group" together with a feed callback.
880 900
881 Sets the maximum number of outstanding requests to $nreqs. If you do 901 Sets the maximum number of outstanding requests to $nreqs. If you do
885 905
886 The default value is very large, so there is no practical limit on 906 The default value is very large, so there is no practical limit on
887 the number of outstanding requests. 907 the number of outstanding requests.
888 908
889 You can still queue as many requests as you want. Therefore, 909 You can still queue as many requests as you want. Therefore,
890 "max_oustsanding" is mainly useful in simple scripts (with low 910 "max_outstanding" is mainly useful in simple scripts (with low
891 values) or as a stop gap to shield against fatal memory overflow 911 values) or as a stop gap to shield against fatal memory overflow
892 (with large values). 912 (with large values).
893 913
894 STATISTICAL INFORMATION 914 STATISTICAL INFORMATION
895 IO::AIO::nreqs 915 IO::AIO::nreqs
945 965
946KNOWN BUGS 966KNOWN BUGS
947 Known bugs will be fixed in the next release. 967 Known bugs will be fixed in the next release.
948 968
949SEE ALSO 969SEE ALSO
950 Coro::AIO. 970 AnyEvent::AIO for easy integration into event loops, Coro::AIO for a
971 more natural syntax.
951 972
952AUTHOR 973AUTHOR
953 Marc Lehmann <schmorp@schmorp.de> 974 Marc Lehmann <schmorp@schmorp.de>
954 http://home.schmorp.de/ 975 http://home.schmorp.de/
955 976

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines