ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/README
(Generate patch)

Comparing IO-AIO/README (file contents):
Revision 1.26 by root, Thu Oct 4 12:50:35 2007 UTC vs.
Revision 1.34 by root, Thu Nov 20 09:01:40 2008 UTC

24 $req->cancel; # cancel request if still in queue 24 $req->cancel; # cancel request if still in queue
25 25
26 my $grp = aio_group sub { print "all stats done\n" }; 26 my $grp = aio_group sub { print "all stats done\n" };
27 add $grp aio_stat "..." for ...; 27 add $grp aio_stat "..." for ...;
28 28
29 # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
30 use AnyEvent::AIO;
31
29 # AnyEvent integration 32 # EV integration
30 open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!"; 33 my $w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
31 my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });
32 34
33 # Event integration 35 # Event integration
34 Event->io (fd => IO::AIO::poll_fileno, 36 Event->io (fd => IO::AIO::poll_fileno,
35 poll => 'r', 37 poll => 'r',
36 cb => \&IO::AIO::poll_cb); 38 cb => \&IO::AIO::poll_cb);
173 All the "aio_*" calls are more or less thin wrappers around the syscall 175 All the "aio_*" calls are more or less thin wrappers around the syscall
174 with the same name (sans "aio_"). The arguments are similar or 176 with the same name (sans "aio_"). The arguments are similar or
175 identical, and they all accept an additional (and optional) $callback 177 identical, and they all accept an additional (and optional) $callback
176 argument which must be a code reference. This code reference will get 178 argument which must be a code reference. This code reference will get
177 called with the syscall return code (e.g. most syscalls return -1 on 179 called with the syscall return code (e.g. most syscalls return -1 on
178 error, unlike perl, which usually delivers "false") as it's sole 180 error, unlike perl, which usually delivers "false") as its sole argument
179 argument when the given syscall has been executed asynchronously. 181 after the given syscall has been executed asynchronously.
180 182
181 All functions expecting a filehandle keep a copy of the filehandle 183 All functions expecting a filehandle keep a copy of the filehandle
182 internally until the request has finished. 184 internally until the request has finished.
183 185
184 All functions return request objects of type IO::AIO::REQ that allow 186 All functions return request objects of type IO::AIO::REQ that allow
197 the user environment, d) use Glib::filename_from_unicode on unicode 199 the user environment, d) use Glib::filename_from_unicode on unicode
198 filenames or e) use something else to ensure your scalar has the correct 200 filenames or e) use something else to ensure your scalar has the correct
199 contents. 201 contents.
200 202
201 This works, btw. independent of the internal UTF-8 bit, which IO::AIO 203 This works, btw. independent of the internal UTF-8 bit, which IO::AIO
202 handles correctly wether it is set or not. 204 handles correctly whether it is set or not.
203 205
204 $prev_pri = aioreq_pri [$pri] 206 $prev_pri = aioreq_pri [$pri]
205 Returns the priority value that would be used for the next request 207 Returns the priority value that would be used for the next request
206 and, if $pri is given, sets the priority for the next aio request. 208 and, if $pri is given, sets the priority for the next aio request.
207 209
260 262
261 aio_close $fh, $callback->($status) 263 aio_close $fh, $callback->($status)
262 Asynchronously close a file and call the callback with the result 264 Asynchronously close a file and call the callback with the result
263 code. 265 code.
264 266
265 Unlike the other functions operating on files, this function uses 267 Unfortunately, you can't do this to perl. Perl *insists* very
266 the PerlIO layer to close the filehandle. The reason is that the 268 strongly on closing the file descriptor associated with the
267 PerlIO API insists on closing the underlying fd itself, no matter 269 filehandle itself.
268 what, and doesn't allow modifications to the fd. Unfortunately, it
269 is not clear that you can call PerlIO from different threads
270 (actually, its quite clear that this won't work in some cases), so
271 while it likely works perfectly with simple file handles (such as
272 the ones created by "aio_open") it might fail in interesting ways
273 for others.
274 270
275 Having said that, aio_close tries to clean up the filehandle as much 271 Therefore, "aio_close" will not close the filehandle - instead it
276 as possible before handing it to an io thread, and generally does 272 will use dup2 to overwrite the file descriptor with the write-end of
277 work. 273 a pipe (the pipe fd will be created on demand and will be cached).
274
275 Or in other words: the file descriptor will be closed, but it will
276 not be free for reuse until the perl filehandle is closed.
278 277
279 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 278 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
280 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 279 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
281 Reads or writes $length bytes from the specified $fh and $offset 280 Reads or writes $length bytes from the specified $fh and $offset
282 into the scalar given by $data and offset $dataoffset and calls the 281 into the scalar given by $data and offset $dataoffset and calls the
452 aio_copy $srcpath, $dstpath, $callback->($status) 451 aio_copy $srcpath, $dstpath, $callback->($status)
453 Try to copy the *file* (directories not supported as either source 452 Try to copy the *file* (directories not supported as either source
454 or destination) from $srcpath to $dstpath and call the callback with 453 or destination) from $srcpath to $dstpath and call the callback with
455 the 0 (error) or -1 ok. 454 the 0 (error) or -1 ok.
456 455
457 This is a composite request that it creates the destination file 456 This is a composite request that creates the destination file with
458 with mode 0200 and copies the contents of the source file into it 457 mode 0200 and copies the contents of the source file into it using
459 using "aio_sendfile", followed by restoring atime, mtime, access 458 "aio_sendfile", followed by restoring atime, mtime, access mode and
460 mode and uid/gid, in that order. 459 uid/gid, in that order.
461 460
462 If an error occurs, the partial destination file will be unlinked, 461 If an error occurs, the partial destination file will be unlinked,
463 if possible, except when setting atime, mtime, access mode and 462 if possible, except when setting atime, mtime, access mode and
464 uid/gid, where errors are being ignored. 463 uid/gid, where errors are being ignored.
465 464
466 aio_move $srcpath, $dstpath, $callback->($status) 465 aio_move $srcpath, $dstpath, $callback->($status)
467 Try to move the *file* (directories not supported as either source 466 Try to move the *file* (directories not supported as either source
468 or destination) from $srcpath to $dstpath and call the callback with 467 or destination) from $srcpath to $dstpath and call the callback with
469 the 0 (error) or -1 ok. 468 the 0 (error) or -1 ok.
470 469
471 This is a composite request that tries to rename(2) the file first. 470 This is a composite request that tries to rename(2) the file first;
472 If rename files with "EXDEV", it copies the file with "aio_copy" 471 if rename fails with "EXDEV", it copies the file with "aio_copy"
473 and, if that is successful, unlinking the $srcpath. 472 and, if that is successful, unlinks the $srcpath.
474 473
475 aio_scandir $path, $maxreq, $callback->($dirs, $nondirs) 474 aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
476 Scans a directory (similar to "aio_readdir") but additionally tries 475 Scans a directory (similar to "aio_readdir") but additionally tries
477 to efficiently separate the entries of directory $path into two sets 476 to efficiently separate the entries of directory $path into two sets
478 of names, directories you can recurse into (directories), and ones 477 of names, directories you can recurse into (directories), and ones
529 Delete a directory tree starting (and including) $path, return the 528 Delete a directory tree starting (and including) $path, return the
530 status of the final "rmdir" only. This is a composite request that 529 status of the final "rmdir" only. This is a composite request that
531 uses "aio_scandir" to recurse into and rmdir directories, and unlink 530 uses "aio_scandir" to recurse into and rmdir directories, and unlink
532 everything else. 531 everything else.
533 532
533 aio_sync $callback->($status)
534 Asynchronously call sync and call the callback when finished.
535
534 aio_fsync $fh, $callback->($status) 536 aio_fsync $fh, $callback->($status)
535 Asynchronously call fsync on the given filehandle and call the 537 Asynchronously call fsync on the given filehandle and call the
536 callback with the fsync result code. 538 callback with the fsync result code.
537 539
538 aio_fdatasync $fh, $callback->($status) 540 aio_fdatasync $fh, $callback->($status)
539 Asynchronously call fdatasync on the given filehandle and call the 541 Asynchronously call fdatasync on the given filehandle and call the
540 callback with the fdatasync result code. 542 callback with the fdatasync result code.
541 543
542 If this call isn't available because your OS lacks it or it couldn't 544 If this call isn't available because your OS lacks it or it couldn't
543 be detected, it will be emulated by calling "fsync" instead. 545 be detected, it will be emulated by calling "fsync" instead.
546
547 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
548 Sync the data portion of the file specified by $offset and $length
549 to disk (but NOT the metadata), by calling the Linux-specific
550 sync_file_range call. If sync_file_range is not available or it
551 returns ENOSYS, then fdatasync or fsync is being substituted.
552
553 $flags can be a combination of
554 "IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE",
555 "IO::AIO::SYNC_FILE_RANGE_WRITE" and
556 "IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER": refer to the sync_file_range
557 manpage for details.
558
559 aio_pathsync $path, $callback->($status)
560 This request tries to open, fsync and close the given path. This is
561 a composite request intended to sync directories after directory
562 operations (E.g. rename). This might not work on all operating
563 systems or have any specific effect, but usually it makes sure that
564 directory changes get written to disc. It works for anything that
565 can be opened for read-only, not just directories.
566
567 Passes 0 when everything went ok, and -1 on error.
544 568
545 aio_group $callback->(...) 569 aio_group $callback->(...)
546 This is a very special aio request: Instead of doing something, it 570 This is a very special aio request: Instead of doing something, it
547 is a container for other aio requests, which is useful if you want 571 is a container for other aio requests, which is useful if you want
548 to bundle many requests into a single, composite, request with a 572 to bundle many requests into a single, composite, request with a
630 }; 654 };
631 655
632 This makes it very easy to create composite requests (see the source of 656 This makes it very easy to create composite requests (see the source of
633 "aio_move" for an application) that work and feel like simple requests. 657 "aio_move" for an application) that work and feel like simple requests.
634 658
635 * The IO::AIO::GRP objects will be cleaned up during calls to 659 * The IO::AIO::GRP objects will be cleaned up during calls to
636 "IO::AIO::poll_cb", just like any other request. 660 "IO::AIO::poll_cb", just like any other request.
661
637 * They can be canceled like any other request. Canceling will cancel not 662 * They can be canceled like any other request. Canceling will cancel
638 only the request itself, but also all requests it contains. 663 not only the request itself, but also all requests it contains.
664
639 * They can also can also be added to other IO::AIO::GRP objects. 665 * They can also can also be added to other IO::AIO::GRP objects.
666
640 * You must not add requests to a group from within the group callback 667 * You must not add requests to a group from within the group callback
641 (or any later time). 668 (or any later time).
642 669
643 Their lifetime, simplified, looks like this: when they are empty, they 670 Their lifetime, simplified, looks like this: when they are empty, they
644 will finish very quickly. If they contain only requests that are in the 671 will finish very quickly. If they contain only requests that are in the
645 "done" state, they will also finish. Otherwise they will continue to 672 "done" state, they will also finish. Otherwise they will continue to
646 exist. 673 exist.
647 674
648 That means after creating a group you have some time to add requests. 675 That means after creating a group you have some time to add requests
649 And in the callbacks of those requests, you can add further requests to 676 (precisely before the callback has been invoked, which is only done
650 the group. And only when all those requests have finished will the the 677 within the "poll_cb"). And in the callbacks of those requests, you can
651 group itself finish. 678 add further requests to the group. And only when all those requests have
679 finished will the the group itself finish.
652 680
653 add $grp ... 681 add $grp ...
654 $grp->add (...) 682 $grp->add (...)
655 Add one or more requests to the group. Any type of IO::AIO::REQ can 683 Add one or more requests to the group. Any type of IO::AIO::REQ can
656 be added, including other groups, as long as you do not create 684 be added, including other groups, as long as you do not create
663 request itself. Useful when you queued a lot of events but got a 691 request itself. Useful when you queued a lot of events but got a
664 result early. 692 result early.
665 693
666 $grp->result (...) 694 $grp->result (...)
667 Set the result value(s) that will be passed to the group callback 695 Set the result value(s) that will be passed to the group callback
668 when all subrequests have finished and set thre groups errno to the 696 when all subrequests have finished and set the groups errno to the
669 current value of errno (just like calling "errno" without an error 697 current value of errno (just like calling "errno" without an error
670 number). By default, no argument will be passed and errno is zero. 698 number). By default, no argument will be passed and errno is zero.
671 699
672 $grp->errno ([$errno]) 700 $grp->errno ([$errno])
673 Sets the group errno value to $errno, or the current value of errno 701 Sets the group errno value to $errno, or the current value of errno
699 does not impose any limits). 727 does not impose any limits).
700 728
701 If the feed does not queue more requests when called, it will be 729 If the feed does not queue more requests when called, it will be
702 automatically removed from the group. 730 automatically removed from the group.
703 731
704 If the feed limit is 0, it will be set to 2 automatically. 732 If the feed limit is 0 when this method is called, it will be set to
733 2 automatically.
705 734
706 Example: 735 Example:
707 736
708 # stat all files in @files, but only ever use four aio requests concurrently: 737 # stat all files in @files, but only ever use four aio requests concurrently:
709 738
719 limit $grp $num 748 limit $grp $num
720 Sets the feeder limit for the group: The feeder will be called 749 Sets the feeder limit for the group: The feeder will be called
721 whenever the group contains less than this many requests. 750 whenever the group contains less than this many requests.
722 751
723 Setting the limit to 0 will pause the feeding process. 752 Setting the limit to 0 will pause the feeding process.
753
754 The default value for the limit is 0, but note that setting a feeder
755 automatically bumps it up to 2.
724 756
725 SUPPORT FUNCTIONS 757 SUPPORT FUNCTIONS
726 EVENT PROCESSING AND EVENT LOOP INTEGRATION 758 EVENT PROCESSING AND EVENT LOOP INTEGRATION
727 $fileno = IO::AIO::poll_fileno 759 $fileno = IO::AIO::poll_fileno
728 Return the *request result pipe file descriptor*. This filehandle 760 Return the *request result pipe file descriptor*. This filehandle
732 764
733 See "poll_cb" for an example. 765 See "poll_cb" for an example.
734 766
735 IO::AIO::poll_cb 767 IO::AIO::poll_cb
736 Process some outstanding events on the result pipe. You have to call 768 Process some outstanding events on the result pipe. You have to call
737 this regularly. Returns the number of events processed. Returns 769 this regularly. Returns 0 if all events could be processed, or -1 if
738 immediately when no events are outstanding. The amount of events 770 it returned earlier for whatever reason. Returns immediately when no
771 events are outstanding. The amount of events processed depends on
739 processed depends on the settings of "IO::AIO::max_poll_req" and 772 the settings of "IO::AIO::max_poll_req" and
740 "IO::AIO::max_poll_time". 773 "IO::AIO::max_poll_time".
741 774
742 If not all requests were processed for whatever reason, the 775 If not all requests were processed for whatever reason, the
743 filehandle will still be ready when "poll_cb" returns. 776 filehandle will still be ready when "poll_cb" returns, so normally
777 you don't have to do anything special to have it called later.
744 778
745 Example: Install an Event watcher that automatically calls 779 Example: Install an Event watcher that automatically calls
746 IO::AIO::poll_cb with high priority: 780 IO::AIO::poll_cb with high priority:
747 781
748 Event->io (fd => IO::AIO::poll_fileno, 782 Event->io (fd => IO::AIO::poll_fileno,
854 888
855 The default is probably ok in most situations, especially if thread 889 The default is probably ok in most situations, especially if thread
856 creation is fast. If thread creation is very slow on your system you 890 creation is fast. If thread creation is very slow on your system you
857 might want to use larger values. 891 might want to use larger values.
858 892
859 $oldmaxreqs = IO::AIO::max_outstanding $maxreqs 893 IO::AIO::max_outstanding $maxreqs
860 This is a very bad function to use in interactive programs because 894 This is a very bad function to use in interactive programs because
861 it blocks, and a bad way to reduce concurrency because it is 895 it blocks, and a bad way to reduce concurrency because it is
862 inexact: Better use an "aio_group" together with a feed callback. 896 inexact: Better use an "aio_group" together with a feed callback.
863 897
864 Sets the maximum number of outstanding requests to $nreqs. If you do 898 Sets the maximum number of outstanding requests to $nreqs. If you do
868 902
869 The default value is very large, so there is no practical limit on 903 The default value is very large, so there is no practical limit on
870 the number of outstanding requests. 904 the number of outstanding requests.
871 905
872 You can still queue as many requests as you want. Therefore, 906 You can still queue as many requests as you want. Therefore,
873 "max_oustsanding" is mainly useful in simple scripts (with low 907 "max_outstanding" is mainly useful in simple scripts (with low
874 values) or as a stop gap to shield against fatal memory overflow 908 values) or as a stop gap to shield against fatal memory overflow
875 (with large values). 909 (with large values).
876 910
877 STATISTICAL INFORMATION 911 STATISTICAL INFORMATION
878 IO::AIO::nreqs 912 IO::AIO::nreqs
928 962
929KNOWN BUGS 963KNOWN BUGS
930 Known bugs will be fixed in the next release. 964 Known bugs will be fixed in the next release.
931 965
932SEE ALSO 966SEE ALSO
933 Coro::AIO. 967 AnyEvent::AIO for easy integration into event loops, Coro::AIO for a
968 more natural syntax.
934 969
935AUTHOR 970AUTHOR
936 Marc Lehmann <schmorp@schmorp.de> 971 Marc Lehmann <schmorp@schmorp.de>
937 http://home.schmorp.de/ 972 http://home.schmorp.de/
938 973

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines