ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/README
(Generate patch)

Comparing IO-AIO/README (file contents):
Revision 1.28 by root, Sun Mar 30 06:31:49 2008 UTC vs.
Revision 1.34 by root, Thu Nov 20 09:01:40 2008 UTC

24 $req->cancel; # cancel request if still in queue 24 $req->cancel; # cancel request if still in queue
25 25
26 my $grp = aio_group sub { print "all stats done\n" }; 26 my $grp = aio_group sub { print "all stats done\n" };
27 add $grp aio_stat "..." for ...; 27 add $grp aio_stat "..." for ...;
28 28
29 # AnyEvent integration (EV, Event, Glib, Tk, urxvt, pureperl...) 29 # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
30 open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!"; 30 use AnyEvent::AIO;
31 my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb });
32 31
33 # EV integration 32 # EV integration
34 my $w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb; 33 my $w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
35 34
36 # Event integration 35 # Event integration
176 All the "aio_*" calls are more or less thin wrappers around the syscall 175 All the "aio_*" calls are more or less thin wrappers around the syscall
177 with the same name (sans "aio_"). The arguments are similar or 176 with the same name (sans "aio_"). The arguments are similar or
178 identical, and they all accept an additional (and optional) $callback 177 identical, and they all accept an additional (and optional) $callback
179 argument which must be a code reference. This code reference will get 178 argument which must be a code reference. This code reference will get
180 called with the syscall return code (e.g. most syscalls return -1 on 179 called with the syscall return code (e.g. most syscalls return -1 on
181 error, unlike perl, which usually delivers "false") as it's sole 180 error, unlike perl, which usually delivers "false") as its sole argument
182 argument when the given syscall has been executed asynchronously. 181 after the given syscall has been executed asynchronously.
183 182
184 All functions expecting a filehandle keep a copy of the filehandle 183 All functions expecting a filehandle keep a copy of the filehandle
185 internally until the request has finished. 184 internally until the request has finished.
186 185
187 All functions return request objects of type IO::AIO::REQ that allow 186 All functions return request objects of type IO::AIO::REQ that allow
200 the user environment, d) use Glib::filename_from_unicode on unicode 199 the user environment, d) use Glib::filename_from_unicode on unicode
201 filenames or e) use something else to ensure your scalar has the correct 200 filenames or e) use something else to ensure your scalar has the correct
202 contents. 201 contents.
203 202
204 This works, btw. independent of the internal UTF-8 bit, which IO::AIO 203 This works, btw. independent of the internal UTF-8 bit, which IO::AIO
205 handles correctly wether it is set or not. 204 handles correctly whether it is set or not.
206 205
207 $prev_pri = aioreq_pri [$pri] 206 $prev_pri = aioreq_pri [$pri]
208 Returns the priority value that would be used for the next request 207 Returns the priority value that would be used for the next request
209 and, if $pri is given, sets the priority for the next aio request. 208 and, if $pri is given, sets the priority for the next aio request.
210 209
265 Asynchronously close a file and call the callback with the result 264 Asynchronously close a file and call the callback with the result
266 code. 265 code.
267 266
268 Unfortunately, you can't do this to perl. Perl *insists* very 267 Unfortunately, you can't do this to perl. Perl *insists* very
269 strongly on closing the file descriptor associated with the 268 strongly on closing the file descriptor associated with the
270 filehandle itself. Here is what aio_close will try: 269 filehandle itself.
271 270
272 1. dup()licate the fd 271 Therefore, "aio_close" will not close the filehandle - instead it
273 2. asynchronously close() the duplicated fd 272 will use dup2 to overwrite the file descriptor with the write-end of
274 3. dup()licate the fd once more 273 a pipe (the pipe fd will be created on demand and will be cached).
275 4. let perl close() the filehandle
276 5. asynchronously close the duplicated fd
277 274
278 The idea is that the first close() flushes stuff to disk that 275 Or in other words: the file descriptor will be closed, but it will
279 closing an fd will flush, so when perl closes the fd, nothing much 276 not be free for reuse until the perl filehandle is closed.
280 will need to be flushed. The second async. close() will then flush
281 stuff to disk that closing the last fd to the file will flush.
282
283 Just FYI, SuSv3 has this to say on close:
284
285 All outstanding record locks owned by the process on the file
286 associated with the file descriptor shall be removed.
287
288 If fildes refers to a socket, close() shall cause the socket to be
289 destroyed. ... close() shall block for up to the current linger
290 interval until all data is transmitted.
291 [this actually sounds like a specification bug, but who knows]
292
293 And at least Linux additionally actually flushes stuff on every
294 close, even when the file itself is still open.
295
296 Sounds enourmously inefficient and complicated? Yes... please show
297 me how to nuke perl's fd out of existence...
298 277
299 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 278 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
300 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 279 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
301 Reads or writes $length bytes from the specified $fh and $offset 280 Reads or writes $length bytes from the specified $fh and $offset
302 into the scalar given by $data and offset $dataoffset and calls the 281 into the scalar given by $data and offset $dataoffset and calls the
472 aio_copy $srcpath, $dstpath, $callback->($status) 451 aio_copy $srcpath, $dstpath, $callback->($status)
473 Try to copy the *file* (directories not supported as either source 452 Try to copy the *file* (directories not supported as either source
474 or destination) from $srcpath to $dstpath and call the callback with 453 or destination) from $srcpath to $dstpath and call the callback with
475 the 0 (error) or -1 ok. 454 the 0 (error) or -1 ok.
476 455
477 This is a composite request that it creates the destination file 456 This is a composite request that creates the destination file with
478 with mode 0200 and copies the contents of the source file into it 457 mode 0200 and copies the contents of the source file into it using
479 using "aio_sendfile", followed by restoring atime, mtime, access 458 "aio_sendfile", followed by restoring atime, mtime, access mode and
480 mode and uid/gid, in that order. 459 uid/gid, in that order.
481 460
482 If an error occurs, the partial destination file will be unlinked, 461 If an error occurs, the partial destination file will be unlinked,
483 if possible, except when setting atime, mtime, access mode and 462 if possible, except when setting atime, mtime, access mode and
484 uid/gid, where errors are being ignored. 463 uid/gid, where errors are being ignored.
485 464
486 aio_move $srcpath, $dstpath, $callback->($status) 465 aio_move $srcpath, $dstpath, $callback->($status)
487 Try to move the *file* (directories not supported as either source 466 Try to move the *file* (directories not supported as either source
488 or destination) from $srcpath to $dstpath and call the callback with 467 or destination) from $srcpath to $dstpath and call the callback with
489 the 0 (error) or -1 ok. 468 the 0 (error) or -1 ok.
490 469
491 This is a composite request that tries to rename(2) the file first. 470 This is a composite request that tries to rename(2) the file first;
492 If rename files with "EXDEV", it copies the file with "aio_copy" 471 if rename fails with "EXDEV", it copies the file with "aio_copy"
493 and, if that is successful, unlinking the $srcpath. 472 and, if that is successful, unlinks the $srcpath.
494 473
495 aio_scandir $path, $maxreq, $callback->($dirs, $nondirs) 474 aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
496 Scans a directory (similar to "aio_readdir") but additionally tries 475 Scans a directory (similar to "aio_readdir") but additionally tries
497 to efficiently separate the entries of directory $path into two sets 476 to efficiently separate the entries of directory $path into two sets
498 of names, directories you can recurse into (directories), and ones 477 of names, directories you can recurse into (directories), and ones
563 callback with the fdatasync result code. 542 callback with the fdatasync result code.
564 543
565 If this call isn't available because your OS lacks it or it couldn't 544 If this call isn't available because your OS lacks it or it couldn't
566 be detected, it will be emulated by calling "fsync" instead. 545 be detected, it will be emulated by calling "fsync" instead.
567 546
547 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
548 Sync the data portion of the file specified by $offset and $length
549 to disk (but NOT the metadata), by calling the Linux-specific
550 sync_file_range call. If sync_file_range is not available or it
551 returns ENOSYS, then fdatasync or fsync is being substituted.
552
553 $flags can be a combination of
554 "IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE",
555 "IO::AIO::SYNC_FILE_RANGE_WRITE" and
556 "IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER": refer to the sync_file_range
557 manpage for details.
558
568 aio_pathsync $path, $callback->($status) 559 aio_pathsync $path, $callback->($status)
569 This request tries to open, fsync and close the given path. This is 560 This request tries to open, fsync and close the given path. This is
570 a composite request intended tosync directories after directory 561 a composite request intended to sync directories after directory
571 operations (E.g. rename). This might not work on all operating 562 operations (E.g. rename). This might not work on all operating
572 systems or have any specific effect, but usually it makes sure that 563 systems or have any specific effect, but usually it makes sure that
573 directory changes get written to disc. It works for anything that 564 directory changes get written to disc. It works for anything that
574 can be opened for read-only, not just directories. 565 can be opened for read-only, not just directories.
575 566
679 Their lifetime, simplified, looks like this: when they are empty, they 670 Their lifetime, simplified, looks like this: when they are empty, they
680 will finish very quickly. If they contain only requests that are in the 671 will finish very quickly. If they contain only requests that are in the
681 "done" state, they will also finish. Otherwise they will continue to 672 "done" state, they will also finish. Otherwise they will continue to
682 exist. 673 exist.
683 674
684 That means after creating a group you have some time to add requests. 675 That means after creating a group you have some time to add requests
685 And in the callbacks of those requests, you can add further requests to 676 (precisely before the callback has been invoked, which is only done
686 the group. And only when all those requests have finished will the the 677 within the "poll_cb"). And in the callbacks of those requests, you can
687 group itself finish. 678 add further requests to the group. And only when all those requests have
679 finished will the the group itself finish.
688 680
689 add $grp ... 681 add $grp ...
690 $grp->add (...) 682 $grp->add (...)
691 Add one or more requests to the group. Any type of IO::AIO::REQ can 683 Add one or more requests to the group. Any type of IO::AIO::REQ can
692 be added, including other groups, as long as you do not create 684 be added, including other groups, as long as you do not create
735 does not impose any limits). 727 does not impose any limits).
736 728
737 If the feed does not queue more requests when called, it will be 729 If the feed does not queue more requests when called, it will be
738 automatically removed from the group. 730 automatically removed from the group.
739 731
740 If the feed limit is 0, it will be set to 2 automatically. 732 If the feed limit is 0 when this method is called, it will be set to
733 2 automatically.
741 734
742 Example: 735 Example:
743 736
744 # stat all files in @files, but only ever use four aio requests concurrently: 737 # stat all files in @files, but only ever use four aio requests concurrently:
745 738
755 limit $grp $num 748 limit $grp $num
756 Sets the feeder limit for the group: The feeder will be called 749 Sets the feeder limit for the group: The feeder will be called
757 whenever the group contains less than this many requests. 750 whenever the group contains less than this many requests.
758 751
759 Setting the limit to 0 will pause the feeding process. 752 Setting the limit to 0 will pause the feeding process.
753
754 The default value for the limit is 0, but note that setting a feeder
755 automatically bumps it up to 2.
760 756
761 SUPPORT FUNCTIONS 757 SUPPORT FUNCTIONS
762 EVENT PROCESSING AND EVENT LOOP INTEGRATION 758 EVENT PROCESSING AND EVENT LOOP INTEGRATION
763 $fileno = IO::AIO::poll_fileno 759 $fileno = IO::AIO::poll_fileno
764 Return the *request result pipe file descriptor*. This filehandle 760 Return the *request result pipe file descriptor*. This filehandle
768 764
769 See "poll_cb" for an example. 765 See "poll_cb" for an example.
770 766
771 IO::AIO::poll_cb 767 IO::AIO::poll_cb
772 Process some outstanding events on the result pipe. You have to call 768 Process some outstanding events on the result pipe. You have to call
773 this regularly. Returns the number of events processed. Returns 769 this regularly. Returns 0 if all events could be processed, or -1 if
774 immediately when no events are outstanding. The amount of events 770 it returned earlier for whatever reason. Returns immediately when no
771 events are outstanding. The amount of events processed depends on
775 processed depends on the settings of "IO::AIO::max_poll_req" and 772 the settings of "IO::AIO::max_poll_req" and
776 "IO::AIO::max_poll_time". 773 "IO::AIO::max_poll_time".
777 774
778 If not all requests were processed for whatever reason, the 775 If not all requests were processed for whatever reason, the
779 filehandle will still be ready when "poll_cb" returns. 776 filehandle will still be ready when "poll_cb" returns, so normally
777 you don't have to do anything special to have it called later.
780 778
781 Example: Install an Event watcher that automatically calls 779 Example: Install an Event watcher that automatically calls
782 IO::AIO::poll_cb with high priority: 780 IO::AIO::poll_cb with high priority:
783 781
784 Event->io (fd => IO::AIO::poll_fileno, 782 Event->io (fd => IO::AIO::poll_fileno,
890 888
891 The default is probably ok in most situations, especially if thread 889 The default is probably ok in most situations, especially if thread
892 creation is fast. If thread creation is very slow on your system you 890 creation is fast. If thread creation is very slow on your system you
893 might want to use larger values. 891 might want to use larger values.
894 892
895 $oldmaxreqs = IO::AIO::max_outstanding $maxreqs 893 IO::AIO::max_outstanding $maxreqs
896 This is a very bad function to use in interactive programs because 894 This is a very bad function to use in interactive programs because
897 it blocks, and a bad way to reduce concurrency because it is 895 it blocks, and a bad way to reduce concurrency because it is
898 inexact: Better use an "aio_group" together with a feed callback. 896 inexact: Better use an "aio_group" together with a feed callback.
899 897
900 Sets the maximum number of outstanding requests to $nreqs. If you do 898 Sets the maximum number of outstanding requests to $nreqs. If you do
904 902
905 The default value is very large, so there is no practical limit on 903 The default value is very large, so there is no practical limit on
906 the number of outstanding requests. 904 the number of outstanding requests.
907 905
908 You can still queue as many requests as you want. Therefore, 906 You can still queue as many requests as you want. Therefore,
909 "max_oustsanding" is mainly useful in simple scripts (with low 907 "max_outstanding" is mainly useful in simple scripts (with low
910 values) or as a stop gap to shield against fatal memory overflow 908 values) or as a stop gap to shield against fatal memory overflow
911 (with large values). 909 (with large values).
912 910
913 STATISTICAL INFORMATION 911 STATISTICAL INFORMATION
914 IO::AIO::nreqs 912 IO::AIO::nreqs
964 962
965KNOWN BUGS 963KNOWN BUGS
966 Known bugs will be fixed in the next release. 964 Known bugs will be fixed in the next release.
967 965
968SEE ALSO 966SEE ALSO
969 Coro::AIO. 967 AnyEvent::AIO for easy integration into event loops, Coro::AIO for a
968 more natural syntax.
970 969
971AUTHOR 970AUTHOR
972 Marc Lehmann <schmorp@schmorp.de> 971 Marc Lehmann <schmorp@schmorp.de>
973 http://home.schmorp.de/ 972 http://home.schmorp.de/
974 973

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines