ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/README
(Generate patch)

Comparing IO-AIO/README (file contents):
Revision 1.30 by root, Sat May 10 22:58:16 2008 UTC vs.
Revision 1.36 by root, Sun Jun 7 18:31:18 2009 UTC

175 All the "aio_*" calls are more or less thin wrappers around the syscall 175 All the "aio_*" calls are more or less thin wrappers around the syscall
176 with the same name (sans "aio_"). The arguments are similar or 176 with the same name (sans "aio_"). The arguments are similar or
177 identical, and they all accept an additional (and optional) $callback 177 identical, and they all accept an additional (and optional) $callback
178 argument which must be a code reference. This code reference will get 178 argument which must be a code reference. This code reference will get
179 called with the syscall return code (e.g. most syscalls return -1 on 179 called with the syscall return code (e.g. most syscalls return -1 on
180 error, unlike perl, which usually delivers "false") as it's sole 180 error, unlike perl, which usually delivers "false") as its sole argument
181 argument when the given syscall has been executed asynchronously. 181 after the given syscall has been executed asynchronously.
182 182
183 All functions expecting a filehandle keep a copy of the filehandle 183 All functions expecting a filehandle keep a copy of the filehandle
184 internally until the request has finished. 184 internally until the request has finished.
185 185
186 All functions return request objects of type IO::AIO::REQ that allow 186 All functions return request objects of type IO::AIO::REQ that allow
199 the user environment, d) use Glib::filename_from_unicode on unicode 199 the user environment, d) use Glib::filename_from_unicode on unicode
200 filenames or e) use something else to ensure your scalar has the correct 200 filenames or e) use something else to ensure your scalar has the correct
201 contents. 201 contents.
202 202
203 This works, btw. independent of the internal UTF-8 bit, which IO::AIO 203 This works, btw. independent of the internal UTF-8 bit, which IO::AIO
204 handles correctly wether it is set or not. 204 handles correctly whether it is set or not.
205 205
206 $prev_pri = aioreq_pri [$pri] 206 $prev_pri = aioreq_pri [$pri]
207 Returns the priority value that would be used for the next request 207 Returns the priority value that would be used for the next request
208 and, if $pri is given, sets the priority for the next aio request. 208 and, if $pri is given, sets the priority for the next aio request.
209 209
275 Or in other words: the file descriptor will be closed, but it will 275 Or in other words: the file descriptor will be closed, but it will
276 not be free for reuse until the perl filehandle is closed. 276 not be free for reuse until the perl filehandle is closed.
277 277
278 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 278 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
279 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 279 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
280 Reads or writes $length bytes from the specified $fh and $offset 280 Reads or writes $length bytes from or to the specified $fh and
281 into the scalar given by $data and offset $dataoffset and calls the 281 $offset into the scalar given by $data and offset $dataoffset and
282 callback without the actual number of bytes read (or -1 on error, 282 calls the callback without the actual number of bytes read (or -1 on
283 just like the syscall). 283 error, just like the syscall).
284
285 "aio_read" will, like "sysread", shrink or grow the $data scalar to
286 offset plus the actual number of bytes read.
284 287
285 If $offset is undefined, then the current file descriptor offset 288 If $offset is undefined, then the current file descriptor offset
286 will be used (and updated), otherwise the file descriptor offset 289 will be used (and updated), otherwise the file descriptor offset
287 will not be changed by these calls. 290 will not be changed by these calls.
288 291
439 aio_readdir $pathname, $callback->($entries) 442 aio_readdir $pathname, $callback->($entries)
440 Unlike the POSIX call of the same name, "aio_readdir" reads an 443 Unlike the POSIX call of the same name, "aio_readdir" reads an
441 entire directory (i.e. opendir + readdir + closedir). The entries 444 entire directory (i.e. opendir + readdir + closedir). The entries
442 will not be sorted, and will NOT include the "." and ".." entries. 445 will not be sorted, and will NOT include the "." and ".." entries.
443 446
444 The callback a single argument which is either "undef" or an 447 The callback is passed a single argument which is either "undef" or
445 array-ref with the filenames. 448 an array-ref with the filenames.
449
450 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
451 Quite similar to "aio_readdir", but the $flags argument allows to
452 tune behaviour and output format. In case of an error, $entries will
453 be "undef".
454
455 The flags are a combination of the following constants, ORed
456 together (the flags will also be passed to the callback, possibly
457 modified):
458
459 IO::AIO::READDIR_DENTS
460 When this flag is off, then the callback gets an arrayref with
461 of names only (as with "aio_readdir"), otherwise it gets an
462 arrayref with "[$name, $type, $inode]" arrayrefs, each
463 describing a single directory entry in more detail.
464
465 $name is the name of the entry.
466
467 $type is one of the "IO::AIO::DT_xxx" constants:
468
469 "IO::AIO::DT_UNKNOWN", "IO::AIO::DT_FIFO", "IO::AIO::DT_CHR",
470 "IO::AIO::DT_DIR", "IO::AIO::DT_BLK", "IO::AIO::DT_REG",
471 "IO::AIO::DT_LNK", "IO::AIO::DT_SOCK", "IO::AIO::DT_WHT".
472
473 "IO::AIO::DT_UNKNOWN" means just that: readdir does not know. If
474 you need to know, you have to run stat yourself. Also, for speed
475 reasons, the $type scalars are read-only: you can not modify
476 them.
477
478 $inode is the inode number (which might not be exact on systems
479 with 64 bit inode numbers and 32 bit perls). On systems that do
480 not deliver the inode information, this will always be zero.
481
482 IO::AIO::READDIR_DIRS_FIRST
483 When this flag is set, then the names will be returned in an
484 order where likely directories come first. This is useful when
485 you need to quickly find directories, or you want to find all
486 directories while avoiding to stat() each entry.
487
488 If the system returns type information in readdir, then this is
489 used to find directories directly. Otherwise, likely directories
490 are files beginning with ".", or otherwise files with no dots,
491 of which files with short names are tried first.
492
493 IO::AIO::READDIR_STAT_ORDER
494 When this flag is set, then the names will be returned in an
495 order suitable for stat()'ing each one. That is, when you plan
496 to stat() all files in the given directory, then the returned
497 order will likely be fastest.
498
499 If both this flag and "IO::AIO::READDIR_DIRS_FIRST" are
500 specified, then the likely dirs come first, resulting in a less
501 optimal stat order.
502
503 IO::AIO::READDIR_FOUND_UNKNOWN
504 This flag should not be set when calling "aio_readdirx".
505 Instead, it is being set by "aio_readdirx", when any of the
506 $type's found were "IO::AIO::DT_UNKNOWN". The absense of this
507 flag therefore indicates that all $type's are known, which can
508 be used to speed up some algorithms.
446 509
447 aio_load $path, $data, $callback->($status) 510 aio_load $path, $data, $callback->($status)
448 This is a composite request that tries to fully load the given file 511 This is a composite request that tries to fully load the given file
449 into memory. Status is the same as with aio_read. 512 into memory. Status is the same as with aio_read.
450 513
451 aio_copy $srcpath, $dstpath, $callback->($status) 514 aio_copy $srcpath, $dstpath, $callback->($status)
452 Try to copy the *file* (directories not supported as either source 515 Try to copy the *file* (directories not supported as either source
453 or destination) from $srcpath to $dstpath and call the callback with 516 or destination) from $srcpath to $dstpath and call the callback with
454 the 0 (error) or -1 ok. 517 the 0 (error) or -1 ok.
455 518
456 This is a composite request that it creates the destination file 519 This is a composite request that creates the destination file with
457 with mode 0200 and copies the contents of the source file into it 520 mode 0200 and copies the contents of the source file into it using
458 using "aio_sendfile", followed by restoring atime, mtime, access 521 "aio_sendfile", followed by restoring atime, mtime, access mode and
459 mode and uid/gid, in that order. 522 uid/gid, in that order.
460 523
461 If an error occurs, the partial destination file will be unlinked, 524 If an error occurs, the partial destination file will be unlinked,
462 if possible, except when setting atime, mtime, access mode and 525 if possible, except when setting atime, mtime, access mode and
463 uid/gid, where errors are being ignored. 526 uid/gid, where errors are being ignored.
464 527
465 aio_move $srcpath, $dstpath, $callback->($status) 528 aio_move $srcpath, $dstpath, $callback->($status)
466 Try to move the *file* (directories not supported as either source 529 Try to move the *file* (directories not supported as either source
467 or destination) from $srcpath to $dstpath and call the callback with 530 or destination) from $srcpath to $dstpath and call the callback with
468 the 0 (error) or -1 ok. 531 the 0 (error) or -1 ok.
469 532
470 This is a composite request that tries to rename(2) the file first. 533 This is a composite request that tries to rename(2) the file first;
471 If rename files with "EXDEV", it copies the file with "aio_copy" 534 if rename fails with "EXDEV", it copies the file with "aio_copy"
472 and, if that is successful, unlinking the $srcpath. 535 and, if that is successful, unlinks the $srcpath.
473 536
474 aio_scandir $path, $maxreq, $callback->($dirs, $nondirs) 537 aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
475 Scans a directory (similar to "aio_readdir") but additionally tries 538 Scans a directory (similar to "aio_readdir") but additionally tries
476 to efficiently separate the entries of directory $path into two sets 539 to efficiently separate the entries of directory $path into two sets
477 of names, directories you can recurse into (directories), and ones 540 of names, directories you can recurse into (directories), and ones
497 Implementation notes. 560 Implementation notes.
498 561
499 The "aio_readdir" cannot be avoided, but "stat()"'ing every entry 562 The "aio_readdir" cannot be avoided, but "stat()"'ing every entry
500 can. 563 can.
501 564
565 If readdir returns file type information, then this is used directly
566 to find directories.
567
502 After reading the directory, the modification time, size etc. of the 568 Otherwise, after reading the directory, the modification time, size
503 directory before and after the readdir is checked, and if they match 569 etc. of the directory before and after the readdir is checked, and
504 (and isn't the current time), the link count will be used to decide 570 if they match (and isn't the current time), the link count will be
505 how many entries are directories (if >= 2). Otherwise, no knowledge 571 used to decide how many entries are directories (if >= 2).
506 of the number of subdirectories will be assumed. 572 Otherwise, no knowledge of the number of subdirectories will be
573 assumed.
507 574
508 Then entries will be sorted into likely directories (everything 575 Then entries will be sorted into likely directories a non-initial
509 without a non-initial dot currently) and likely non-directories 576 dot currently) and likely non-directories (see "aio_readdirx"). Then
510 (everything else). Then every entry plus an appended "/." will be 577 every entry plus an appended "/." will be "stat"'ed, likely
511 "stat"'ed, likely directories first. If that succeeds, it assumes 578 directories first, in order of their inode numbers. If that
512 that the entry is a directory or a symlink to directory (which will 579 succeeds, it assumes that the entry is a directory or a symlink to
513 be checked seperately). This is often faster than stat'ing the entry 580 directory (which will be checked seperately). This is often faster
514 itself because filesystems might detect the type of the entry 581 than stat'ing the entry itself because filesystems might detect the
515 without reading the inode data (e.g. ext2fs filetype feature). 582 type of the entry without reading the inode data (e.g. ext2fs
583 filetype feature), even on systems that cannot return the filetype
584 information on readdir.
516 585
517 If the known number of directories (link count - 2) has been 586 If the known number of directories (link count - 2) has been
518 reached, the rest of the entries is assumed to be non-directories. 587 reached, the rest of the entries is assumed to be non-directories.
519 588
520 This only works with certainty on POSIX (= UNIX) filesystems, which 589 This only works with certainty on POSIX (= UNIX) filesystems, which
542 callback with the fdatasync result code. 611 callback with the fdatasync result code.
543 612
544 If this call isn't available because your OS lacks it or it couldn't 613 If this call isn't available because your OS lacks it or it couldn't
545 be detected, it will be emulated by calling "fsync" instead. 614 be detected, it will be emulated by calling "fsync" instead.
546 615
616 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
617 Sync the data portion of the file specified by $offset and $length
618 to disk (but NOT the metadata), by calling the Linux-specific
619 sync_file_range call. If sync_file_range is not available or it
620 returns ENOSYS, then fdatasync or fsync is being substituted.
621
622 $flags can be a combination of
623 "IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE",
624 "IO::AIO::SYNC_FILE_RANGE_WRITE" and
625 "IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER": refer to the sync_file_range
626 manpage for details.
627
547 aio_pathsync $path, $callback->($status) 628 aio_pathsync $path, $callback->($status)
548 This request tries to open, fsync and close the given path. This is 629 This request tries to open, fsync and close the given path. This is
549 a composite request intended tosync directories after directory 630 a composite request intended to sync directories after directory
550 operations (E.g. rename). This might not work on all operating 631 operations (E.g. rename). This might not work on all operating
551 systems or have any specific effect, but usually it makes sure that 632 systems or have any specific effect, but usually it makes sure that
552 directory changes get written to disc. It works for anything that 633 directory changes get written to disc. It works for anything that
553 can be opened for read-only, not just directories. 634 can be opened for read-only, not just directories.
554 635
658 Their lifetime, simplified, looks like this: when they are empty, they 739 Their lifetime, simplified, looks like this: when they are empty, they
659 will finish very quickly. If they contain only requests that are in the 740 will finish very quickly. If they contain only requests that are in the
660 "done" state, they will also finish. Otherwise they will continue to 741 "done" state, they will also finish. Otherwise they will continue to
661 exist. 742 exist.
662 743
663 That means after creating a group you have some time to add requests. 744 That means after creating a group you have some time to add requests
664 And in the callbacks of those requests, you can add further requests to 745 (precisely before the callback has been invoked, which is only done
665 the group. And only when all those requests have finished will the the 746 within the "poll_cb"). And in the callbacks of those requests, you can
666 group itself finish. 747 add further requests to the group. And only when all those requests have
748 finished will the the group itself finish.
667 749
668 add $grp ... 750 add $grp ...
669 $grp->add (...) 751 $grp->add (...)
670 Add one or more requests to the group. Any type of IO::AIO::REQ can 752 Add one or more requests to the group. Any type of IO::AIO::REQ can
671 be added, including other groups, as long as you do not create 753 be added, including other groups, as long as you do not create
714 does not impose any limits). 796 does not impose any limits).
715 797
716 If the feed does not queue more requests when called, it will be 798 If the feed does not queue more requests when called, it will be
717 automatically removed from the group. 799 automatically removed from the group.
718 800
719 If the feed limit is 0, it will be set to 2 automatically. 801 If the feed limit is 0 when this method is called, it will be set to
802 2 automatically.
720 803
721 Example: 804 Example:
722 805
723 # stat all files in @files, but only ever use four aio requests concurrently: 806 # stat all files in @files, but only ever use four aio requests concurrently:
724 807
734 limit $grp $num 817 limit $grp $num
735 Sets the feeder limit for the group: The feeder will be called 818 Sets the feeder limit for the group: The feeder will be called
736 whenever the group contains less than this many requests. 819 whenever the group contains less than this many requests.
737 820
738 Setting the limit to 0 will pause the feeding process. 821 Setting the limit to 0 will pause the feeding process.
822
823 The default value for the limit is 0, but note that setting a feeder
824 automatically bumps it up to 2.
739 825
740 SUPPORT FUNCTIONS 826 SUPPORT FUNCTIONS
741 EVENT PROCESSING AND EVENT LOOP INTEGRATION 827 EVENT PROCESSING AND EVENT LOOP INTEGRATION
742 $fileno = IO::AIO::poll_fileno 828 $fileno = IO::AIO::poll_fileno
743 Return the *request result pipe file descriptor*. This filehandle 829 Return the *request result pipe file descriptor*. This filehandle
747 833
748 See "poll_cb" for an example. 834 See "poll_cb" for an example.
749 835
750 IO::AIO::poll_cb 836 IO::AIO::poll_cb
751 Process some outstanding events on the result pipe. You have to call 837 Process some outstanding events on the result pipe. You have to call
752 this regularly. Returns the number of events processed. Returns 838 this regularly. Returns 0 if all events could be processed, or -1 if
753 immediately when no events are outstanding. The amount of events 839 it returned earlier for whatever reason. Returns immediately when no
840 events are outstanding. The amount of events processed depends on
754 processed depends on the settings of "IO::AIO::max_poll_req" and 841 the settings of "IO::AIO::max_poll_req" and
755 "IO::AIO::max_poll_time". 842 "IO::AIO::max_poll_time".
756 843
757 If not all requests were processed for whatever reason, the 844 If not all requests were processed for whatever reason, the
758 filehandle will still be ready when "poll_cb" returns. 845 filehandle will still be ready when "poll_cb" returns, so normally
846 you don't have to do anything special to have it called later.
759 847
760 Example: Install an Event watcher that automatically calls 848 Example: Install an Event watcher that automatically calls
761 IO::AIO::poll_cb with high priority: 849 IO::AIO::poll_cb with high priority:
762 850
763 Event->io (fd => IO::AIO::poll_fileno, 851 Event->io (fd => IO::AIO::poll_fileno,

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines