--- IO-AIO/README 2007/01/06 02:47:11 1.22 +++ IO-AIO/README 2008/03/30 06:31:49 1.28 @@ -26,10 +26,13 @@ my $grp = aio_group sub { print "all stats done\n" }; add $grp aio_stat "..." for ...; - # AnyEvent integration + # AnyEvent integration (EV, Event, Glib, Tk, urxvt, pureperl...) open my $fh, "<&=" . IO::AIO::poll_fileno or die "$!"; my $w = AnyEvent->io (fh => $fh, poll => 'r', cb => sub { IO::AIO::poll_cb }); + # EV integration + my $w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb; + # Event integration Event->io (fd => IO::AIO::poll_fileno, poll => 'r', @@ -63,10 +66,9 @@ While most of this works on all types of file descriptors (for example sockets), using these functions on file descriptors that support - nonblocking operation (again, sockets, pipes etc.) is very inefficient - or might not work (aio_read fails on sockets/pipes/fifos). Use an event - loop for that (such as the Event module): IO::AIO will naturally fit - into such an event loop itself. + nonblocking operation (again, sockets, pipes etc.) is very inefficient. + Use an event loop for that (such as the Event module): IO::AIO will + naturally fit into such an event loop itself. In this version, a number of threads are started that execute your requests and signal their completion. You don't need thread support in @@ -78,10 +80,10 @@ aio_write, so the remaining functionality would have to be implemented using threads anyway. - Although the module will work with in the presence of other (Perl-) - threads, it is currently not reentrant in any way, so use appropriate - locking yourself, always call "poll_cb" from within the same thread, or - never call "poll_cb" (or other "aio_" functions) recursively. + Although the module will work in the presence of other (Perl-) threads, + it is currently not reentrant in any way, so use appropriate locking + yourself, always call "poll_cb" from within the same thread, or never + call "poll_cb" (or other "aio_" functions) recursively. EXAMPLE This is a simple example that uses the Event module and loads @@ -244,7 +246,9 @@ Likewise, $mode specifies the mode of the newly created file, if it didn't exist and "O_CREAT" has been given, just like perl's "sysopen", except that it is mandatory (i.e. use 0 if you don't - create new files, and 0666 or 0777 if you do). + create new files, and 0666 or 0777 if you do). Note that the $mode + will be modified by the umask in effect then the request is being + executed, so better never change the umask. Example: @@ -259,24 +263,59 @@ aio_close $fh, $callback->($status) Asynchronously close a file and call the callback with the result - code. *WARNING:* although accepted, you should not pass in a perl - filehandle here, as perl will likely close the file descriptor - another time when the filehandle is destroyed. Normally, you can - safely call perls "close" or just let filehandles go out of scope. + code. + + Unfortunately, you can't do this to perl. Perl *insists* very + strongly on closing the file descriptor associated with the + filehandle itself. Here is what aio_close will try: + + 1. dup()licate the fd + 2. asynchronously close() the duplicated fd + 3. dup()licate the fd once more + 4. let perl close() the filehandle + 5. asynchronously close the duplicated fd + + The idea is that the first close() flushes stuff to disk that + closing an fd will flush, so when perl closes the fd, nothing much + will need to be flushed. The second async. close() will then flush + stuff to disk that closing the last fd to the file will flush. + + Just FYI, SuSv3 has this to say on close: + + All outstanding record locks owned by the process on the file + associated with the file descriptor shall be removed. + + If fildes refers to a socket, close() shall cause the socket to be + destroyed. ... close() shall block for up to the current linger + interval until all data is transmitted. + [this actually sounds like a specification bug, but who knows] - This is supposed to be a bug in the API, so that might change. It's - therefore best to avoid this function. + And at least Linux additionally actually flushes stuff on every + close, even when the file itself is still open. + + Sounds enourmously inefficient and complicated? Yes... please show + me how to nuke perl's fd out of existence... aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) - Reads or writes "length" bytes from the specified "fh" and "offset" - into the scalar given by "data" and offset "dataoffset" and calls - the callback without the actual number of bytes read (or -1 on - error, just like the syscall). + Reads or writes $length bytes from the specified $fh and $offset + into the scalar given by $data and offset $dataoffset and calls the + callback without the actual number of bytes read (or -1 on error, + just like the syscall). + + If $offset is undefined, then the current file descriptor offset + will be used (and updated), otherwise the file descriptor offset + will not be changed by these calls. + + If $length is undefined in "aio_write", use the remaining length of + $data. + + If $dataoffset is less than zero, it will be counted from the end of + $data. The $data scalar *MUST NOT* be modified in any way while the request - is outstanding. Modifying it can result in segfaults or WW3 (if the - necessary/optional hardware is installed). + is outstanding. Modifying it can result in segfaults or World War + III (if the necessary/optional hardware is installed). Example: Read 15 bytes at offset 7 into scalar $buffer, starting at offset 0 within the scalar: @@ -344,6 +383,40 @@ print "size is ", -s _, "\n"; }; + aio_utime $fh_or_path, $atime, $mtime, $callback->($status) + Works like perl's "utime" function (including the special case of + $atime and $mtime being undef). Fractional times are supported if + the underlying syscalls support them. + + When called with a pathname, uses utimes(2) if available, otherwise + utime(2). If called on a file descriptor, uses futimes(2) if + available, otherwise returns ENOSYS, so this is not portable. + + Examples: + + # set atime and mtime to current time (basically touch(1)): + aio_utime "path", undef, undef; + # set atime to current time and mtime to beginning of the epoch: + aio_utime "path", time, undef; # undef==0 + + aio_chown $fh_or_path, $uid, $gid, $callback->($status) + Works like perl's "chown" function, except that "undef" for either + $uid or $gid is being interpreted as "do not change" (but -1 can + also be used). + + Examples: + + # same as "chown root path" in the shell: + aio_chown "path", 0, -1; + # same as above: + aio_chown "path", 0, undef; + + aio_truncate $fh_or_path, $offset, $callback->($status) + Works like truncate(2) or ftruncate(2). + + aio_chmod $fh_or_path, $mode, $callback->($status) + Works like perl's "chmod" function. + aio_unlink $pathname, $callback->($status) Asynchronously unlink (delete) a file and call the callback with the result code. @@ -375,6 +448,11 @@ Asynchronously rename the object at $srcpath to $dstpath, just as rename(2) and call the callback with the result code. + aio_mkdir $pathname, $mode, $callback->($status) + Asynchronously mkdir (create) a directory and call the callback with + the result code. $mode will be modified by the umask at the time the + request is executed, so do not change your umask. + aio_rmdir $pathname, $callback->($status) Asynchronously rmdir (delete) a directory and call the callback with the result code. @@ -467,6 +545,15 @@ efficiency as those tend to return 0 or 1 as link counts, which disables the directory counting heuristic. + aio_rmtree $path, $callback->($status) + Delete a directory tree starting (and including) $path, return the + status of the final "rmdir" only. This is a composite request that + uses "aio_scandir" to recurse into and rmdir directories, and unlink + everything else. + + aio_sync $callback->($status) + Asynchronously call sync and call the callback when finished. + aio_fsync $fh, $callback->($status) Asynchronously call fsync on the given filehandle and call the callback with the fsync result code. @@ -478,6 +565,16 @@ If this call isn't available because your OS lacks it or it couldn't be detected, it will be emulated by calling "fsync" instead. + aio_pathsync $path, $callback->($status) + This request tries to open, fsync and close the given path. This is + a composite request intended tosync directories after directory + operations (E.g. rename). This might not work on all operating + systems or have any specific effect, but usually it makes sure that + directory changes get written to disc. It works for anything that + can be opened for read-only, not just directories. + + Passes 0 when everything went ok, and -1 on error. + aio_group $callback->(...) This is a very special aio request: Instead of doing something, it is a container for other aio requests, which is useful if you want @@ -568,13 +665,16 @@ This makes it very easy to create composite requests (see the source of "aio_move" for an application) that work and feel like simple requests. - * The IO::AIO::GRP objects will be cleaned up during calls to - "IO::AIO::poll_cb", just like any other request. - * They can be canceled like any other request. Canceling will cancel not - only the request itself, but also all requests it contains. - * They can also can also be added to other IO::AIO::GRP objects. - * You must not add requests to a group from within the group callback - (or any later time). + * The IO::AIO::GRP objects will be cleaned up during calls to + "IO::AIO::poll_cb", just like any other request. + + * They can be canceled like any other request. Canceling will cancel + not only the request itself, but also all requests it contains. + + * They can also can also be added to other IO::AIO::GRP objects. + + * You must not add requests to a group from within the group callback + (or any later time). Their lifetime, simplified, looks like this: when they are empty, they will finish very quickly. If they contain only requests that are in the @@ -601,7 +701,7 @@ $grp->result (...) Set the result value(s) that will be passed to the group callback - when all subrequests have finished and set thre groups errno to the + when all subrequests have finished and set the groups errno to the current value of errno (just like calling "errno" without an error number). By default, no argument will be passed and errno is zero. @@ -797,7 +897,7 @@ it blocks, and a bad way to reduce concurrency because it is inexact: Better use an "aio_group" together with a feed callback. - Sets the maximum number of outstanding requests to $nreqs. If you to + Sets the maximum number of outstanding requests to $nreqs. If you do queue up more than this number of requests, the next call to the "poll_cb" (and "poll_some" and other functions calling "poll_cb") function will block until the limit is no longer exceeded. @@ -853,7 +953,7 @@ scalars and other data passed into aio requests will also be locked and will consume memory till the request has entered the done state. - This is now awfully much, so queuing lots of requests is not usually a + This is not awfully much, so queuing lots of requests is not usually a problem. Per-thread usage: