--- IO-AIO/README 2005/07/10 17:07:44 1.1 +++ IO-AIO/README 2005/07/11 03:29:39 1.6 @@ -4,27 +4,60 @@ SYNOPSIS use IO::AIO; + aio_open "/etc/passwd", O_RDONLY, 0, sub { + my ($fh) = @_; + ... + }; + + aio_unlink "/tmp/file", sub { }; + + aio_read $fh, 30000, 1024, $buffer, 0, sub { + $_[0] > 0 or die "read error: $!"; + }; + + # Event + Event->io (fd => IO::AIO::poll_fileno, + poll => 'r', + cb => \&IO::AIO::poll_cb); + + # Glib/Gtk2 + add_watch Glib::IO IO::AIO::poll_fileno, + in => sub { IO::AIO::poll_cb, 1 }; + + # Tk + Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "", + readable => \&IO::AIO::poll_cb); + + # Danga::Socket + Danga::Socket->AddOtherFds (IO::AIO::poll_fileno => + \&IO::AIO::poll_cb); + DESCRIPTION This module implements asynchronous I/O using whatever means your - operating system supports. Currently, it falls back to Linux::AIO if - that module is available, or uses pthreads to emulato aio functionality. + operating system supports. - Currently, in this module a number of threads are started that execute - your read/writes and signal their completion. You don't need thread - support in your libc or perl, and the threads created by this module - will not be visible to the pthreads library. + Currently, a number of threads are started that execute your read/writes + and signal their completion. You don't need thread support in your libc + or perl, and the threads created by this module will not be visible to + the pthreads library. In the future, this module might make use of the + native aio functions available on many operating systems. However, they + are often not well-supported (Linux doesn't allow them on normal files + currently, for example), and they would only support aio_read and + aio_write, so the remaining functionality would have to be implemented + using threads anyway. Although the module will work with in the presence of other threads, it - is not reentrant, so use appropriate locking yourself. + is currently not reentrant, so use appropriate locking yourself. - API NOTES +FUNCTIONS + AIO FUNCTIONS All the "aio_*" calls are more or less thin wrappers around the syscall with the same name (sans "aio_"). The arguments are similar or - identical, and they all accept an additional $callback argument which - must be a code reference. This code reference will get called with the - syscall return code (e.g. most syscalls return -1 on error, unlike perl, - which usually delivers "false") as it's sole argument when the given - syscall has been executed asynchronously. + identical, and they all accept an additional (and optional) $callback + argument which must be a code reference. This code reference will get + called with the syscall return code (e.g. most syscalls return -1 on + error, unlike perl, which usually delivers "false") as it's sole + argument when the given syscall has been executed asynchronously. All functions that expect a filehandle will also accept a file descriptor. @@ -34,67 +67,12 @@ directory could have changed. Alternatively, you can make sure that you never change the current working directory. - IO::AIO::min_parallel $nthreads - Set the minimum number of AIO threads to $nthreads. The default is - 1, which means a single asynchronous operation can be done at one - time (the number of outstanding operations, however, is unlimited). - - It is recommended to keep the number of threads low, as some linux - kernel versions will scale negatively with the number of threads - (higher parallelity => MUCH higher latency). - - Under normal circumstances you don't need to call this function, as - this module automatically starts a single async thread. - - IO::AIO::max_parallel $nthreads - Sets the maximum number of AIO threads to $nthreads. If more than - the specified number of threads are currently running, kill them. - This function blocks until the limit is reached. - - This module automatically runs "max_parallel 0" at program end, to - ensure that all threads are killed and that there are no outstanding - requests. - - Under normal circumstances you don't need to call this function. - - $fileno = IO::AIO::poll_fileno - Return the *request result pipe filehandle*. This filehandle must be - polled for reading by some mechanism outside this module (e.g. Event - or select, see below). If the pipe becomes readable you have to call - "poll_cb" to check the results. - - See "poll_cb" for an example. - - IO::AIO::poll_cb - Process all outstanding events on the result pipe. You have to call - this regularly. Returns the number of events processed. Returns - immediately when no events are outstanding. - - You can use Event to multiplex, e.g.: - - Event->io (fd => IO::AIO::poll_fileno, - poll => 'r', async => 1, - cb => \&IO::AIO::poll_cb); - - IO::AIO::poll_wait - Wait till the result filehandle becomes ready for reading (simply - does a select on the filehandle. This is useful if you want to - synchronously wait for some requests to finish). - - See "nreqs" for an example. - - IO::AIO::nreqs - Returns the number of requests currently outstanding. - - Example: wait till there are no outstanding requests anymore: - - IO::AIO::poll_wait, IO::AIO::poll_cb - while IO::AIO::nreqs; - aio_open $pathname, $flags, $mode, $callback - Asynchronously open or create a file and call the callback with the - filedescriptor (NOT a perl filehandle, sorry for that, but watch - out, this might change in the future). + Asynchronously open or create a file and call the callback with a + newly created filehandle for the file. + + The pathname passed to "aio_open" must be absolute. See API NOTES, + above, for an explanation. The $mode argument is a bitmask. See the "Fcntl" module for a list. They are the same as used in "sysopen". @@ -102,10 +80,8 @@ Example: aio_open "/etc/passwd", O_RDONLY, 0, sub { - if ($_[0] >= 0) { - open my $fh, "<&$_[0]"; # create a copy for perl - aio_close $_[0], sub { }; # close the aio handle - print "open successful, fh is $fh\n"; + if ($_[0]) { + print "open successful, fh is $_[0]\n"; ... } else { die "open failed: $!\n"; @@ -114,7 +90,10 @@ aio_close $fh, $callback Asynchronously close a file and call the callback with the result - code. + code. *WARNING:* although accepted, you should not pass in a perl + filehandle here, as perl will likely close the file descriptor + itself when the filehandle is destroyed. Normally, you can safely + call perls "close" or just let filehandles go out of scope. aio_read $fh,$offset,$length, $data,$dataoffset,$callback aio_write $fh,$offset,$length, $data,$dataoffset,$callback @@ -123,17 +102,19 @@ the callback without the actual number of bytes read (or -1 on error, just like the syscall). - Example: Read 15 bytes at offset 7 into scalar $buffer, strating at + Example: Read 15 bytes at offset 7 into scalar $buffer, starting at offset 0 within the scalar: aio_read $fh, 7, 15, $buffer, 0, sub { - $_[0] >= 0 or die "read error: $!"; - print "read <$buffer>\n"; + $_[0] > 0 or die "read error: $!"; + print "read $_[0] bytes: <$buffer>\n"; }; aio_readahead $fh,$offset,$length, $callback Asynchronously reads the specified byte range into the page cache, - using the "readahead" syscall. + using the "readahead" syscall. If that syscall doesn't exist (likely + if your OS isn't Linux) the status will be -1 and $! is set to + ENOSYS. readahead() populates the page cache with data from a file so that subsequent reads from that file will not block on disk I/O. The @@ -178,14 +159,95 @@ Asynchronously call fdatasync on the given filehandle and call the callback with the fdatasync result code. -BUGS - This module has been extensively tested in a large and very busy - webserver for many years now. + SUPPORT FUNCTIONS + $fileno = IO::AIO::poll_fileno + Return the *request result pipe filehandle*. This filehandle must be + polled for reading by some mechanism outside this module (e.g. Event + or select, see below). If the pipe becomes readable you have to call + "poll_cb" to check the results. + + See "poll_cb" for an example. + + IO::AIO::poll_cb + Process all outstanding events on the result pipe. You have to call + this regularly. Returns the number of events processed. Returns + immediately when no events are outstanding. + + You can use Event to multiplex, e.g.: + + Event->io (fd => IO::AIO::poll_fileno, + poll => 'r', async => 1, + cb => \&IO::AIO::poll_cb); + + IO::AIO::poll_wait + Wait till the result filehandle becomes ready for reading (simply + does a select on the filehandle. This is useful if you want to + synchronously wait for some requests to finish). + + See "nreqs" for an example. + + IO::AIO::nreqs + Returns the number of requests currently outstanding. + + Example: wait till there are no outstanding requests anymore: + + IO::AIO::poll_wait, IO::AIO::poll_cb + while IO::AIO::nreqs; + + IO::AIO::flush + Wait till all outstanding AIO requests have been handled. - - aio_open gives a fd, but all other functions expect a perl filehandle. + Strictly equivalent to: + + IO::AIO::poll_wait, IO::AIO::poll_cb + while IO::AIO::nreqs; + + IO::AIO::poll + Waits until some requests have been handled. + + Strictly equivalent to: + + IO::AIO::poll_wait, IO::AIO::poll_cb + if IO::AIO::nreqs; + + IO::AIO::min_parallel $nthreads + Set the minimum number of AIO threads to $nthreads. The default is + 1, which means a single asynchronous operation can be done at one + time (the number of outstanding operations, however, is unlimited). + + It is recommended to keep the number of threads low, as some Linux + kernel versions will scale negatively with the number of threads + (higher parallelity => MUCH higher latency). With current Linux 2.6 + versions, 4-32 threads should be fine. + + Under normal circumstances you don't need to call this function, as + this module automatically starts some threads (the exact number + might change, and is currently 4). + + IO::AIO::max_parallel $nthreads + Sets the maximum number of AIO threads to $nthreads. If more than + the specified number of threads are currently running, kill them. + This function blocks until the limit is reached. + + This module automatically runs "max_parallel 0" at program end, to + ensure that all threads are killed and that there are no outstanding + requests. + + Under normal circumstances you don't need to call this function. + + $oldnreqs = IO::AIO::max_outstanding $nreqs + Sets the maximum number of outstanding requests to $nreqs. If you + try to queue up more than this number of requests, the caller will + block until some requests have been handled. + + The default is very large, so normally there is no practical limit. + If you queue up many requests in a loop it it often improves speed + if you set this to a relatively low number, such as 100. + + Under normal circumstances you don't need to call this function. SEE ALSO - Coro. + Coro, Linux::AIO. AUTHOR Marc Lehmann