--- IO-AIO/README 2005/07/11 03:29:39 1.6 +++ IO-AIO/README 2005/08/23 00:05:00 1.12 @@ -22,7 +22,7 @@ # Glib/Gtk2 add_watch Glib::IO IO::AIO::poll_fileno, - in => sub { IO::AIO::poll_cb, 1 }; + in => sub { IO::AIO::poll_cb; 1 }; # Tk Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "", @@ -47,7 +47,9 @@ using threads anyway. Although the module will work with in the presence of other threads, it - is currently not reentrant, so use appropriate locking yourself. + is currently not reentrant, so use appropriate locking yourself, always + call "poll_cb" from within the same thread, or never call "poll_cb" (or + other "aio_" functions) recursively. FUNCTIONS AIO FUNCTIONS @@ -59,13 +61,21 @@ error, unlike perl, which usually delivers "false") as it's sole argument when the given syscall has been executed asynchronously. - All functions that expect a filehandle will also accept a file - descriptor. + All functions expecting a filehandle keep a copy of the filehandle + internally until the request has finished. - The filenames you pass to these routines *must* be absolute. The reason - is that at the time the request is being executed, the current working - directory could have changed. Alternatively, you can make sure that you - never change the current working directory. + The pathnames you pass to these routines *must* be absolute and encoded + in byte form. The reason for the former is that at the time the request + is being executed, the current working directory could have changed. + Alternatively, you can make sure that you never change the current + working directory. + + To encode pathnames to byte form, either make sure you either: a) always + pass in filenames you got from outside (command line, readdir etc.), b) + are ASCII or ISO 8859-1, c) use the Encode module and encode your + pathnames to the locale (or other) encoding in effect in the user + environment, d) use Glib::filename_from_unicode on unicode filenames or + e) use something else. aio_open $pathname, $flags, $mode, $callback Asynchronously open or create a file and call the callback with a @@ -74,8 +84,13 @@ The pathname passed to "aio_open" must be absolute. See API NOTES, above, for an explanation. - The $mode argument is a bitmask. See the "Fcntl" module for a list. - They are the same as used in "sysopen". + The $flags argument is a bitmask. See the "Fcntl" module for a list. + They are the same as used by "sysopen". + + Likewise, $mode specifies the mode of the newly created file, if it + didn't exist and "O_CREAT" has been given, just like perl's + "sysopen", except that it is mandatory (i.e. use 0 if you don't + create new files, and 0666 or 0777 if you do). Example: @@ -92,8 +107,11 @@ Asynchronously close a file and call the callback with the result code. *WARNING:* although accepted, you should not pass in a perl filehandle here, as perl will likely close the file descriptor - itself when the filehandle is destroyed. Normally, you can safely - call perls "close" or just let filehandles go out of scope. + another time when the filehandle is destroyed. Normally, you can + safely call perls "close" or just let filehandles go out of scope. + + This is supposed to be a bug in the API, so that might change. It's + therefore best to avoid this function. aio_read $fh,$offset,$length, $data,$dataoffset,$callback aio_write $fh,$offset,$length, $data,$dataoffset,$callback @@ -102,6 +120,10 @@ the callback without the actual number of bytes read (or -1 on error, just like the syscall). + The $data scalar *MUST NOT* be modified in any way while the request + is outstanding. Modifying it can result in segfaults or WW3 (if the + necessary/optional hardware is installed). + Example: Read 15 bytes at offset 7 into scalar $buffer, starting at offset 0 within the scalar: @@ -110,22 +132,43 @@ print "read $_[0] bytes: <$buffer>\n"; }; - aio_readahead $fh,$offset,$length, $callback - Asynchronously reads the specified byte range into the page cache, - using the "readahead" syscall. If that syscall doesn't exist (likely - if your OS isn't Linux) the status will be -1 and $! is set to - ENOSYS. + aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback + Tries to copy $length bytes from $in_fh to $out_fh. It starts + reading at byte offset $in_offset, and starts writing at the current + file offset of $out_fh. Because of that, it is not safe to issue + more than one "aio_sendfile" per $out_fh, as they will interfere + with each other. + + This call tries to make use of a native "sendfile" syscall to + provide zero-copy operation. For this to work, $out_fh should refer + to a socket, and $in_fh should refer to mmap'able file. + + If the native sendfile call fails or is not implemented, it will be + emulated, so you can call "aio_sendfile" on any type of filehandle + regardless of the limitations of the operating system. + + Please note, however, that "aio_sendfile" can read more bytes from + $in_fh than are written, and there is no way to find out how many + bytes have been read from "aio_sendfile" alone, as "aio_sendfile" + only provides the number of bytes written to $out_fh. Only if the + result value equals $length one can assume that $length bytes have + been read. - readahead() populates the page cache with data from a file so that - subsequent reads from that file will not block on disk I/O. The + aio_readahead $fh,$offset,$length, $callback + "aio_readahead" populates the page cache with data from a file so + that subsequent reads from that file will not block on disk I/O. The $offset argument specifies the starting point from which data is to be read and $length specifies the number of bytes to be read. I/O is performed in whole pages, so that offset is effectively rounded down to a page boundary and bytes are read up to the next page boundary - greater than or equal to (off-set+length). aio_readahead() does not + greater than or equal to (off-set+length). "aio_readahead" does not read beyond the end of the file. The current file offset of the file is left unchanged. + If that syscall doesn't exist (likely if your OS isn't Linux) it + will be emulated by simply reading the data, which would have a + similar effect. + aio_stat $fh_or_path, $callback aio_lstat $fh, $callback Works like perl's "stat" or "lstat" in void context. The callback @@ -151,6 +194,10 @@ Asynchronously unlink (delete) a file and call the callback with the result code. + aio_rmdir $pathname, $callback + Asynchronously rmdir (delete) a directory and call the callback with + the result code. + aio_fsync $fh, $callback Asynchronously call fsync on the given filehandle and call the callback with the fsync result code. @@ -159,12 +206,15 @@ Asynchronously call fdatasync on the given filehandle and call the callback with the fdatasync result code. + If this call isn't available because your OS lacks it or it couldn't + be detected, it will be emulated by calling "fsync" instead. + SUPPORT FUNCTIONS $fileno = IO::AIO::poll_fileno - Return the *request result pipe filehandle*. This filehandle must be - polled for reading by some mechanism outside this module (e.g. Event - or select, see below). If the pipe becomes readable you have to call - "poll_cb" to check the results. + Return the *request result pipe file descriptor*. This filehandle + must be polled for reading by some mechanism outside this module + (e.g. Event or select, see below or the SYNOPSIS). If the pipe + becomes readable you have to call "poll_cb" to check the results. See "poll_cb" for an example. @@ -173,7 +223,8 @@ this regularly. Returns the number of events processed. Returns immediately when no events are outstanding. - You can use Event to multiplex, e.g.: + Example: Install an Event watcher that automatically calls + IO::AIO::poll_cb with high priority: Event->io (fd => IO::AIO::poll_fileno, poll => 'r', async => 1, @@ -181,13 +232,14 @@ IO::AIO::poll_wait Wait till the result filehandle becomes ready for reading (simply - does a select on the filehandle. This is useful if you want to + does a "select" on the filehandle. This is useful if you want to synchronously wait for some requests to finish). See "nreqs" for an example. IO::AIO::nreqs - Returns the number of requests currently outstanding. + Returns the number of requests currently outstanding (i.e. for which + their callback has not been invoked yet). Example: wait till there are no outstanding requests anymore: @@ -211,23 +263,30 @@ if IO::AIO::nreqs; IO::AIO::min_parallel $nthreads - Set the minimum number of AIO threads to $nthreads. The default is - 1, which means a single asynchronous operation can be done at one - time (the number of outstanding operations, however, is unlimited). + Set the minimum number of AIO threads to $nthreads. The current + default is 4, which means four asynchronous operations can be done + at one time (the number of outstanding operations, however, is + unlimited). + + IO::AIO starts threads only on demand, when an AIO request is queued + and no free thread exists. It is recommended to keep the number of threads low, as some Linux kernel versions will scale negatively with the number of threads (higher parallelity => MUCH higher latency). With current Linux 2.6 versions, 4-32 threads should be fine. - Under normal circumstances you don't need to call this function, as - this module automatically starts some threads (the exact number - might change, and is currently 4). + Under most circumstances you don't need to call this function, as + the module selects a default that is suitable for low to moderate + load. IO::AIO::max_parallel $nthreads Sets the maximum number of AIO threads to $nthreads. If more than - the specified number of threads are currently running, kill them. - This function blocks until the limit is reached. + the specified number of threads are currently running, this function + kills them. This function blocks until the limit is reached. + + While $nthreads are zero, aio requests get queued but not executed + until the number of threads has been increased again. This module automatically runs "max_parallel 0" at program end, to ensure that all threads are killed and that there are no outstanding @@ -241,11 +300,20 @@ block until some requests have been handled. The default is very large, so normally there is no practical limit. - If you queue up many requests in a loop it it often improves speed - if you set this to a relatively low number, such as 100. + If you queue up many requests in a loop it often improves speed if + you set this to a relatively low number, such as 100. Under normal circumstances you don't need to call this function. + FORK BEHAVIOUR + Before the fork, IO::AIO enters a quiescent state where no requests can + be added in other threads and no results will be processed. After the + fork the parent simply leaves the quiescent state and continues + request/result processing, while the child clears the request/result + queue (so the requests started before the fork will only be handled in + the parent). Threats will be started on demand until the limit ste in + the parent process has been reached again. + SEE ALSO Coro, Linux::AIO.