ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.22 by root, Wed Jul 20 21:55:27 2005 UTC vs.
Revision 1.39 by root, Sun Aug 28 11:05:50 2005 UTC

56 56
57=cut 57=cut
58 58
59package IO::AIO; 59package IO::AIO;
60 60
61no warnings;
62
61use base 'Exporter'; 63use base 'Exporter';
62 64
63use Fcntl (); 65use Fcntl ();
64 66
65BEGIN { 67BEGIN {
66 $VERSION = 0.9; 68 $VERSION = 1.6;
67 69
68 @EXPORT = qw(aio_read aio_write aio_open aio_close aio_stat aio_lstat aio_unlink 70 @EXPORT = qw(aio_sendfile aio_read aio_write aio_open aio_close aio_stat
71 aio_lstat aio_unlink aio_rmdir aio_readdir aio_symlink
69 aio_fsync aio_fdatasync aio_readahead); 72 aio_fsync aio_fdatasync aio_readahead);
70 @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel max_outstanding nreqs); 73 @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel
74 max_outstanding nreqs);
71 75
72 require XSLoader; 76 require XSLoader;
73 XSLoader::load IO::AIO, $VERSION; 77 XSLoader::load IO::AIO, $VERSION;
74} 78}
75 79
83which must be a code reference. This code reference will get called with 87which must be a code reference. This code reference will get called with
84the syscall return code (e.g. most syscalls return C<-1> on error, unlike 88the syscall return code (e.g. most syscalls return C<-1> on error, unlike
85perl, which usually delivers "false") as it's sole argument when the given 89perl, which usually delivers "false") as it's sole argument when the given
86syscall has been executed asynchronously. 90syscall has been executed asynchronously.
87 91
88All functions that expect a filehandle will also accept a file descriptor. 92All functions expecting a filehandle keep a copy of the filehandle
93internally until the request has finished.
89 94
90The filenames you pass to these routines I<must> be absolute. The reason 95The pathnames you pass to these routines I<must> be absolute and
91for this is that at the time the request is being executed, the current 96encoded in byte form. The reason for the former is that at the time the
92working directory could have changed. Alternatively, you can make sure 97request is being executed, the current working directory could have
93that you never change the current working directory. 98changed. Alternatively, you can make sure that you never change the
99current working directory.
100
101To encode pathnames to byte form, either make sure you either: a)
102always pass in filenames you got from outside (command line, readdir
103etc.), b) are ASCII or ISO 8859-1, c) use the Encode module and encode
104your pathnames to the locale (or other) encoding in effect in the user
105environment, d) use Glib::filename_from_unicode on unicode filenames or e)
106use something else.
94 107
95=over 4 108=over 4
96 109
97=item aio_open $pathname, $flags, $mode, $callback 110=item aio_open $pathname, $flags, $mode, $callback
98 111
139Reads or writes C<length> bytes from the specified C<fh> and C<offset> 152Reads or writes C<length> bytes from the specified C<fh> and C<offset>
140into the scalar given by C<data> and offset C<dataoffset> and calls the 153into the scalar given by C<data> and offset C<dataoffset> and calls the
141callback without the actual number of bytes read (or -1 on error, just 154callback without the actual number of bytes read (or -1 on error, just
142like the syscall). 155like the syscall).
143 156
157The C<$data> scalar I<MUST NOT> be modified in any way while the request
158is outstanding. Modifying it can result in segfaults or WW3 (if the
159necessary/optional hardware is installed).
160
144Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at 161Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at
145offset C<0> within the scalar: 162offset C<0> within the scalar:
146 163
147 aio_read $fh, 7, 15, $buffer, 0, sub { 164 aio_read $fh, 7, 15, $buffer, 0, sub {
148 $_[0] > 0 or die "read error: $!"; 165 $_[0] > 0 or die "read error: $!";
149 print "read $_[0] bytes: <$buffer>\n"; 166 print "read $_[0] bytes: <$buffer>\n";
150 }; 167 };
151 168
169=item aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback
170
171Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
172reading at byte offset C<$in_offset>, and starts writing at the current
173file offset of C<$out_fh>. Because of that, it is not safe to issue more
174than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each
175other.
176
177This call tries to make use of a native C<sendfile> syscall to provide
178zero-copy operation. For this to work, C<$out_fh> should refer to a
179socket, and C<$in_fh> should refer to mmap'able file.
180
181If the native sendfile call fails or is not implemented, it will be
182emulated, so you can call C<aio_sendfile> on any type of filehandle
183regardless of the limitations of the operating system.
184
185Please note, however, that C<aio_sendfile> can read more bytes from
186C<$in_fh> than are written, and there is no way to find out how many
187bytes have been read from C<aio_sendfile> alone, as C<aio_sendfile> only
188provides the number of bytes written to C<$out_fh>. Only if the result
189value equals C<$length> one can assume that C<$length> bytes have been
190read.
191
152=item aio_readahead $fh,$offset,$length, $callback 192=item aio_readahead $fh,$offset,$length, $callback
153
154Asynchronously reads the specified byte range into the page cache, using
155the C<readahead> syscall. If that syscall doesn't exist (likely if your OS
156isn't Linux) the status will be C<-1> and C<$!> is set to C<ENOSYS>.
157 193
158C<aio_readahead> populates the page cache with data from a file so that 194C<aio_readahead> populates the page cache with data from a file so that
159subsequent reads from that file will not block on disk I/O. The C<$offset> 195subsequent reads from that file will not block on disk I/O. The C<$offset>
160argument specifies the starting point from which data is to be read and 196argument specifies the starting point from which data is to be read and
161C<$length> specifies the number of bytes to be read. I/O is performed in 197C<$length> specifies the number of bytes to be read. I/O is performed in
162whole pages, so that offset is effectively rounded down to a page boundary 198whole pages, so that offset is effectively rounded down to a page boundary
163and bytes are read up to the next page boundary greater than or equal to 199and bytes are read up to the next page boundary greater than or equal to
164(off-set+length). C<aio_readahead> does not read beyond the end of the 200(off-set+length). C<aio_readahead> does not read beyond the end of the
165file. The current file offset of the file is left unchanged. 201file. The current file offset of the file is left unchanged.
166 202
203If that syscall doesn't exist (likely if your OS isn't Linux) it will be
204emulated by simply reading the data, which would have a similar effect.
205
167=item aio_stat $fh_or_path, $callback 206=item aio_stat $fh_or_path, $callback
168 207
169=item aio_lstat $fh, $callback 208=item aio_lstat $fh, $callback
170 209
171Works like perl's C<stat> or C<lstat> in void context. The callback will 210Works like perl's C<stat> or C<lstat> in void context. The callback will
189=item aio_unlink $pathname, $callback 228=item aio_unlink $pathname, $callback
190 229
191Asynchronously unlink (delete) a file and call the callback with the 230Asynchronously unlink (delete) a file and call the callback with the
192result code. 231result code.
193 232
233=item aio_rmdir $pathname, $callback
234
235Asynchronously rmdir (delete) a directory and call the callback with the
236result code.
237
238=item aio_readdir $pathname $callback
239
240Unlike the POSIX call of the same name, C<aio_readdir> reads an entire
241directory (i.e. opendir + readdir + closedir). The entries will not be
242sorted, and will B<NOT> include the C<.> and C<..> entries.
243
244The callback a single argument which is either C<undef> or an array-ref
245with the filenames.
246
194=item aio_fsync $fh, $callback 247=item aio_fsync $fh, $callback
195 248
196Asynchronously call fsync on the given filehandle and call the callback 249Asynchronously call fsync on the given filehandle and call the callback
197with the fsync result code. 250with the fsync result code.
198 251
199=item aio_fdatasync $fh, $callback 252=item aio_fdatasync $fh, $callback
200 253
201Asynchronously call fdatasync on the given filehandle and call the 254Asynchronously call fdatasync on the given filehandle and call the
202callback with the fdatasync result code. Might set C<$!> to C<ENOSYS> if 255callback with the fdatasync result code.
203C<fdatasync> is not available. 256
257If this call isn't available because your OS lacks it or it couldn't be
258detected, it will be emulated by calling C<fsync> instead.
204 259
205=back 260=back
206 261
207=head2 SUPPORT FUNCTIONS 262=head2 SUPPORT FUNCTIONS
208 263
266 IO::AIO::poll_wait, IO::AIO::poll_cb 321 IO::AIO::poll_wait, IO::AIO::poll_cb
267 if IO::AIO::nreqs; 322 if IO::AIO::nreqs;
268 323
269=item IO::AIO::min_parallel $nthreads 324=item IO::AIO::min_parallel $nthreads
270 325
271Set the minimum number of AIO threads to C<$nthreads>. The default is 326Set the minimum number of AIO threads to C<$nthreads>. The current default
272C<1>, which means a single asynchronous operation can be done at one time 327is C<4>, which means four asynchronous operations can be done at one time
273(the number of outstanding operations, however, is unlimited). 328(the number of outstanding operations, however, is unlimited).
329
330IO::AIO starts threads only on demand, when an AIO request is queued and
331no free thread exists.
274 332
275It is recommended to keep the number of threads low, as some Linux 333It is recommended to keep the number of threads low, as some Linux
276kernel versions will scale negatively with the number of threads (higher 334kernel versions will scale negatively with the number of threads (higher
277parallelity => MUCH higher latency). With current Linux 2.6 versions, 4-32 335parallelity => MUCH higher latency). With current Linux 2.6 versions, 4-32
278threads should be fine. 336threads should be fine.
279 337
280Under normal circumstances you don't need to call this function, as this 338Under most circumstances you don't need to call this function, as the
281module automatically starts some threads (the exact number might change, 339module selects a default that is suitable for low to moderate load.
282and is currently 4).
283 340
284=item IO::AIO::max_parallel $nthreads 341=item IO::AIO::max_parallel $nthreads
285 342
286Sets the maximum number of AIO threads to C<$nthreads>. If more than 343Sets the maximum number of AIO threads to C<$nthreads>. If more than the
287the specified number of threads are currently running, kill them. This 344specified number of threads are currently running, this function kills
288function blocks until the limit is reached. 345them. This function blocks until the limit is reached.
346
347While C<$nthreads> are zero, aio requests get queued but not executed
348until the number of threads has been increased again.
289 349
290This module automatically runs C<max_parallel 0> at program end, to ensure 350This module automatically runs C<max_parallel 0> at program end, to ensure
291that all threads are killed and that there are no outstanding requests. 351that all threads are killed and that there are no outstanding requests.
292 352
293Under normal circumstances you don't need to call this function. 353Under normal circumstances you don't need to call this function.
297Sets the maximum number of outstanding requests to C<$nreqs>. If you 357Sets the maximum number of outstanding requests to C<$nreqs>. If you
298try to queue up more than this number of requests, the caller will block until 358try to queue up more than this number of requests, the caller will block until
299some requests have been handled. 359some requests have been handled.
300 360
301The default is very large, so normally there is no practical limit. If you 361The default is very large, so normally there is no practical limit. If you
302queue up many requests in a loop it it often improves speed if you set 362queue up many requests in a loop it often improves speed if you set
303this to a relatively low number, such as C<100>. 363this to a relatively low number, such as C<100>.
304 364
305Under normal circumstances you don't need to call this function. 365Under normal circumstances you don't need to call this function.
306 366
307=back 367=back
310 370
311# support function to convert a fd into a perl filehandle 371# support function to convert a fd into a perl filehandle
312sub _fd2fh { 372sub _fd2fh {
313 return undef if $_[0] < 0; 373 return undef if $_[0] < 0;
314 374
315 # try to be perl5.6-compatible 375 # try to generate nice filehandles
316 local *AIO_FH; 376 my $sym = "IO::AIO::fd#$_[0]";
317 open AIO_FH, "+<&=$_[0]" 377 local *$sym;
378
379 open *$sym, "+<&=$_[0]" # usually works under any unix
380 or open *$sym, "<&=$_[0]" # cygwin needs this
381 or open *$sym, ">&=$_[0]" # or this
318 or return undef; 382 or return undef;
319 383
320 *AIO_FH 384 *$sym
321} 385}
322 386
323min_parallel 4; 387min_parallel 4;
324 388
325END { 389END {
326 max_parallel 0; 390 max_parallel 0;
327} 391}
328 392
3291; 3931;
330 394
395=head2 FORK BEHAVIOUR
396
397Before the fork, IO::AIO enters a quiescent state where no requests
398can be added in other threads and no results will be processed. After
399the fork the parent simply leaves the quiescent state and continues
400request/result processing, while the child clears the request/result
401queue (so the requests started before the fork will only be handled in
402the parent). Threats will be started on demand until the limit ste in the
403parent process has been reached again.
404
331=head1 SEE ALSO 405=head1 SEE ALSO
332 406
333L<Coro>, L<Linux::AIO>. 407L<Coro>, L<Linux::AIO>.
334 408
335=head1 AUTHOR 409=head1 AUTHOR

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines