ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/README
(Generate patch)

Comparing IO-AIO/README (file contents):
Revision 1.47 by root, Fri May 27 00:44:49 2011 UTC vs.
Revision 1.49 by root, Mon Jul 18 03:09:06 2011 UTC

… …
168 aio_unlink $pathname, $callback->($status) 168 aio_unlink $pathname, $callback->($status)
169 aio_mknod $path, $mode, $dev, $callback->($status) 169 aio_mknod $path, $mode, $dev, $callback->($status)
170 aio_link $srcpath, $dstpath, $callback->($status) 170 aio_link $srcpath, $dstpath, $callback->($status)
171 aio_symlink $srcpath, $dstpath, $callback->($status) 171 aio_symlink $srcpath, $dstpath, $callback->($status)
172 aio_readlink $path, $callback->($link) 172 aio_readlink $path, $callback->($link)
173 aio_realpath $path, $callback->($link)
173 aio_rename $srcpath, $dstpath, $callback->($status) 174 aio_rename $srcpath, $dstpath, $callback->($status)
174 aio_mkdir $pathname, $mode, $callback->($status) 175 aio_mkdir $pathname, $mode, $callback->($status)
175 aio_rmdir $pathname, $callback->($status) 176 aio_rmdir $pathname, $callback->($status)
176 aio_readdir $pathname, $callback->($entries) 177 aio_readdir $pathname, $callback->($entries)
177 aio_readdirx $pathname, $flags, $callback->($entries, $flags) 178 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
… …
367 aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval) 368 aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
368 Tries to copy $length bytes from $in_fh to $out_fh. It starts 369 Tries to copy $length bytes from $in_fh to $out_fh. It starts
369 reading at byte offset $in_offset, and starts writing at the current 370 reading at byte offset $in_offset, and starts writing at the current
370 file offset of $out_fh. Because of that, it is not safe to issue 371 file offset of $out_fh. Because of that, it is not safe to issue
371 more than one "aio_sendfile" per $out_fh, as they will interfere 372 more than one "aio_sendfile" per $out_fh, as they will interfere
372 with each other. 373 with each other. The same $in_fh works fine though, as this function
374 does not move or use the file offset of $in_fh.
373 375
374 Please note that "aio_sendfile" can read more bytes from $in_fh than 376 Please note that "aio_sendfile" can read more bytes from $in_fh than
375 are written, and there is no way to find out how many bytes have 377 are written, and there is no way to find out how many more bytes
376 been read from "aio_sendfile" alone, as "aio_sendfile" only provides 378 have been read from "aio_sendfile" alone, as "aio_sendfile" only
377 the number of bytes written to $out_fh. Only if the result value 379 provides the number of bytes written to $out_fh. Only if the result
378 equals $length one can assume that $length bytes have been read. 380 value equals $length one can assume that $length bytes have been
381 read.
379 382
380 Unlike with other "aio_" functions, it makes a lot of sense to use 383 Unlike with other "aio_" functions, it makes a lot of sense to use
381 "aio_sendfile" on non-blocking sockets, as long as one end 384 "aio_sendfile" on non-blocking sockets, as long as one end
382 (typically the $in_fh) is a file - the file I/O will then be 385 (typically the $in_fh) is a file - the file I/O will then be
383 asynchronous, while the socket I/O will be non-blocking. Note, 386 asynchronous, while the socket I/O will be non-blocking. Note,
384 however, that you can run into a trap where "aio_sendfile" reads 387 however, that you can run into a trap where "aio_sendfile" reads
385 some data with readahead, then fails to write all data, and when the 388 some data with readahead, then fails to write all data, and when the
386 socket is ready the next time, the data in the cache is already 389 socket is ready the next time, the data in the cache is already
387 lost, forcing "aio_sendfile" to again hit the disk. Explicit 390 lost, forcing "aio_sendfile" to again hit the disk. Explicit
388 "aio_read" + "aio_write" let's you control resource usage much 391 "aio_read" + "aio_write" let's you better control resource usage.
389 better.
390 392
391 This call tries to make use of a native "sendfile" syscall to 393 This call tries to make use of a native "sendfile"-like syscall to
392 provide zero-copy operation. For this to work, $out_fh should refer 394 provide zero-copy operation. For this to work, $out_fh should refer
393 to a socket, and $in_fh should refer to an mmap'able file. 395 to a socket, and $in_fh should refer to an mmap'able file.
394 396
395 If a native sendfile cannot be found or it fails with "ENOSYS", 397 If a native sendfile cannot be found or it fails with "ENOSYS",
396 "ENOTSUP", "EOPNOTSUPP", "EAFNOSUPPORT", "EPROTOTYPE" or "ENOTSOCK", 398 "EINVAL", "ENOTSUP", "EOPNOTSUPP", "EAFNOSUPPORT", "EPROTOTYPE" or
397 it will be emulated, so you can call "aio_sendfile" on any type of 399 "ENOTSOCK", it will be emulated, so you can call "aio_sendfile" on
398 filehandle regardless of the limitations of the operating system. 400 any type of filehandle regardless of the limitations of the
401 operating system.
402
403 As native sendfile syscalls (as practically any non-POSIX interface
404 hacked together in a hurry to improve benchmark numbers) tend to be
405 rather buggy on many systems, this implementation tries to work
406 around some known bugs in Linux and FreeBSD kernels (probably
407 others, too), but that might fail, so you really really should check
408 the return value of "aio_sendfile" - fewre bytes than expected might
409 have been transferred.
399 410
400 aio_readahead $fh,$offset,$length, $callback->($retval) 411 aio_readahead $fh,$offset,$length, $callback->($retval)
401 "aio_readahead" populates the page cache with data from a file so 412 "aio_readahead" populates the page cache with data from a file so
402 that subsequent reads from that file will not block on disk I/O. The 413 that subsequent reads from that file will not block on disk I/O. The
403 $offset argument specifies the starting point from which data is to 414 $offset argument specifies the starting point from which data is to
… …
547 aio_readlink $path, $callback->($link) 558 aio_readlink $path, $callback->($link)
548 Asynchronously read the symlink specified by $path and pass it to 559 Asynchronously read the symlink specified by $path and pass it to
549 the callback. If an error occurs, nothing or undef gets passed to 560 the callback. If an error occurs, nothing or undef gets passed to
550 the callback. 561 the callback.
551 562
563 aio_realpath $path, $callback->($path)
564 Asynchronously make the path absolute and resolve any symlinks in
565 $path. The resulting path only consists of directories (Same as
566 Cwd::realpath).
567
568 This request can be used to get the absolute path of the current
569 working directory by passing it a path of . (a single dot).
570
552 aio_rename $srcpath, $dstpath, $callback->($status) 571 aio_rename $srcpath, $dstpath, $callback->($status)
553 Asynchronously rename the object at $srcpath to $dstpath, just as 572 Asynchronously rename the object at $srcpath to $dstpath, just as
554 rename(2) and call the callback with the result code. 573 rename(2) and call the callback with the result code.
555 574
556 aio_mkdir $pathname, $mode, $callback->($status) 575 aio_mkdir $pathname, $mode, $callback->($status)
… …
1170 IO::AIO::idle_timeout $seconds 1189 IO::AIO::idle_timeout $seconds
1171 Sets the minimum idle timeout (default 10) after which worker 1190 Sets the minimum idle timeout (default 10) after which worker
1172 threads are allowed to exit. SEe "IO::AIO::max_idle". 1191 threads are allowed to exit. SEe "IO::AIO::max_idle".
1173 1192
1174 IO::AIO::max_outstanding $maxreqs 1193 IO::AIO::max_outstanding $maxreqs
1194 Sets the maximum number of outstanding requests to $nreqs. If you do
1195 queue up more than this number of requests, the next call to
1196 "IO::AIO::poll_cb" (and other functions calling "poll_cb", such as
1197 "IO::AIO::flush" or "IO::AIO::poll") will block until the limit is
1198 no longer exceeded.
1199
1200 In other words, this setting does not enforce a queue limit, but can
1201 be used to make poll functions block if the limit is exceeded.
1202
1175 This is a very bad function to use in interactive programs because 1203 This is a very bad function to use in interactive programs because
1176 it blocks, and a bad way to reduce concurrency because it is 1204 it blocks, and a bad way to reduce concurrency because it is
1177 inexact: Better use an "aio_group" together with a feed callback. 1205 inexact: Better use an "aio_group" together with a feed callback.
1178 1206
1179 Sets the maximum number of outstanding requests to $nreqs. If you do 1207 It's main use is in scripts without an event loop - when you want to
1180 queue up more than this number of requests, the next call to the 1208 stat a lot of files, you can write somehting like this:
1181 "poll_cb" (and "poll_some" and other functions calling "poll_cb")
1182 function will block until the limit is no longer exceeded.
1183 1209
1184 The default value is very large, so there is no practical limit on 1210 IO::AIO::max_outstanding 32;
1211
1212 for my $path (...) {
1213 aio_stat $path , ...;
1214 IO::AIO::poll_cb;
1215 }
1216
1217 IO::AIO::flush;
1218
1219 The call to "poll_cb" inside the loop will normally return
1220 instantly, but as soon as more thna 32 reqeusts are in-flight, it
1221 will block until some requests have been handled. This keeps the
1222 loop from pushing a large number of "aio_stat" requests onto the
1223 queue.
1224
1225 The default value for "max_outstanding" is very large, so there is
1185 the number of outstanding requests. 1226 no practical limit on the number of outstanding requests.
1186
1187 You can still queue as many requests as you want. Therefore,
1188 "max_outstanding" is mainly useful in simple scripts (with low
1189 values) or as a stop gap to shield against fatal memory overflow
1190 (with large values).
1191 1227
1192 STATISTICAL INFORMATION 1228 STATISTICAL INFORMATION
1193 IO::AIO::nreqs 1229 IO::AIO::nreqs
1194 Returns the number of requests currently in the ready, execute or 1230 Returns the number of requests currently in the ready, execute or
1195 pending states (i.e. for which their callback has not been invoked 1231 pending states (i.e. for which their callback has not been invoked
… …
1341 # Danga::Socket integration 1377 # Danga::Socket integration
1342 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno => 1378 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
1343 \&IO::AIO::poll_cb); 1379 \&IO::AIO::poll_cb);
1344 1380
1345 FORK BEHAVIOUR 1381 FORK BEHAVIOUR
1346 This module should do "the right thing" when the process using it forks: 1382 Usage of pthreads in a program changes the semantics of fork
1383 considerably. Specifically, only async-safe functions can be called
1384 after fork. Perl doesn't know about this, so in general, you cannot call
1385 fork with defined behaviour in perl if pthreads are involved. IO::AIO
1386 uses pthreads, so this applies, but many other extensions and (for
1387 inexplicable reasons) perl itself often is linked against pthreads, so
1388 this limitation applies to quite a lot of perls.
1347 1389
1348 Before the fork, IO::AIO enters a quiescent state where no requests can 1390 This module no longer tries to fight your OS, or POSIX. That means
1349 be added in other threads and no results will be processed. After the 1391 IO::AIO only works in the process that loaded it. Forking is fully
1350 fork the parent simply leaves the quiescent state and continues 1392 supported, but using IO::AIO in the child is not.
1351 request/result processing, while the child frees the request/result
1352 queue (so that the requests started before the fork will only be handled
1353 in the parent). Threads will be started on demand until the limit set in
1354 the parent process has been reached again.
1355 1393
1356 In short: the parent will, after a short pause, continue as if fork had 1394 You might get around by not *using* IO::AIO before (or after) forking.
1357 not been called, while the child will act as if IO::AIO has not been 1395 You could also try to call the IO::AIO::reinit function in the child:
1358 used yet. 1396
1397 IO::AIO::reinit
1398 Abondons all current requests and I/O threads and simply
1399 reinitialises all data structures. This is not an operation
1400 suppported by any standards, but happens to work on GNU/Linux and
1401 some newer BSD systems.
1402
1403 The only reasonable use for this function is to call it after
1404 forking, if "IO::AIO" was used in the parent. Calling it while
1405 IO::AIO is active in the process will result in undefined behaviour.
1406 Calling it at any time will also result in any undefined (by POSIX)
1407 behaviour.
1359 1408
1360 MEMORY USAGE 1409 MEMORY USAGE
1361 Per-request usage: 1410 Per-request usage:
1362 1411
1363 Each aio request uses - depending on your architecture - around 100-200 1412 Each aio request uses - depending on your architecture - around 100-200

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines