ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
Revision: 1.241
Committed: Fri Dec 28 07:33:41 2012 UTC (11 years, 5 months ago) by root
Branch: MAIN
Changes since 1.240: +10 -0 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.1 =head1 NAME
2    
3     IO::AIO - Asynchronous Input/Output
4    
5     =head1 SYNOPSIS
6    
7     use IO::AIO;
8    
9 root 1.181 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
10 root 1.94 my $fh = shift
11     or die "/etc/passwd: $!";
12 root 1.6 ...
13     };
14    
15     aio_unlink "/tmp/file", sub { };
16    
17     aio_read $fh, 30000, 1024, $buffer, 0, sub {
18 root 1.8 $_[0] > 0 or die "read error: $!";
19 root 1.6 };
20    
21 root 1.56 # version 2+ has request and group objects
22     use IO::AIO 2;
23 root 1.52
24 root 1.68 aioreq_pri 4; # give next request a very high priority
25 root 1.52 my $req = aio_unlink "/tmp/file", sub { };
26     $req->cancel; # cancel request if still in queue
27    
28 root 1.56 my $grp = aio_group sub { print "all stats done\n" };
29     add $grp aio_stat "..." for ...;
30    
31 root 1.1 =head1 DESCRIPTION
32    
33     This module implements asynchronous I/O using whatever means your
34 root 1.156 operating system supports. It is implemented as an interface to C<libeio>
35     (L<http://software.schmorp.de/pkg/libeio.html>).
36 root 1.1
37 root 1.85 Asynchronous means that operations that can normally block your program
38     (e.g. reading from disk) will be done asynchronously: the operation
39     will still block, but you can do something else in the meantime. This
40     is extremely useful for programs that need to stay interactive even
41     when doing heavy I/O (GUI programs, high performance network servers
42     etc.), but can also be used to easily do operations in parallel that are
43     normally done sequentially, e.g. stat'ing many files, which is much faster
44     on a RAID volume or over NFS when you do a number of stat operations
45     concurrently.
46    
47 root 1.108 While most of this works on all types of file descriptors (for
48     example sockets), using these functions on file descriptors that
49 root 1.156 support nonblocking operation (again, sockets, pipes etc.) is
50     very inefficient. Use an event loop for that (such as the L<EV>
51 root 1.108 module): IO::AIO will naturally fit into such an event loop itself.
52 root 1.85
53 root 1.72 In this version, a number of threads are started that execute your
54     requests and signal their completion. You don't need thread support
55     in perl, and the threads created by this module will not be visible
56     to perl. In the future, this module might make use of the native aio
57     functions available on many operating systems. However, they are often
58 root 1.85 not well-supported or restricted (GNU/Linux doesn't allow them on normal
59 root 1.72 files currently, for example), and they would only support aio_read and
60     aio_write, so the remaining functionality would have to be implemented
61     using threads anyway.
62    
63 root 1.108 Although the module will work in the presence of other (Perl-) threads,
64     it is currently not reentrant in any way, so use appropriate locking
65     yourself, always call C<poll_cb> from within the same thread, or never
66     call C<poll_cb> (or other C<aio_> functions) recursively.
67 root 1.72
68 root 1.86 =head2 EXAMPLE
69    
70 root 1.156 This is a simple example that uses the EV module and loads
71 root 1.86 F</etc/passwd> asynchronously:
72    
73     use Fcntl;
74 root 1.156 use EV;
75 root 1.86 use IO::AIO;
76    
77 root 1.156 # register the IO::AIO callback with EV
78     my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
79 root 1.86
80     # queue the request to open /etc/passwd
81 root 1.181 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
82 root 1.94 my $fh = shift
83 root 1.86 or die "error while opening: $!";
84    
85     # stat'ing filehandles is generally non-blocking
86     my $size = -s $fh;
87    
88     # queue a request to read the file
89     my $contents;
90     aio_read $fh, 0, $size, $contents, 0, sub {
91     $_[0] == $size
92     or die "short read: $!";
93    
94     close $fh;
95    
96     # file contents now in $contents
97     print $contents;
98    
99     # exit event loop and program
100 root 1.156 EV::unloop;
101 root 1.86 };
102     };
103    
104     # possibly queue up other requests, or open GUI windows,
105     # check for sockets etc. etc.
106    
107     # process events as long as there are some:
108 root 1.156 EV::loop;
109 root 1.86
110 root 1.72 =head1 REQUEST ANATOMY AND LIFETIME
111    
112     Every C<aio_*> function creates a request. which is a C data structure not
113     directly visible to Perl.
114    
115     If called in non-void context, every request function returns a Perl
116     object representing the request. In void context, nothing is returned,
117     which saves a bit of memory.
118    
119     The perl object is a fairly standard ref-to-hash object. The hash contents
120     are not used by IO::AIO so you are free to store anything you like in it.
121    
122     During their existance, aio requests travel through the following states,
123     in order:
124    
125     =over 4
126    
127     =item ready
128    
129     Immediately after a request is created it is put into the ready state,
130     waiting for a thread to execute it.
131    
132     =item execute
133    
134     A thread has accepted the request for processing and is currently
135     executing it (e.g. blocking in read).
136    
137     =item pending
138    
139     The request has been executed and is waiting for result processing.
140    
141     While request submission and execution is fully asynchronous, result
142     processing is not and relies on the perl interpreter calling C<poll_cb>
143     (or another function with the same effect).
144    
145     =item result
146    
147     The request results are processed synchronously by C<poll_cb>.
148    
149     The C<poll_cb> function will process all outstanding aio requests by
150     calling their callbacks, freeing memory associated with them and managing
151     any groups they are contained in.
152    
153     =item done
154    
155     Request has reached the end of its lifetime and holds no resources anymore
156     (except possibly for the Perl object, but its connection to the actual
157     aio request is severed and calling its methods will either do nothing or
158     result in a runtime error).
159 root 1.1
160 root 1.88 =back
161    
162 root 1.1 =cut
163    
164     package IO::AIO;
165    
166 root 1.117 use Carp ();
167    
168 root 1.161 use common::sense;
169 root 1.23
170 root 1.1 use base 'Exporter';
171    
172     BEGIN {
173 root 1.238 our $VERSION = '4.18';
174 root 1.1
175 root 1.220 our @AIO_REQ = qw(aio_sendfile aio_seek aio_read aio_write aio_open aio_close
176 root 1.148 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
177 root 1.206 aio_scandir aio_symlink aio_readlink aio_realpath aio_sync
178 root 1.236 aio_fsync aio_syncfs aio_fdatasync aio_sync_file_range aio_allocate
179 root 1.222 aio_pathsync aio_readahead aio_fiemap
180 root 1.120 aio_rename aio_link aio_move aio_copy aio_group
181     aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
182 root 1.170 aio_chmod aio_utime aio_truncate
183 root 1.182 aio_msync aio_mtouch aio_mlock aio_mlockall
184 root 1.208 aio_statvfs
185     aio_wd);
186 root 1.120
187 root 1.123 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice));
188 root 1.67 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
189 root 1.188 min_parallel max_parallel max_idle idle_timeout
190 root 1.86 nreqs nready npending nthreads
191 root 1.157 max_poll_time max_poll_reqs
192 root 1.182 sendfile fadvise madvise
193     mmap munmap munlock munlockall);
194 root 1.1
195 root 1.143 push @AIO_REQ, qw(aio_busy); # not exported
196    
197 root 1.54 @IO::AIO::GRP::ISA = 'IO::AIO::REQ';
198    
199 root 1.1 require XSLoader;
200 root 1.51 XSLoader::load ("IO::AIO", $VERSION);
201 root 1.1 }
202    
203 root 1.5 =head1 FUNCTIONS
204 root 1.1
205 root 1.175 =head2 QUICK OVERVIEW
206    
207 root 1.230 This section simply lists the prototypes most of the functions for
208     quick reference. See the following sections for function-by-function
209 root 1.175 documentation.
210    
211 root 1.208 aio_wd $pathname, $callback->($wd)
212 root 1.175 aio_open $pathname, $flags, $mode, $callback->($fh)
213     aio_close $fh, $callback->($status)
214 root 1.220 aio_seek $fh,$offset,$whence, $callback->($offs)
215 root 1.175 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
216     aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
217     aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
218     aio_readahead $fh,$offset,$length, $callback->($retval)
219     aio_stat $fh_or_path, $callback->($status)
220     aio_lstat $fh, $callback->($status)
221     aio_statvfs $fh_or_path, $callback->($statvfs)
222     aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
223     aio_chown $fh_or_path, $uid, $gid, $callback->($status)
224 root 1.220 aio_chmod $fh_or_path, $mode, $callback->($status)
225 root 1.175 aio_truncate $fh_or_path, $offset, $callback->($status)
226 root 1.229 aio_allocate $fh, $mode, $offset, $len, $callback->($status)
227 root 1.230 aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
228 root 1.175 aio_unlink $pathname, $callback->($status)
229 root 1.209 aio_mknod $pathname, $mode, $dev, $callback->($status)
230 root 1.175 aio_link $srcpath, $dstpath, $callback->($status)
231     aio_symlink $srcpath, $dstpath, $callback->($status)
232 root 1.209 aio_readlink $pathname, $callback->($link)
233     aio_realpath $pathname, $callback->($link)
234 root 1.175 aio_rename $srcpath, $dstpath, $callback->($status)
235     aio_mkdir $pathname, $mode, $callback->($status)
236     aio_rmdir $pathname, $callback->($status)
237     aio_readdir $pathname, $callback->($entries)
238     aio_readdirx $pathname, $flags, $callback->($entries, $flags)
239     IO::AIO::READDIR_DENTS IO::AIO::READDIR_DIRS_FIRST
240     IO::AIO::READDIR_STAT_ORDER IO::AIO::READDIR_FOUND_UNKNOWN
241 root 1.215 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
242 root 1.209 aio_load $pathname, $data, $callback->($status)
243 root 1.175 aio_copy $srcpath, $dstpath, $callback->($status)
244     aio_move $srcpath, $dstpath, $callback->($status)
245 root 1.209 aio_rmtree $pathname, $callback->($status)
246 root 1.175 aio_sync $callback->($status)
247 root 1.206 aio_syncfs $fh, $callback->($status)
248 root 1.175 aio_fsync $fh, $callback->($status)
249     aio_fdatasync $fh, $callback->($status)
250     aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
251 root 1.209 aio_pathsync $pathname, $callback->($status)
252 root 1.175 aio_msync $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
253     aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
254 root 1.182 aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
255     aio_mlockall $flags, $callback->($status)
256 root 1.175 aio_group $callback->(...)
257     aio_nop $callback->()
258    
259     $prev_pri = aioreq_pri [$pri]
260     aioreq_nice $pri_adjust
261    
262     IO::AIO::poll_wait
263     IO::AIO::poll_cb
264     IO::AIO::poll
265     IO::AIO::flush
266     IO::AIO::max_poll_reqs $nreqs
267     IO::AIO::max_poll_time $seconds
268     IO::AIO::min_parallel $nthreads
269     IO::AIO::max_parallel $nthreads
270     IO::AIO::max_idle $nthreads
271 root 1.188 IO::AIO::idle_timeout $seconds
272 root 1.175 IO::AIO::max_outstanding $maxreqs
273     IO::AIO::nreqs
274     IO::AIO::nready
275     IO::AIO::npending
276    
277     IO::AIO::sendfile $ofh, $ifh, $offset, $count
278     IO::AIO::fadvise $fh, $offset, $len, $advice
279 root 1.226 IO::AIO::mmap $scalar, $length, $prot, $flags[, $fh[, $offset]]
280     IO::AIO::munmap $scalar
281 root 1.184 IO::AIO::madvise $scalar, $offset, $length, $advice
282     IO::AIO::mprotect $scalar, $offset, $length, $protect
283 root 1.182 IO::AIO::munlock $scalar, $offset = 0, $length = undef
284 root 1.175 IO::AIO::munlockall
285    
286 root 1.219 =head2 API NOTES
287 root 1.1
288 root 1.5 All the C<aio_*> calls are more or less thin wrappers around the syscall
289     with the same name (sans C<aio_>). The arguments are similar or identical,
290 root 1.14 and they all accept an additional (and optional) C<$callback> argument
291 root 1.212 which must be a code reference. This code reference will be called after
292     the syscall has been executed in an asynchronous fashion. The results
293     of the request will be passed as arguments to the callback (and, if an
294     error occured, in C<$!>) - for most requests the syscall return code (e.g.
295     most syscalls return C<-1> on error, unlike perl, which usually delivers
296     "false").
297    
298     Some requests (such as C<aio_readdir>) pass the actual results and
299     communicate failures by passing C<undef>.
300 root 1.1
301 root 1.23 All functions expecting a filehandle keep a copy of the filehandle
302     internally until the request has finished.
303 root 1.1
304 root 1.87 All functions return request objects of type L<IO::AIO::REQ> that allow
305     further manipulation of those requests while they are in-flight.
306 root 1.52
307 root 1.209 The pathnames you pass to these routines I<should> be absolute. The
308     reason for this is that at the time the request is being executed, the
309 root 1.212 current working directory could have changed. Alternatively, you can
310     make sure that you never change the current working directory anywhere
311     in the program and then use relative paths. You can also take advantage
312     of IO::AIOs working directory abstraction, that lets you specify paths
313     relative to some previously-opened "working directory object" - see the
314     description of the C<IO::AIO::WD> class later in this document.
315 root 1.28
316 root 1.87 To encode pathnames as octets, either make sure you either: a) always pass
317     in filenames you got from outside (command line, readdir etc.) without
318 root 1.212 tinkering, b) are in your native filesystem encoding, c) use the Encode
319     module and encode your pathnames to the locale (or other) encoding in
320     effect in the user environment, d) use Glib::filename_from_unicode on
321     unicode filenames or e) use something else to ensure your scalar has the
322     correct contents.
323 root 1.87
324     This works, btw. independent of the internal UTF-8 bit, which IO::AIO
325 root 1.136 handles correctly whether it is set or not.
326 root 1.1
327 root 1.219 =head2 AIO REQUEST FUNCTIONS
328    
329 root 1.5 =over 4
330 root 1.1
331 root 1.80 =item $prev_pri = aioreq_pri [$pri]
332 root 1.68
333 root 1.80 Returns the priority value that would be used for the next request and, if
334     C<$pri> is given, sets the priority for the next aio request.
335 root 1.68
336 root 1.80 The default priority is C<0>, the minimum and maximum priorities are C<-4>
337     and C<4>, respectively. Requests with higher priority will be serviced
338     first.
339    
340     The priority will be reset to C<0> after each call to one of the C<aio_*>
341 root 1.68 functions.
342    
343 root 1.69 Example: open a file with low priority, then read something from it with
344     higher priority so the read request is serviced before other low priority
345     open requests (potentially spamming the cache):
346    
347     aioreq_pri -3;
348     aio_open ..., sub {
349     return unless $_[0];
350    
351     aioreq_pri -2;
352     aio_read $_[0], ..., sub {
353     ...
354     };
355     };
356    
357 root 1.106
358 root 1.69 =item aioreq_nice $pri_adjust
359    
360     Similar to C<aioreq_pri>, but subtracts the given value from the current
361 root 1.87 priority, so the effect is cumulative.
362 root 1.69
363 root 1.106
364 root 1.40 =item aio_open $pathname, $flags, $mode, $callback->($fh)
365 root 1.1
366 root 1.2 Asynchronously open or create a file and call the callback with a newly
367 root 1.233 created filehandle for the file (or C<undef> in case of an error).
368 root 1.1
369     The pathname passed to C<aio_open> must be absolute. See API NOTES, above,
370     for an explanation.
371    
372 root 1.20 The C<$flags> argument is a bitmask. See the C<Fcntl> module for a
373     list. They are the same as used by C<sysopen>.
374    
375     Likewise, C<$mode> specifies the mode of the newly created file, if it
376     didn't exist and C<O_CREAT> has been given, just like perl's C<sysopen>,
377     except that it is mandatory (i.e. use C<0> if you don't create new files,
378 root 1.101 and C<0666> or C<0777> if you do). Note that the C<$mode> will be modified
379     by the umask in effect then the request is being executed, so better never
380     change the umask.
381 root 1.1
382     Example:
383    
384 root 1.181 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
385 root 1.2 if ($_[0]) {
386     print "open successful, fh is $_[0]\n";
387 root 1.1 ...
388     } else {
389     die "open failed: $!\n";
390     }
391     };
392    
393 root 1.194 In addition to all the common open modes/flags (C<O_RDONLY>, C<O_WRONLY>,
394     C<O_RDWR>, C<O_CREAT>, C<O_TRUNC>, C<O_EXCL> and C<O_APPEND>), the
395     following POSIX and non-POSIX constants are available (missing ones on
396     your system are, as usual, C<0>):
397    
398     C<O_ASYNC>, C<O_DIRECT>, C<O_NOATIME>, C<O_CLOEXEC>, C<O_NOCTTY>, C<O_NOFOLLOW>,
399     C<O_NONBLOCK>, C<O_EXEC>, C<O_SEARCH>, C<O_DIRECTORY>, C<O_DSYNC>,
400     C<O_RSYNC>, C<O_SYNC> and C<O_TTY_INIT>.
401    
402 root 1.106
403 root 1.40 =item aio_close $fh, $callback->($status)
404 root 1.1
405 root 1.2 Asynchronously close a file and call the callback with the result
406 root 1.116 code.
407    
408 root 1.117 Unfortunately, you can't do this to perl. Perl I<insists> very strongly on
409 root 1.121 closing the file descriptor associated with the filehandle itself.
410 root 1.117
411 root 1.121 Therefore, C<aio_close> will not close the filehandle - instead it will
412     use dup2 to overwrite the file descriptor with the write-end of a pipe
413     (the pipe fd will be created on demand and will be cached).
414 root 1.117
415 root 1.121 Or in other words: the file descriptor will be closed, but it will not be
416     free for reuse until the perl filehandle is closed.
417 root 1.117
418     =cut
419    
420 root 1.220 =item aio_seek $fh, $offset, $whence, $callback->($offs)
421    
422 root 1.221 Seeks the filehandle to the new C<$offset>, similarly to perl's
423 root 1.220 C<sysseek>. The C<$whence> can use the traditional values (C<0> for
424     C<IO::AIO::SEEK_SET>, C<1> for C<IO::AIO::SEEK_CUR> or C<2> for
425     C<IO::AIO::SEEK_END>).
426    
427     The resulting absolute offset will be passed to the callback, or C<-1> in
428     case of an error.
429    
430     In theory, the C<$whence> constants could be different than the
431     corresponding values from L<Fcntl>, but perl guarantees they are the same,
432     so don't panic.
433    
434 root 1.225 As a GNU/Linux (and maybe Solaris) extension, also the constants
435     C<IO::AIO::SEEK_DATA> and C<IO::AIO::SEEK_HOLE> are available, if they
436     could be found. No guarantees about suitability for use in C<aio_seek> or
437     Perl's C<sysseek> can be made though, although I would naively assume they
438     "just work".
439    
440 root 1.40 =item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
441 root 1.1
442 root 1.40 =item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
443 root 1.1
444 root 1.145 Reads or writes C<$length> bytes from or to the specified C<$fh> and
445     C<$offset> into the scalar given by C<$data> and offset C<$dataoffset>
446     and calls the callback without the actual number of bytes read (or -1 on
447     error, just like the syscall).
448 root 1.109
449 root 1.146 C<aio_read> will, like C<sysread>, shrink or grow the C<$data> scalar to
450     offset plus the actual number of bytes read.
451    
452 root 1.112 If C<$offset> is undefined, then the current file descriptor offset will
453     be used (and updated), otherwise the file descriptor offset will not be
454     changed by these calls.
455 root 1.109
456 root 1.145 If C<$length> is undefined in C<aio_write>, use the remaining length of
457     C<$data>.
458 root 1.109
459     If C<$dataoffset> is less than zero, it will be counted from the end of
460     C<$data>.
461 root 1.1
462 root 1.31 The C<$data> scalar I<MUST NOT> be modified in any way while the request
463 root 1.108 is outstanding. Modifying it can result in segfaults or World War III (if
464     the necessary/optional hardware is installed).
465 root 1.31
466 root 1.17 Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at
467 root 1.1 offset C<0> within the scalar:
468    
469     aio_read $fh, 7, 15, $buffer, 0, sub {
470 root 1.9 $_[0] > 0 or die "read error: $!";
471     print "read $_[0] bytes: <$buffer>\n";
472 root 1.1 };
473    
474 root 1.106
475 root 1.40 =item aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
476 root 1.35
477     Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
478     reading at byte offset C<$in_offset>, and starts writing at the current
479     file offset of C<$out_fh>. Because of that, it is not safe to issue more
480     than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each
481 root 1.196 other. The same C<$in_fh> works fine though, as this function does not
482     move or use the file offset of C<$in_fh>.
483 root 1.35
484 root 1.185 Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than
485 root 1.196 are written, and there is no way to find out how many more bytes have been
486     read from C<aio_sendfile> alone, as C<aio_sendfile> only provides the
487     number of bytes written to C<$out_fh>. Only if the result value equals
488     C<$length> one can assume that C<$length> bytes have been read.
489 root 1.185
490     Unlike with other C<aio_> functions, it makes a lot of sense to use
491     C<aio_sendfile> on non-blocking sockets, as long as one end (typically
492     the C<$in_fh>) is a file - the file I/O will then be asynchronous, while
493 root 1.196 the socket I/O will be non-blocking. Note, however, that you can run
494     into a trap where C<aio_sendfile> reads some data with readahead, then
495     fails to write all data, and when the socket is ready the next time, the
496     data in the cache is already lost, forcing C<aio_sendfile> to again hit
497     the disk. Explicit C<aio_read> + C<aio_write> let's you better control
498     resource usage.
499    
500     This call tries to make use of a native C<sendfile>-like syscall to
501     provide zero-copy operation. For this to work, C<$out_fh> should refer to
502     a socket, and C<$in_fh> should refer to an mmap'able file.
503 root 1.35
504 root 1.170 If a native sendfile cannot be found or it fails with C<ENOSYS>,
505 root 1.196 C<EINVAL>, C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or
506     C<ENOTSOCK>, it will be emulated, so you can call C<aio_sendfile> on any
507     type of filehandle regardless of the limitations of the operating system.
508    
509     As native sendfile syscalls (as practically any non-POSIX interface hacked
510     together in a hurry to improve benchmark numbers) tend to be rather buggy
511     on many systems, this implementation tries to work around some known bugs
512     in Linux and FreeBSD kernels (probably others, too), but that might fail,
513     so you really really should check the return value of C<aio_sendfile> -
514     fewre bytes than expected might have been transferred.
515 root 1.35
516 root 1.106
517 root 1.40 =item aio_readahead $fh,$offset,$length, $callback->($retval)
518 root 1.1
519 root 1.20 C<aio_readahead> populates the page cache with data from a file so that
520 root 1.1 subsequent reads from that file will not block on disk I/O. The C<$offset>
521     argument specifies the starting point from which data is to be read and
522     C<$length> specifies the number of bytes to be read. I/O is performed in
523     whole pages, so that offset is effectively rounded down to a page boundary
524     and bytes are read up to the next page boundary greater than or equal to
525 root 1.20 (off-set+length). C<aio_readahead> does not read beyond the end of the
526 root 1.1 file. The current file offset of the file is left unchanged.
527    
528 root 1.26 If that syscall doesn't exist (likely if your OS isn't Linux) it will be
529     emulated by simply reading the data, which would have a similar effect.
530    
531 root 1.106
532 root 1.40 =item aio_stat $fh_or_path, $callback->($status)
533 root 1.1
534 root 1.40 =item aio_lstat $fh, $callback->($status)
535 root 1.1
536     Works like perl's C<stat> or C<lstat> in void context. The callback will
537     be called after the stat and the results will be available using C<stat _>
538     or C<-s _> etc...
539    
540     The pathname passed to C<aio_stat> must be absolute. See API NOTES, above,
541     for an explanation.
542    
543     Currently, the stats are always 64-bit-stats, i.e. instead of returning an
544     error when stat'ing a large file, the results will be silently truncated
545     unless perl itself is compiled with large file support.
546    
547 root 1.187 To help interpret the mode and dev/rdev stat values, IO::AIO offers the
548     following constants and functions (if not implemented, the constants will
549     be C<0> and the functions will either C<croak> or fall back on traditional
550     behaviour).
551    
552     C<S_IFMT>, C<S_IFIFO>, C<S_IFCHR>, C<S_IFBLK>, C<S_IFLNK>, C<S_IFREG>,
553     C<S_IFDIR>, C<S_IFWHT>, C<S_IFSOCK>, C<IO::AIO::major $dev_t>,
554     C<IO::AIO::minor $dev_t>, C<IO::AIO::makedev $major, $minor>.
555    
556 root 1.1 Example: Print the length of F</etc/passwd>:
557    
558     aio_stat "/etc/passwd", sub {
559     $_[0] and die "stat failed: $!";
560     print "size is ", -s _, "\n";
561     };
562    
563 root 1.106
564 root 1.175 =item aio_statvfs $fh_or_path, $callback->($statvfs)
565 root 1.172
566     Works like the POSIX C<statvfs> or C<fstatvfs> syscalls, depending on
567     whether a file handle or path was passed.
568    
569     On success, the callback is passed a hash reference with the following
570     members: C<bsize>, C<frsize>, C<blocks>, C<bfree>, C<bavail>, C<files>,
571     C<ffree>, C<favail>, C<fsid>, C<flag> and C<namemax>. On failure, C<undef>
572     is passed.
573    
574     The following POSIX IO::AIO::ST_* constants are defined: C<ST_RDONLY> and
575     C<ST_NOSUID>.
576    
577     The following non-POSIX IO::AIO::ST_* flag masks are defined to
578     their correct value when available, or to C<0> on systems that do
579     not support them: C<ST_NODEV>, C<ST_NOEXEC>, C<ST_SYNCHRONOUS>,
580     C<ST_MANDLOCK>, C<ST_WRITE>, C<ST_APPEND>, C<ST_IMMUTABLE>, C<ST_NOATIME>,
581     C<ST_NODIRATIME> and C<ST_RELATIME>.
582    
583     Example: stat C</wd> and dump out the data if successful.
584    
585     aio_statvfs "/wd", sub {
586     my $f = $_[0]
587     or die "statvfs: $!";
588    
589     use Data::Dumper;
590     say Dumper $f;
591     };
592    
593     # result:
594     {
595     bsize => 1024,
596     bfree => 4333064312,
597     blocks => 10253828096,
598     files => 2050765568,
599     flag => 4096,
600     favail => 2042092649,
601     bavail => 4333064312,
602     ffree => 2042092649,
603     namemax => 255,
604     frsize => 1024,
605     fsid => 1810
606     }
607    
608 root 1.235 Here is a (likely partial) list of fsid values used by Linux - it is safe
609     to hardcode these when the $^O is C<linux>:
610    
611     0x0000adf5 adfs
612     0x0000adff affs
613     0x5346414f afs
614     0x09041934 anon-inode filesystem
615     0x00000187 autofs
616     0x42465331 befs
617     0x1badface bfs
618     0x42494e4d binfmt_misc
619     0x9123683e btrfs
620     0x0027e0eb cgroupfs
621     0xff534d42 cifs
622     0x73757245 coda
623     0x012ff7b7 coh
624     0x28cd3d45 cramfs
625     0x453dcd28 cramfs-wend (wrong endianness)
626     0x64626720 debugfs
627     0x00001373 devfs
628     0x00001cd1 devpts
629     0x0000f15f ecryptfs
630     0x00414a53 efs
631     0x0000137d ext
632     0x0000ef53 ext2/ext3
633     0x0000ef51 ext2
634     0x00004006 fat
635     0x65735546 fuseblk
636     0x65735543 fusectl
637     0x0bad1dea futexfs
638     0x01161970 gfs2
639     0x47504653 gpfs
640     0x00004244 hfs
641     0xf995e849 hpfs
642     0x958458f6 hugetlbfs
643     0x2bad1dea inotifyfs
644     0x00009660 isofs
645     0x000072b6 jffs2
646     0x3153464a jfs
647     0x6b414653 k-afs
648     0x0bd00bd0 lustre
649     0x0000137f minix
650     0x0000138f minix 30 char names
651     0x00002468 minix v2
652     0x00002478 minix v2 30 char names
653     0x00004d5a minix v3
654     0x19800202 mqueue
655     0x00004d44 msdos
656     0x0000564c novell
657     0x00006969 nfs
658     0x6e667364 nfsd
659     0x00003434 nilfs
660     0x5346544e ntfs
661     0x00009fa1 openprom
662     0x7461636F ocfs2
663     0x00009fa0 proc
664     0x6165676c pstorefs
665     0x0000002f qnx4
666     0x858458f6 ramfs
667     0x52654973 reiserfs
668     0x00007275 romfs
669     0x67596969 rpc_pipefs
670     0x73636673 securityfs
671     0xf97cff8c selinux
672     0x0000517b smb
673     0x534f434b sockfs
674     0x73717368 squashfs
675     0x62656572 sysfs
676     0x012ff7b6 sysv2
677     0x012ff7b5 sysv4
678     0x01021994 tmpfs
679     0x15013346 udf
680     0x00011954 ufs
681     0x54190100 ufs byteswapped
682     0x00009fa2 usbdevfs
683     0x01021997 v9fs
684     0xa501fcf5 vxfs
685     0xabba1974 xenfs
686     0x012ff7b4 xenix
687     0x58465342 xfs
688     0x012fd16d xia
689 root 1.172
690 root 1.106 =item aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
691    
692     Works like perl's C<utime> function (including the special case of $atime
693     and $mtime being undef). Fractional times are supported if the underlying
694     syscalls support them.
695    
696     When called with a pathname, uses utimes(2) if available, otherwise
697     utime(2). If called on a file descriptor, uses futimes(2) if available,
698     otherwise returns ENOSYS, so this is not portable.
699    
700     Examples:
701    
702 root 1.107 # set atime and mtime to current time (basically touch(1)):
703 root 1.106 aio_utime "path", undef, undef;
704     # set atime to current time and mtime to beginning of the epoch:
705     aio_utime "path", time, undef; # undef==0
706    
707    
708     =item aio_chown $fh_or_path, $uid, $gid, $callback->($status)
709    
710     Works like perl's C<chown> function, except that C<undef> for either $uid
711     or $gid is being interpreted as "do not change" (but -1 can also be used).
712    
713     Examples:
714    
715     # same as "chown root path" in the shell:
716     aio_chown "path", 0, -1;
717     # same as above:
718     aio_chown "path", 0, undef;
719    
720    
721 root 1.110 =item aio_truncate $fh_or_path, $offset, $callback->($status)
722    
723     Works like truncate(2) or ftruncate(2).
724    
725    
726 root 1.229 =item aio_allocate $fh, $mode, $offset, $len, $callback->($status)
727    
728     Allocates or freed disk space according to the C<$mode> argument. See the
729     linux C<fallocate> docuemntation for details.
730    
731     C<$mode> can currently be C<0> or C<IO::AIO::FALLOC_FL_KEEP_SIZE>
732     to allocate space, or C<IO::AIO::FALLOC_FL_PUNCH_HOLE |
733     IO::AIO::FALLOC_FL_KEEP_SIZE>, to deallocate a file range.
734    
735     The file system block size used by C<fallocate> is presumably the
736     C<f_bsize> returned by C<statvfs>.
737    
738     If C<fallocate> isn't available or cannot be emulated (currently no
739     emulation will be attempted), passes C<-1> and sets C<$!> to C<ENOSYS>.
740    
741    
742 root 1.106 =item aio_chmod $fh_or_path, $mode, $callback->($status)
743    
744     Works like perl's C<chmod> function.
745    
746    
747 root 1.40 =item aio_unlink $pathname, $callback->($status)
748 root 1.1
749     Asynchronously unlink (delete) a file and call the callback with the
750     result code.
751    
752 root 1.106
753 root 1.209 =item aio_mknod $pathname, $mode, $dev, $callback->($status)
754 root 1.82
755 root 1.86 [EXPERIMENTAL]
756    
757 root 1.83 Asynchronously create a device node (or fifo). See mknod(2).
758    
759 root 1.86 The only (POSIX-) portable way of calling this function is:
760 root 1.83
761 root 1.209 aio_mknod $pathname, IO::AIO::S_IFIFO | $mode, 0, sub { ...
762 root 1.82
763 root 1.187 See C<aio_stat> for info about some potentially helpful extra constants
764     and functions.
765 root 1.106
766 root 1.50 =item aio_link $srcpath, $dstpath, $callback->($status)
767    
768     Asynchronously create a new link to the existing object at C<$srcpath> at
769     the path C<$dstpath> and call the callback with the result code.
770    
771 root 1.106
772 root 1.50 =item aio_symlink $srcpath, $dstpath, $callback->($status)
773    
774     Asynchronously create a new symbolic link to the existing object at C<$srcpath> at
775     the path C<$dstpath> and call the callback with the result code.
776    
777 root 1.106
778 root 1.209 =item aio_readlink $pathname, $callback->($link)
779 root 1.90
780     Asynchronously read the symlink specified by C<$path> and pass it to
781     the callback. If an error occurs, nothing or undef gets passed to the
782     callback.
783    
784 root 1.106
785 root 1.209 =item aio_realpath $pathname, $callback->($path)
786 root 1.201
787     Asynchronously make the path absolute and resolve any symlinks in
788 root 1.239 C<$path>. The resulting path only consists of directories (same as
789 root 1.202 L<Cwd::realpath>).
790 root 1.201
791     This request can be used to get the absolute path of the current working
792     directory by passing it a path of F<.> (a single dot).
793    
794    
795 root 1.50 =item aio_rename $srcpath, $dstpath, $callback->($status)
796    
797     Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as
798     rename(2) and call the callback with the result code.
799    
800 root 1.241 On systems that support the AIO::WD working directory abstraction
801     natively, the case C<[$wd, "."]> as C<$srcpath> is specialcased - instead
802     of failing, C<rename> is called on the absolute path of C<$wd>.
803    
804 root 1.106
805 root 1.101 =item aio_mkdir $pathname, $mode, $callback->($status)
806    
807     Asynchronously mkdir (create) a directory and call the callback with
808     the result code. C<$mode> will be modified by the umask at the time the
809     request is executed, so do not change your umask.
810    
811 root 1.106
812 root 1.40 =item aio_rmdir $pathname, $callback->($status)
813 root 1.27
814     Asynchronously rmdir (delete) a directory and call the callback with the
815     result code.
816    
817 root 1.241 On systems that support the AIO::WD working directory abstraction
818     natively, the case C<[$wd, "."]> is specialcased - instead of failing,
819     C<rmdir> is called on the absolute path of C<$wd>.
820    
821 root 1.106
822 root 1.46 =item aio_readdir $pathname, $callback->($entries)
823 root 1.37
824     Unlike the POSIX call of the same name, C<aio_readdir> reads an entire
825     directory (i.e. opendir + readdir + closedir). The entries will not be
826     sorted, and will B<NOT> include the C<.> and C<..> entries.
827    
828 root 1.148 The callback is passed a single argument which is either C<undef> or an
829     array-ref with the filenames.
830    
831    
832     =item aio_readdirx $pathname, $flags, $callback->($entries, $flags)
833    
834 root 1.207 Quite similar to C<aio_readdir>, but the C<$flags> argument allows one to
835     tune behaviour and output format. In case of an error, C<$entries> will be
836 root 1.148 C<undef>.
837    
838     The flags are a combination of the following constants, ORed together (the
839     flags will also be passed to the callback, possibly modified):
840    
841     =over 4
842    
843 root 1.150 =item IO::AIO::READDIR_DENTS
844 root 1.148
845 root 1.190 When this flag is off, then the callback gets an arrayref consisting of
846     names only (as with C<aio_readdir>), otherwise it gets an arrayref with
847 root 1.150 C<[$name, $type, $inode]> arrayrefs, each describing a single directory
848 root 1.148 entry in more detail.
849    
850     C<$name> is the name of the entry.
851    
852 root 1.150 C<$type> is one of the C<IO::AIO::DT_xxx> constants:
853 root 1.148
854 root 1.150 C<IO::AIO::DT_UNKNOWN>, C<IO::AIO::DT_FIFO>, C<IO::AIO::DT_CHR>, C<IO::AIO::DT_DIR>,
855     C<IO::AIO::DT_BLK>, C<IO::AIO::DT_REG>, C<IO::AIO::DT_LNK>, C<IO::AIO::DT_SOCK>,
856     C<IO::AIO::DT_WHT>.
857 root 1.148
858 root 1.150 C<IO::AIO::DT_UNKNOWN> means just that: readdir does not know. If you need to
859 root 1.148 know, you have to run stat yourself. Also, for speed reasons, the C<$type>
860     scalars are read-only: you can not modify them.
861    
862 root 1.150 C<$inode> is the inode number (which might not be exact on systems with 64
863 root 1.155 bit inode numbers and 32 bit perls). This field has unspecified content on
864     systems that do not deliver the inode information.
865 root 1.150
866     =item IO::AIO::READDIR_DIRS_FIRST
867 root 1.148
868     When this flag is set, then the names will be returned in an order where
869 root 1.193 likely directories come first, in optimal stat order. This is useful when
870     you need to quickly find directories, or you want to find all directories
871     while avoiding to stat() each entry.
872 root 1.148
873 root 1.149 If the system returns type information in readdir, then this is used
874 root 1.193 to find directories directly. Otherwise, likely directories are names
875     beginning with ".", or otherwise names with no dots, of which names with
876 root 1.149 short names are tried first.
877    
878 root 1.150 =item IO::AIO::READDIR_STAT_ORDER
879 root 1.148
880     When this flag is set, then the names will be returned in an order
881     suitable for stat()'ing each one. That is, when you plan to stat()
882     all files in the given directory, then the returned order will likely
883     be fastest.
884    
885 root 1.150 If both this flag and C<IO::AIO::READDIR_DIRS_FIRST> are specified, then
886     the likely dirs come first, resulting in a less optimal stat order.
887 root 1.148
888 root 1.150 =item IO::AIO::READDIR_FOUND_UNKNOWN
889 root 1.148
890     This flag should not be set when calling C<aio_readdirx>. Instead, it
891     is being set by C<aio_readdirx>, when any of the C<$type>'s found were
892 root 1.207 C<IO::AIO::DT_UNKNOWN>. The absence of this flag therefore indicates that all
893 root 1.148 C<$type>'s are known, which can be used to speed up some algorithms.
894    
895     =back
896 root 1.37
897 root 1.106
898 root 1.209 =item aio_load $pathname, $data, $callback->($status)
899 root 1.98
900     This is a composite request that tries to fully load the given file into
901     memory. Status is the same as with aio_read.
902    
903     =cut
904    
905     sub aio_load($$;$) {
906 root 1.123 my ($path, undef, $cb) = @_;
907     my $data = \$_[1];
908 root 1.98
909 root 1.123 my $pri = aioreq_pri;
910     my $grp = aio_group $cb;
911    
912     aioreq_pri $pri;
913     add $grp aio_open $path, O_RDONLY, 0, sub {
914     my $fh = shift
915     or return $grp->result (-1);
916 root 1.98
917     aioreq_pri $pri;
918 root 1.123 add $grp aio_read $fh, 0, (-s $fh), $$data, 0, sub {
919     $grp->result ($_[0]);
920 root 1.98 };
921 root 1.123 };
922 root 1.98
923 root 1.123 $grp
924 root 1.98 }
925    
926 root 1.82 =item aio_copy $srcpath, $dstpath, $callback->($status)
927    
928     Try to copy the I<file> (directories not supported as either source or
929     destination) from C<$srcpath> to C<$dstpath> and call the callback with
930 root 1.165 a status of C<0> (ok) or C<-1> (error, see C<$!>).
931 root 1.82
932 root 1.134 This is a composite request that creates the destination file with
933 root 1.82 mode 0200 and copies the contents of the source file into it using
934     C<aio_sendfile>, followed by restoring atime, mtime, access mode and
935     uid/gid, in that order.
936    
937     If an error occurs, the partial destination file will be unlinked, if
938     possible, except when setting atime, mtime, access mode and uid/gid, where
939     errors are being ignored.
940    
941     =cut
942    
943     sub aio_copy($$;$) {
944 root 1.123 my ($src, $dst, $cb) = @_;
945 root 1.82
946 root 1.123 my $pri = aioreq_pri;
947     my $grp = aio_group $cb;
948 root 1.82
949 root 1.123 aioreq_pri $pri;
950     add $grp aio_open $src, O_RDONLY, 0, sub {
951     if (my $src_fh = $_[0]) {
952 root 1.166 my @stat = stat $src_fh; # hmm, might block over nfs?
953 root 1.95
954 root 1.123 aioreq_pri $pri;
955     add $grp aio_open $dst, O_CREAT | O_WRONLY | O_TRUNC, 0200, sub {
956     if (my $dst_fh = $_[0]) {
957     aioreq_pri $pri;
958     add $grp aio_sendfile $dst_fh, $src_fh, 0, $stat[7], sub {
959     if ($_[0] == $stat[7]) {
960     $grp->result (0);
961     close $src_fh;
962    
963 root 1.147 my $ch = sub {
964     aioreq_pri $pri;
965     add $grp aio_chmod $dst_fh, $stat[2] & 07777, sub {
966     aioreq_pri $pri;
967     add $grp aio_chown $dst_fh, $stat[4], $stat[5], sub {
968     aioreq_pri $pri;
969     add $grp aio_close $dst_fh;
970     }
971     };
972     };
973 root 1.123
974     aioreq_pri $pri;
975 root 1.147 add $grp aio_utime $dst_fh, $stat[8], $stat[9], sub {
976     if ($_[0] < 0 && $! == ENOSYS) {
977     aioreq_pri $pri;
978     add $grp aio_utime $dst, $stat[8], $stat[9], $ch;
979     } else {
980     $ch->();
981     }
982     };
983 root 1.123 } else {
984     $grp->result (-1);
985     close $src_fh;
986     close $dst_fh;
987    
988     aioreq $pri;
989     add $grp aio_unlink $dst;
990     }
991     };
992     } else {
993     $grp->result (-1);
994     }
995     },
996 root 1.82
997 root 1.123 } else {
998     $grp->result (-1);
999     }
1000     };
1001 root 1.82
1002 root 1.123 $grp
1003 root 1.82 }
1004    
1005     =item aio_move $srcpath, $dstpath, $callback->($status)
1006    
1007     Try to move the I<file> (directories not supported as either source or
1008     destination) from C<$srcpath> to C<$dstpath> and call the callback with
1009 root 1.165 a status of C<0> (ok) or C<-1> (error, see C<$!>).
1010 root 1.82
1011 root 1.137 This is a composite request that tries to rename(2) the file first; if
1012     rename fails with C<EXDEV>, it copies the file with C<aio_copy> and, if
1013     that is successful, unlinks the C<$srcpath>.
1014 root 1.82
1015     =cut
1016    
1017     sub aio_move($$;$) {
1018 root 1.123 my ($src, $dst, $cb) = @_;
1019 root 1.82
1020 root 1.123 my $pri = aioreq_pri;
1021     my $grp = aio_group $cb;
1022 root 1.82
1023 root 1.123 aioreq_pri $pri;
1024     add $grp aio_rename $src, $dst, sub {
1025     if ($_[0] && $! == EXDEV) {
1026     aioreq_pri $pri;
1027     add $grp aio_copy $src, $dst, sub {
1028     $grp->result ($_[0]);
1029 root 1.95
1030 root 1.196 unless ($_[0]) {
1031 root 1.123 aioreq_pri $pri;
1032     add $grp aio_unlink $src;
1033     }
1034     };
1035     } else {
1036     $grp->result ($_[0]);
1037     }
1038     };
1039 root 1.82
1040 root 1.123 $grp
1041 root 1.82 }
1042    
1043 root 1.209 =item aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
1044 root 1.40
1045 root 1.52 Scans a directory (similar to C<aio_readdir>) but additionally tries to
1046 root 1.76 efficiently separate the entries of directory C<$path> into two sets of
1047     names, directories you can recurse into (directories), and ones you cannot
1048     recurse into (everything else, including symlinks to directories).
1049 root 1.52
1050 root 1.61 C<aio_scandir> is a composite request that creates of many sub requests_
1051     C<$maxreq> specifies the maximum number of outstanding aio requests that
1052     this function generates. If it is C<< <= 0 >>, then a suitable default
1053 root 1.81 will be chosen (currently 4).
1054 root 1.40
1055     On error, the callback is called without arguments, otherwise it receives
1056     two array-refs with path-relative entry names.
1057    
1058     Example:
1059    
1060     aio_scandir $dir, 0, sub {
1061     my ($dirs, $nondirs) = @_;
1062     print "real directories: @$dirs\n";
1063     print "everything else: @$nondirs\n";
1064     };
1065    
1066     Implementation notes.
1067    
1068     The C<aio_readdir> cannot be avoided, but C<stat()>'ing every entry can.
1069    
1070 root 1.149 If readdir returns file type information, then this is used directly to
1071     find directories.
1072    
1073     Otherwise, after reading the directory, the modification time, size etc.
1074     of the directory before and after the readdir is checked, and if they
1075     match (and isn't the current time), the link count will be used to decide
1076     how many entries are directories (if >= 2). Otherwise, no knowledge of the
1077     number of subdirectories will be assumed.
1078    
1079     Then entries will be sorted into likely directories a non-initial dot
1080     currently) and likely non-directories (see C<aio_readdirx>). Then every
1081     entry plus an appended C</.> will be C<stat>'ed, likely directories first,
1082     in order of their inode numbers. If that succeeds, it assumes that the
1083     entry is a directory or a symlink to directory (which will be checked
1084 root 1.207 separately). This is often faster than stat'ing the entry itself because
1085 root 1.52 filesystems might detect the type of the entry without reading the inode
1086 root 1.149 data (e.g. ext2fs filetype feature), even on systems that cannot return
1087     the filetype information on readdir.
1088 root 1.52
1089     If the known number of directories (link count - 2) has been reached, the
1090     rest of the entries is assumed to be non-directories.
1091    
1092     This only works with certainty on POSIX (= UNIX) filesystems, which
1093     fortunately are the vast majority of filesystems around.
1094    
1095     It will also likely work on non-POSIX filesystems with reduced efficiency
1096     as those tend to return 0 or 1 as link counts, which disables the
1097     directory counting heuristic.
1098 root 1.40
1099     =cut
1100    
1101 root 1.100 sub aio_scandir($$;$) {
1102 root 1.123 my ($path, $maxreq, $cb) = @_;
1103    
1104     my $pri = aioreq_pri;
1105 root 1.40
1106 root 1.123 my $grp = aio_group $cb;
1107 root 1.80
1108 root 1.123 $maxreq = 4 if $maxreq <= 0;
1109 root 1.55
1110 root 1.210 # get a wd object
1111 root 1.123 aioreq_pri $pri;
1112 root 1.210 add $grp aio_wd $path, sub {
1113 root 1.212 $_[0]
1114     or return $grp->result ();
1115    
1116 root 1.210 my $wd = [shift, "."];
1117 root 1.40
1118 root 1.210 # stat once
1119 root 1.80 aioreq_pri $pri;
1120 root 1.210 add $grp aio_stat $wd, sub {
1121     return $grp->result () if $_[0];
1122     my $now = time;
1123     my $hash1 = join ":", (stat _)[0,1,3,7,9];
1124 root 1.40
1125 root 1.210 # read the directory entries
1126 root 1.80 aioreq_pri $pri;
1127 root 1.210 add $grp aio_readdirx $wd, READDIR_DIRS_FIRST, sub {
1128     my $entries = shift
1129     or return $grp->result ();
1130    
1131     # stat the dir another time
1132     aioreq_pri $pri;
1133     add $grp aio_stat $wd, sub {
1134     my $hash2 = join ":", (stat _)[0,1,3,7,9];
1135 root 1.95
1136 root 1.210 my $ndirs;
1137 root 1.95
1138 root 1.210 # take the slow route if anything looks fishy
1139     if ($hash1 ne $hash2 or (stat _)[9] == $now) {
1140     $ndirs = -1;
1141     } else {
1142     # if nlink == 2, we are finished
1143     # for non-posix-fs's, we rely on nlink < 2
1144     $ndirs = (stat _)[3] - 2
1145     or return $grp->result ([], $entries);
1146     }
1147 root 1.123
1148 root 1.210 my (@dirs, @nondirs);
1149 root 1.40
1150 root 1.210 my $statgrp = add $grp aio_group sub {
1151     $grp->result (\@dirs, \@nondirs);
1152     };
1153 root 1.40
1154 root 1.210 limit $statgrp $maxreq;
1155     feed $statgrp sub {
1156     return unless @$entries;
1157     my $entry = shift @$entries;
1158    
1159     aioreq_pri $pri;
1160     $wd->[1] = "$entry/.";
1161     add $statgrp aio_stat $wd, sub {
1162     if ($_[0] < 0) {
1163     push @nondirs, $entry;
1164     } else {
1165     # need to check for real directory
1166     aioreq_pri $pri;
1167     $wd->[1] = $entry;
1168     add $statgrp aio_lstat $wd, sub {
1169     if (-d _) {
1170     push @dirs, $entry;
1171    
1172     unless (--$ndirs) {
1173     push @nondirs, @$entries;
1174     feed $statgrp;
1175     }
1176     } else {
1177     push @nondirs, $entry;
1178 root 1.74 }
1179 root 1.40 }
1180     }
1181 root 1.210 };
1182 root 1.74 };
1183 root 1.40 };
1184     };
1185     };
1186 root 1.123 };
1187 root 1.55
1188 root 1.123 $grp
1189 root 1.40 }
1190    
1191 root 1.209 =item aio_rmtree $pathname, $callback->($status)
1192 root 1.99
1193 root 1.100 Delete a directory tree starting (and including) C<$path>, return the
1194 root 1.239 status of the final C<rmdir> only. This is a composite request that
1195 root 1.100 uses C<aio_scandir> to recurse into and rmdir directories, and unlink
1196     everything else.
1197 root 1.99
1198     =cut
1199    
1200     sub aio_rmtree;
1201 root 1.100 sub aio_rmtree($;$) {
1202 root 1.123 my ($path, $cb) = @_;
1203 root 1.99
1204 root 1.123 my $pri = aioreq_pri;
1205     my $grp = aio_group $cb;
1206 root 1.99
1207 root 1.123 aioreq_pri $pri;
1208     add $grp aio_scandir $path, 0, sub {
1209     my ($dirs, $nondirs) = @_;
1210 root 1.99
1211 root 1.123 my $dirgrp = aio_group sub {
1212     add $grp aio_rmdir $path, sub {
1213     $grp->result ($_[0]);
1214 root 1.99 };
1215 root 1.123 };
1216 root 1.99
1217 root 1.123 (aioreq_pri $pri), add $dirgrp aio_rmtree "$path/$_" for @$dirs;
1218     (aioreq_pri $pri), add $dirgrp aio_unlink "$path/$_" for @$nondirs;
1219 root 1.99
1220 root 1.123 add $grp $dirgrp;
1221     };
1222 root 1.99
1223 root 1.123 $grp
1224 root 1.99 }
1225    
1226 root 1.119 =item aio_sync $callback->($status)
1227    
1228     Asynchronously call sync and call the callback when finished.
1229    
1230 root 1.40 =item aio_fsync $fh, $callback->($status)
1231 root 1.1
1232     Asynchronously call fsync on the given filehandle and call the callback
1233     with the fsync result code.
1234    
1235 root 1.40 =item aio_fdatasync $fh, $callback->($status)
1236 root 1.1
1237     Asynchronously call fdatasync on the given filehandle and call the
1238 root 1.26 callback with the fdatasync result code.
1239    
1240     If this call isn't available because your OS lacks it or it couldn't be
1241     detected, it will be emulated by calling C<fsync> instead.
1242 root 1.1
1243 root 1.206 =item aio_syncfs $fh, $callback->($status)
1244    
1245     Asynchronously call the syncfs syscall to sync the filesystem associated
1246     to the given filehandle and call the callback with the syncfs result
1247     code. If syncfs is not available, calls sync(), but returns C<-1> and sets
1248     errno to C<ENOSYS> nevertheless.
1249    
1250 root 1.142 =item aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
1251    
1252     Sync the data portion of the file specified by C<$offset> and C<$length>
1253     to disk (but NOT the metadata), by calling the Linux-specific
1254     sync_file_range call. If sync_file_range is not available or it returns
1255     ENOSYS, then fdatasync or fsync is being substituted.
1256    
1257     C<$flags> can be a combination of C<IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE>,
1258     C<IO::AIO::SYNC_FILE_RANGE_WRITE> and
1259     C<IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER>: refer to the sync_file_range
1260     manpage for details.
1261    
1262 root 1.209 =item aio_pathsync $pathname, $callback->($status)
1263 root 1.120
1264     This request tries to open, fsync and close the given path. This is a
1265 root 1.135 composite request intended to sync directories after directory operations
1266 root 1.120 (E.g. rename). This might not work on all operating systems or have any
1267     specific effect, but usually it makes sure that directory changes get
1268     written to disc. It works for anything that can be opened for read-only,
1269     not just directories.
1270    
1271 root 1.162 Future versions of this function might fall back to other methods when
1272     C<fsync> on the directory fails (such as calling C<sync>).
1273    
1274 root 1.120 Passes C<0> when everything went ok, and C<-1> on error.
1275    
1276     =cut
1277    
1278     sub aio_pathsync($;$) {
1279 root 1.123 my ($path, $cb) = @_;
1280    
1281     my $pri = aioreq_pri;
1282     my $grp = aio_group $cb;
1283 root 1.120
1284 root 1.123 aioreq_pri $pri;
1285     add $grp aio_open $path, O_RDONLY, 0, sub {
1286     my ($fh) = @_;
1287     if ($fh) {
1288     aioreq_pri $pri;
1289     add $grp aio_fsync $fh, sub {
1290     $grp->result ($_[0]);
1291 root 1.120
1292     aioreq_pri $pri;
1293 root 1.123 add $grp aio_close $fh;
1294     };
1295     } else {
1296     $grp->result (-1);
1297     }
1298     };
1299 root 1.120
1300 root 1.123 $grp
1301 root 1.120 }
1302    
1303 root 1.170 =item aio_msync $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
1304    
1305     This is a rather advanced IO::AIO call, which only works on mmap(2)ed
1306 root 1.176 scalars (see the C<IO::AIO::mmap> function, although it also works on data
1307     scalars managed by the L<Sys::Mmap> or L<Mmap> modules, note that the
1308     scalar must only be modified in-place while an aio operation is pending on
1309     it).
1310 root 1.170
1311     It calls the C<msync> function of your OS, if available, with the memory
1312     area starting at C<$offset> in the string and ending C<$length> bytes
1313     later. If C<$length> is negative, counts from the end, and if C<$length>
1314     is C<undef>, then it goes till the end of the string. The flags can be
1315     a combination of C<IO::AIO::MS_ASYNC>, C<IO::AIO::MS_INVALIDATE> and
1316     C<IO::AIO::MS_SYNC>.
1317    
1318     =item aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
1319    
1320     This is a rather advanced IO::AIO call, which works best on mmap(2)ed
1321     scalars.
1322    
1323     It touches (reads or writes) all memory pages in the specified
1324 root 1.239 range inside the scalar. All caveats and parameters are the same
1325 root 1.170 as for C<aio_msync>, above, except for flags, which must be either
1326     C<0> (which reads all pages and ensures they are instantiated) or
1327 root 1.239 C<IO::AIO::MT_MODIFY>, which modifies the memory pages (by reading and
1328 root 1.170 writing an octet from it, which dirties the page).
1329    
1330 root 1.182 =item aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
1331    
1332     This is a rather advanced IO::AIO call, which works best on mmap(2)ed
1333     scalars.
1334    
1335     It reads in all the pages of the underlying storage into memory (if any)
1336     and locks them, so they are not getting swapped/paged out or removed.
1337    
1338     If C<$length> is undefined, then the scalar will be locked till the end.
1339    
1340     On systems that do not implement C<mlock>, this function returns C<-1>
1341     and sets errno to C<ENOSYS>.
1342    
1343     Note that the corresponding C<munlock> is synchronous and is
1344     documented under L<MISCELLANEOUS FUNCTIONS>.
1345    
1346 root 1.183 Example: open a file, mmap and mlock it - both will be undone when
1347     C<$data> gets destroyed.
1348    
1349     open my $fh, "<", $path or die "$path: $!";
1350     my $data;
1351     IO::AIO::mmap $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh;
1352     aio_mlock $data; # mlock in background
1353    
1354 root 1.182 =item aio_mlockall $flags, $callback->($status)
1355    
1356     Calls the C<mlockall> function with the given C<$flags> (a combination of
1357     C<IO::AIO::MCL_CURRENT> and C<IO::AIO::MCL_FUTURE>).
1358    
1359     On systems that do not implement C<mlockall>, this function returns C<-1>
1360     and sets errno to C<ENOSYS>.
1361    
1362     Note that the corresponding C<munlockall> is synchronous and is
1363     documented under L<MISCELLANEOUS FUNCTIONS>.
1364    
1365 root 1.183 Example: asynchronously lock all current and future pages into memory.
1366    
1367     aio_mlockall IO::AIO::MCL_FUTURE;
1368    
1369 root 1.223 =item aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
1370    
1371 root 1.234 Queries the extents of the given file (by calling the Linux C<FIEMAP>
1372     ioctl, see L<http://cvs.schmorp.de/IO-AIO/doc/fiemap.txt> for details). If
1373     the ioctl is not available on your OS, then this request will fail with
1374 root 1.223 C<ENOSYS>.
1375    
1376     C<$start> is the starting offset to query extents for, C<$length> is the
1377     size of the range to query - if it is C<undef>, then the whole file will
1378     be queried.
1379    
1380     C<$flags> is a combination of flags (C<IO::AIO::FIEMAP_FLAG_SYNC> or
1381     C<IO::AIO::FIEMAP_FLAG_XATTR> - C<IO::AIO::FIEMAP_FLAGS_COMPAT> is also
1382     exported), and is normally C<0> or C<IO::AIO::FIEMAP_FLAG_SYNC> to query
1383     the data portion.
1384    
1385     C<$count> is the maximum number of extent records to return. If it is
1386 root 1.232 C<undef>, then IO::AIO queries all extents of the range. As a very special
1387 root 1.223 case, if it is C<0>, then the callback receives the number of extents
1388 root 1.232 instead of the extents themselves (which is unreliable, see below).
1389 root 1.223
1390     If an error occurs, the callback receives no arguments. The special
1391     C<errno> value C<IO::AIO::EBADR> is available to test for flag errors.
1392    
1393     Otherwise, the callback receives an array reference with extent
1394     structures. Each extent structure is an array reference itself, with the
1395     following members:
1396    
1397     [$logical, $physical, $length, $flags]
1398    
1399     Flags is any combination of the following flag values (typically either C<0>
1400 root 1.231 or C<IO::AIO::FIEMAP_EXTENT_LAST> (1)):
1401 root 1.223
1402     C<IO::AIO::FIEMAP_EXTENT_LAST>, C<IO::AIO::FIEMAP_EXTENT_UNKNOWN>,
1403     C<IO::AIO::FIEMAP_EXTENT_DELALLOC>, C<IO::AIO::FIEMAP_EXTENT_ENCODED>,
1404     C<IO::AIO::FIEMAP_EXTENT_DATA_ENCRYPTED>, C<IO::AIO::FIEMAP_EXTENT_NOT_ALIGNED>,
1405     C<IO::AIO::FIEMAP_EXTENT_DATA_INLINE>, C<IO::AIO::FIEMAP_EXTENT_DATA_TAIL>,
1406     C<IO::AIO::FIEMAP_EXTENT_UNWRITTEN>, C<IO::AIO::FIEMAP_EXTENT_MERGED> or
1407     C<IO::AIO::FIEMAP_EXTENT_SHARED>.
1408    
1409 root 1.232 At the time of this writing (Linux 3.2), this requets is unreliable unless
1410     C<$count> is C<undef>, as the kernel has all sorts of bugs preventing
1411     it to return all extents of a range for files with large number of
1412     extents. The code works around all these issues if C<$count> is undef.
1413    
1414 root 1.58 =item aio_group $callback->(...)
1415 root 1.54
1416 root 1.55 This is a very special aio request: Instead of doing something, it is a
1417     container for other aio requests, which is useful if you want to bundle
1418 root 1.71 many requests into a single, composite, request with a definite callback
1419     and the ability to cancel the whole request with its subrequests.
1420 root 1.55
1421     Returns an object of class L<IO::AIO::GRP>. See its documentation below
1422     for more info.
1423    
1424     Example:
1425    
1426     my $grp = aio_group sub {
1427     print "all stats done\n";
1428     };
1429    
1430     add $grp
1431     (aio_stat ...),
1432     (aio_stat ...),
1433     ...;
1434    
1435 root 1.63 =item aio_nop $callback->()
1436    
1437     This is a special request - it does nothing in itself and is only used for
1438     side effects, such as when you want to add a dummy request to a group so
1439     that finishing the requests in the group depends on executing the given
1440     code.
1441    
1442 root 1.64 While this request does nothing, it still goes through the execution
1443     phase and still requires a worker thread. Thus, the callback will not
1444     be executed immediately but only after other requests in the queue have
1445     entered their execution phase. This can be used to measure request
1446     latency.
1447    
1448 root 1.71 =item IO::AIO::aio_busy $fractional_seconds, $callback->() *NOT EXPORTED*
1449 root 1.54
1450     Mainly used for debugging and benchmarking, this aio request puts one of
1451     the request workers to sleep for the given time.
1452    
1453 root 1.56 While it is theoretically handy to have simple I/O scheduling requests
1454 root 1.71 like sleep and file handle readable/writable, the overhead this creates is
1455     immense (it blocks a thread for a long time) so do not use this function
1456     except to put your application under artificial I/O pressure.
1457 root 1.56
1458 root 1.5 =back
1459    
1460 root 1.209
1461     =head2 IO::AIO::WD - multiple working directories
1462    
1463     Your process only has one current working directory, which is used by all
1464     threads. This makes it hard to use relative paths (some other component
1465     could call C<chdir> at any time, and it is hard to control when the path
1466     will be used by IO::AIO).
1467    
1468     One solution for this is to always use absolute paths. This usually works,
1469     but can be quite slow (the kernel has to walk the whole path on every
1470     access), and can also be a hassle to implement.
1471    
1472     Newer POSIX systems have a number of functions (openat, fdopendir,
1473     futimensat and so on) that make it possible to specify working directories
1474     per operation.
1475    
1476     For portability, and because the clowns who "designed", or shall I write,
1477     perpetrated this new interface were obviously half-drunk, this abstraction
1478     cannot be perfect, though.
1479    
1480     IO::AIO allows you to convert directory paths into a so-called IO::AIO::WD
1481     object. This object stores the canonicalised, absolute version of the
1482     path, and on systems that allow it, also a directory file descriptor.
1483    
1484     Everywhere where a pathname is accepted by IO::AIO (e.g. in C<aio_stat>
1485     or C<aio_unlink>), one can specify an array reference with an IO::AIO::WD
1486 root 1.214 object and a pathname instead (or the IO::AIO::WD object alone, which
1487     gets interpreted as C<[$wd, "."]>). If the pathname is absolute, the
1488 root 1.213 IO::AIO::WD object is ignored, otherwise the pathname is resolved relative
1489 root 1.209 to that IO::AIO::WD object.
1490    
1491     For example, to get a wd object for F</etc> and then stat F<passwd>
1492     inside, you would write:
1493    
1494     aio_wd "/etc", sub {
1495     my $etcdir = shift;
1496    
1497     # although $etcdir can be undef on error, there is generally no reason
1498     # to check for errors here, as aio_stat will fail with ENOENT
1499     # when $etcdir is undef.
1500    
1501     aio_stat [$etcdir, "passwd"], sub {
1502     # yay
1503     };
1504     };
1505    
1506 root 1.214 That C<aio_wd> is a request and not a normal function shows that creating
1507     an IO::AIO::WD object is itself a potentially blocking operation, which is
1508     why it is done asynchronously.
1509    
1510     To stat the directory obtained with C<aio_wd> above, one could write
1511     either of the following three request calls:
1512    
1513     aio_lstat "/etc" , sub { ... # pathname as normal string
1514     aio_lstat [$wd, "."], sub { ... # "." relative to $wd (i.e. $wd itself)
1515     aio_lstat $wd , sub { ... # shorthand for the previous
1516 root 1.209
1517     As with normal pathnames, IO::AIO keeps a copy of the working directory
1518     object and the pathname string, so you could write the following without
1519     causing any issues due to C<$path> getting reused:
1520    
1521     my $path = [$wd, undef];
1522    
1523     for my $name (qw(abc def ghi)) {
1524     $path->[1] = $name;
1525     aio_stat $path, sub {
1526     # ...
1527     };
1528     }
1529    
1530     There are some caveats: when directories get renamed (or deleted), the
1531     pathname string doesn't change, so will point to the new directory (or
1532     nowhere at all), while the directory fd, if available on the system,
1533     will still point to the original directory. Most functions accepting a
1534     pathname will use the directory fd on newer systems, and the string on
1535     older systems. Some functions (such as realpath) will always rely on the
1536     string form of the pathname.
1537    
1538 root 1.239 So this functionality is mainly useful to get some protection against
1539 root 1.209 C<chdir>, to easily get an absolute path out of a relative path for future
1540     reference, and to speed up doing many operations in the same directory
1541     (e.g. when stat'ing all files in a directory).
1542    
1543     The following functions implement this working directory abstraction:
1544    
1545     =over 4
1546    
1547     =item aio_wd $pathname, $callback->($wd)
1548    
1549     Asynchonously canonicalise the given pathname and convert it to an
1550     IO::AIO::WD object representing it. If possible and supported on the
1551     system, also open a directory fd to speed up pathname resolution relative
1552     to this working directory.
1553    
1554     If something goes wrong, then C<undef> is passwd to the callback instead
1555     of a working directory object and C<$!> is set appropriately. Since
1556     passing C<undef> as working directory component of a pathname fails the
1557     request with C<ENOENT>, there is often no need for error checking in the
1558     C<aio_wd> callback, as future requests using the value will fail in the
1559     expected way.
1560    
1561     If this call isn't available because your OS lacks it or it couldn't be
1562     detected, it will be emulated by calling C<fsync> instead.
1563    
1564     =item IO::AIO::CWD
1565    
1566     This is a compiletime constant (object) that represents the process
1567     current working directory.
1568    
1569 root 1.239 Specifying this object as working directory object for a pathname is as if
1570     the pathname would be specified directly, without a directory object. For
1571     example, these calls are functionally identical:
1572 root 1.209
1573     aio_stat "somefile", sub { ... };
1574     aio_stat [IO::AIO::CWD, "somefile"], sub { ... };
1575    
1576     =back
1577    
1578 root 1.239 To recover the path associated with an IO::AIO::WD object, you can use
1579     C<aio_realpath>:
1580    
1581     aio_realpath $wd, sub {
1582     warn "path is $_[0]\n";
1583     };
1584    
1585 root 1.241 Currently, C<aio_statvfs> always, and C<aio_rename> and C<aio_rmdir>
1586     sometimes, fall back to using an absolue path.
1587 root 1.209
1588 root 1.53 =head2 IO::AIO::REQ CLASS
1589 root 1.52
1590     All non-aggregate C<aio_*> functions return an object of this class when
1591     called in non-void context.
1592    
1593     =over 4
1594    
1595 root 1.65 =item cancel $req
1596 root 1.52
1597     Cancels the request, if possible. Has the effect of skipping execution
1598     when entering the B<execute> state and skipping calling the callback when
1599     entering the the B<result> state, but will leave the request otherwise
1600 root 1.151 untouched (with the exception of readdir). That means that requests that
1601     currently execute will not be stopped and resources held by the request
1602     will not be freed prematurely.
1603 root 1.52
1604 root 1.65 =item cb $req $callback->(...)
1605    
1606     Replace (or simply set) the callback registered to the request.
1607    
1608 root 1.52 =back
1609    
1610 root 1.55 =head2 IO::AIO::GRP CLASS
1611    
1612     This class is a subclass of L<IO::AIO::REQ>, so all its methods apply to
1613     objects of this class, too.
1614    
1615     A IO::AIO::GRP object is a special request that can contain multiple other
1616     aio requests.
1617    
1618     You create one by calling the C<aio_group> constructing function with a
1619     callback that will be called when all contained requests have entered the
1620     C<done> state:
1621    
1622     my $grp = aio_group sub {
1623     print "all requests are done\n";
1624     };
1625    
1626     You add requests by calling the C<add> method with one or more
1627     C<IO::AIO::REQ> objects:
1628    
1629     $grp->add (aio_unlink "...");
1630    
1631 root 1.58 add $grp aio_stat "...", sub {
1632     $_[0] or return $grp->result ("error");
1633    
1634     # add another request dynamically, if first succeeded
1635     add $grp aio_open "...", sub {
1636     $grp->result ("ok");
1637     };
1638     };
1639 root 1.55
1640     This makes it very easy to create composite requests (see the source of
1641     C<aio_move> for an application) that work and feel like simple requests.
1642    
1643 root 1.62 =over 4
1644    
1645     =item * The IO::AIO::GRP objects will be cleaned up during calls to
1646 root 1.55 C<IO::AIO::poll_cb>, just like any other request.
1647    
1648 root 1.62 =item * They can be canceled like any other request. Canceling will cancel not
1649 root 1.59 only the request itself, but also all requests it contains.
1650 root 1.55
1651 root 1.62 =item * They can also can also be added to other IO::AIO::GRP objects.
1652 root 1.55
1653 root 1.62 =item * You must not add requests to a group from within the group callback (or
1654 root 1.60 any later time).
1655    
1656 root 1.62 =back
1657    
1658 root 1.55 Their lifetime, simplified, looks like this: when they are empty, they
1659     will finish very quickly. If they contain only requests that are in the
1660     C<done> state, they will also finish. Otherwise they will continue to
1661     exist.
1662    
1663 root 1.133 That means after creating a group you have some time to add requests
1664     (precisely before the callback has been invoked, which is only done within
1665     the C<poll_cb>). And in the callbacks of those requests, you can add
1666     further requests to the group. And only when all those requests have
1667     finished will the the group itself finish.
1668 root 1.57
1669 root 1.55 =over 4
1670    
1671 root 1.65 =item add $grp ...
1672    
1673 root 1.55 =item $grp->add (...)
1674    
1675 root 1.57 Add one or more requests to the group. Any type of L<IO::AIO::REQ> can
1676     be added, including other groups, as long as you do not create circular
1677     dependencies.
1678    
1679     Returns all its arguments.
1680 root 1.55
1681 root 1.74 =item $grp->cancel_subs
1682    
1683     Cancel all subrequests and clears any feeder, but not the group request
1684     itself. Useful when you queued a lot of events but got a result early.
1685    
1686 root 1.168 The group request will finish normally (you cannot add requests to the
1687     group).
1688    
1689 root 1.58 =item $grp->result (...)
1690    
1691     Set the result value(s) that will be passed to the group callback when all
1692 root 1.120 subrequests have finished and set the groups errno to the current value
1693 root 1.80 of errno (just like calling C<errno> without an error number). By default,
1694     no argument will be passed and errno is zero.
1695    
1696     =item $grp->errno ([$errno])
1697    
1698     Sets the group errno value to C<$errno>, or the current value of errno
1699     when the argument is missing.
1700    
1701     Every aio request has an associated errno value that is restored when
1702     the callback is invoked. This method lets you change this value from its
1703     default (0).
1704    
1705     Calling C<result> will also set errno, so make sure you either set C<$!>
1706     before the call to C<result>, or call c<errno> after it.
1707 root 1.58
1708 root 1.65 =item feed $grp $callback->($grp)
1709 root 1.60
1710     Sets a feeder/generator on this group: every group can have an attached
1711     generator that generates requests if idle. The idea behind this is that,
1712     although you could just queue as many requests as you want in a group,
1713 root 1.139 this might starve other requests for a potentially long time. For example,
1714 root 1.211 C<aio_scandir> might generate hundreds of thousands of C<aio_stat>
1715     requests, delaying any later requests for a long time.
1716 root 1.60
1717     To avoid this, and allow incremental generation of requests, you can
1718     instead a group and set a feeder on it that generates those requests. The
1719 root 1.68 feed callback will be called whenever there are few enough (see C<limit>,
1720 root 1.60 below) requests active in the group itself and is expected to queue more
1721     requests.
1722    
1723 root 1.68 The feed callback can queue as many requests as it likes (i.e. C<add> does
1724     not impose any limits).
1725 root 1.60
1726 root 1.65 If the feed does not queue more requests when called, it will be
1727 root 1.60 automatically removed from the group.
1728    
1729 root 1.138 If the feed limit is C<0> when this method is called, it will be set to
1730     C<2> automatically.
1731 root 1.60
1732     Example:
1733    
1734     # stat all files in @files, but only ever use four aio requests concurrently:
1735    
1736     my $grp = aio_group sub { print "finished\n" };
1737 root 1.68 limit $grp 4;
1738 root 1.65 feed $grp sub {
1739 root 1.60 my $file = pop @files
1740     or return;
1741    
1742     add $grp aio_stat $file, sub { ... };
1743 root 1.65 };
1744 root 1.60
1745 root 1.68 =item limit $grp $num
1746 root 1.60
1747     Sets the feeder limit for the group: The feeder will be called whenever
1748     the group contains less than this many requests.
1749    
1750     Setting the limit to C<0> will pause the feeding process.
1751    
1752 root 1.138 The default value for the limit is C<0>, but note that setting a feeder
1753     automatically bumps it up to C<2>.
1754    
1755 root 1.55 =back
1756    
1757 root 1.5 =head2 SUPPORT FUNCTIONS
1758    
1759 root 1.86 =head3 EVENT PROCESSING AND EVENT LOOP INTEGRATION
1760    
1761 root 1.5 =over 4
1762    
1763     =item $fileno = IO::AIO::poll_fileno
1764    
1765 root 1.20 Return the I<request result pipe file descriptor>. This filehandle must be
1766 root 1.156 polled for reading by some mechanism outside this module (e.g. EV, Glib,
1767     select and so on, see below or the SYNOPSIS). If the pipe becomes readable
1768     you have to call C<poll_cb> to check the results.
1769 root 1.5
1770     See C<poll_cb> for an example.
1771    
1772     =item IO::AIO::poll_cb
1773    
1774 root 1.240 Process some requests that have reached the result phase (i.e. they have
1775     been executed but the results are not yet reported). You have to call
1776     this "regularly" to finish outstanding requests.
1777    
1778     Returns C<0> if all events could be processed (or there were no
1779     events to process), or C<-1> if it returned earlier for whatever
1780     reason. Returns immediately when no events are outstanding. The amount
1781     of events processed depends on the settings of C<IO::AIO::max_poll_req>,
1782     C<IO::AIO::max_poll_time> and C<IO::AIO::max_outstanding>.
1783    
1784     If not all requests were processed for whatever reason, the poll file
1785     descriptor will still be ready when C<poll_cb> returns, so normally you
1786     don't have to do anything special to have it called later.
1787 root 1.78
1788 root 1.192 Apart from calling C<IO::AIO::poll_cb> when the event filehandle becomes
1789     ready, it can be beneficial to call this function from loops which submit
1790     a lot of requests, to make sure the results get processed when they become
1791     available and not just when the loop is finished and the event loop takes
1792     over again. This function returns very fast when there are no outstanding
1793     requests.
1794    
1795 root 1.20 Example: Install an Event watcher that automatically calls
1796 root 1.156 IO::AIO::poll_cb with high priority (more examples can be found in the
1797     SYNOPSIS section, at the top of this document):
1798 root 1.5
1799     Event->io (fd => IO::AIO::poll_fileno,
1800     poll => 'r', async => 1,
1801     cb => \&IO::AIO::poll_cb);
1802    
1803 root 1.175 =item IO::AIO::poll_wait
1804    
1805 root 1.240 Wait until either at least one request is in the result phase or no
1806     requests are outstanding anymore.
1807    
1808     This is useful if you want to synchronously wait for some requests to
1809     become ready, without actually handling them.
1810 root 1.175
1811     See C<nreqs> for an example.
1812    
1813     =item IO::AIO::poll
1814    
1815     Waits until some requests have been handled.
1816    
1817     Returns the number of requests processed, but is otherwise strictly
1818     equivalent to:
1819    
1820     IO::AIO::poll_wait, IO::AIO::poll_cb
1821    
1822     =item IO::AIO::flush
1823    
1824     Wait till all outstanding AIO requests have been handled.
1825    
1826     Strictly equivalent to:
1827    
1828     IO::AIO::poll_wait, IO::AIO::poll_cb
1829     while IO::AIO::nreqs;
1830    
1831 root 1.86 =item IO::AIO::max_poll_reqs $nreqs
1832    
1833     =item IO::AIO::max_poll_time $seconds
1834    
1835     These set the maximum number of requests (default C<0>, meaning infinity)
1836     that are being processed by C<IO::AIO::poll_cb> in one call, respectively
1837     the maximum amount of time (default C<0>, meaning infinity) spent in
1838     C<IO::AIO::poll_cb> to process requests (more correctly the mininum amount
1839     of time C<poll_cb> is allowed to use).
1840 root 1.78
1841 root 1.89 Setting C<max_poll_time> to a non-zero value creates an overhead of one
1842     syscall per request processed, which is not normally a problem unless your
1843     callbacks are really really fast or your OS is really really slow (I am
1844     not mentioning Solaris here). Using C<max_poll_reqs> incurs no overhead.
1845    
1846 root 1.86 Setting these is useful if you want to ensure some level of
1847     interactiveness when perl is not fast enough to process all requests in
1848     time.
1849 root 1.78
1850 root 1.86 For interactive programs, values such as C<0.01> to C<0.1> should be fine.
1851 root 1.78
1852     Example: Install an Event watcher that automatically calls
1853 root 1.89 IO::AIO::poll_cb with low priority, to ensure that other parts of the
1854 root 1.78 program get the CPU sometimes even under high AIO load.
1855    
1856 root 1.86 # try not to spend much more than 0.1s in poll_cb
1857     IO::AIO::max_poll_time 0.1;
1858    
1859     # use a low priority so other tasks have priority
1860 root 1.78 Event->io (fd => IO::AIO::poll_fileno,
1861     poll => 'r', nice => 1,
1862 root 1.86 cb => &IO::AIO::poll_cb);
1863 root 1.78
1864 root 1.104 =back
1865    
1866 root 1.86 =head3 CONTROLLING THE NUMBER OF THREADS
1867 root 1.13
1868 root 1.105 =over
1869    
1870 root 1.5 =item IO::AIO::min_parallel $nthreads
1871    
1872 root 1.61 Set the minimum number of AIO threads to C<$nthreads>. The current
1873     default is C<8>, which means eight asynchronous operations can execute
1874     concurrently at any one time (the number of outstanding requests,
1875     however, is unlimited).
1876 root 1.5
1877 root 1.34 IO::AIO starts threads only on demand, when an AIO request is queued and
1878 root 1.86 no free thread exists. Please note that queueing up a hundred requests can
1879     create demand for a hundred threads, even if it turns out that everything
1880     is in the cache and could have been processed faster by a single thread.
1881 root 1.34
1882 root 1.61 It is recommended to keep the number of threads relatively low, as some
1883     Linux kernel versions will scale negatively with the number of threads
1884     (higher parallelity => MUCH higher latency). With current Linux 2.6
1885     versions, 4-32 threads should be fine.
1886 root 1.5
1887 root 1.34 Under most circumstances you don't need to call this function, as the
1888     module selects a default that is suitable for low to moderate load.
1889 root 1.5
1890     =item IO::AIO::max_parallel $nthreads
1891    
1892 root 1.34 Sets the maximum number of AIO threads to C<$nthreads>. If more than the
1893     specified number of threads are currently running, this function kills
1894     them. This function blocks until the limit is reached.
1895    
1896     While C<$nthreads> are zero, aio requests get queued but not executed
1897     until the number of threads has been increased again.
1898 root 1.5
1899     This module automatically runs C<max_parallel 0> at program end, to ensure
1900     that all threads are killed and that there are no outstanding requests.
1901    
1902     Under normal circumstances you don't need to call this function.
1903    
1904 root 1.86 =item IO::AIO::max_idle $nthreads
1905    
1906 root 1.188 Limit the number of threads (default: 4) that are allowed to idle
1907     (i.e., threads that did not get a request to process within the idle
1908     timeout (default: 10 seconds). That means if a thread becomes idle while
1909     C<$nthreads> other threads are also idle, it will free its resources and
1910     exit.
1911 root 1.86
1912     This is useful when you allow a large number of threads (e.g. 100 or 1000)
1913     to allow for extremely high load situations, but want to free resources
1914     under normal circumstances (1000 threads can easily consume 30MB of RAM).
1915    
1916     The default is probably ok in most situations, especially if thread
1917     creation is fast. If thread creation is very slow on your system you might
1918     want to use larger values.
1919    
1920 root 1.188 =item IO::AIO::idle_timeout $seconds
1921    
1922     Sets the minimum idle timeout (default 10) after which worker threads are
1923     allowed to exit. SEe C<IO::AIO::max_idle>.
1924    
1925 root 1.123 =item IO::AIO::max_outstanding $maxreqs
1926 root 1.5
1927 root 1.195 Sets the maximum number of outstanding requests to C<$nreqs>. If
1928     you do queue up more than this number of requests, the next call to
1929     C<IO::AIO::poll_cb> (and other functions calling C<poll_cb>, such as
1930     C<IO::AIO::flush> or C<IO::AIO::poll>) will block until the limit is no
1931     longer exceeded.
1932    
1933     In other words, this setting does not enforce a queue limit, but can be
1934     used to make poll functions block if the limit is exceeded.
1935    
1936 root 1.79 This is a very bad function to use in interactive programs because it
1937     blocks, and a bad way to reduce concurrency because it is inexact: Better
1938     use an C<aio_group> together with a feed callback.
1939    
1940 root 1.195 It's main use is in scripts without an event loop - when you want to stat
1941     a lot of files, you can write somehting like this:
1942    
1943     IO::AIO::max_outstanding 32;
1944    
1945     for my $path (...) {
1946     aio_stat $path , ...;
1947     IO::AIO::poll_cb;
1948     }
1949    
1950     IO::AIO::flush;
1951    
1952     The call to C<poll_cb> inside the loop will normally return instantly, but
1953     as soon as more thna C<32> reqeusts are in-flight, it will block until
1954     some requests have been handled. This keeps the loop from pushing a large
1955     number of C<aio_stat> requests onto the queue.
1956    
1957     The default value for C<max_outstanding> is very large, so there is no
1958     practical limit on the number of outstanding requests.
1959 root 1.5
1960 root 1.104 =back
1961    
1962 root 1.86 =head3 STATISTICAL INFORMATION
1963    
1964 root 1.104 =over
1965    
1966 root 1.86 =item IO::AIO::nreqs
1967    
1968     Returns the number of requests currently in the ready, execute or pending
1969     states (i.e. for which their callback has not been invoked yet).
1970    
1971     Example: wait till there are no outstanding requests anymore:
1972    
1973     IO::AIO::poll_wait, IO::AIO::poll_cb
1974     while IO::AIO::nreqs;
1975    
1976     =item IO::AIO::nready
1977    
1978     Returns the number of requests currently in the ready state (not yet
1979     executed).
1980    
1981     =item IO::AIO::npending
1982    
1983     Returns the number of requests currently in the pending state (executed,
1984     but not yet processed by poll_cb).
1985    
1986 root 1.5 =back
1987    
1988 root 1.157 =head3 MISCELLANEOUS FUNCTIONS
1989    
1990     IO::AIO implements some functions that might be useful, but are not
1991     asynchronous.
1992    
1993     =over 4
1994    
1995     =item IO::AIO::sendfile $ofh, $ifh, $offset, $count
1996    
1997     Calls the C<eio_sendfile_sync> function, which is like C<aio_sendfile>,
1998     but is blocking (this makes most sense if you know the input data is
1999     likely cached already and the output filehandle is set to non-blocking
2000     operations).
2001    
2002     Returns the number of bytes copied, or C<-1> on error.
2003    
2004     =item IO::AIO::fadvise $fh, $offset, $len, $advice
2005    
2006 root 1.184 Simply calls the C<posix_fadvise> function (see its
2007 root 1.157 manpage for details). The following advice constants are
2008 root 1.207 available: C<IO::AIO::FADV_NORMAL>, C<IO::AIO::FADV_SEQUENTIAL>,
2009 root 1.157 C<IO::AIO::FADV_RANDOM>, C<IO::AIO::FADV_NOREUSE>,
2010     C<IO::AIO::FADV_WILLNEED>, C<IO::AIO::FADV_DONTNEED>.
2011    
2012     On systems that do not implement C<posix_fadvise>, this function returns
2013     ENOSYS, otherwise the return value of C<posix_fadvise>.
2014    
2015 root 1.184 =item IO::AIO::madvise $scalar, $offset, $len, $advice
2016    
2017     Simply calls the C<posix_madvise> function (see its
2018     manpage for details). The following advice constants are
2019 root 1.207 available: C<IO::AIO::MADV_NORMAL>, C<IO::AIO::MADV_SEQUENTIAL>,
2020 root 1.184 C<IO::AIO::MADV_RANDOM>, C<IO::AIO::MADV_WILLNEED>, C<IO::AIO::MADV_DONTNEED>.
2021    
2022     On systems that do not implement C<posix_madvise>, this function returns
2023     ENOSYS, otherwise the return value of C<posix_madvise>.
2024    
2025     =item IO::AIO::mprotect $scalar, $offset, $len, $protect
2026    
2027     Simply calls the C<mprotect> function on the preferably AIO::mmap'ed
2028     $scalar (see its manpage for details). The following protect
2029 root 1.207 constants are available: C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_READ>,
2030 root 1.184 C<IO::AIO::PROT_WRITE>, C<IO::AIO::PROT_EXEC>.
2031    
2032     On systems that do not implement C<mprotect>, this function returns
2033     ENOSYS, otherwise the return value of C<mprotect>.
2034    
2035 root 1.176 =item IO::AIO::mmap $scalar, $length, $prot, $flags, $fh[, $offset]
2036    
2037     Memory-maps a file (or anonymous memory range) and attaches it to the
2038 root 1.228 given C<$scalar>, which will act like a string scalar. Returns true on
2039     success, and false otherwise.
2040 root 1.176
2041     The only operations allowed on the scalar are C<substr>/C<vec> that don't
2042     change the string length, and most read-only operations such as copying it
2043     or searching it with regexes and so on.
2044    
2045     Anything else is unsafe and will, at best, result in memory leaks.
2046    
2047     The memory map associated with the C<$scalar> is automatically removed
2048     when the C<$scalar> is destroyed, or when the C<IO::AIO::mmap> or
2049     C<IO::AIO::munmap> functions are called.
2050    
2051     This calls the C<mmap>(2) function internally. See your system's manual
2052     page for details on the C<$length>, C<$prot> and C<$flags> parameters.
2053    
2054     The C<$length> must be larger than zero and smaller than the actual
2055     filesize.
2056    
2057     C<$prot> is a combination of C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_EXEC>,
2058     C<IO::AIO::PROT_READ> and/or C<IO::AIO::PROT_WRITE>,
2059    
2060     C<$flags> can be a combination of C<IO::AIO::MAP_SHARED> or
2061     C<IO::AIO::MAP_PRIVATE>, or a number of system-specific flags (when
2062     not available, the are defined as 0): C<IO::AIO::MAP_ANONYMOUS>
2063     (which is set to C<MAP_ANON> if your system only provides this
2064     constant), C<IO::AIO::MAP_HUGETLB>, C<IO::AIO::MAP_LOCKED>,
2065     C<IO::AIO::MAP_NORESERVE>, C<IO::AIO::MAP_POPULATE> or
2066     C<IO::AIO::MAP_NONBLOCK>
2067    
2068     If C<$fh> is C<undef>, then a file descriptor of C<-1> is passed.
2069    
2070 root 1.179 C<$offset> is the offset from the start of the file - it generally must be
2071     a multiple of C<IO::AIO::PAGESIZE> and defaults to C<0>.
2072    
2073 root 1.177 Example:
2074    
2075     use Digest::MD5;
2076     use IO::AIO;
2077    
2078     open my $fh, "<verybigfile"
2079     or die "$!";
2080    
2081     IO::AIO::mmap my $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh
2082     or die "verybigfile: $!";
2083    
2084     my $fast_md5 = md5 $data;
2085    
2086 root 1.176 =item IO::AIO::munmap $scalar
2087    
2088     Removes a previous mmap and undefines the C<$scalar>.
2089    
2090 root 1.182 =item IO::AIO::munlock $scalar, $offset = 0, $length = undef
2091 root 1.174
2092 root 1.182 Calls the C<munlock> function, undoing the effects of a previous
2093     C<aio_mlock> call (see its description for details).
2094 root 1.174
2095     =item IO::AIO::munlockall
2096    
2097     Calls the C<munlockall> function.
2098    
2099     On systems that do not implement C<munlockall>, this function returns
2100     ENOSYS, otherwise the return value of C<munlockall>.
2101    
2102 root 1.225 =item IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
2103    
2104     Calls the GNU/Linux C<splice(2)> syscall, if available. If C<$r_off> or
2105     C<$w_off> are C<undef>, then C<NULL> is passed for these, otherwise they
2106     should be the file offset.
2107    
2108 root 1.227 C<$r_fh> and C<$w_fh> should not refer to the same file, as splice might
2109     silently corrupt the data in this case.
2110    
2111 root 1.225 The following symbol flag values are available: C<IO::AIO::SPLICE_F_MOVE>,
2112     C<IO::AIO::SPLICE_F_NONBLOCK>, C<IO::AIO::SPLICE_F_MORE> and
2113     C<IO::AIO::SPLICE_F_GIFT>.
2114    
2115     See the C<splice(2)> manpage for details.
2116    
2117     =item IO::AIO::tee $r_fh, $w_fh, $length, $flags
2118    
2119     Calls the GNU/Linux C<tee(2)> syscall, see it's manpage and the
2120     description for C<IO::AIO::splice> above for details.
2121    
2122 root 1.157 =back
2123    
2124 root 1.1 =cut
2125    
2126 root 1.61 min_parallel 8;
2127 root 1.1
2128 root 1.95 END { flush }
2129 root 1.82
2130 root 1.1 1;
2131    
2132 root 1.175 =head1 EVENT LOOP INTEGRATION
2133    
2134     It is recommended to use L<AnyEvent::AIO> to integrate IO::AIO
2135     automatically into many event loops:
2136    
2137     # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
2138     use AnyEvent::AIO;
2139    
2140     You can also integrate IO::AIO manually into many event loops, here are
2141     some examples of how to do this:
2142    
2143     # EV integration
2144     my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
2145    
2146     # Event integration
2147     Event->io (fd => IO::AIO::poll_fileno,
2148     poll => 'r',
2149     cb => \&IO::AIO::poll_cb);
2150    
2151     # Glib/Gtk2 integration
2152     add_watch Glib::IO IO::AIO::poll_fileno,
2153     in => sub { IO::AIO::poll_cb; 1 };
2154    
2155     # Tk integration
2156     Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
2157     readable => \&IO::AIO::poll_cb);
2158    
2159     # Danga::Socket integration
2160     Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
2161     \&IO::AIO::poll_cb);
2162    
2163 root 1.27 =head2 FORK BEHAVIOUR
2164    
2165 root 1.197 Usage of pthreads in a program changes the semantics of fork
2166     considerably. Specifically, only async-safe functions can be called after
2167     fork. Perl doesn't know about this, so in general, you cannot call fork
2168 root 1.204 with defined behaviour in perl if pthreads are involved. IO::AIO uses
2169     pthreads, so this applies, but many other extensions and (for inexplicable
2170     reasons) perl itself often is linked against pthreads, so this limitation
2171     applies to quite a lot of perls.
2172    
2173     This module no longer tries to fight your OS, or POSIX. That means IO::AIO
2174     only works in the process that loaded it. Forking is fully supported, but
2175     using IO::AIO in the child is not.
2176    
2177     You might get around by not I<using> IO::AIO before (or after)
2178     forking. You could also try to call the L<IO::AIO::reinit> function in the
2179     child:
2180    
2181     =over 4
2182    
2183     =item IO::AIO::reinit
2184    
2185 root 1.207 Abandons all current requests and I/O threads and simply reinitialises all
2186     data structures. This is not an operation supported by any standards, but
2187 root 1.204 happens to work on GNU/Linux and some newer BSD systems.
2188    
2189     The only reasonable use for this function is to call it after forking, if
2190     C<IO::AIO> was used in the parent. Calling it while IO::AIO is active in
2191     the process will result in undefined behaviour. Calling it at any time
2192     will also result in any undefined (by POSIX) behaviour.
2193    
2194     =back
2195 root 1.52
2196 root 1.60 =head2 MEMORY USAGE
2197    
2198 root 1.72 Per-request usage:
2199    
2200     Each aio request uses - depending on your architecture - around 100-200
2201     bytes of memory. In addition, stat requests need a stat buffer (possibly
2202     a few hundred bytes), readdir requires a result buffer and so on. Perl
2203     scalars and other data passed into aio requests will also be locked and
2204     will consume memory till the request has entered the done state.
2205 root 1.60
2206 root 1.111 This is not awfully much, so queuing lots of requests is not usually a
2207 root 1.60 problem.
2208    
2209 root 1.72 Per-thread usage:
2210    
2211     In the execution phase, some aio requests require more memory for
2212     temporary buffers, and each thread requires a stack and other data
2213     structures (usually around 16k-128k, depending on the OS).
2214    
2215     =head1 KNOWN BUGS
2216    
2217 root 1.73 Known bugs will be fixed in the next release.
2218 root 1.60
2219 root 1.1 =head1 SEE ALSO
2220    
2221 root 1.125 L<AnyEvent::AIO> for easy integration into event loops, L<Coro::AIO> for a
2222     more natural syntax.
2223 root 1.1
2224     =head1 AUTHOR
2225    
2226     Marc Lehmann <schmorp@schmorp.de>
2227     http://home.schmorp.de/
2228    
2229     =cut
2230