ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
Revision: 1.248
Committed: Sun Jul 27 22:10:53 2014 UTC (9 years, 9 months ago) by root
Branch: MAIN
Changes since 1.247: +6 -4 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.1 =head1 NAME
2    
3     IO::AIO - Asynchronous Input/Output
4    
5     =head1 SYNOPSIS
6    
7     use IO::AIO;
8    
9 root 1.181 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
10 root 1.94 my $fh = shift
11     or die "/etc/passwd: $!";
12 root 1.6 ...
13     };
14    
15     aio_unlink "/tmp/file", sub { };
16    
17     aio_read $fh, 30000, 1024, $buffer, 0, sub {
18 root 1.8 $_[0] > 0 or die "read error: $!";
19 root 1.6 };
20    
21 root 1.56 # version 2+ has request and group objects
22     use IO::AIO 2;
23 root 1.52
24 root 1.68 aioreq_pri 4; # give next request a very high priority
25 root 1.52 my $req = aio_unlink "/tmp/file", sub { };
26     $req->cancel; # cancel request if still in queue
27    
28 root 1.56 my $grp = aio_group sub { print "all stats done\n" };
29     add $grp aio_stat "..." for ...;
30    
31 root 1.1 =head1 DESCRIPTION
32    
33     This module implements asynchronous I/O using whatever means your
34 root 1.156 operating system supports. It is implemented as an interface to C<libeio>
35     (L<http://software.schmorp.de/pkg/libeio.html>).
36 root 1.1
37 root 1.85 Asynchronous means that operations that can normally block your program
38     (e.g. reading from disk) will be done asynchronously: the operation
39     will still block, but you can do something else in the meantime. This
40     is extremely useful for programs that need to stay interactive even
41     when doing heavy I/O (GUI programs, high performance network servers
42     etc.), but can also be used to easily do operations in parallel that are
43     normally done sequentially, e.g. stat'ing many files, which is much faster
44     on a RAID volume or over NFS when you do a number of stat operations
45     concurrently.
46    
47 root 1.108 While most of this works on all types of file descriptors (for
48     example sockets), using these functions on file descriptors that
49 root 1.156 support nonblocking operation (again, sockets, pipes etc.) is
50     very inefficient. Use an event loop for that (such as the L<EV>
51 root 1.108 module): IO::AIO will naturally fit into such an event loop itself.
52 root 1.85
53 root 1.72 In this version, a number of threads are started that execute your
54     requests and signal their completion. You don't need thread support
55     in perl, and the threads created by this module will not be visible
56     to perl. In the future, this module might make use of the native aio
57     functions available on many operating systems. However, they are often
58 root 1.85 not well-supported or restricted (GNU/Linux doesn't allow them on normal
59 root 1.72 files currently, for example), and they would only support aio_read and
60     aio_write, so the remaining functionality would have to be implemented
61     using threads anyway.
62    
63 root 1.108 Although the module will work in the presence of other (Perl-) threads,
64     it is currently not reentrant in any way, so use appropriate locking
65     yourself, always call C<poll_cb> from within the same thread, or never
66     call C<poll_cb> (or other C<aio_> functions) recursively.
67 root 1.72
68 root 1.86 =head2 EXAMPLE
69    
70 root 1.156 This is a simple example that uses the EV module and loads
71 root 1.86 F</etc/passwd> asynchronously:
72    
73 root 1.156 use EV;
74 root 1.86 use IO::AIO;
75    
76 root 1.156 # register the IO::AIO callback with EV
77     my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
78 root 1.86
79     # queue the request to open /etc/passwd
80 root 1.181 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
81 root 1.94 my $fh = shift
82 root 1.86 or die "error while opening: $!";
83    
84     # stat'ing filehandles is generally non-blocking
85     my $size = -s $fh;
86    
87     # queue a request to read the file
88     my $contents;
89     aio_read $fh, 0, $size, $contents, 0, sub {
90     $_[0] == $size
91     or die "short read: $!";
92    
93     close $fh;
94    
95     # file contents now in $contents
96     print $contents;
97    
98     # exit event loop and program
99 root 1.156 EV::unloop;
100 root 1.86 };
101     };
102    
103     # possibly queue up other requests, or open GUI windows,
104     # check for sockets etc. etc.
105    
106     # process events as long as there are some:
107 root 1.156 EV::loop;
108 root 1.86
109 root 1.72 =head1 REQUEST ANATOMY AND LIFETIME
110    
111     Every C<aio_*> function creates a request. which is a C data structure not
112     directly visible to Perl.
113    
114     If called in non-void context, every request function returns a Perl
115     object representing the request. In void context, nothing is returned,
116     which saves a bit of memory.
117    
118     The perl object is a fairly standard ref-to-hash object. The hash contents
119     are not used by IO::AIO so you are free to store anything you like in it.
120    
121     During their existance, aio requests travel through the following states,
122     in order:
123    
124     =over 4
125    
126     =item ready
127    
128     Immediately after a request is created it is put into the ready state,
129     waiting for a thread to execute it.
130    
131     =item execute
132    
133     A thread has accepted the request for processing and is currently
134     executing it (e.g. blocking in read).
135    
136     =item pending
137    
138     The request has been executed and is waiting for result processing.
139    
140     While request submission and execution is fully asynchronous, result
141     processing is not and relies on the perl interpreter calling C<poll_cb>
142     (or another function with the same effect).
143    
144     =item result
145    
146     The request results are processed synchronously by C<poll_cb>.
147    
148     The C<poll_cb> function will process all outstanding aio requests by
149     calling their callbacks, freeing memory associated with them and managing
150     any groups they are contained in.
151    
152     =item done
153    
154     Request has reached the end of its lifetime and holds no resources anymore
155     (except possibly for the Perl object, but its connection to the actual
156     aio request is severed and calling its methods will either do nothing or
157     result in a runtime error).
158 root 1.1
159 root 1.88 =back
160    
161 root 1.1 =cut
162    
163     package IO::AIO;
164    
165 root 1.117 use Carp ();
166    
167 root 1.161 use common::sense;
168 root 1.23
169 root 1.1 use base 'Exporter';
170    
171     BEGIN {
172 root 1.247 our $VERSION = 4.31;
173 root 1.1
174 root 1.220 our @AIO_REQ = qw(aio_sendfile aio_seek aio_read aio_write aio_open aio_close
175 root 1.148 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
176 root 1.206 aio_scandir aio_symlink aio_readlink aio_realpath aio_sync
177 root 1.236 aio_fsync aio_syncfs aio_fdatasync aio_sync_file_range aio_allocate
178 root 1.222 aio_pathsync aio_readahead aio_fiemap
179 root 1.120 aio_rename aio_link aio_move aio_copy aio_group
180     aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
181 root 1.170 aio_chmod aio_utime aio_truncate
182 root 1.182 aio_msync aio_mtouch aio_mlock aio_mlockall
183 root 1.208 aio_statvfs
184     aio_wd);
185 root 1.120
186 root 1.123 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice));
187 root 1.67 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
188 root 1.188 min_parallel max_parallel max_idle idle_timeout
189 root 1.86 nreqs nready npending nthreads
190 root 1.157 max_poll_time max_poll_reqs
191 root 1.182 sendfile fadvise madvise
192     mmap munmap munlock munlockall);
193 root 1.1
194 root 1.143 push @AIO_REQ, qw(aio_busy); # not exported
195    
196 root 1.54 @IO::AIO::GRP::ISA = 'IO::AIO::REQ';
197    
198 root 1.1 require XSLoader;
199 root 1.51 XSLoader::load ("IO::AIO", $VERSION);
200 root 1.1 }
201    
202 root 1.5 =head1 FUNCTIONS
203 root 1.1
204 root 1.175 =head2 QUICK OVERVIEW
205    
206 root 1.230 This section simply lists the prototypes most of the functions for
207     quick reference. See the following sections for function-by-function
208 root 1.175 documentation.
209    
210 root 1.208 aio_wd $pathname, $callback->($wd)
211 root 1.175 aio_open $pathname, $flags, $mode, $callback->($fh)
212     aio_close $fh, $callback->($status)
213 root 1.220 aio_seek $fh,$offset,$whence, $callback->($offs)
214 root 1.175 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
215     aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
216     aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
217     aio_readahead $fh,$offset,$length, $callback->($retval)
218     aio_stat $fh_or_path, $callback->($status)
219     aio_lstat $fh, $callback->($status)
220     aio_statvfs $fh_or_path, $callback->($statvfs)
221     aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
222     aio_chown $fh_or_path, $uid, $gid, $callback->($status)
223 root 1.220 aio_chmod $fh_or_path, $mode, $callback->($status)
224 root 1.175 aio_truncate $fh_or_path, $offset, $callback->($status)
225 root 1.229 aio_allocate $fh, $mode, $offset, $len, $callback->($status)
226 root 1.230 aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
227 root 1.175 aio_unlink $pathname, $callback->($status)
228 root 1.209 aio_mknod $pathname, $mode, $dev, $callback->($status)
229 root 1.175 aio_link $srcpath, $dstpath, $callback->($status)
230     aio_symlink $srcpath, $dstpath, $callback->($status)
231 root 1.209 aio_readlink $pathname, $callback->($link)
232     aio_realpath $pathname, $callback->($link)
233 root 1.175 aio_rename $srcpath, $dstpath, $callback->($status)
234     aio_mkdir $pathname, $mode, $callback->($status)
235     aio_rmdir $pathname, $callback->($status)
236     aio_readdir $pathname, $callback->($entries)
237     aio_readdirx $pathname, $flags, $callback->($entries, $flags)
238     IO::AIO::READDIR_DENTS IO::AIO::READDIR_DIRS_FIRST
239     IO::AIO::READDIR_STAT_ORDER IO::AIO::READDIR_FOUND_UNKNOWN
240 root 1.215 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
241 root 1.209 aio_load $pathname, $data, $callback->($status)
242 root 1.175 aio_copy $srcpath, $dstpath, $callback->($status)
243     aio_move $srcpath, $dstpath, $callback->($status)
244 root 1.209 aio_rmtree $pathname, $callback->($status)
245 root 1.175 aio_sync $callback->($status)
246 root 1.206 aio_syncfs $fh, $callback->($status)
247 root 1.175 aio_fsync $fh, $callback->($status)
248     aio_fdatasync $fh, $callback->($status)
249     aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
250 root 1.209 aio_pathsync $pathname, $callback->($status)
251 root 1.175 aio_msync $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
252     aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
253 root 1.182 aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
254     aio_mlockall $flags, $callback->($status)
255 root 1.175 aio_group $callback->(...)
256     aio_nop $callback->()
257    
258     $prev_pri = aioreq_pri [$pri]
259     aioreq_nice $pri_adjust
260    
261     IO::AIO::poll_wait
262     IO::AIO::poll_cb
263     IO::AIO::poll
264     IO::AIO::flush
265     IO::AIO::max_poll_reqs $nreqs
266     IO::AIO::max_poll_time $seconds
267     IO::AIO::min_parallel $nthreads
268     IO::AIO::max_parallel $nthreads
269     IO::AIO::max_idle $nthreads
270 root 1.188 IO::AIO::idle_timeout $seconds
271 root 1.175 IO::AIO::max_outstanding $maxreqs
272     IO::AIO::nreqs
273     IO::AIO::nready
274     IO::AIO::npending
275    
276     IO::AIO::sendfile $ofh, $ifh, $offset, $count
277     IO::AIO::fadvise $fh, $offset, $len, $advice
278 root 1.226 IO::AIO::mmap $scalar, $length, $prot, $flags[, $fh[, $offset]]
279     IO::AIO::munmap $scalar
280 root 1.184 IO::AIO::madvise $scalar, $offset, $length, $advice
281     IO::AIO::mprotect $scalar, $offset, $length, $protect
282 root 1.182 IO::AIO::munlock $scalar, $offset = 0, $length = undef
283 root 1.175 IO::AIO::munlockall
284    
285 root 1.219 =head2 API NOTES
286 root 1.1
287 root 1.5 All the C<aio_*> calls are more or less thin wrappers around the syscall
288     with the same name (sans C<aio_>). The arguments are similar or identical,
289 root 1.14 and they all accept an additional (and optional) C<$callback> argument
290 root 1.212 which must be a code reference. This code reference will be called after
291     the syscall has been executed in an asynchronous fashion. The results
292     of the request will be passed as arguments to the callback (and, if an
293     error occured, in C<$!>) - for most requests the syscall return code (e.g.
294     most syscalls return C<-1> on error, unlike perl, which usually delivers
295     "false").
296    
297     Some requests (such as C<aio_readdir>) pass the actual results and
298     communicate failures by passing C<undef>.
299 root 1.1
300 root 1.23 All functions expecting a filehandle keep a copy of the filehandle
301     internally until the request has finished.
302 root 1.1
303 root 1.87 All functions return request objects of type L<IO::AIO::REQ> that allow
304     further manipulation of those requests while they are in-flight.
305 root 1.52
306 root 1.209 The pathnames you pass to these routines I<should> be absolute. The
307     reason for this is that at the time the request is being executed, the
308 root 1.212 current working directory could have changed. Alternatively, you can
309     make sure that you never change the current working directory anywhere
310     in the program and then use relative paths. You can also take advantage
311     of IO::AIOs working directory abstraction, that lets you specify paths
312     relative to some previously-opened "working directory object" - see the
313     description of the C<IO::AIO::WD> class later in this document.
314 root 1.28
315 root 1.87 To encode pathnames as octets, either make sure you either: a) always pass
316     in filenames you got from outside (command line, readdir etc.) without
317 root 1.212 tinkering, b) are in your native filesystem encoding, c) use the Encode
318     module and encode your pathnames to the locale (or other) encoding in
319     effect in the user environment, d) use Glib::filename_from_unicode on
320     unicode filenames or e) use something else to ensure your scalar has the
321     correct contents.
322 root 1.87
323     This works, btw. independent of the internal UTF-8 bit, which IO::AIO
324 root 1.136 handles correctly whether it is set or not.
325 root 1.1
326 root 1.219 =head2 AIO REQUEST FUNCTIONS
327    
328 root 1.5 =over 4
329 root 1.1
330 root 1.80 =item $prev_pri = aioreq_pri [$pri]
331 root 1.68
332 root 1.80 Returns the priority value that would be used for the next request and, if
333     C<$pri> is given, sets the priority for the next aio request.
334 root 1.68
335 root 1.80 The default priority is C<0>, the minimum and maximum priorities are C<-4>
336     and C<4>, respectively. Requests with higher priority will be serviced
337     first.
338    
339     The priority will be reset to C<0> after each call to one of the C<aio_*>
340 root 1.68 functions.
341    
342 root 1.69 Example: open a file with low priority, then read something from it with
343     higher priority so the read request is serviced before other low priority
344     open requests (potentially spamming the cache):
345    
346     aioreq_pri -3;
347     aio_open ..., sub {
348     return unless $_[0];
349    
350     aioreq_pri -2;
351     aio_read $_[0], ..., sub {
352     ...
353     };
354     };
355    
356 root 1.106
357 root 1.69 =item aioreq_nice $pri_adjust
358    
359     Similar to C<aioreq_pri>, but subtracts the given value from the current
360 root 1.87 priority, so the effect is cumulative.
361 root 1.69
362 root 1.106
363 root 1.40 =item aio_open $pathname, $flags, $mode, $callback->($fh)
364 root 1.1
365 root 1.2 Asynchronously open or create a file and call the callback with a newly
366 root 1.233 created filehandle for the file (or C<undef> in case of an error).
367 root 1.1
368     The pathname passed to C<aio_open> must be absolute. See API NOTES, above,
369     for an explanation.
370    
371 root 1.20 The C<$flags> argument is a bitmask. See the C<Fcntl> module for a
372     list. They are the same as used by C<sysopen>.
373    
374     Likewise, C<$mode> specifies the mode of the newly created file, if it
375     didn't exist and C<O_CREAT> has been given, just like perl's C<sysopen>,
376     except that it is mandatory (i.e. use C<0> if you don't create new files,
377 root 1.101 and C<0666> or C<0777> if you do). Note that the C<$mode> will be modified
378     by the umask in effect then the request is being executed, so better never
379     change the umask.
380 root 1.1
381     Example:
382    
383 root 1.181 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
384 root 1.2 if ($_[0]) {
385     print "open successful, fh is $_[0]\n";
386 root 1.1 ...
387     } else {
388     die "open failed: $!\n";
389     }
390     };
391    
392 root 1.194 In addition to all the common open modes/flags (C<O_RDONLY>, C<O_WRONLY>,
393     C<O_RDWR>, C<O_CREAT>, C<O_TRUNC>, C<O_EXCL> and C<O_APPEND>), the
394     following POSIX and non-POSIX constants are available (missing ones on
395     your system are, as usual, C<0>):
396    
397     C<O_ASYNC>, C<O_DIRECT>, C<O_NOATIME>, C<O_CLOEXEC>, C<O_NOCTTY>, C<O_NOFOLLOW>,
398     C<O_NONBLOCK>, C<O_EXEC>, C<O_SEARCH>, C<O_DIRECTORY>, C<O_DSYNC>,
399     C<O_RSYNC>, C<O_SYNC> and C<O_TTY_INIT>.
400    
401 root 1.106
402 root 1.40 =item aio_close $fh, $callback->($status)
403 root 1.1
404 root 1.2 Asynchronously close a file and call the callback with the result
405 root 1.116 code.
406    
407 root 1.117 Unfortunately, you can't do this to perl. Perl I<insists> very strongly on
408 root 1.121 closing the file descriptor associated with the filehandle itself.
409 root 1.117
410 root 1.121 Therefore, C<aio_close> will not close the filehandle - instead it will
411     use dup2 to overwrite the file descriptor with the write-end of a pipe
412     (the pipe fd will be created on demand and will be cached).
413 root 1.117
414 root 1.121 Or in other words: the file descriptor will be closed, but it will not be
415     free for reuse until the perl filehandle is closed.
416 root 1.117
417     =cut
418    
419 root 1.220 =item aio_seek $fh, $offset, $whence, $callback->($offs)
420    
421 root 1.221 Seeks the filehandle to the new C<$offset>, similarly to perl's
422 root 1.220 C<sysseek>. The C<$whence> can use the traditional values (C<0> for
423     C<IO::AIO::SEEK_SET>, C<1> for C<IO::AIO::SEEK_CUR> or C<2> for
424     C<IO::AIO::SEEK_END>).
425    
426     The resulting absolute offset will be passed to the callback, or C<-1> in
427     case of an error.
428    
429     In theory, the C<$whence> constants could be different than the
430     corresponding values from L<Fcntl>, but perl guarantees they are the same,
431     so don't panic.
432    
433 root 1.225 As a GNU/Linux (and maybe Solaris) extension, also the constants
434     C<IO::AIO::SEEK_DATA> and C<IO::AIO::SEEK_HOLE> are available, if they
435     could be found. No guarantees about suitability for use in C<aio_seek> or
436     Perl's C<sysseek> can be made though, although I would naively assume they
437     "just work".
438    
439 root 1.40 =item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
440 root 1.1
441 root 1.40 =item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
442 root 1.1
443 root 1.145 Reads or writes C<$length> bytes from or to the specified C<$fh> and
444     C<$offset> into the scalar given by C<$data> and offset C<$dataoffset>
445     and calls the callback without the actual number of bytes read (or -1 on
446     error, just like the syscall).
447 root 1.109
448 root 1.146 C<aio_read> will, like C<sysread>, shrink or grow the C<$data> scalar to
449     offset plus the actual number of bytes read.
450    
451 root 1.112 If C<$offset> is undefined, then the current file descriptor offset will
452     be used (and updated), otherwise the file descriptor offset will not be
453     changed by these calls.
454 root 1.109
455 root 1.145 If C<$length> is undefined in C<aio_write>, use the remaining length of
456     C<$data>.
457 root 1.109
458     If C<$dataoffset> is less than zero, it will be counted from the end of
459     C<$data>.
460 root 1.1
461 root 1.31 The C<$data> scalar I<MUST NOT> be modified in any way while the request
462 root 1.108 is outstanding. Modifying it can result in segfaults or World War III (if
463     the necessary/optional hardware is installed).
464 root 1.31
465 root 1.17 Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at
466 root 1.1 offset C<0> within the scalar:
467    
468     aio_read $fh, 7, 15, $buffer, 0, sub {
469 root 1.9 $_[0] > 0 or die "read error: $!";
470     print "read $_[0] bytes: <$buffer>\n";
471 root 1.1 };
472    
473 root 1.106
474 root 1.40 =item aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
475 root 1.35
476     Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
477     reading at byte offset C<$in_offset>, and starts writing at the current
478     file offset of C<$out_fh>. Because of that, it is not safe to issue more
479     than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each
480 root 1.196 other. The same C<$in_fh> works fine though, as this function does not
481     move or use the file offset of C<$in_fh>.
482 root 1.35
483 root 1.185 Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than
484 root 1.196 are written, and there is no way to find out how many more bytes have been
485     read from C<aio_sendfile> alone, as C<aio_sendfile> only provides the
486     number of bytes written to C<$out_fh>. Only if the result value equals
487     C<$length> one can assume that C<$length> bytes have been read.
488 root 1.185
489     Unlike with other C<aio_> functions, it makes a lot of sense to use
490     C<aio_sendfile> on non-blocking sockets, as long as one end (typically
491     the C<$in_fh>) is a file - the file I/O will then be asynchronous, while
492 root 1.196 the socket I/O will be non-blocking. Note, however, that you can run
493     into a trap where C<aio_sendfile> reads some data with readahead, then
494     fails to write all data, and when the socket is ready the next time, the
495     data in the cache is already lost, forcing C<aio_sendfile> to again hit
496     the disk. Explicit C<aio_read> + C<aio_write> let's you better control
497     resource usage.
498    
499     This call tries to make use of a native C<sendfile>-like syscall to
500     provide zero-copy operation. For this to work, C<$out_fh> should refer to
501     a socket, and C<$in_fh> should refer to an mmap'able file.
502 root 1.35
503 root 1.170 If a native sendfile cannot be found or it fails with C<ENOSYS>,
504 root 1.196 C<EINVAL>, C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or
505     C<ENOTSOCK>, it will be emulated, so you can call C<aio_sendfile> on any
506     type of filehandle regardless of the limitations of the operating system.
507    
508     As native sendfile syscalls (as practically any non-POSIX interface hacked
509     together in a hurry to improve benchmark numbers) tend to be rather buggy
510     on many systems, this implementation tries to work around some known bugs
511     in Linux and FreeBSD kernels (probably others, too), but that might fail,
512     so you really really should check the return value of C<aio_sendfile> -
513     fewre bytes than expected might have been transferred.
514 root 1.35
515 root 1.106
516 root 1.40 =item aio_readahead $fh,$offset,$length, $callback->($retval)
517 root 1.1
518 root 1.20 C<aio_readahead> populates the page cache with data from a file so that
519 root 1.1 subsequent reads from that file will not block on disk I/O. The C<$offset>
520     argument specifies the starting point from which data is to be read and
521     C<$length> specifies the number of bytes to be read. I/O is performed in
522     whole pages, so that offset is effectively rounded down to a page boundary
523     and bytes are read up to the next page boundary greater than or equal to
524 root 1.20 (off-set+length). C<aio_readahead> does not read beyond the end of the
525 root 1.1 file. The current file offset of the file is left unchanged.
526    
527 root 1.26 If that syscall doesn't exist (likely if your OS isn't Linux) it will be
528     emulated by simply reading the data, which would have a similar effect.
529    
530 root 1.106
531 root 1.40 =item aio_stat $fh_or_path, $callback->($status)
532 root 1.1
533 root 1.40 =item aio_lstat $fh, $callback->($status)
534 root 1.1
535     Works like perl's C<stat> or C<lstat> in void context. The callback will
536     be called after the stat and the results will be available using C<stat _>
537     or C<-s _> etc...
538    
539     The pathname passed to C<aio_stat> must be absolute. See API NOTES, above,
540     for an explanation.
541    
542     Currently, the stats are always 64-bit-stats, i.e. instead of returning an
543     error when stat'ing a large file, the results will be silently truncated
544     unless perl itself is compiled with large file support.
545    
546 root 1.187 To help interpret the mode and dev/rdev stat values, IO::AIO offers the
547     following constants and functions (if not implemented, the constants will
548     be C<0> and the functions will either C<croak> or fall back on traditional
549     behaviour).
550    
551     C<S_IFMT>, C<S_IFIFO>, C<S_IFCHR>, C<S_IFBLK>, C<S_IFLNK>, C<S_IFREG>,
552     C<S_IFDIR>, C<S_IFWHT>, C<S_IFSOCK>, C<IO::AIO::major $dev_t>,
553     C<IO::AIO::minor $dev_t>, C<IO::AIO::makedev $major, $minor>.
554    
555 root 1.1 Example: Print the length of F</etc/passwd>:
556    
557     aio_stat "/etc/passwd", sub {
558     $_[0] and die "stat failed: $!";
559     print "size is ", -s _, "\n";
560     };
561    
562 root 1.106
563 root 1.175 =item aio_statvfs $fh_or_path, $callback->($statvfs)
564 root 1.172
565     Works like the POSIX C<statvfs> or C<fstatvfs> syscalls, depending on
566     whether a file handle or path was passed.
567    
568     On success, the callback is passed a hash reference with the following
569     members: C<bsize>, C<frsize>, C<blocks>, C<bfree>, C<bavail>, C<files>,
570     C<ffree>, C<favail>, C<fsid>, C<flag> and C<namemax>. On failure, C<undef>
571     is passed.
572    
573     The following POSIX IO::AIO::ST_* constants are defined: C<ST_RDONLY> and
574     C<ST_NOSUID>.
575    
576     The following non-POSIX IO::AIO::ST_* flag masks are defined to
577     their correct value when available, or to C<0> on systems that do
578     not support them: C<ST_NODEV>, C<ST_NOEXEC>, C<ST_SYNCHRONOUS>,
579     C<ST_MANDLOCK>, C<ST_WRITE>, C<ST_APPEND>, C<ST_IMMUTABLE>, C<ST_NOATIME>,
580     C<ST_NODIRATIME> and C<ST_RELATIME>.
581    
582     Example: stat C</wd> and dump out the data if successful.
583    
584     aio_statvfs "/wd", sub {
585     my $f = $_[0]
586     or die "statvfs: $!";
587    
588     use Data::Dumper;
589     say Dumper $f;
590     };
591    
592     # result:
593     {
594     bsize => 1024,
595     bfree => 4333064312,
596     blocks => 10253828096,
597     files => 2050765568,
598     flag => 4096,
599     favail => 2042092649,
600     bavail => 4333064312,
601     ffree => 2042092649,
602     namemax => 255,
603     frsize => 1024,
604     fsid => 1810
605     }
606    
607 root 1.244 Here is a (likely partial - send me updates!) list of fsid values used by
608     Linux - it is safe to hardcode these when C<$^O> is C<linux>:
609 root 1.235
610     0x0000adf5 adfs
611     0x0000adff affs
612     0x5346414f afs
613     0x09041934 anon-inode filesystem
614     0x00000187 autofs
615     0x42465331 befs
616     0x1badface bfs
617     0x42494e4d binfmt_misc
618     0x9123683e btrfs
619     0x0027e0eb cgroupfs
620     0xff534d42 cifs
621     0x73757245 coda
622     0x012ff7b7 coh
623     0x28cd3d45 cramfs
624     0x453dcd28 cramfs-wend (wrong endianness)
625     0x64626720 debugfs
626     0x00001373 devfs
627     0x00001cd1 devpts
628     0x0000f15f ecryptfs
629     0x00414a53 efs
630     0x0000137d ext
631     0x0000ef53 ext2/ext3
632     0x0000ef51 ext2
633     0x00004006 fat
634     0x65735546 fuseblk
635     0x65735543 fusectl
636     0x0bad1dea futexfs
637     0x01161970 gfs2
638     0x47504653 gpfs
639     0x00004244 hfs
640     0xf995e849 hpfs
641     0x958458f6 hugetlbfs
642     0x2bad1dea inotifyfs
643     0x00009660 isofs
644     0x000072b6 jffs2
645     0x3153464a jfs
646     0x6b414653 k-afs
647     0x0bd00bd0 lustre
648     0x0000137f minix
649     0x0000138f minix 30 char names
650     0x00002468 minix v2
651     0x00002478 minix v2 30 char names
652     0x00004d5a minix v3
653     0x19800202 mqueue
654     0x00004d44 msdos
655     0x0000564c novell
656     0x00006969 nfs
657     0x6e667364 nfsd
658     0x00003434 nilfs
659     0x5346544e ntfs
660     0x00009fa1 openprom
661     0x7461636F ocfs2
662     0x00009fa0 proc
663     0x6165676c pstorefs
664     0x0000002f qnx4
665     0x858458f6 ramfs
666     0x52654973 reiserfs
667     0x00007275 romfs
668     0x67596969 rpc_pipefs
669     0x73636673 securityfs
670     0xf97cff8c selinux
671     0x0000517b smb
672     0x534f434b sockfs
673     0x73717368 squashfs
674     0x62656572 sysfs
675     0x012ff7b6 sysv2
676     0x012ff7b5 sysv4
677     0x01021994 tmpfs
678     0x15013346 udf
679     0x00011954 ufs
680     0x54190100 ufs byteswapped
681     0x00009fa2 usbdevfs
682     0x01021997 v9fs
683     0xa501fcf5 vxfs
684     0xabba1974 xenfs
685     0x012ff7b4 xenix
686     0x58465342 xfs
687     0x012fd16d xia
688 root 1.172
689 root 1.106 =item aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
690    
691     Works like perl's C<utime> function (including the special case of $atime
692     and $mtime being undef). Fractional times are supported if the underlying
693     syscalls support them.
694    
695     When called with a pathname, uses utimes(2) if available, otherwise
696     utime(2). If called on a file descriptor, uses futimes(2) if available,
697     otherwise returns ENOSYS, so this is not portable.
698    
699     Examples:
700    
701 root 1.107 # set atime and mtime to current time (basically touch(1)):
702 root 1.106 aio_utime "path", undef, undef;
703     # set atime to current time and mtime to beginning of the epoch:
704     aio_utime "path", time, undef; # undef==0
705    
706    
707     =item aio_chown $fh_or_path, $uid, $gid, $callback->($status)
708    
709     Works like perl's C<chown> function, except that C<undef> for either $uid
710     or $gid is being interpreted as "do not change" (but -1 can also be used).
711    
712     Examples:
713    
714     # same as "chown root path" in the shell:
715     aio_chown "path", 0, -1;
716     # same as above:
717     aio_chown "path", 0, undef;
718    
719    
720 root 1.110 =item aio_truncate $fh_or_path, $offset, $callback->($status)
721    
722     Works like truncate(2) or ftruncate(2).
723    
724    
725 root 1.229 =item aio_allocate $fh, $mode, $offset, $len, $callback->($status)
726    
727     Allocates or freed disk space according to the C<$mode> argument. See the
728     linux C<fallocate> docuemntation for details.
729    
730     C<$mode> can currently be C<0> or C<IO::AIO::FALLOC_FL_KEEP_SIZE>
731     to allocate space, or C<IO::AIO::FALLOC_FL_PUNCH_HOLE |
732     IO::AIO::FALLOC_FL_KEEP_SIZE>, to deallocate a file range.
733    
734     The file system block size used by C<fallocate> is presumably the
735     C<f_bsize> returned by C<statvfs>.
736    
737     If C<fallocate> isn't available or cannot be emulated (currently no
738     emulation will be attempted), passes C<-1> and sets C<$!> to C<ENOSYS>.
739    
740    
741 root 1.106 =item aio_chmod $fh_or_path, $mode, $callback->($status)
742    
743     Works like perl's C<chmod> function.
744    
745    
746 root 1.40 =item aio_unlink $pathname, $callback->($status)
747 root 1.1
748     Asynchronously unlink (delete) a file and call the callback with the
749     result code.
750    
751 root 1.106
752 root 1.209 =item aio_mknod $pathname, $mode, $dev, $callback->($status)
753 root 1.82
754 root 1.86 [EXPERIMENTAL]
755    
756 root 1.83 Asynchronously create a device node (or fifo). See mknod(2).
757    
758 root 1.86 The only (POSIX-) portable way of calling this function is:
759 root 1.83
760 root 1.209 aio_mknod $pathname, IO::AIO::S_IFIFO | $mode, 0, sub { ...
761 root 1.82
762 root 1.187 See C<aio_stat> for info about some potentially helpful extra constants
763     and functions.
764 root 1.106
765 root 1.50 =item aio_link $srcpath, $dstpath, $callback->($status)
766    
767     Asynchronously create a new link to the existing object at C<$srcpath> at
768     the path C<$dstpath> and call the callback with the result code.
769    
770 root 1.106
771 root 1.50 =item aio_symlink $srcpath, $dstpath, $callback->($status)
772    
773     Asynchronously create a new symbolic link to the existing object at C<$srcpath> at
774     the path C<$dstpath> and call the callback with the result code.
775    
776 root 1.106
777 root 1.209 =item aio_readlink $pathname, $callback->($link)
778 root 1.90
779     Asynchronously read the symlink specified by C<$path> and pass it to
780     the callback. If an error occurs, nothing or undef gets passed to the
781     callback.
782    
783 root 1.106
784 root 1.209 =item aio_realpath $pathname, $callback->($path)
785 root 1.201
786     Asynchronously make the path absolute and resolve any symlinks in
787 root 1.239 C<$path>. The resulting path only consists of directories (same as
788 root 1.202 L<Cwd::realpath>).
789 root 1.201
790     This request can be used to get the absolute path of the current working
791     directory by passing it a path of F<.> (a single dot).
792    
793    
794 root 1.50 =item aio_rename $srcpath, $dstpath, $callback->($status)
795    
796     Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as
797     rename(2) and call the callback with the result code.
798    
799 root 1.241 On systems that support the AIO::WD working directory abstraction
800     natively, the case C<[$wd, "."]> as C<$srcpath> is specialcased - instead
801     of failing, C<rename> is called on the absolute path of C<$wd>.
802    
803 root 1.106
804 root 1.101 =item aio_mkdir $pathname, $mode, $callback->($status)
805    
806     Asynchronously mkdir (create) a directory and call the callback with
807     the result code. C<$mode> will be modified by the umask at the time the
808     request is executed, so do not change your umask.
809    
810 root 1.106
811 root 1.40 =item aio_rmdir $pathname, $callback->($status)
812 root 1.27
813     Asynchronously rmdir (delete) a directory and call the callback with the
814     result code.
815    
816 root 1.241 On systems that support the AIO::WD working directory abstraction
817     natively, the case C<[$wd, "."]> is specialcased - instead of failing,
818     C<rmdir> is called on the absolute path of C<$wd>.
819    
820 root 1.106
821 root 1.46 =item aio_readdir $pathname, $callback->($entries)
822 root 1.37
823     Unlike the POSIX call of the same name, C<aio_readdir> reads an entire
824     directory (i.e. opendir + readdir + closedir). The entries will not be
825     sorted, and will B<NOT> include the C<.> and C<..> entries.
826    
827 root 1.148 The callback is passed a single argument which is either C<undef> or an
828     array-ref with the filenames.
829    
830    
831     =item aio_readdirx $pathname, $flags, $callback->($entries, $flags)
832    
833 root 1.207 Quite similar to C<aio_readdir>, but the C<$flags> argument allows one to
834     tune behaviour and output format. In case of an error, C<$entries> will be
835 root 1.148 C<undef>.
836    
837     The flags are a combination of the following constants, ORed together (the
838     flags will also be passed to the callback, possibly modified):
839    
840     =over 4
841    
842 root 1.150 =item IO::AIO::READDIR_DENTS
843 root 1.148
844 root 1.190 When this flag is off, then the callback gets an arrayref consisting of
845     names only (as with C<aio_readdir>), otherwise it gets an arrayref with
846 root 1.150 C<[$name, $type, $inode]> arrayrefs, each describing a single directory
847 root 1.148 entry in more detail.
848    
849     C<$name> is the name of the entry.
850    
851 root 1.150 C<$type> is one of the C<IO::AIO::DT_xxx> constants:
852 root 1.148
853 root 1.150 C<IO::AIO::DT_UNKNOWN>, C<IO::AIO::DT_FIFO>, C<IO::AIO::DT_CHR>, C<IO::AIO::DT_DIR>,
854     C<IO::AIO::DT_BLK>, C<IO::AIO::DT_REG>, C<IO::AIO::DT_LNK>, C<IO::AIO::DT_SOCK>,
855     C<IO::AIO::DT_WHT>.
856 root 1.148
857 root 1.150 C<IO::AIO::DT_UNKNOWN> means just that: readdir does not know. If you need to
858 root 1.148 know, you have to run stat yourself. Also, for speed reasons, the C<$type>
859     scalars are read-only: you can not modify them.
860    
861 root 1.150 C<$inode> is the inode number (which might not be exact on systems with 64
862 root 1.155 bit inode numbers and 32 bit perls). This field has unspecified content on
863     systems that do not deliver the inode information.
864 root 1.150
865     =item IO::AIO::READDIR_DIRS_FIRST
866 root 1.148
867     When this flag is set, then the names will be returned in an order where
868 root 1.193 likely directories come first, in optimal stat order. This is useful when
869     you need to quickly find directories, or you want to find all directories
870     while avoiding to stat() each entry.
871 root 1.148
872 root 1.149 If the system returns type information in readdir, then this is used
873 root 1.193 to find directories directly. Otherwise, likely directories are names
874     beginning with ".", or otherwise names with no dots, of which names with
875 root 1.149 short names are tried first.
876    
877 root 1.150 =item IO::AIO::READDIR_STAT_ORDER
878 root 1.148
879     When this flag is set, then the names will be returned in an order
880     suitable for stat()'ing each one. That is, when you plan to stat()
881     all files in the given directory, then the returned order will likely
882     be fastest.
883    
884 root 1.150 If both this flag and C<IO::AIO::READDIR_DIRS_FIRST> are specified, then
885     the likely dirs come first, resulting in a less optimal stat order.
886 root 1.148
887 root 1.150 =item IO::AIO::READDIR_FOUND_UNKNOWN
888 root 1.148
889     This flag should not be set when calling C<aio_readdirx>. Instead, it
890     is being set by C<aio_readdirx>, when any of the C<$type>'s found were
891 root 1.207 C<IO::AIO::DT_UNKNOWN>. The absence of this flag therefore indicates that all
892 root 1.148 C<$type>'s are known, which can be used to speed up some algorithms.
893    
894     =back
895 root 1.37
896 root 1.106
897 root 1.209 =item aio_load $pathname, $data, $callback->($status)
898 root 1.98
899     This is a composite request that tries to fully load the given file into
900     memory. Status is the same as with aio_read.
901    
902     =cut
903    
904     sub aio_load($$;$) {
905 root 1.123 my ($path, undef, $cb) = @_;
906     my $data = \$_[1];
907 root 1.98
908 root 1.123 my $pri = aioreq_pri;
909     my $grp = aio_group $cb;
910    
911     aioreq_pri $pri;
912     add $grp aio_open $path, O_RDONLY, 0, sub {
913     my $fh = shift
914     or return $grp->result (-1);
915 root 1.98
916     aioreq_pri $pri;
917 root 1.123 add $grp aio_read $fh, 0, (-s $fh), $$data, 0, sub {
918     $grp->result ($_[0]);
919 root 1.98 };
920 root 1.123 };
921 root 1.98
922 root 1.123 $grp
923 root 1.98 }
924    
925 root 1.82 =item aio_copy $srcpath, $dstpath, $callback->($status)
926    
927     Try to copy the I<file> (directories not supported as either source or
928     destination) from C<$srcpath> to C<$dstpath> and call the callback with
929 root 1.165 a status of C<0> (ok) or C<-1> (error, see C<$!>).
930 root 1.82
931 root 1.134 This is a composite request that creates the destination file with
932 root 1.82 mode 0200 and copies the contents of the source file into it using
933     C<aio_sendfile>, followed by restoring atime, mtime, access mode and
934     uid/gid, in that order.
935    
936     If an error occurs, the partial destination file will be unlinked, if
937     possible, except when setting atime, mtime, access mode and uid/gid, where
938     errors are being ignored.
939    
940     =cut
941    
942     sub aio_copy($$;$) {
943 root 1.123 my ($src, $dst, $cb) = @_;
944 root 1.82
945 root 1.123 my $pri = aioreq_pri;
946     my $grp = aio_group $cb;
947 root 1.82
948 root 1.123 aioreq_pri $pri;
949     add $grp aio_open $src, O_RDONLY, 0, sub {
950     if (my $src_fh = $_[0]) {
951 root 1.166 my @stat = stat $src_fh; # hmm, might block over nfs?
952 root 1.95
953 root 1.123 aioreq_pri $pri;
954     add $grp aio_open $dst, O_CREAT | O_WRONLY | O_TRUNC, 0200, sub {
955     if (my $dst_fh = $_[0]) {
956     aioreq_pri $pri;
957     add $grp aio_sendfile $dst_fh, $src_fh, 0, $stat[7], sub {
958     if ($_[0] == $stat[7]) {
959     $grp->result (0);
960     close $src_fh;
961    
962 root 1.147 my $ch = sub {
963     aioreq_pri $pri;
964     add $grp aio_chmod $dst_fh, $stat[2] & 07777, sub {
965     aioreq_pri $pri;
966     add $grp aio_chown $dst_fh, $stat[4], $stat[5], sub {
967     aioreq_pri $pri;
968     add $grp aio_close $dst_fh;
969     }
970     };
971     };
972 root 1.123
973     aioreq_pri $pri;
974 root 1.147 add $grp aio_utime $dst_fh, $stat[8], $stat[9], sub {
975     if ($_[0] < 0 && $! == ENOSYS) {
976     aioreq_pri $pri;
977     add $grp aio_utime $dst, $stat[8], $stat[9], $ch;
978     } else {
979     $ch->();
980     }
981     };
982 root 1.123 } else {
983     $grp->result (-1);
984     close $src_fh;
985     close $dst_fh;
986    
987     aioreq $pri;
988     add $grp aio_unlink $dst;
989     }
990     };
991     } else {
992     $grp->result (-1);
993     }
994     },
995 root 1.82
996 root 1.123 } else {
997     $grp->result (-1);
998     }
999     };
1000 root 1.82
1001 root 1.123 $grp
1002 root 1.82 }
1003    
1004     =item aio_move $srcpath, $dstpath, $callback->($status)
1005    
1006     Try to move the I<file> (directories not supported as either source or
1007     destination) from C<$srcpath> to C<$dstpath> and call the callback with
1008 root 1.165 a status of C<0> (ok) or C<-1> (error, see C<$!>).
1009 root 1.82
1010 root 1.137 This is a composite request that tries to rename(2) the file first; if
1011     rename fails with C<EXDEV>, it copies the file with C<aio_copy> and, if
1012     that is successful, unlinks the C<$srcpath>.
1013 root 1.82
1014     =cut
1015    
1016     sub aio_move($$;$) {
1017 root 1.123 my ($src, $dst, $cb) = @_;
1018 root 1.82
1019 root 1.123 my $pri = aioreq_pri;
1020     my $grp = aio_group $cb;
1021 root 1.82
1022 root 1.123 aioreq_pri $pri;
1023     add $grp aio_rename $src, $dst, sub {
1024     if ($_[0] && $! == EXDEV) {
1025     aioreq_pri $pri;
1026     add $grp aio_copy $src, $dst, sub {
1027     $grp->result ($_[0]);
1028 root 1.95
1029 root 1.196 unless ($_[0]) {
1030 root 1.123 aioreq_pri $pri;
1031     add $grp aio_unlink $src;
1032     }
1033     };
1034     } else {
1035     $grp->result ($_[0]);
1036     }
1037     };
1038 root 1.82
1039 root 1.123 $grp
1040 root 1.82 }
1041    
1042 root 1.209 =item aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
1043 root 1.40
1044 root 1.52 Scans a directory (similar to C<aio_readdir>) but additionally tries to
1045 root 1.76 efficiently separate the entries of directory C<$path> into two sets of
1046     names, directories you can recurse into (directories), and ones you cannot
1047     recurse into (everything else, including symlinks to directories).
1048 root 1.52
1049 root 1.61 C<aio_scandir> is a composite request that creates of many sub requests_
1050     C<$maxreq> specifies the maximum number of outstanding aio requests that
1051     this function generates. If it is C<< <= 0 >>, then a suitable default
1052 root 1.81 will be chosen (currently 4).
1053 root 1.40
1054     On error, the callback is called without arguments, otherwise it receives
1055     two array-refs with path-relative entry names.
1056    
1057     Example:
1058    
1059     aio_scandir $dir, 0, sub {
1060     my ($dirs, $nondirs) = @_;
1061     print "real directories: @$dirs\n";
1062     print "everything else: @$nondirs\n";
1063     };
1064    
1065     Implementation notes.
1066    
1067     The C<aio_readdir> cannot be avoided, but C<stat()>'ing every entry can.
1068    
1069 root 1.149 If readdir returns file type information, then this is used directly to
1070     find directories.
1071    
1072     Otherwise, after reading the directory, the modification time, size etc.
1073     of the directory before and after the readdir is checked, and if they
1074     match (and isn't the current time), the link count will be used to decide
1075     how many entries are directories (if >= 2). Otherwise, no knowledge of the
1076     number of subdirectories will be assumed.
1077    
1078     Then entries will be sorted into likely directories a non-initial dot
1079     currently) and likely non-directories (see C<aio_readdirx>). Then every
1080     entry plus an appended C</.> will be C<stat>'ed, likely directories first,
1081     in order of their inode numbers. If that succeeds, it assumes that the
1082     entry is a directory or a symlink to directory (which will be checked
1083 root 1.207 separately). This is often faster than stat'ing the entry itself because
1084 root 1.52 filesystems might detect the type of the entry without reading the inode
1085 root 1.149 data (e.g. ext2fs filetype feature), even on systems that cannot return
1086     the filetype information on readdir.
1087 root 1.52
1088     If the known number of directories (link count - 2) has been reached, the
1089     rest of the entries is assumed to be non-directories.
1090    
1091     This only works with certainty on POSIX (= UNIX) filesystems, which
1092     fortunately are the vast majority of filesystems around.
1093    
1094     It will also likely work on non-POSIX filesystems with reduced efficiency
1095     as those tend to return 0 or 1 as link counts, which disables the
1096     directory counting heuristic.
1097 root 1.40
1098     =cut
1099    
1100 root 1.100 sub aio_scandir($$;$) {
1101 root 1.123 my ($path, $maxreq, $cb) = @_;
1102    
1103     my $pri = aioreq_pri;
1104 root 1.40
1105 root 1.123 my $grp = aio_group $cb;
1106 root 1.80
1107 root 1.123 $maxreq = 4 if $maxreq <= 0;
1108 root 1.55
1109 root 1.210 # get a wd object
1110 root 1.123 aioreq_pri $pri;
1111 root 1.210 add $grp aio_wd $path, sub {
1112 root 1.212 $_[0]
1113     or return $grp->result ();
1114    
1115 root 1.210 my $wd = [shift, "."];
1116 root 1.40
1117 root 1.210 # stat once
1118 root 1.80 aioreq_pri $pri;
1119 root 1.210 add $grp aio_stat $wd, sub {
1120     return $grp->result () if $_[0];
1121     my $now = time;
1122     my $hash1 = join ":", (stat _)[0,1,3,7,9];
1123 root 1.40
1124 root 1.210 # read the directory entries
1125 root 1.80 aioreq_pri $pri;
1126 root 1.210 add $grp aio_readdirx $wd, READDIR_DIRS_FIRST, sub {
1127     my $entries = shift
1128     or return $grp->result ();
1129    
1130     # stat the dir another time
1131     aioreq_pri $pri;
1132     add $grp aio_stat $wd, sub {
1133     my $hash2 = join ":", (stat _)[0,1,3,7,9];
1134 root 1.95
1135 root 1.210 my $ndirs;
1136 root 1.95
1137 root 1.210 # take the slow route if anything looks fishy
1138     if ($hash1 ne $hash2 or (stat _)[9] == $now) {
1139     $ndirs = -1;
1140     } else {
1141     # if nlink == 2, we are finished
1142     # for non-posix-fs's, we rely on nlink < 2
1143     $ndirs = (stat _)[3] - 2
1144     or return $grp->result ([], $entries);
1145     }
1146 root 1.123
1147 root 1.210 my (@dirs, @nondirs);
1148 root 1.40
1149 root 1.210 my $statgrp = add $grp aio_group sub {
1150     $grp->result (\@dirs, \@nondirs);
1151     };
1152 root 1.40
1153 root 1.210 limit $statgrp $maxreq;
1154     feed $statgrp sub {
1155     return unless @$entries;
1156     my $entry = shift @$entries;
1157    
1158     aioreq_pri $pri;
1159     $wd->[1] = "$entry/.";
1160     add $statgrp aio_stat $wd, sub {
1161     if ($_[0] < 0) {
1162     push @nondirs, $entry;
1163     } else {
1164     # need to check for real directory
1165     aioreq_pri $pri;
1166     $wd->[1] = $entry;
1167     add $statgrp aio_lstat $wd, sub {
1168     if (-d _) {
1169     push @dirs, $entry;
1170    
1171     unless (--$ndirs) {
1172     push @nondirs, @$entries;
1173     feed $statgrp;
1174     }
1175     } else {
1176     push @nondirs, $entry;
1177 root 1.74 }
1178 root 1.40 }
1179     }
1180 root 1.210 };
1181 root 1.74 };
1182 root 1.40 };
1183     };
1184     };
1185 root 1.123 };
1186 root 1.55
1187 root 1.123 $grp
1188 root 1.40 }
1189    
1190 root 1.209 =item aio_rmtree $pathname, $callback->($status)
1191 root 1.99
1192 root 1.100 Delete a directory tree starting (and including) C<$path>, return the
1193 root 1.239 status of the final C<rmdir> only. This is a composite request that
1194 root 1.100 uses C<aio_scandir> to recurse into and rmdir directories, and unlink
1195     everything else.
1196 root 1.99
1197     =cut
1198    
1199     sub aio_rmtree;
1200 root 1.100 sub aio_rmtree($;$) {
1201 root 1.123 my ($path, $cb) = @_;
1202 root 1.99
1203 root 1.123 my $pri = aioreq_pri;
1204     my $grp = aio_group $cb;
1205 root 1.99
1206 root 1.123 aioreq_pri $pri;
1207     add $grp aio_scandir $path, 0, sub {
1208     my ($dirs, $nondirs) = @_;
1209 root 1.99
1210 root 1.123 my $dirgrp = aio_group sub {
1211     add $grp aio_rmdir $path, sub {
1212     $grp->result ($_[0]);
1213 root 1.99 };
1214 root 1.123 };
1215 root 1.99
1216 root 1.123 (aioreq_pri $pri), add $dirgrp aio_rmtree "$path/$_" for @$dirs;
1217     (aioreq_pri $pri), add $dirgrp aio_unlink "$path/$_" for @$nondirs;
1218 root 1.99
1219 root 1.123 add $grp $dirgrp;
1220     };
1221 root 1.99
1222 root 1.123 $grp
1223 root 1.99 }
1224    
1225 root 1.119 =item aio_sync $callback->($status)
1226    
1227     Asynchronously call sync and call the callback when finished.
1228    
1229 root 1.40 =item aio_fsync $fh, $callback->($status)
1230 root 1.1
1231     Asynchronously call fsync on the given filehandle and call the callback
1232     with the fsync result code.
1233    
1234 root 1.40 =item aio_fdatasync $fh, $callback->($status)
1235 root 1.1
1236     Asynchronously call fdatasync on the given filehandle and call the
1237 root 1.26 callback with the fdatasync result code.
1238    
1239     If this call isn't available because your OS lacks it or it couldn't be
1240     detected, it will be emulated by calling C<fsync> instead.
1241 root 1.1
1242 root 1.206 =item aio_syncfs $fh, $callback->($status)
1243    
1244     Asynchronously call the syncfs syscall to sync the filesystem associated
1245     to the given filehandle and call the callback with the syncfs result
1246     code. If syncfs is not available, calls sync(), but returns C<-1> and sets
1247     errno to C<ENOSYS> nevertheless.
1248    
1249 root 1.142 =item aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
1250    
1251     Sync the data portion of the file specified by C<$offset> and C<$length>
1252     to disk (but NOT the metadata), by calling the Linux-specific
1253     sync_file_range call. If sync_file_range is not available or it returns
1254     ENOSYS, then fdatasync or fsync is being substituted.
1255    
1256     C<$flags> can be a combination of C<IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE>,
1257     C<IO::AIO::SYNC_FILE_RANGE_WRITE> and
1258     C<IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER>: refer to the sync_file_range
1259     manpage for details.
1260    
1261 root 1.209 =item aio_pathsync $pathname, $callback->($status)
1262 root 1.120
1263     This request tries to open, fsync and close the given path. This is a
1264 root 1.135 composite request intended to sync directories after directory operations
1265 root 1.120 (E.g. rename). This might not work on all operating systems or have any
1266     specific effect, but usually it makes sure that directory changes get
1267     written to disc. It works for anything that can be opened for read-only,
1268     not just directories.
1269    
1270 root 1.162 Future versions of this function might fall back to other methods when
1271     C<fsync> on the directory fails (such as calling C<sync>).
1272    
1273 root 1.120 Passes C<0> when everything went ok, and C<-1> on error.
1274    
1275     =cut
1276    
1277     sub aio_pathsync($;$) {
1278 root 1.123 my ($path, $cb) = @_;
1279    
1280     my $pri = aioreq_pri;
1281     my $grp = aio_group $cb;
1282 root 1.120
1283 root 1.123 aioreq_pri $pri;
1284     add $grp aio_open $path, O_RDONLY, 0, sub {
1285     my ($fh) = @_;
1286     if ($fh) {
1287     aioreq_pri $pri;
1288     add $grp aio_fsync $fh, sub {
1289     $grp->result ($_[0]);
1290 root 1.120
1291     aioreq_pri $pri;
1292 root 1.123 add $grp aio_close $fh;
1293     };
1294     } else {
1295     $grp->result (-1);
1296     }
1297     };
1298 root 1.120
1299 root 1.123 $grp
1300 root 1.120 }
1301    
1302 root 1.170 =item aio_msync $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
1303    
1304     This is a rather advanced IO::AIO call, which only works on mmap(2)ed
1305 root 1.176 scalars (see the C<IO::AIO::mmap> function, although it also works on data
1306     scalars managed by the L<Sys::Mmap> or L<Mmap> modules, note that the
1307     scalar must only be modified in-place while an aio operation is pending on
1308     it).
1309 root 1.170
1310     It calls the C<msync> function of your OS, if available, with the memory
1311     area starting at C<$offset> in the string and ending C<$length> bytes
1312     later. If C<$length> is negative, counts from the end, and if C<$length>
1313     is C<undef>, then it goes till the end of the string. The flags can be
1314     a combination of C<IO::AIO::MS_ASYNC>, C<IO::AIO::MS_INVALIDATE> and
1315     C<IO::AIO::MS_SYNC>.
1316    
1317     =item aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
1318    
1319     This is a rather advanced IO::AIO call, which works best on mmap(2)ed
1320     scalars.
1321    
1322     It touches (reads or writes) all memory pages in the specified
1323 root 1.239 range inside the scalar. All caveats and parameters are the same
1324 root 1.170 as for C<aio_msync>, above, except for flags, which must be either
1325     C<0> (which reads all pages and ensures they are instantiated) or
1326 root 1.239 C<IO::AIO::MT_MODIFY>, which modifies the memory pages (by reading and
1327 root 1.170 writing an octet from it, which dirties the page).
1328    
1329 root 1.182 =item aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
1330    
1331     This is a rather advanced IO::AIO call, which works best on mmap(2)ed
1332     scalars.
1333    
1334     It reads in all the pages of the underlying storage into memory (if any)
1335     and locks them, so they are not getting swapped/paged out or removed.
1336    
1337     If C<$length> is undefined, then the scalar will be locked till the end.
1338    
1339     On systems that do not implement C<mlock>, this function returns C<-1>
1340     and sets errno to C<ENOSYS>.
1341    
1342     Note that the corresponding C<munlock> is synchronous and is
1343     documented under L<MISCELLANEOUS FUNCTIONS>.
1344    
1345 root 1.183 Example: open a file, mmap and mlock it - both will be undone when
1346     C<$data> gets destroyed.
1347    
1348     open my $fh, "<", $path or die "$path: $!";
1349     my $data;
1350     IO::AIO::mmap $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh;
1351     aio_mlock $data; # mlock in background
1352    
1353 root 1.182 =item aio_mlockall $flags, $callback->($status)
1354    
1355     Calls the C<mlockall> function with the given C<$flags> (a combination of
1356     C<IO::AIO::MCL_CURRENT> and C<IO::AIO::MCL_FUTURE>).
1357    
1358     On systems that do not implement C<mlockall>, this function returns C<-1>
1359     and sets errno to C<ENOSYS>.
1360    
1361     Note that the corresponding C<munlockall> is synchronous and is
1362     documented under L<MISCELLANEOUS FUNCTIONS>.
1363    
1364 root 1.183 Example: asynchronously lock all current and future pages into memory.
1365    
1366     aio_mlockall IO::AIO::MCL_FUTURE;
1367    
1368 root 1.223 =item aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
1369    
1370 root 1.234 Queries the extents of the given file (by calling the Linux C<FIEMAP>
1371     ioctl, see L<http://cvs.schmorp.de/IO-AIO/doc/fiemap.txt> for details). If
1372     the ioctl is not available on your OS, then this request will fail with
1373 root 1.223 C<ENOSYS>.
1374    
1375     C<$start> is the starting offset to query extents for, C<$length> is the
1376     size of the range to query - if it is C<undef>, then the whole file will
1377     be queried.
1378    
1379     C<$flags> is a combination of flags (C<IO::AIO::FIEMAP_FLAG_SYNC> or
1380     C<IO::AIO::FIEMAP_FLAG_XATTR> - C<IO::AIO::FIEMAP_FLAGS_COMPAT> is also
1381     exported), and is normally C<0> or C<IO::AIO::FIEMAP_FLAG_SYNC> to query
1382     the data portion.
1383    
1384     C<$count> is the maximum number of extent records to return. If it is
1385 root 1.232 C<undef>, then IO::AIO queries all extents of the range. As a very special
1386 root 1.223 case, if it is C<0>, then the callback receives the number of extents
1387 root 1.232 instead of the extents themselves (which is unreliable, see below).
1388 root 1.223
1389     If an error occurs, the callback receives no arguments. The special
1390     C<errno> value C<IO::AIO::EBADR> is available to test for flag errors.
1391    
1392     Otherwise, the callback receives an array reference with extent
1393     structures. Each extent structure is an array reference itself, with the
1394     following members:
1395    
1396     [$logical, $physical, $length, $flags]
1397    
1398     Flags is any combination of the following flag values (typically either C<0>
1399 root 1.231 or C<IO::AIO::FIEMAP_EXTENT_LAST> (1)):
1400 root 1.223
1401     C<IO::AIO::FIEMAP_EXTENT_LAST>, C<IO::AIO::FIEMAP_EXTENT_UNKNOWN>,
1402     C<IO::AIO::FIEMAP_EXTENT_DELALLOC>, C<IO::AIO::FIEMAP_EXTENT_ENCODED>,
1403     C<IO::AIO::FIEMAP_EXTENT_DATA_ENCRYPTED>, C<IO::AIO::FIEMAP_EXTENT_NOT_ALIGNED>,
1404     C<IO::AIO::FIEMAP_EXTENT_DATA_INLINE>, C<IO::AIO::FIEMAP_EXTENT_DATA_TAIL>,
1405     C<IO::AIO::FIEMAP_EXTENT_UNWRITTEN>, C<IO::AIO::FIEMAP_EXTENT_MERGED> or
1406     C<IO::AIO::FIEMAP_EXTENT_SHARED>.
1407    
1408 root 1.232 At the time of this writing (Linux 3.2), this requets is unreliable unless
1409     C<$count> is C<undef>, as the kernel has all sorts of bugs preventing
1410     it to return all extents of a range for files with large number of
1411     extents. The code works around all these issues if C<$count> is undef.
1412    
1413 root 1.58 =item aio_group $callback->(...)
1414 root 1.54
1415 root 1.55 This is a very special aio request: Instead of doing something, it is a
1416     container for other aio requests, which is useful if you want to bundle
1417 root 1.71 many requests into a single, composite, request with a definite callback
1418     and the ability to cancel the whole request with its subrequests.
1419 root 1.55
1420     Returns an object of class L<IO::AIO::GRP>. See its documentation below
1421     for more info.
1422    
1423     Example:
1424    
1425     my $grp = aio_group sub {
1426     print "all stats done\n";
1427     };
1428    
1429     add $grp
1430     (aio_stat ...),
1431     (aio_stat ...),
1432     ...;
1433    
1434 root 1.63 =item aio_nop $callback->()
1435    
1436     This is a special request - it does nothing in itself and is only used for
1437     side effects, such as when you want to add a dummy request to a group so
1438     that finishing the requests in the group depends on executing the given
1439     code.
1440    
1441 root 1.64 While this request does nothing, it still goes through the execution
1442     phase and still requires a worker thread. Thus, the callback will not
1443     be executed immediately but only after other requests in the queue have
1444     entered their execution phase. This can be used to measure request
1445     latency.
1446    
1447 root 1.71 =item IO::AIO::aio_busy $fractional_seconds, $callback->() *NOT EXPORTED*
1448 root 1.54
1449     Mainly used for debugging and benchmarking, this aio request puts one of
1450     the request workers to sleep for the given time.
1451    
1452 root 1.56 While it is theoretically handy to have simple I/O scheduling requests
1453 root 1.71 like sleep and file handle readable/writable, the overhead this creates is
1454     immense (it blocks a thread for a long time) so do not use this function
1455     except to put your application under artificial I/O pressure.
1456 root 1.56
1457 root 1.5 =back
1458    
1459 root 1.209
1460     =head2 IO::AIO::WD - multiple working directories
1461    
1462     Your process only has one current working directory, which is used by all
1463     threads. This makes it hard to use relative paths (some other component
1464     could call C<chdir> at any time, and it is hard to control when the path
1465     will be used by IO::AIO).
1466    
1467     One solution for this is to always use absolute paths. This usually works,
1468     but can be quite slow (the kernel has to walk the whole path on every
1469     access), and can also be a hassle to implement.
1470    
1471     Newer POSIX systems have a number of functions (openat, fdopendir,
1472     futimensat and so on) that make it possible to specify working directories
1473     per operation.
1474    
1475     For portability, and because the clowns who "designed", or shall I write,
1476     perpetrated this new interface were obviously half-drunk, this abstraction
1477     cannot be perfect, though.
1478    
1479     IO::AIO allows you to convert directory paths into a so-called IO::AIO::WD
1480     object. This object stores the canonicalised, absolute version of the
1481     path, and on systems that allow it, also a directory file descriptor.
1482    
1483     Everywhere where a pathname is accepted by IO::AIO (e.g. in C<aio_stat>
1484     or C<aio_unlink>), one can specify an array reference with an IO::AIO::WD
1485 root 1.214 object and a pathname instead (or the IO::AIO::WD object alone, which
1486     gets interpreted as C<[$wd, "."]>). If the pathname is absolute, the
1487 root 1.213 IO::AIO::WD object is ignored, otherwise the pathname is resolved relative
1488 root 1.209 to that IO::AIO::WD object.
1489    
1490     For example, to get a wd object for F</etc> and then stat F<passwd>
1491     inside, you would write:
1492    
1493     aio_wd "/etc", sub {
1494     my $etcdir = shift;
1495    
1496     # although $etcdir can be undef on error, there is generally no reason
1497     # to check for errors here, as aio_stat will fail with ENOENT
1498     # when $etcdir is undef.
1499    
1500     aio_stat [$etcdir, "passwd"], sub {
1501     # yay
1502     };
1503     };
1504    
1505 root 1.214 That C<aio_wd> is a request and not a normal function shows that creating
1506     an IO::AIO::WD object is itself a potentially blocking operation, which is
1507     why it is done asynchronously.
1508    
1509     To stat the directory obtained with C<aio_wd> above, one could write
1510     either of the following three request calls:
1511    
1512     aio_lstat "/etc" , sub { ... # pathname as normal string
1513     aio_lstat [$wd, "."], sub { ... # "." relative to $wd (i.e. $wd itself)
1514     aio_lstat $wd , sub { ... # shorthand for the previous
1515 root 1.209
1516     As with normal pathnames, IO::AIO keeps a copy of the working directory
1517     object and the pathname string, so you could write the following without
1518     causing any issues due to C<$path> getting reused:
1519    
1520     my $path = [$wd, undef];
1521    
1522     for my $name (qw(abc def ghi)) {
1523     $path->[1] = $name;
1524     aio_stat $path, sub {
1525     # ...
1526     };
1527     }
1528    
1529     There are some caveats: when directories get renamed (or deleted), the
1530     pathname string doesn't change, so will point to the new directory (or
1531     nowhere at all), while the directory fd, if available on the system,
1532     will still point to the original directory. Most functions accepting a
1533     pathname will use the directory fd on newer systems, and the string on
1534     older systems. Some functions (such as realpath) will always rely on the
1535     string form of the pathname.
1536    
1537 root 1.239 So this functionality is mainly useful to get some protection against
1538 root 1.209 C<chdir>, to easily get an absolute path out of a relative path for future
1539     reference, and to speed up doing many operations in the same directory
1540     (e.g. when stat'ing all files in a directory).
1541    
1542     The following functions implement this working directory abstraction:
1543    
1544     =over 4
1545    
1546     =item aio_wd $pathname, $callback->($wd)
1547    
1548     Asynchonously canonicalise the given pathname and convert it to an
1549     IO::AIO::WD object representing it. If possible and supported on the
1550     system, also open a directory fd to speed up pathname resolution relative
1551     to this working directory.
1552    
1553     If something goes wrong, then C<undef> is passwd to the callback instead
1554     of a working directory object and C<$!> is set appropriately. Since
1555     passing C<undef> as working directory component of a pathname fails the
1556     request with C<ENOENT>, there is often no need for error checking in the
1557     C<aio_wd> callback, as future requests using the value will fail in the
1558     expected way.
1559    
1560     =item IO::AIO::CWD
1561    
1562     This is a compiletime constant (object) that represents the process
1563     current working directory.
1564    
1565 root 1.239 Specifying this object as working directory object for a pathname is as if
1566     the pathname would be specified directly, without a directory object. For
1567     example, these calls are functionally identical:
1568 root 1.209
1569     aio_stat "somefile", sub { ... };
1570     aio_stat [IO::AIO::CWD, "somefile"], sub { ... };
1571    
1572     =back
1573    
1574 root 1.239 To recover the path associated with an IO::AIO::WD object, you can use
1575     C<aio_realpath>:
1576    
1577     aio_realpath $wd, sub {
1578     warn "path is $_[0]\n";
1579     };
1580    
1581 root 1.241 Currently, C<aio_statvfs> always, and C<aio_rename> and C<aio_rmdir>
1582     sometimes, fall back to using an absolue path.
1583 root 1.209
1584 root 1.53 =head2 IO::AIO::REQ CLASS
1585 root 1.52
1586     All non-aggregate C<aio_*> functions return an object of this class when
1587     called in non-void context.
1588    
1589     =over 4
1590    
1591 root 1.65 =item cancel $req
1592 root 1.52
1593     Cancels the request, if possible. Has the effect of skipping execution
1594     when entering the B<execute> state and skipping calling the callback when
1595     entering the the B<result> state, but will leave the request otherwise
1596 root 1.151 untouched (with the exception of readdir). That means that requests that
1597     currently execute will not be stopped and resources held by the request
1598     will not be freed prematurely.
1599 root 1.52
1600 root 1.65 =item cb $req $callback->(...)
1601    
1602     Replace (or simply set) the callback registered to the request.
1603    
1604 root 1.52 =back
1605    
1606 root 1.55 =head2 IO::AIO::GRP CLASS
1607    
1608     This class is a subclass of L<IO::AIO::REQ>, so all its methods apply to
1609     objects of this class, too.
1610    
1611     A IO::AIO::GRP object is a special request that can contain multiple other
1612     aio requests.
1613    
1614     You create one by calling the C<aio_group> constructing function with a
1615     callback that will be called when all contained requests have entered the
1616     C<done> state:
1617    
1618     my $grp = aio_group sub {
1619     print "all requests are done\n";
1620     };
1621    
1622     You add requests by calling the C<add> method with one or more
1623     C<IO::AIO::REQ> objects:
1624    
1625     $grp->add (aio_unlink "...");
1626    
1627 root 1.58 add $grp aio_stat "...", sub {
1628     $_[0] or return $grp->result ("error");
1629    
1630     # add another request dynamically, if first succeeded
1631     add $grp aio_open "...", sub {
1632     $grp->result ("ok");
1633     };
1634     };
1635 root 1.55
1636     This makes it very easy to create composite requests (see the source of
1637     C<aio_move> for an application) that work and feel like simple requests.
1638    
1639 root 1.62 =over 4
1640    
1641     =item * The IO::AIO::GRP objects will be cleaned up during calls to
1642 root 1.55 C<IO::AIO::poll_cb>, just like any other request.
1643    
1644 root 1.62 =item * They can be canceled like any other request. Canceling will cancel not
1645 root 1.59 only the request itself, but also all requests it contains.
1646 root 1.55
1647 root 1.62 =item * They can also can also be added to other IO::AIO::GRP objects.
1648 root 1.55
1649 root 1.62 =item * You must not add requests to a group from within the group callback (or
1650 root 1.60 any later time).
1651    
1652 root 1.62 =back
1653    
1654 root 1.55 Their lifetime, simplified, looks like this: when they are empty, they
1655     will finish very quickly. If they contain only requests that are in the
1656     C<done> state, they will also finish. Otherwise they will continue to
1657     exist.
1658    
1659 root 1.133 That means after creating a group you have some time to add requests
1660     (precisely before the callback has been invoked, which is only done within
1661     the C<poll_cb>). And in the callbacks of those requests, you can add
1662     further requests to the group. And only when all those requests have
1663     finished will the the group itself finish.
1664 root 1.57
1665 root 1.55 =over 4
1666    
1667 root 1.65 =item add $grp ...
1668    
1669 root 1.55 =item $grp->add (...)
1670    
1671 root 1.57 Add one or more requests to the group. Any type of L<IO::AIO::REQ> can
1672     be added, including other groups, as long as you do not create circular
1673     dependencies.
1674    
1675     Returns all its arguments.
1676 root 1.55
1677 root 1.74 =item $grp->cancel_subs
1678    
1679     Cancel all subrequests and clears any feeder, but not the group request
1680     itself. Useful when you queued a lot of events but got a result early.
1681    
1682 root 1.168 The group request will finish normally (you cannot add requests to the
1683     group).
1684    
1685 root 1.58 =item $grp->result (...)
1686    
1687     Set the result value(s) that will be passed to the group callback when all
1688 root 1.120 subrequests have finished and set the groups errno to the current value
1689 root 1.80 of errno (just like calling C<errno> without an error number). By default,
1690     no argument will be passed and errno is zero.
1691    
1692     =item $grp->errno ([$errno])
1693    
1694     Sets the group errno value to C<$errno>, or the current value of errno
1695     when the argument is missing.
1696    
1697     Every aio request has an associated errno value that is restored when
1698     the callback is invoked. This method lets you change this value from its
1699     default (0).
1700    
1701     Calling C<result> will also set errno, so make sure you either set C<$!>
1702     before the call to C<result>, or call c<errno> after it.
1703 root 1.58
1704 root 1.65 =item feed $grp $callback->($grp)
1705 root 1.60
1706     Sets a feeder/generator on this group: every group can have an attached
1707     generator that generates requests if idle. The idea behind this is that,
1708     although you could just queue as many requests as you want in a group,
1709 root 1.139 this might starve other requests for a potentially long time. For example,
1710 root 1.211 C<aio_scandir> might generate hundreds of thousands of C<aio_stat>
1711     requests, delaying any later requests for a long time.
1712 root 1.60
1713     To avoid this, and allow incremental generation of requests, you can
1714     instead a group and set a feeder on it that generates those requests. The
1715 root 1.68 feed callback will be called whenever there are few enough (see C<limit>,
1716 root 1.60 below) requests active in the group itself and is expected to queue more
1717     requests.
1718    
1719 root 1.68 The feed callback can queue as many requests as it likes (i.e. C<add> does
1720     not impose any limits).
1721 root 1.60
1722 root 1.65 If the feed does not queue more requests when called, it will be
1723 root 1.60 automatically removed from the group.
1724    
1725 root 1.138 If the feed limit is C<0> when this method is called, it will be set to
1726     C<2> automatically.
1727 root 1.60
1728     Example:
1729    
1730     # stat all files in @files, but only ever use four aio requests concurrently:
1731    
1732     my $grp = aio_group sub { print "finished\n" };
1733 root 1.68 limit $grp 4;
1734 root 1.65 feed $grp sub {
1735 root 1.60 my $file = pop @files
1736     or return;
1737    
1738     add $grp aio_stat $file, sub { ... };
1739 root 1.65 };
1740 root 1.60
1741 root 1.68 =item limit $grp $num
1742 root 1.60
1743     Sets the feeder limit for the group: The feeder will be called whenever
1744     the group contains less than this many requests.
1745    
1746     Setting the limit to C<0> will pause the feeding process.
1747    
1748 root 1.138 The default value for the limit is C<0>, but note that setting a feeder
1749     automatically bumps it up to C<2>.
1750    
1751 root 1.55 =back
1752    
1753 root 1.5 =head2 SUPPORT FUNCTIONS
1754    
1755 root 1.86 =head3 EVENT PROCESSING AND EVENT LOOP INTEGRATION
1756    
1757 root 1.5 =over 4
1758    
1759     =item $fileno = IO::AIO::poll_fileno
1760    
1761 root 1.20 Return the I<request result pipe file descriptor>. This filehandle must be
1762 root 1.156 polled for reading by some mechanism outside this module (e.g. EV, Glib,
1763     select and so on, see below or the SYNOPSIS). If the pipe becomes readable
1764     you have to call C<poll_cb> to check the results.
1765 root 1.5
1766     See C<poll_cb> for an example.
1767    
1768     =item IO::AIO::poll_cb
1769    
1770 root 1.240 Process some requests that have reached the result phase (i.e. they have
1771     been executed but the results are not yet reported). You have to call
1772     this "regularly" to finish outstanding requests.
1773    
1774     Returns C<0> if all events could be processed (or there were no
1775     events to process), or C<-1> if it returned earlier for whatever
1776     reason. Returns immediately when no events are outstanding. The amount
1777     of events processed depends on the settings of C<IO::AIO::max_poll_req>,
1778     C<IO::AIO::max_poll_time> and C<IO::AIO::max_outstanding>.
1779    
1780     If not all requests were processed for whatever reason, the poll file
1781     descriptor will still be ready when C<poll_cb> returns, so normally you
1782     don't have to do anything special to have it called later.
1783 root 1.78
1784 root 1.192 Apart from calling C<IO::AIO::poll_cb> when the event filehandle becomes
1785     ready, it can be beneficial to call this function from loops which submit
1786     a lot of requests, to make sure the results get processed when they become
1787     available and not just when the loop is finished and the event loop takes
1788     over again. This function returns very fast when there are no outstanding
1789     requests.
1790    
1791 root 1.20 Example: Install an Event watcher that automatically calls
1792 root 1.156 IO::AIO::poll_cb with high priority (more examples can be found in the
1793     SYNOPSIS section, at the top of this document):
1794 root 1.5
1795     Event->io (fd => IO::AIO::poll_fileno,
1796     poll => 'r', async => 1,
1797     cb => \&IO::AIO::poll_cb);
1798    
1799 root 1.175 =item IO::AIO::poll_wait
1800    
1801 root 1.240 Wait until either at least one request is in the result phase or no
1802     requests are outstanding anymore.
1803    
1804     This is useful if you want to synchronously wait for some requests to
1805     become ready, without actually handling them.
1806 root 1.175
1807     See C<nreqs> for an example.
1808    
1809     =item IO::AIO::poll
1810    
1811     Waits until some requests have been handled.
1812    
1813     Returns the number of requests processed, but is otherwise strictly
1814     equivalent to:
1815    
1816     IO::AIO::poll_wait, IO::AIO::poll_cb
1817    
1818     =item IO::AIO::flush
1819    
1820     Wait till all outstanding AIO requests have been handled.
1821    
1822     Strictly equivalent to:
1823    
1824     IO::AIO::poll_wait, IO::AIO::poll_cb
1825     while IO::AIO::nreqs;
1826    
1827 root 1.86 =item IO::AIO::max_poll_reqs $nreqs
1828    
1829     =item IO::AIO::max_poll_time $seconds
1830    
1831     These set the maximum number of requests (default C<0>, meaning infinity)
1832     that are being processed by C<IO::AIO::poll_cb> in one call, respectively
1833     the maximum amount of time (default C<0>, meaning infinity) spent in
1834     C<IO::AIO::poll_cb> to process requests (more correctly the mininum amount
1835     of time C<poll_cb> is allowed to use).
1836 root 1.78
1837 root 1.89 Setting C<max_poll_time> to a non-zero value creates an overhead of one
1838     syscall per request processed, which is not normally a problem unless your
1839     callbacks are really really fast or your OS is really really slow (I am
1840     not mentioning Solaris here). Using C<max_poll_reqs> incurs no overhead.
1841    
1842 root 1.86 Setting these is useful if you want to ensure some level of
1843     interactiveness when perl is not fast enough to process all requests in
1844     time.
1845 root 1.78
1846 root 1.86 For interactive programs, values such as C<0.01> to C<0.1> should be fine.
1847 root 1.78
1848     Example: Install an Event watcher that automatically calls
1849 root 1.89 IO::AIO::poll_cb with low priority, to ensure that other parts of the
1850 root 1.78 program get the CPU sometimes even under high AIO load.
1851    
1852 root 1.86 # try not to spend much more than 0.1s in poll_cb
1853     IO::AIO::max_poll_time 0.1;
1854    
1855     # use a low priority so other tasks have priority
1856 root 1.78 Event->io (fd => IO::AIO::poll_fileno,
1857     poll => 'r', nice => 1,
1858 root 1.86 cb => &IO::AIO::poll_cb);
1859 root 1.78
1860 root 1.104 =back
1861    
1862 root 1.86 =head3 CONTROLLING THE NUMBER OF THREADS
1863 root 1.13
1864 root 1.105 =over
1865    
1866 root 1.5 =item IO::AIO::min_parallel $nthreads
1867    
1868 root 1.61 Set the minimum number of AIO threads to C<$nthreads>. The current
1869     default is C<8>, which means eight asynchronous operations can execute
1870     concurrently at any one time (the number of outstanding requests,
1871     however, is unlimited).
1872 root 1.5
1873 root 1.34 IO::AIO starts threads only on demand, when an AIO request is queued and
1874 root 1.86 no free thread exists. Please note that queueing up a hundred requests can
1875     create demand for a hundred threads, even if it turns out that everything
1876     is in the cache and could have been processed faster by a single thread.
1877 root 1.34
1878 root 1.61 It is recommended to keep the number of threads relatively low, as some
1879     Linux kernel versions will scale negatively with the number of threads
1880     (higher parallelity => MUCH higher latency). With current Linux 2.6
1881     versions, 4-32 threads should be fine.
1882 root 1.5
1883 root 1.34 Under most circumstances you don't need to call this function, as the
1884     module selects a default that is suitable for low to moderate load.
1885 root 1.5
1886     =item IO::AIO::max_parallel $nthreads
1887    
1888 root 1.34 Sets the maximum number of AIO threads to C<$nthreads>. If more than the
1889     specified number of threads are currently running, this function kills
1890     them. This function blocks until the limit is reached.
1891    
1892     While C<$nthreads> are zero, aio requests get queued but not executed
1893     until the number of threads has been increased again.
1894 root 1.5
1895     This module automatically runs C<max_parallel 0> at program end, to ensure
1896     that all threads are killed and that there are no outstanding requests.
1897    
1898     Under normal circumstances you don't need to call this function.
1899    
1900 root 1.86 =item IO::AIO::max_idle $nthreads
1901    
1902 root 1.188 Limit the number of threads (default: 4) that are allowed to idle
1903     (i.e., threads that did not get a request to process within the idle
1904     timeout (default: 10 seconds). That means if a thread becomes idle while
1905     C<$nthreads> other threads are also idle, it will free its resources and
1906     exit.
1907 root 1.86
1908     This is useful when you allow a large number of threads (e.g. 100 or 1000)
1909     to allow for extremely high load situations, but want to free resources
1910     under normal circumstances (1000 threads can easily consume 30MB of RAM).
1911    
1912     The default is probably ok in most situations, especially if thread
1913     creation is fast. If thread creation is very slow on your system you might
1914     want to use larger values.
1915    
1916 root 1.188 =item IO::AIO::idle_timeout $seconds
1917    
1918     Sets the minimum idle timeout (default 10) after which worker threads are
1919     allowed to exit. SEe C<IO::AIO::max_idle>.
1920    
1921 root 1.123 =item IO::AIO::max_outstanding $maxreqs
1922 root 1.5
1923 root 1.195 Sets the maximum number of outstanding requests to C<$nreqs>. If
1924     you do queue up more than this number of requests, the next call to
1925     C<IO::AIO::poll_cb> (and other functions calling C<poll_cb>, such as
1926     C<IO::AIO::flush> or C<IO::AIO::poll>) will block until the limit is no
1927     longer exceeded.
1928    
1929     In other words, this setting does not enforce a queue limit, but can be
1930     used to make poll functions block if the limit is exceeded.
1931    
1932 root 1.79 This is a very bad function to use in interactive programs because it
1933     blocks, and a bad way to reduce concurrency because it is inexact: Better
1934     use an C<aio_group> together with a feed callback.
1935    
1936 root 1.248 Its main use is in scripts without an event loop - when you want to stat
1937 root 1.195 a lot of files, you can write somehting like this:
1938    
1939     IO::AIO::max_outstanding 32;
1940    
1941     for my $path (...) {
1942     aio_stat $path , ...;
1943     IO::AIO::poll_cb;
1944     }
1945    
1946     IO::AIO::flush;
1947    
1948     The call to C<poll_cb> inside the loop will normally return instantly, but
1949     as soon as more thna C<32> reqeusts are in-flight, it will block until
1950     some requests have been handled. This keeps the loop from pushing a large
1951     number of C<aio_stat> requests onto the queue.
1952    
1953     The default value for C<max_outstanding> is very large, so there is no
1954     practical limit on the number of outstanding requests.
1955 root 1.5
1956 root 1.104 =back
1957    
1958 root 1.86 =head3 STATISTICAL INFORMATION
1959    
1960 root 1.104 =over
1961    
1962 root 1.86 =item IO::AIO::nreqs
1963    
1964     Returns the number of requests currently in the ready, execute or pending
1965     states (i.e. for which their callback has not been invoked yet).
1966    
1967     Example: wait till there are no outstanding requests anymore:
1968    
1969     IO::AIO::poll_wait, IO::AIO::poll_cb
1970     while IO::AIO::nreqs;
1971    
1972     =item IO::AIO::nready
1973    
1974     Returns the number of requests currently in the ready state (not yet
1975     executed).
1976    
1977     =item IO::AIO::npending
1978    
1979     Returns the number of requests currently in the pending state (executed,
1980     but not yet processed by poll_cb).
1981    
1982 root 1.5 =back
1983    
1984 root 1.157 =head3 MISCELLANEOUS FUNCTIONS
1985    
1986 root 1.248 IO::AIO implements some functions that are useful when you want to use
1987     some "Advanced I/O" function not available to in Perl, without going the
1988     "Asynchronous I/O" route. Many of these have an asynchronous C<aio_*>
1989     counterpart.
1990 root 1.157
1991     =over 4
1992    
1993     =item IO::AIO::sendfile $ofh, $ifh, $offset, $count
1994    
1995     Calls the C<eio_sendfile_sync> function, which is like C<aio_sendfile>,
1996     but is blocking (this makes most sense if you know the input data is
1997     likely cached already and the output filehandle is set to non-blocking
1998     operations).
1999    
2000     Returns the number of bytes copied, or C<-1> on error.
2001    
2002     =item IO::AIO::fadvise $fh, $offset, $len, $advice
2003    
2004 root 1.184 Simply calls the C<posix_fadvise> function (see its
2005 root 1.157 manpage for details). The following advice constants are
2006 root 1.207 available: C<IO::AIO::FADV_NORMAL>, C<IO::AIO::FADV_SEQUENTIAL>,
2007 root 1.157 C<IO::AIO::FADV_RANDOM>, C<IO::AIO::FADV_NOREUSE>,
2008     C<IO::AIO::FADV_WILLNEED>, C<IO::AIO::FADV_DONTNEED>.
2009    
2010     On systems that do not implement C<posix_fadvise>, this function returns
2011     ENOSYS, otherwise the return value of C<posix_fadvise>.
2012    
2013 root 1.184 =item IO::AIO::madvise $scalar, $offset, $len, $advice
2014    
2015     Simply calls the C<posix_madvise> function (see its
2016     manpage for details). The following advice constants are
2017 root 1.207 available: C<IO::AIO::MADV_NORMAL>, C<IO::AIO::MADV_SEQUENTIAL>,
2018 root 1.184 C<IO::AIO::MADV_RANDOM>, C<IO::AIO::MADV_WILLNEED>, C<IO::AIO::MADV_DONTNEED>.
2019    
2020     On systems that do not implement C<posix_madvise>, this function returns
2021     ENOSYS, otherwise the return value of C<posix_madvise>.
2022    
2023     =item IO::AIO::mprotect $scalar, $offset, $len, $protect
2024    
2025     Simply calls the C<mprotect> function on the preferably AIO::mmap'ed
2026     $scalar (see its manpage for details). The following protect
2027 root 1.207 constants are available: C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_READ>,
2028 root 1.184 C<IO::AIO::PROT_WRITE>, C<IO::AIO::PROT_EXEC>.
2029    
2030     On systems that do not implement C<mprotect>, this function returns
2031     ENOSYS, otherwise the return value of C<mprotect>.
2032    
2033 root 1.176 =item IO::AIO::mmap $scalar, $length, $prot, $flags, $fh[, $offset]
2034    
2035     Memory-maps a file (or anonymous memory range) and attaches it to the
2036 root 1.228 given C<$scalar>, which will act like a string scalar. Returns true on
2037     success, and false otherwise.
2038 root 1.176
2039     The only operations allowed on the scalar are C<substr>/C<vec> that don't
2040     change the string length, and most read-only operations such as copying it
2041     or searching it with regexes and so on.
2042    
2043     Anything else is unsafe and will, at best, result in memory leaks.
2044    
2045     The memory map associated with the C<$scalar> is automatically removed
2046     when the C<$scalar> is destroyed, or when the C<IO::AIO::mmap> or
2047     C<IO::AIO::munmap> functions are called.
2048    
2049     This calls the C<mmap>(2) function internally. See your system's manual
2050     page for details on the C<$length>, C<$prot> and C<$flags> parameters.
2051    
2052     The C<$length> must be larger than zero and smaller than the actual
2053     filesize.
2054    
2055     C<$prot> is a combination of C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_EXEC>,
2056     C<IO::AIO::PROT_READ> and/or C<IO::AIO::PROT_WRITE>,
2057    
2058     C<$flags> can be a combination of C<IO::AIO::MAP_SHARED> or
2059     C<IO::AIO::MAP_PRIVATE>, or a number of system-specific flags (when
2060     not available, the are defined as 0): C<IO::AIO::MAP_ANONYMOUS>
2061     (which is set to C<MAP_ANON> if your system only provides this
2062     constant), C<IO::AIO::MAP_HUGETLB>, C<IO::AIO::MAP_LOCKED>,
2063     C<IO::AIO::MAP_NORESERVE>, C<IO::AIO::MAP_POPULATE> or
2064     C<IO::AIO::MAP_NONBLOCK>
2065    
2066     If C<$fh> is C<undef>, then a file descriptor of C<-1> is passed.
2067    
2068 root 1.179 C<$offset> is the offset from the start of the file - it generally must be
2069     a multiple of C<IO::AIO::PAGESIZE> and defaults to C<0>.
2070    
2071 root 1.177 Example:
2072    
2073     use Digest::MD5;
2074     use IO::AIO;
2075    
2076     open my $fh, "<verybigfile"
2077     or die "$!";
2078    
2079     IO::AIO::mmap my $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh
2080     or die "verybigfile: $!";
2081    
2082     my $fast_md5 = md5 $data;
2083    
2084 root 1.176 =item IO::AIO::munmap $scalar
2085    
2086     Removes a previous mmap and undefines the C<$scalar>.
2087    
2088 root 1.182 =item IO::AIO::munlock $scalar, $offset = 0, $length = undef
2089 root 1.174
2090 root 1.182 Calls the C<munlock> function, undoing the effects of a previous
2091     C<aio_mlock> call (see its description for details).
2092 root 1.174
2093     =item IO::AIO::munlockall
2094    
2095     Calls the C<munlockall> function.
2096    
2097     On systems that do not implement C<munlockall>, this function returns
2098     ENOSYS, otherwise the return value of C<munlockall>.
2099    
2100 root 1.225 =item IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
2101    
2102     Calls the GNU/Linux C<splice(2)> syscall, if available. If C<$r_off> or
2103     C<$w_off> are C<undef>, then C<NULL> is passed for these, otherwise they
2104     should be the file offset.
2105    
2106 root 1.227 C<$r_fh> and C<$w_fh> should not refer to the same file, as splice might
2107     silently corrupt the data in this case.
2108    
2109 root 1.225 The following symbol flag values are available: C<IO::AIO::SPLICE_F_MOVE>,
2110     C<IO::AIO::SPLICE_F_NONBLOCK>, C<IO::AIO::SPLICE_F_MORE> and
2111     C<IO::AIO::SPLICE_F_GIFT>.
2112    
2113     See the C<splice(2)> manpage for details.
2114    
2115     =item IO::AIO::tee $r_fh, $w_fh, $length, $flags
2116    
2117 root 1.248 Calls the GNU/Linux C<tee(2)> syscall, see its manpage and the
2118 root 1.225 description for C<IO::AIO::splice> above for details.
2119    
2120 root 1.243 =item $actual_size = IO::AIO::pipesize $r_fh[, $new_size]
2121    
2122     Attempts to query or change the pipe buffer size. Obviously works only
2123     on pipes, and currently works only on GNU/Linux systems, and fails with
2124     C<-1>/C<ENOSYS> everywhere else. If anybody knows how to influence pipe buffer
2125     size on other systems, drop me a note.
2126    
2127 root 1.157 =back
2128    
2129 root 1.1 =cut
2130    
2131 root 1.61 min_parallel 8;
2132 root 1.1
2133 root 1.95 END { flush }
2134 root 1.82
2135 root 1.1 1;
2136    
2137 root 1.175 =head1 EVENT LOOP INTEGRATION
2138    
2139     It is recommended to use L<AnyEvent::AIO> to integrate IO::AIO
2140     automatically into many event loops:
2141    
2142     # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
2143     use AnyEvent::AIO;
2144    
2145     You can also integrate IO::AIO manually into many event loops, here are
2146     some examples of how to do this:
2147    
2148     # EV integration
2149     my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
2150    
2151     # Event integration
2152     Event->io (fd => IO::AIO::poll_fileno,
2153     poll => 'r',
2154     cb => \&IO::AIO::poll_cb);
2155    
2156     # Glib/Gtk2 integration
2157     add_watch Glib::IO IO::AIO::poll_fileno,
2158     in => sub { IO::AIO::poll_cb; 1 };
2159    
2160     # Tk integration
2161     Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
2162     readable => \&IO::AIO::poll_cb);
2163    
2164     # Danga::Socket integration
2165     Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
2166     \&IO::AIO::poll_cb);
2167    
2168 root 1.27 =head2 FORK BEHAVIOUR
2169    
2170 root 1.197 Usage of pthreads in a program changes the semantics of fork
2171     considerably. Specifically, only async-safe functions can be called after
2172     fork. Perl doesn't know about this, so in general, you cannot call fork
2173 root 1.204 with defined behaviour in perl if pthreads are involved. IO::AIO uses
2174     pthreads, so this applies, but many other extensions and (for inexplicable
2175     reasons) perl itself often is linked against pthreads, so this limitation
2176     applies to quite a lot of perls.
2177    
2178     This module no longer tries to fight your OS, or POSIX. That means IO::AIO
2179     only works in the process that loaded it. Forking is fully supported, but
2180     using IO::AIO in the child is not.
2181    
2182     You might get around by not I<using> IO::AIO before (or after)
2183     forking. You could also try to call the L<IO::AIO::reinit> function in the
2184     child:
2185    
2186     =over 4
2187    
2188     =item IO::AIO::reinit
2189    
2190 root 1.207 Abandons all current requests and I/O threads and simply reinitialises all
2191     data structures. This is not an operation supported by any standards, but
2192 root 1.204 happens to work on GNU/Linux and some newer BSD systems.
2193    
2194     The only reasonable use for this function is to call it after forking, if
2195     C<IO::AIO> was used in the parent. Calling it while IO::AIO is active in
2196     the process will result in undefined behaviour. Calling it at any time
2197     will also result in any undefined (by POSIX) behaviour.
2198    
2199     =back
2200 root 1.52
2201 root 1.60 =head2 MEMORY USAGE
2202    
2203 root 1.72 Per-request usage:
2204    
2205     Each aio request uses - depending on your architecture - around 100-200
2206     bytes of memory. In addition, stat requests need a stat buffer (possibly
2207     a few hundred bytes), readdir requires a result buffer and so on. Perl
2208     scalars and other data passed into aio requests will also be locked and
2209     will consume memory till the request has entered the done state.
2210 root 1.60
2211 root 1.111 This is not awfully much, so queuing lots of requests is not usually a
2212 root 1.60 problem.
2213    
2214 root 1.72 Per-thread usage:
2215    
2216     In the execution phase, some aio requests require more memory for
2217     temporary buffers, and each thread requires a stack and other data
2218     structures (usually around 16k-128k, depending on the OS).
2219    
2220     =head1 KNOWN BUGS
2221    
2222 root 1.73 Known bugs will be fixed in the next release.
2223 root 1.60
2224 root 1.1 =head1 SEE ALSO
2225    
2226 root 1.125 L<AnyEvent::AIO> for easy integration into event loops, L<Coro::AIO> for a
2227     more natural syntax.
2228 root 1.1
2229     =head1 AUTHOR
2230    
2231     Marc Lehmann <schmorp@schmorp.de>
2232     http://home.schmorp.de/
2233    
2234     =cut
2235