ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
Revision: 1.322
Committed: Sun Mar 30 13:03:20 2025 UTC (7 weeks, 2 days ago) by root
Branch: MAIN
CVS Tags: HEAD
Changes since 1.321: +8 -11 lines
Log Message:
*** empty log message ***

File Contents

# Content
1 =head1 NAME
2
3 IO::AIO - Asynchronous/Advanced Input/Output
4
5 =head1 SYNOPSIS
6
7 use IO::AIO;
8
9 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
10 my $fh = shift
11 or die "/etc/passwd: $!";
12 ...
13 };
14
15 aio_unlink "/tmp/file", sub { };
16
17 aio_read $fh, 30000, 1024, $buffer, 0, sub {
18 $_[0] > 0 or die "read error: $!";
19 };
20
21 # version 2+ has request and group objects
22 use IO::AIO 2;
23
24 aioreq_pri 4; # give next request a very high priority
25 my $req = aio_unlink "/tmp/file", sub { };
26 $req->cancel; # cancel request if still in queue
27
28 my $grp = aio_group sub { print "all stats done\n" };
29 add $grp aio_stat "..." for ...;
30
31 =head1 DESCRIPTION
32
33 This module implements asynchronous I/O using whatever means your
34 operating system supports. It is implemented as an interface to C<libeio>
35 (L<http://software.schmorp.de/pkg/libeio.html>).
36
37 Asynchronous means that operations that can normally block your program
38 (e.g. reading from disk) will be done asynchronously: the operation
39 will still block, but you can do something else in the meantime. This
40 is extremely useful for programs that need to stay interactive even
41 when doing heavy I/O (GUI programs, high performance network servers
42 etc.), but can also be used to easily do operations in parallel that are
43 normally done sequentially, e.g. stat'ing many files, which is much faster
44 on a RAID volume or over NFS when you do a number of stat operations
45 concurrently.
46
47 While most of this works on all types of file descriptors (for
48 example sockets), using these functions on file descriptors that
49 support nonblocking operation (again, sockets, pipes etc.) is
50 very inefficient. Use an event loop for that (such as the L<EV>
51 module): IO::AIO will naturally fit into such an event loop itself.
52
53 In this version, a number of threads are started that execute your
54 requests and signal their completion. You don't need thread support
55 in perl, and the threads created by this module will not be visible
56 to perl. In the future, this module might make use of the native aio
57 functions available on many operating systems. However, they are often
58 not well-supported or restricted (GNU/Linux doesn't allow them on normal
59 files currently, for example), and they would only support aio_read and
60 aio_write, so the remaining functionality would have to be implemented
61 using threads anyway.
62
63 In addition to asynchronous I/O, this module also exports some rather
64 arcane interfaces, such as C<madvise> or linux's C<splice> system call,
65 which is why the C<A> in C<AIO> can also mean I<advanced>.
66
67 Although the module will work in the presence of other (Perl-) threads,
68 it is currently not reentrant in any way, so use appropriate locking
69 yourself, always call C<poll_cb> from within the same thread, or never
70 call C<poll_cb> (or other C<aio_> functions) recursively.
71
72 =head2 EXAMPLE
73
74 This is a simple example that uses the EV module and loads
75 F</etc/passwd> asynchronously:
76
77 use EV;
78 use IO::AIO;
79
80 # register the IO::AIO callback with EV
81 my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
82
83 # queue the request to open /etc/passwd
84 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
85 my $fh = shift
86 or die "error while opening: $!";
87
88 # stat'ing filehandles is generally non-blocking
89 my $size = -s $fh;
90
91 # queue a request to read the file
92 my $contents;
93 aio_read $fh, 0, $size, $contents, 0, sub {
94 $_[0] == $size
95 or die "short read: $!";
96
97 close $fh;
98
99 # file contents now in $contents
100 print $contents;
101
102 # exit event loop and program
103 EV::break;
104 };
105 };
106
107 # possibly queue up other requests, or open GUI windows,
108 # check for sockets etc. etc.
109
110 # process events as long as there are some:
111 EV::run;
112
113 =head1 REQUEST ANATOMY AND LIFETIME
114
115 Every C<aio_*> function creates a request. which is a C data structure not
116 directly visible to Perl.
117
118 If called in non-void context, every request function returns a Perl
119 object representing the request. In void context, nothing is returned,
120 which saves a bit of memory.
121
122 The perl object is a fairly standard ref-to-hash object. The hash contents
123 are not used by IO::AIO so you are free to store anything you like in it.
124
125 During their existance, aio requests travel through the following states,
126 in order:
127
128 =over 4
129
130 =item ready
131
132 Immediately after a request is created it is put into the ready state,
133 waiting for a thread to execute it.
134
135 =item execute
136
137 A thread has accepted the request for processing and is currently
138 executing it (e.g. blocking in read).
139
140 =item pending
141
142 The request has been executed and is waiting for result processing.
143
144 While request submission and execution is fully asynchronous, result
145 processing is not and relies on the perl interpreter calling C<poll_cb>
146 (or another function with the same effect).
147
148 =item result
149
150 The request results are processed synchronously by C<poll_cb>.
151
152 The C<poll_cb> function will process all outstanding aio requests by
153 calling their callbacks, freeing memory associated with them and managing
154 any groups they are contained in.
155
156 =item done
157
158 Request has reached the end of its lifetime and holds no resources anymore
159 (except possibly for the Perl object, but its connection to the actual
160 aio request is severed and calling its methods will either do nothing or
161 result in a runtime error).
162
163 =back
164
165 =cut
166
167 package IO::AIO;
168
169 use Carp ();
170
171 use common::sense;
172
173 use base 'Exporter';
174
175 BEGIN {
176 our $VERSION = 4.81;
177
178 our @AIO_REQ = qw(aio_sendfile aio_seek aio_read aio_write aio_open aio_close
179 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
180 aio_scandir aio_symlink aio_readlink aio_realpath aio_fcntl aio_ioctl
181 aio_sync aio_fsync aio_syncfs aio_fdatasync aio_sync_file_range
182 aio_pathsync aio_readahead aio_fiemap aio_allocate
183 aio_rename aio_rename2 aio_link aio_move aio_copy aio_group
184 aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
185 aio_chmod aio_utime aio_truncate
186 aio_msync aio_mtouch aio_mlock aio_mlockall
187 aio_statvfs
188 aio_slurp
189 aio_wd);
190
191 our @EXPORT = (@AIO_REQ, qw(aioreq_pri aioreq_nice));
192 our @EXPORT_OK = qw(poll_fileno poll_cb poll_wait flush
193 min_parallel max_parallel max_idle idle_timeout
194 nreqs nready npending nthreads
195 max_poll_time max_poll_reqs
196 sendfile fadvise madvise
197 mmap munmap mremap munlock munlockall
198
199 accept4 tee splice pipe2 pipesize
200 fexecve mount umount memfd_create eventfd
201 timerfd_create timerfd_settime timerfd_gettime
202 pidfd_open pidfd_send_signal pidfd_getfd);
203
204 push @AIO_REQ, qw(aio_busy); # not exported
205
206 @IO::AIO::GRP::ISA = 'IO::AIO::REQ';
207
208 require XSLoader;
209 XSLoader::load ("IO::AIO", $VERSION);
210 }
211
212 =head1 FUNCTIONS
213
214 =head2 QUICK OVERVIEW
215
216 This section simply lists the prototypes most of the functions for
217 quick reference. See the following sections for function-by-function
218 documentation.
219
220 aio_wd $pathname, $callback->($wd)
221 aio_open $pathname, $flags, $mode, $callback->($fh)
222 aio_close $fh, $callback->($status)
223 aio_seek $fh,$offset,$whence, $callback->($offs)
224 aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
225 aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
226 aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
227 aio_readahead $fh,$offset,$length, $callback->($retval)
228 aio_stat $fh_or_path, $callback->($status)
229 aio_lstat $fh, $callback->($status)
230 aio_statvfs $fh_or_path, $callback->($statvfs)
231 aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
232 aio_chown $fh_or_path, $uid, $gid, $callback->($status)
233 aio_chmod $fh_or_path, $mode, $callback->($status)
234 aio_truncate $fh_or_path, $offset, $callback->($status)
235 aio_allocate $fh, $mode, $offset, $len, $callback->($status)
236 aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
237 aio_unlink $pathname, $callback->($status)
238 aio_mknod $pathname, $mode, $dev, $callback->($status)
239 aio_link $srcpath, $dstpath, $callback->($status)
240 aio_symlink $srcpath, $dstpath, $callback->($status)
241 aio_readlink $pathname, $callback->($link)
242 aio_realpath $pathname, $callback->($path)
243 aio_rename $srcpath, $dstpath, $callback->($status)
244 aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
245 aio_mkdir $pathname, $mode, $callback->($status)
246 aio_rmdir $pathname, $callback->($status)
247 aio_readdir $pathname, $callback->($entries)
248 aio_readdirx $pathname, $flags, $callback->($entries, $flags)
249 IO::AIO::READDIR_DENTS IO::AIO::READDIR_DIRS_FIRST
250 IO::AIO::READDIR_STAT_ORDER IO::AIO::READDIR_FOUND_UNKNOWN
251 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
252 aio_load $pathname, $data, $callback->($status)
253 aio_copy $srcpath, $dstpath, $callback->($status)
254 aio_move $srcpath, $dstpath, $callback->($status)
255 aio_rmtree $pathname, $callback->($status)
256 aio_fcntl $fh, $cmd, $arg, $callback->($status)
257 aio_ioctl $fh, $request, $buf, $callback->($status)
258 aio_sync $callback->($status)
259 aio_syncfs $fh, $callback->($status)
260 aio_fsync $fh, $callback->($status)
261 aio_fdatasync $fh, $callback->($status)
262 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
263 aio_pathsync $pathname, $callback->($status)
264 aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC, $callback->($status)
265 aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
266 aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
267 aio_mlockall $flags, $callback->($status)
268 aio_group $callback->(...)
269 aio_nop $callback->()
270
271 $prev_pri = aioreq_pri [$pri]
272 aioreq_nice $pri_adjust
273
274 IO::AIO::poll_wait
275 IO::AIO::poll_cb
276 IO::AIO::poll
277 IO::AIO::flush
278 IO::AIO::max_poll_reqs $nreqs
279 IO::AIO::max_poll_time $seconds
280 IO::AIO::min_parallel $nthreads
281 IO::AIO::max_parallel $nthreads
282 IO::AIO::max_idle $nthreads
283 IO::AIO::idle_timeout $seconds
284 IO::AIO::max_outstanding $maxreqs
285 IO::AIO::nreqs
286 IO::AIO::nready
287 IO::AIO::npending
288 IO::AIO::reinit
289
290 $nfd = IO::AIO::get_fdlimit
291 IO::AIO::min_fdlimit $nfd
292
293 IO::AIO::sendfile $ofh, $ifh, $offset, $count
294 IO::AIO::fadvise $fh, $offset, $len, $advice
295 IO::AIO::fexecve $fh, $argv, $envp
296
297 IO::AIO::mmap $scalar, $length, $prot, $flags[, $fh[, $offset]]
298 IO::AIO::munmap $scalar
299 IO::AIO::mremap $scalar, $new_length, $flags[, $new_address]
300 IO::AIO::madvise $scalar, $offset, $length, $advice
301 IO::AIO::mprotect $scalar, $offset, $length, $protect
302 IO::AIO::munlock $scalar, $offset = 0, $length = undef
303 IO::AIO::munlockall
304
305 # stat extensions, should work with built-in stat and aio_stat
306 $counter = IO::AIO::st_gen
307 $seconds = IO::AIO::st_atime, IO::AIO::st_mtime, IO::AIO::st_ctime, IO::AIO::st_btime
308 [$seconds, $nanoseconds] = IO::AIO::st_atime2, IO::AIO::st_mtime2, IO::AIO::st_ctime2, IO::AIO::st_btime2
309 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime
310 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime2 # returns [sec, nsec] arrayrefs
311 $int_seconds = IO::AIO::st_atimesec, IO::AIO::st_mtimesec, IO::AIO::st_ctimesec, IO::AIO::st_btimesec
312 $nanoseconds = IO::AIO::st_atimensec, IO::AIO::st_mtimensec, IO::AIO::st_ctimensec, IO::AIO::st_btimensec
313 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtimesec
314 ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtimensec
315
316 # very much unportable syscalls
317 IO::AIO::accept4 $r_fh, $sockaddr, $sockaddr_len, $flags
318 IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
319 IO::AIO::tee $r_fh, $w_fh, $length, $flags
320
321 $actual_size = IO::AIO::pipesize $r_fh[, $new_size]
322 ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
323
324 $fh = IO::AIO::eventfd [$initval, [$flags]]
325 $fh = IO::AIO::memfd_create $pathname[, $flags]
326
327 $fh = IO::AIO::timerfd_create $clockid[, $flags]
328 ($cur_interval, $cur_value) = IO::AIO::timerfd_settime $fh, $flags, $new_interval, $nbw_value
329 ($cur_interval, $cur_value) = IO::AIO::timerfd_gettime $fh
330
331 $fh = IO::AIO::pidfd_open $pid[, $flags]
332 $status = IO::AIO::pidfd_send_signal $pidfh, $signal[, $siginfo[, $flags]]
333 $fh = IO::AIO::pidfd_getfd $pidfh, $targetfd[, $flags]
334
335 $retval = IO::AIO::mount $special, $path, $fstype, $flags = 0, $data = undef
336 $retval = IO::AIO::umount $path, $flags = 0
337
338 =head2 API NOTES
339
340 All the C<aio_*> calls are more or less thin wrappers around the syscall
341 with the same name (sans C<aio_>). The arguments are similar or identical,
342 and they all accept an additional (and optional) C<$callback> argument
343 which must be a code reference. This code reference will be called after
344 the syscall has been executed in an asynchronous fashion. The results
345 of the request will be passed as arguments to the callback (and, if an
346 error occured, in C<$!>) - for most requests the syscall return code (e.g.
347 most syscalls return C<-1> on error, unlike perl, which usually delivers
348 "false").
349
350 Some requests (such as C<aio_readdir>) pass the actual results and
351 communicate failures by passing C<undef>.
352
353 All functions expecting a filehandle keep a copy of the filehandle
354 internally until the request has finished.
355
356 All functions return request objects of type L<IO::AIO::REQ> that allow
357 further manipulation of those requests while they are in-flight.
358
359 The pathnames you pass to these routines I<should> be absolute. The
360 reason for this is that at the time the request is being executed, the
361 current working directory could have changed. Alternatively, you can
362 make sure that you never change the current working directory anywhere
363 in the program and then use relative paths. You can also take advantage
364 of IO::AIOs working directory abstraction, that lets you specify paths
365 relative to some previously-opened "working directory object" - see the
366 description of the C<IO::AIO::WD> class later in this document.
367
368 To encode pathnames as octets, either make sure you either: a) always pass
369 in filenames you got from outside (command line, readdir etc.) without
370 tinkering, b) are in your native filesystem encoding, c) use the Encode
371 module and encode your pathnames to the locale (or other) encoding in
372 effect in the user environment, d) use Glib::filename_from_unicode on
373 unicode filenames or e) use something else to ensure your scalar has the
374 correct contents.
375
376 This works, btw. independent of the internal UTF-8 bit, which IO::AIO
377 handles correctly whether it is set or not.
378
379 =head2 AIO REQUEST FUNCTIONS
380
381 =over 4
382
383 =item $prev_pri = aioreq_pri [$pri]
384
385 Returns the priority value that would be used for the next request and, if
386 C<$pri> is given, sets the priority for the next aio request.
387
388 The default priority is C<0>, the minimum and maximum priorities are C<-4>
389 and C<4>, respectively. Requests with higher priority will be serviced
390 first.
391
392 The priority will be reset to C<0> after each call to one of the C<aio_*>
393 functions.
394
395 Example: open a file with low priority, then read something from it with
396 higher priority so the read request is serviced before other low priority
397 open requests (potentially spamming the cache):
398
399 aioreq_pri -3;
400 aio_open ..., sub {
401 return unless $_[0];
402
403 aioreq_pri -2;
404 aio_read $_[0], ..., sub {
405 ...
406 };
407 };
408
409
410 =item aioreq_nice $pri_adjust
411
412 Similar to C<aioreq_pri>, but subtracts the given value from the current
413 priority, so the effect is cumulative.
414
415
416 =item aio_open $pathname, $flags, $mode, $callback->($fh)
417
418 Asynchronously open or create a file and call the callback with a newly
419 created filehandle for the file (or C<undef> in case of an error).
420
421 The C<$flags> argument is a bitmask. See the C<Fcntl> module for a
422 list. They are the same as used by C<sysopen>.
423
424 Likewise, C<$mode> specifies the mode of the newly created file, if it
425 didn't exist and C<O_CREAT> has been given, just like perl's C<sysopen>,
426 except that it is mandatory (i.e. use C<0> if you don't create new files,
427 and C<0666> or C<0777> if you do). Note that the C<$mode> will be modified
428 by the umask in effect then the request is being executed, so better never
429 change the umask.
430
431 Example:
432
433 aio_open "/etc/passwd", IO::AIO::O_RDONLY, 0, sub {
434 if ($_[0]) {
435 print "open successful, fh is $_[0]\n";
436 ...
437 } else {
438 die "open failed: $!\n";
439 }
440 };
441
442 In addition to all the common open modes/flags (C<O_RDONLY>, C<O_WRONLY>,
443 C<O_RDWR>, C<O_CREAT>, C<O_TRUNC>, C<O_EXCL> and C<O_APPEND>), the
444 following POSIX and non-POSIX constants are available (missing ones on
445 your system are, as usual, C<0>):
446
447 C<O_ASYNC>, C<O_DIRECT>, C<O_NOATIME>, C<O_CLOEXEC>, C<O_NOCTTY>, C<O_NOFOLLOW>,
448 C<O_NONBLOCK>, C<O_EXEC>, C<O_SEARCH>, C<O_DIRECTORY>, C<O_DSYNC>,
449 C<O_RSYNC>, C<O_SYNC>, C<O_PATH>, C<O_TMPFILE>, C<O_TTY_INIT> and C<O_ACCMODE>.
450
451
452 =item aio_close $fh, $callback->($status)
453
454 Asynchronously close a file and call the callback with the result
455 code.
456
457 Unfortunately, you can't do this to perl. Perl I<insists> very strongly on
458 closing the file descriptor associated with the filehandle itself.
459
460 Therefore, C<aio_close> will not close the filehandle - instead it will
461 use dup2 to overwrite the file descriptor with the write-end of a pipe
462 (the pipe fd will be created on demand and will be cached).
463
464 Or in other words: the file descriptor will be closed, but it will not be
465 free for reuse until the perl filehandle is closed.
466
467 =cut
468
469 =item aio_seek $fh, $offset, $whence, $callback->($offs)
470
471 Seeks the filehandle to the new C<$offset>, similarly to perl's
472 C<sysseek>. The C<$whence> can use the traditional values (C<0> for
473 C<IO::AIO::SEEK_SET>, C<1> for C<IO::AIO::SEEK_CUR> or C<2> for
474 C<IO::AIO::SEEK_END>).
475
476 The resulting absolute offset will be passed to the callback, or C<-1> in
477 case of an error.
478
479 In theory, the C<$whence> constants could be different than the
480 corresponding values from L<Fcntl>, but perl guarantees they are the same,
481 so don't panic.
482
483 As a GNU/Linux (and maybe Solaris) extension, also the constants
484 C<IO::AIO::SEEK_DATA> and C<IO::AIO::SEEK_HOLE> are available, if they
485 could be found. No guarantees about suitability for use in C<aio_seek> or
486 Perl's C<sysseek> can be made though, although I would naively assume they
487 "just work".
488
489 =item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
490
491 =item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
492
493 Reads or writes C<$length> bytes from or to the specified C<$fh> and
494 C<$offset> into the scalar given by C<$data> and offset C<$dataoffset> and
495 calls the callback with the actual number of bytes transferred (or -1 on
496 error, just like the syscall).
497
498 C<aio_read> will, like C<sysread>, shrink or grow the C<$data> scalar to
499 offset plus the actual number of bytes read.
500
501 If C<$offset> is undefined, then the current file descriptor offset will
502 be used (and updated), otherwise the file descriptor offset will not be
503 changed by these calls.
504
505 If C<$length> is undefined in C<aio_write>, use the remaining length of
506 C<$data>.
507
508 If C<$dataoffset> is less than zero, it will be counted from the end of
509 C<$data>.
510
511 The C<$data> scalar I<MUST NOT> be modified in any way while the request
512 is outstanding. Modifying it can result in segfaults or World War III (if
513 the necessary/optional hardware is installed).
514
515 Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at
516 offset C<0> within the scalar:
517
518 aio_read $fh, 7, 15, $buffer, 0, sub {
519 $_[0] > 0 or die "read error: $!";
520 print "read $_[0] bytes: <$buffer>\n";
521 };
522
523
524 =item aio_sendfile $out_fh, $in_fh, $in_offset, $length, $callback->($retval)
525
526 Tries to copy C<$length> bytes from C<$in_fh> to C<$out_fh>. It starts
527 reading at byte offset C<$in_offset>, and starts writing at the current
528 file offset of C<$out_fh>. Because of that, it is not safe to issue more
529 than one C<aio_sendfile> per C<$out_fh>, as they will interfere with each
530 other. The same C<$in_fh> works fine though, as this function does not
531 move or use the file offset of C<$in_fh>.
532
533 Please note that C<aio_sendfile> can read more bytes from C<$in_fh> than
534 are written, and there is no way to find out how many more bytes have been
535 read from C<aio_sendfile> alone, as C<aio_sendfile> only provides the
536 number of bytes written to C<$out_fh>. Only if the result value equals
537 C<$length> one can assume that C<$length> bytes have been read.
538
539 Unlike with other C<aio_> functions, it makes a lot of sense to use
540 C<aio_sendfile> on non-blocking sockets, as long as one end (typically
541 the C<$in_fh>) is a file - the file I/O will then be asynchronous, while
542 the socket I/O will be non-blocking. Note, however, that you can run
543 into a trap where C<aio_sendfile> reads some data with readahead, then
544 fails to write all data, and when the socket is ready the next time, the
545 data in the cache is already lost, forcing C<aio_sendfile> to again hit
546 the disk. Explicit C<aio_read> + C<aio_write> let's you better control
547 resource usage.
548
549 This call tries to make use of a native C<sendfile>-like syscall to
550 provide zero-copy operation. For this to work, C<$out_fh> should refer to
551 a socket, and C<$in_fh> should refer to an mmap'able file.
552
553 If a native sendfile cannot be found or it fails with C<ENOSYS>,
554 C<EINVAL>, C<ENOTSUP>, C<EOPNOTSUPP>, C<EAFNOSUPPORT>, C<EPROTOTYPE> or
555 C<ENOTSOCK>, it will be emulated, so you can call C<aio_sendfile> on any
556 type of filehandle regardless of the limitations of the operating system.
557
558 As native sendfile syscalls (as practically any non-POSIX interface hacked
559 together in a hurry to improve benchmark numbers) tend to be rather buggy
560 on many systems, this implementation tries to work around some known bugs
561 in Linux and FreeBSD kernels (probably others, too), but that might fail,
562 so you really really should check the return value of C<aio_sendfile> -
563 fewer bytes than expected might have been transferred.
564
565
566 =item aio_readahead $fh,$offset,$length, $callback->($retval)
567
568 C<aio_readahead> populates the page cache with data from a file so that
569 subsequent reads from that file will not block on disk I/O. The C<$offset>
570 argument specifies the starting point from which data is to be read and
571 C<$length> specifies the number of bytes to be read. I/O is performed in
572 whole pages, so that offset is effectively rounded down to a page boundary
573 and bytes are read up to the next page boundary greater than or equal to
574 (off-set+length). C<aio_readahead> does not read beyond the end of the
575 file. The current file offset of the file is left unchanged.
576
577 If that syscall doesn't exist (likely if your kernel isn't Linux) it will
578 be emulated by simply reading the data, which would have a similar effect.
579
580
581 =item aio_stat $fh_or_path, $callback->($status)
582
583 =item aio_lstat $fh, $callback->($status)
584
585 Works almost exactly like perl's C<stat> or C<lstat> in void context. The
586 callback will be called after the stat and the results will be available
587 using C<stat _> or C<-s _> and other tests (with the exception of C<-B>
588 and C<-T>).
589
590 Currently, the stats are always 64-bit-stats, i.e. instead of returning an
591 error when stat'ing a large file, the results will be silently truncated
592 unless perl itself is compiled with large file support.
593
594 To help interpret the mode and dev/rdev stat values, IO::AIO offers the
595 following constants and functions (if not implemented, the constants will
596 be C<0> and the functions will either C<croak> or fall back on traditional
597 behaviour).
598
599 C<S_IFMT>, C<S_IFIFO>, C<S_IFCHR>, C<S_IFBLK>, C<S_IFLNK>, C<S_IFREG>,
600 C<S_IFDIR>, C<S_IFWHT>, C<S_IFSOCK>, C<IO::AIO::major $dev_t>,
601 C<IO::AIO::minor $dev_t>, C<IO::AIO::makedev $major, $minor>.
602
603 To access higher resolution stat timestamps, see L<SUBSECOND STAT TIME
604 ACCESS>.
605
606 Example: Print the length of F</etc/passwd>:
607
608 aio_stat "/etc/passwd", sub {
609 $_[0] and die "stat failed: $!";
610 print "size is ", -s _, "\n";
611 };
612
613
614 =item aio_statvfs $fh_or_path, $callback->($statvfs)
615
616 Works like the POSIX C<statvfs> or C<fstatvfs> syscalls, depending on
617 whether a file handle or path was passed.
618
619 On success, the callback is passed a hash reference with the following
620 members: C<bsize>, C<frsize>, C<blocks>, C<bfree>, C<bavail>, C<files>,
621 C<ffree>, C<favail>, C<fsid>, C<flag> and C<namemax>. On failure, C<undef>
622 is passed.
623
624 The following POSIX IO::AIO::ST_* constants are defined: C<ST_RDONLY> and
625 C<ST_NOSUID>.
626
627 The following non-POSIX IO::AIO::ST_* flag masks are defined to
628 their correct value when available, or to C<0> on systems that do
629 not support them: C<ST_NODEV>, C<ST_NOEXEC>, C<ST_SYNCHRONOUS>,
630 C<ST_MANDLOCK>, C<ST_WRITE>, C<ST_APPEND>, C<ST_IMMUTABLE>, C<ST_NOATIME>,
631 C<ST_NODIRATIME> and C<ST_RELATIME>.
632
633 Example: stat C</wd> and dump out the data if successful.
634
635 aio_statvfs "/wd", sub {
636 my $f = $_[0]
637 or die "statvfs: $!";
638
639 use Data::Dumper;
640 say Dumper $f;
641 };
642
643 # result:
644 {
645 bsize => 1024,
646 bfree => 4333064312,
647 blocks => 10253828096,
648 files => 2050765568,
649 flag => 4096,
650 favail => 2042092649,
651 bavail => 4333064312,
652 ffree => 2042092649,
653 namemax => 255,
654 frsize => 1024,
655 fsid => 1810
656 }
657
658 =item aio_utime $fh_or_path, $atime, $mtime, $callback->($status)
659
660 Works like perl's C<utime> function (including the special case of $atime
661 and $mtime being undef). Fractional times are supported if the underlying
662 syscalls support them, and can be specified as fractional numbers or
663 C<[$second, $nanosecond]> integer arrayref pairs.
664
665 When called with a pathname, uses utimensat(2) or utimes(2) if available,
666 otherwise utime(2). If called on a file descriptor, uses futimens(2)
667 or futimes(2) if available, otherwise returns ENOSYS, so this is not
668 portable.
669
670 Examples:
671
672 # set atime and mtime to current time (basically touch(1)):
673 aio_utime "path", undef, undef;
674 # set atime to current time and mtime to beginning of the epoch:
675 aio_utime "path", time, undef; # undef==0
676
677
678 =item aio_chown $fh_or_path, $uid, $gid, $callback->($status)
679
680 Works like perl's C<chown> function, except that C<undef> for either $uid
681 or $gid is being interpreted as "do not change" (but -1 can also be used).
682
683 Examples:
684
685 # same as "chown root path" in the shell:
686 aio_chown "path", 0, -1;
687 # same as above:
688 aio_chown "path", 0, undef;
689
690
691 =item aio_truncate $fh_or_path, $offset, $callback->($status)
692
693 Works like truncate(2) or ftruncate(2).
694
695
696 =item aio_allocate $fh, $mode, $offset, $len, $callback->($status)
697
698 Allocates or frees disk space according to the C<$mode> argument. See the
699 linux C<fallocate> documentation for details.
700
701 C<$mode> is usually C<0> or C<IO::AIO::FALLOC_FL_KEEP_SIZE> to allocate
702 space, or C<IO::AIO::FALLOC_FL_PUNCH_HOLE | IO::AIO::FALLOC_FL_KEEP_SIZE>,
703 to deallocate a file range.
704
705 IO::AIO also supports C<FALLOC_FL_COLLAPSE_RANGE>, to remove a range
706 (without leaving a hole), C<FALLOC_FL_ZERO_RANGE>, to zero a range,
707 C<FALLOC_FL_INSERT_RANGE> to insert a range and C<FALLOC_FL_UNSHARE_RANGE>
708 to unshare shared blocks (see your L<fallocate(2)> manpage).
709
710 The file system block size used by C<fallocate> is presumably the
711 C<f_bsize> returned by C<statvfs>, but different filesystems and filetypes
712 can dictate other limitations.
713
714 If C<fallocate> isn't available or cannot be emulated (currently no
715 emulation will be attempted), passes C<-1> and sets C<$!> to C<ENOSYS>.
716
717
718 =item aio_chmod $fh_or_path, $mode, $callback->($status)
719
720 Works like perl's C<chmod> function.
721
722
723 =item aio_unlink $pathname, $callback->($status)
724
725 Asynchronously unlink (delete) a file and call the callback with the
726 result code.
727
728
729 =item aio_mknod $pathname, $mode, $dev, $callback->($status)
730
731 [EXPERIMENTAL]
732
733 Asynchronously create a device node (or fifo). See mknod(2).
734
735 The only (POSIX-) portable way of calling this function is:
736
737 aio_mknod $pathname, IO::AIO::S_IFIFO | $mode, 0, sub { ...
738
739 See C<aio_stat> for info about some potentially helpful extra constants
740 and functions.
741
742 =item aio_link $srcpath, $dstpath, $callback->($status)
743
744 Asynchronously create a new link to the existing object at C<$srcpath> at
745 the path C<$dstpath> and call the callback with the result code.
746
747
748 =item aio_symlink $srcpath, $dstpath, $callback->($status)
749
750 Asynchronously create a new symbolic link to the existing object at C<$srcpath> at
751 the path C<$dstpath> and call the callback with the result code.
752
753
754 =item aio_readlink $pathname, $callback->($link)
755
756 Asynchronously read the symlink specified by C<$path> and pass it to
757 the callback. If an error occurs, nothing or undef gets passed to the
758 callback.
759
760
761 =item aio_realpath $pathname, $callback->($path)
762
763 Asynchronously make the path absolute and resolve any symlinks in
764 C<$path>. The resulting path only consists of directories (same as
765 L<Cwd::realpath>).
766
767 This request can be used to get the absolute path of the current working
768 directory by passing it a path of F<.> (a single dot).
769
770
771 =item aio_rename $srcpath, $dstpath, $callback->($status)
772
773 Asynchronously rename the object at C<$srcpath> to C<$dstpath>, just as
774 rename(2) and call the callback with the result code.
775
776 On systems that support the AIO::WD working directory abstraction
777 natively, the case C<[$wd, "."]> as C<$srcpath> is specialcased - instead
778 of failing, C<rename> is called on the absolute path of C<$wd>.
779
780
781 =item aio_rename2 $srcpath, $dstpath, $flags, $callback->($status)
782
783 Basically a version of C<aio_rename> with an additional C<$flags>
784 argument. Calling this with C<$flags=0> is the same as calling
785 C<aio_rename>.
786
787 Non-zero flags are currently only supported on GNU/Linux systems that
788 support renameat2. Other systems fail with C<ENOSYS> in this case.
789
790 The following constants are available (missing ones are, as usual C<0>),
791 see renameat2(2) for details:
792
793 C<IO::AIO::RENAME_NOREPLACE>, C<IO::AIO::RENAME_EXCHANGE>
794 and C<IO::AIO::RENAME_WHITEOUT>.
795
796
797 =item aio_mkdir $pathname, $mode, $callback->($status)
798
799 Asynchronously mkdir (create) a directory and call the callback with
800 the result code. C<$mode> will be modified by the umask at the time the
801 request is executed, so do not change your umask.
802
803
804 =item aio_rmdir $pathname, $callback->($status)
805
806 Asynchronously rmdir (delete) a directory and call the callback with the
807 result code.
808
809 On systems that support the AIO::WD working directory abstraction
810 natively, the case C<[$wd, "."]> is specialcased - instead of failing,
811 C<rmdir> is called on the absolute path of C<$wd>.
812
813
814 =item aio_readdir $pathname, $callback->($entries)
815
816 Unlike the POSIX call of the same name, C<aio_readdir> reads an entire
817 directory (i.e. opendir + readdir + closedir). The entries will not be
818 sorted, and will B<NOT> include the C<.> and C<..> entries.
819
820 The callback is passed a single argument which is either C<undef> or an
821 array-ref with the filenames.
822
823
824 =item aio_readdirx $pathname, $flags, $callback->($entries, $flags)
825
826 Quite similar to C<aio_readdir>, but the C<$flags> argument allows one to
827 tune behaviour and output format. In case of an error, C<$entries> will be
828 C<undef>.
829
830 The flags are a combination of the following constants, ORed together (the
831 flags will also be passed to the callback, possibly modified):
832
833 =over 4
834
835 =item IO::AIO::READDIR_DENTS
836
837 Normally the callback gets an arrayref consisting of names only (as
838 with C<aio_readdir>). If this flag is set, then the callback gets an
839 arrayref with C<[$name, $type, $inode]> arrayrefs, each describing a
840 single directory entry in more detail:
841
842 C<$name> is the name of the entry.
843
844 C<$type> is one of the C<IO::AIO::DT_xxx> constants:
845
846 C<IO::AIO::DT_UNKNOWN>, C<IO::AIO::DT_FIFO>, C<IO::AIO::DT_CHR>, C<IO::AIO::DT_DIR>,
847 C<IO::AIO::DT_BLK>, C<IO::AIO::DT_REG>, C<IO::AIO::DT_LNK>, C<IO::AIO::DT_SOCK>,
848 C<IO::AIO::DT_WHT>.
849
850 C<IO::AIO::DT_UNKNOWN> means just that: readdir does not know. If you need
851 to know, you have to run stat yourself. Also, for speed/memory reasons,
852 the C<$type> scalars are read-only: you must not modify them.
853
854 C<$inode> is the inode number (which might not be exact on systems with 64
855 bit inode numbers and 32 bit perls). This field has unspecified content on
856 systems that do not deliver the inode information.
857
858 =item IO::AIO::READDIR_DIRS_FIRST
859
860 When this flag is set, then the names will be returned in an order where
861 likely directories come first, in optimal stat order. This is useful when
862 you need to quickly find directories, or you want to find all directories
863 while avoiding to stat() each entry.
864
865 If the system returns type information in readdir, then this is used
866 to find directories directly. Otherwise, likely directories are names
867 beginning with ".", or otherwise names with no dots, of which names with
868 short names are tried first.
869
870 =item IO::AIO::READDIR_STAT_ORDER
871
872 When this flag is set, then the names will be returned in an order
873 suitable for stat()'ing each one. That is, when you plan to stat() most or
874 all files in the given directory, then the returned order will likely be
875 faster.
876
877 If both this flag and C<IO::AIO::READDIR_DIRS_FIRST> are specified,
878 then the likely dirs come first, resulting in a less optimal stat order
879 for stat'ing all entries, but likely a more optimal order for finding
880 subdirectories.
881
882 =item IO::AIO::READDIR_FOUND_UNKNOWN
883
884 This flag should not be set when calling C<aio_readdirx>. Instead, it
885 is being set by C<aio_readdirx>, when any of the C<$type>'s found were
886 C<IO::AIO::DT_UNKNOWN>. The absence of this flag therefore indicates that all
887 C<$type>'s are known, which can be used to speed up some algorithms.
888
889 =back
890
891
892 =item aio_slurp $pathname, $offset, $length, $data, $callback->($status)
893
894 Opens, reads and closes the given file. The data is put into C<$data>,
895 which is resized as required.
896
897 If C<$offset> is negative, then it is counted from the end of the file.
898
899 If C<$length> is zero, then the remaining length of the file is
900 used. Also, in this case, the same limitations to modifying C<$data> apply
901 as when IO::AIO::mmap is used, i.e. it must only be modified in-place
902 with C<substr>. If the size of the file is known, specifying a non-zero
903 C<$length> results in a performance advantage.
904
905 This request is similar to the older C<aio_load> request, but since it is
906 a single request, it might be more efficient to use.
907
908 Example: load F</etc/passwd> into C<$passwd>.
909
910 my $passwd;
911 aio_slurp "/etc/passwd", 0, 0, $passwd, sub {
912 $_[0] >= 0
913 or die "/etc/passwd: $!\n";
914
915 printf "/etc/passwd is %d bytes long, and contains:\n", length $passwd;
916 print $passwd;
917 };
918 IO::AIO::flush;
919
920
921 =item aio_load $pathname, $data, $callback->($status)
922
923 This is a composite request that tries to fully load the given file into
924 memory. Status is the same as with aio_read.
925
926 Using C<aio_slurp> might be more efficient, as it is a single request.
927
928 =cut
929
930 sub aio_load($$;$) {
931 my ($path, undef, $cb) = @_;
932 my $data = \$_[1];
933
934 my $pri = aioreq_pri;
935 my $grp = aio_group $cb;
936
937 aioreq_pri $pri;
938 add $grp aio_open $path, O_RDONLY, 0, sub {
939 my $fh = shift
940 or return $grp->result (-1);
941
942 aioreq_pri $pri;
943 add $grp aio_read $fh, 0, (-s $fh), $$data, 0, sub {
944 $grp->result ($_[0]);
945 };
946 };
947
948 $grp
949 }
950
951 =item aio_copy $srcpath, $dstpath, $callback->($status)
952
953 Try to copy the I<file> (directories not supported as either source or
954 destination) from C<$srcpath> to C<$dstpath> and call the callback with
955 a status of C<0> (ok) or C<-1> (error, see C<$!>).
956
957 Existing destination files will be truncated.
958
959 This is a composite request that creates the destination file with
960 mode 0200 and copies the contents of the source file into it using
961 C<aio_sendfile>, followed by restoring atime, mtime, access mode and
962 uid/gid, in that order.
963
964 If an error occurs, the partial destination file will be unlinked, if
965 possible, except when setting atime, mtime, access mode and uid/gid, where
966 errors are being ignored.
967
968 =cut
969
970 sub aio_copy($$;$) {
971 my ($src, $dst, $cb) = @_;
972
973 my $pri = aioreq_pri;
974 my $grp = aio_group $cb;
975
976 aioreq_pri $pri;
977 add $grp aio_open $src, O_RDONLY, 0, sub {
978 if (my $src_fh = $_[0]) {
979 my @stat = stat $src_fh; # hmm, might block over nfs?
980
981 aioreq_pri $pri;
982 add $grp aio_open $dst, O_CREAT | O_WRONLY | O_TRUNC, 0200, sub {
983 if (my $dst_fh = $_[0]) {
984
985 # best-effort preallocate
986 aioreq_pri $pri;
987 add $grp aio_allocate $dst_fh, IO::AIO::FALLOC_FL_KEEP_SIZE, 0, $stat[7], sub { };
988
989 aioreq_pri $pri;
990 add $grp aio_sendfile $dst_fh, $src_fh, 0, $stat[7], sub {
991 if ($_[0] == $stat[7]) {
992 $grp->result (0);
993 close $src_fh;
994
995 my $ch = sub {
996 aioreq_pri $pri;
997 add $grp aio_chmod $dst_fh, $stat[2] & 07777, sub {
998 aioreq_pri $pri;
999 add $grp aio_chown $dst_fh, $stat[4], $stat[5], sub {
1000 aioreq_pri $pri;
1001 add $grp aio_close $dst_fh;
1002 }
1003 };
1004 };
1005
1006 aioreq_pri $pri;
1007 add $grp aio_utime $dst_fh, $stat[8], $stat[9], sub {
1008 if ($_[0] < 0 && $! == ENOSYS) {
1009 aioreq_pri $pri;
1010 add $grp aio_utime $dst, $stat[8], $stat[9], $ch;
1011 } else {
1012 $ch->();
1013 }
1014 };
1015 } else {
1016 $grp->result (-1);
1017 close $src_fh;
1018 close $dst_fh;
1019
1020 aioreq $pri;
1021 add $grp aio_unlink $dst;
1022 }
1023 };
1024 } else {
1025 $grp->result (-1);
1026 }
1027 },
1028
1029 } else {
1030 $grp->result (-1);
1031 }
1032 };
1033
1034 $grp
1035 }
1036
1037 =item aio_move $srcpath, $dstpath, $callback->($status)
1038
1039 Try to move the I<file> (directories not supported as either source or
1040 destination) from C<$srcpath> to C<$dstpath> and call the callback with
1041 a status of C<0> (ok) or C<-1> (error, see C<$!>).
1042
1043 This is a composite request that tries to rename(2) the file first; if
1044 rename fails with C<EXDEV>, it copies the file with C<aio_copy> and, if
1045 that is successful, unlinks the C<$srcpath>.
1046
1047 =cut
1048
1049 sub aio_move($$;$) {
1050 my ($src, $dst, $cb) = @_;
1051
1052 my $pri = aioreq_pri;
1053 my $grp = aio_group $cb;
1054
1055 aioreq_pri $pri;
1056 add $grp aio_rename $src, $dst, sub {
1057 if ($_[0] && $! == EXDEV) {
1058 aioreq_pri $pri;
1059 add $grp aio_copy $src, $dst, sub {
1060 $grp->result ($_[0]);
1061
1062 unless ($_[0]) {
1063 aioreq_pri $pri;
1064 add $grp aio_unlink $src;
1065 }
1066 };
1067 } else {
1068 $grp->result ($_[0]);
1069 }
1070 };
1071
1072 $grp
1073 }
1074
1075 =item aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
1076
1077 Scans a directory (similar to C<aio_readdir>) but additionally tries to
1078 efficiently separate the entries of directory C<$path> into two sets of
1079 names, directories you can recurse into (directories), and ones you cannot
1080 recurse into (everything else, including symlinks to directories).
1081
1082 C<aio_scandir> is a composite request that generates many sub requests.
1083 C<$maxreq> specifies the maximum number of outstanding aio requests that
1084 this function generates. If it is C<< <= 0 >>, then a suitable default
1085 will be chosen (currently 4).
1086
1087 On error, the callback is called without arguments, otherwise it receives
1088 two array-refs with path-relative entry names.
1089
1090 Example:
1091
1092 aio_scandir $dir, 0, sub {
1093 my ($dirs, $nondirs) = @_;
1094 print "real directories: @$dirs\n";
1095 print "everything else: @$nondirs\n";
1096 };
1097
1098 Implementation notes.
1099
1100 The C<aio_readdir> cannot be avoided, but C<stat()>'ing every entry can.
1101
1102 If readdir returns file type information, then this is used directly to
1103 find directories.
1104
1105 Otherwise, after reading the directory, the modification time, size etc.
1106 of the directory before and after the readdir is checked, and if they
1107 match (and isn't the current time), the link count will be used to decide
1108 how many entries are directories (if >= 2). Otherwise, no knowledge of the
1109 number of subdirectories will be assumed.
1110
1111 Then entries will be sorted into likely directories a non-initial dot
1112 currently) and likely non-directories (see C<aio_readdirx>). Then every
1113 entry plus an appended C</.> will be C<stat>'ed, likely directories first,
1114 in order of their inode numbers. If that succeeds, it assumes that the
1115 entry is a directory or a symlink to directory (which will be checked
1116 separately). This is often faster than stat'ing the entry itself because
1117 filesystems might detect the type of the entry without reading the inode
1118 data (e.g. ext2fs filetype feature), even on systems that cannot return
1119 the filetype information on readdir.
1120
1121 If the known number of directories (link count - 2) has been reached, the
1122 rest of the entries is assumed to be non-directories.
1123
1124 This only works with certainty on POSIX (= UNIX) filesystems, which
1125 fortunately are the vast majority of filesystems around.
1126
1127 It will also likely work on non-POSIX filesystems with reduced efficiency
1128 as those tend to return 0 or 1 as link counts, which disables the
1129 directory counting heuristic.
1130
1131 =cut
1132
1133 sub aio_scandir($$;$) {
1134 my ($path, $maxreq, $cb) = @_;
1135
1136 my $pri = aioreq_pri;
1137
1138 my $grp = aio_group $cb;
1139
1140 $maxreq = 4 if $maxreq <= 0;
1141
1142 # get a wd object
1143 aioreq_pri $pri;
1144 add $grp aio_wd $path, sub {
1145 $_[0]
1146 or return $grp->result ();
1147
1148 my $wd = [shift, "."];
1149
1150 # stat once
1151 aioreq_pri $pri;
1152 add $grp aio_stat $wd, sub {
1153 return $grp->result () if $_[0];
1154 my $now = time;
1155 my $hash1 = join ":", (stat _)[0,1,3,7,9];
1156 my $rdxflags = READDIR_DIRS_FIRST;
1157
1158 if ((stat _)[3] < 2) {
1159 # at least one non-POSIX filesystem exists
1160 # that returns useful DT_type values: btrfs,
1161 # so optimise for this here by requesting dents
1162 $rdxflags |= READDIR_DENTS;
1163 }
1164
1165 # read the directory entries
1166 aioreq_pri $pri;
1167 add $grp aio_readdirx $wd, $rdxflags, sub {
1168 my ($entries, $flags) = @_
1169 or return $grp->result ();
1170
1171 if ($rdxflags & READDIR_DENTS) {
1172 # if we requested type values, see if we can use them directly.
1173
1174 # if there were any DT_UNKNOWN entries then we assume we
1175 # don't know. alternatively, we could assume that if we get
1176 # one DT_DIR, then all directories are indeed marked with
1177 # DT_DIR, but this seems not required for btrfs, and this
1178 # is basically the "btrfs can't get it's act together" code
1179 # branch.
1180 unless ($flags & READDIR_FOUND_UNKNOWN) {
1181 # now we have valid DT_ information for all entries,
1182 # so use it as an optimisation without further stat's.
1183 # they must also all be at the beginning of @$entries
1184 # by now.
1185
1186 my $dirs;
1187
1188 if (@$entries) {
1189 for (0 .. $#$entries) {
1190 if ($entries->[$_][1] != DT_DIR) {
1191 # splice out directories
1192 $dirs = [splice @$entries, 0, $_];
1193 last;
1194 }
1195 }
1196
1197 # if we didn't find any non-dir, then all entries are dirs
1198 unless ($dirs) {
1199 ($dirs, $entries) = ($entries, []);
1200 }
1201 } else {
1202 # directory is empty, so there are no sbdirs
1203 $dirs = [];
1204 }
1205
1206 # either splice'd the directories out or the dir was empty.
1207 # convert dents to filenames
1208 $_ = $_->[0] for @$dirs;
1209 $_ = $_->[0] for @$entries;
1210
1211 return $grp->result ($dirs, $entries);
1212 }
1213
1214 # cannot use, so return to our old ways
1215 # by pretending we only scanned for names.
1216 $_ = $_->[0] for @$entries;
1217 }
1218
1219 # stat the dir another time
1220 aioreq_pri $pri;
1221 add $grp aio_stat $wd, sub {
1222 my $hash2 = join ":", (stat _)[0,1,3,7,9];
1223
1224 my $ndirs;
1225
1226 # take the slow route if anything looks fishy
1227 if ($hash1 ne $hash2 or (stat _)[9] == $now) {
1228 $ndirs = -1;
1229 } else {
1230 # if nlink == 2, we are finished
1231 # for non-posix-fs's, we rely on nlink < 2
1232 $ndirs = (stat _)[3] - 2
1233 or return $grp->result ([], $entries);
1234 }
1235
1236 my (@dirs, @nondirs);
1237
1238 my $statgrp = add $grp aio_group sub {
1239 $grp->result (\@dirs, \@nondirs);
1240 };
1241
1242 limit $statgrp $maxreq;
1243 feed $statgrp sub {
1244 return unless @$entries;
1245 my $entry = shift @$entries;
1246
1247 aioreq_pri $pri;
1248 $wd->[1] = "$entry/.";
1249 add $statgrp aio_stat $wd, sub {
1250 if ($_[0] < 0) {
1251 push @nondirs, $entry;
1252 } else {
1253 # need to check for real directory
1254 aioreq_pri $pri;
1255 $wd->[1] = $entry;
1256 add $statgrp aio_lstat $wd, sub {
1257 if (-d _) {
1258 push @dirs, $entry;
1259
1260 unless (--$ndirs) {
1261 push @nondirs, @$entries;
1262 feed $statgrp;
1263 }
1264 } else {
1265 push @nondirs, $entry;
1266 }
1267 }
1268 }
1269 };
1270 };
1271 };
1272 };
1273 };
1274 };
1275
1276 $grp
1277 }
1278
1279 =item aio_rmtree $pathname, $callback->($status)
1280
1281 Delete a directory tree starting (and including) C<$path>, return the
1282 status of the final C<rmdir> only. This is a composite request that
1283 uses C<aio_scandir> to recurse into and rmdir directories, and unlink
1284 everything else.
1285
1286 =cut
1287
1288 sub aio_rmtree;
1289 sub aio_rmtree($;$) {
1290 my ($path, $cb) = @_;
1291
1292 my $pri = aioreq_pri;
1293 my $grp = aio_group $cb;
1294
1295 aioreq_pri $pri;
1296 add $grp aio_scandir $path, 0, sub {
1297 my ($dirs, $nondirs) = @_;
1298
1299 my $dirgrp = aio_group sub {
1300 add $grp aio_rmdir $path, sub {
1301 $grp->result ($_[0]);
1302 };
1303 };
1304
1305 (aioreq_pri $pri), add $dirgrp aio_rmtree "$path/$_" for @$dirs;
1306 (aioreq_pri $pri), add $dirgrp aio_unlink "$path/$_" for @$nondirs;
1307
1308 add $grp $dirgrp;
1309 };
1310
1311 $grp
1312 }
1313
1314 =item aio_fcntl $fh, $cmd, $arg, $callback->($status)
1315
1316 =item aio_ioctl $fh, $request, $buf, $callback->($status)
1317
1318 These work just like the C<fcntl> and C<ioctl> built-in functions, except
1319 they execute asynchronously and pass the return value to the callback.
1320
1321 Both calls can be used for a lot of things, some of which make more sense
1322 to run asynchronously in their own thread, while some others make less
1323 sense. For example, calls that block waiting for external events, such
1324 as locking, will also lock down an I/O thread while it is waiting, which
1325 can deadlock the whole I/O system. At the same time, there might be no
1326 alternative to using a thread to wait.
1327
1328 So in general, you should only use these calls for things that do
1329 (filesystem) I/O, not for things that wait for other events (network,
1330 other processes), although if you are careful and know what you are doing,
1331 you still can.
1332
1333 The following constants are available and can be used for normal C<ioctl>
1334 and C<fcntl> as well (missing ones are, as usual C<0>):
1335
1336 C<F_DUPFD_CLOEXEC>,
1337
1338 C<F_OFD_GETLK>, C<F_OFD_SETLK>, C<F_OFD_GETLKW>,
1339
1340 C<FIFREEZE>, C<FITHAW>, C<FITRIM>, C<FICLONE>, C<FICLONERANGE>, C<FIDEDUPERANGE>.
1341
1342 C<F_ADD_SEALS>, C<F_GET_SEALS>, C<F_SEAL_SEAL>, C<F_SEAL_SHRINK>, C<F_SEAL_GROW> and
1343 C<F_SEAL_WRITE>.
1344
1345 C<FS_IOC_GETFLAGS>, C<FS_IOC_SETFLAGS>, C<FS_IOC_GETVERSION>, C<FS_IOC_SETVERSION>,
1346 C<FS_IOC_FIEMAP>.
1347
1348 C<FS_IOC_FSGETXATTR>, C<FS_IOC_FSSETXATTR>, C<FS_IOC_SET_ENCRYPTION_POLICY>,
1349 C<FS_IOC_GET_ENCRYPTION_PWSALT>, C<FS_IOC_GET_ENCRYPTION_POLICY>, C<FS_KEY_DESCRIPTOR_SIZE>.
1350
1351 C<FS_SECRM_FL>, C<FS_UNRM_FL>, C<FS_COMPR_FL>, C<FS_SYNC_FL>, C<FS_IMMUTABLE_FL>,
1352 C<FS_APPEND_FL>, C<FS_NODUMP_FL>, C<FS_NOATIME_FL>, C<FS_DIRTY_FL>,
1353 C<FS_COMPRBLK_FL>, C<FS_NOCOMP_FL>, C<FS_ENCRYPT_FL>, C<FS_BTREE_FL>,
1354 C<FS_INDEX_FL>, C<FS_JOURNAL_DATA_FL>, C<FS_NOTAIL_FL>, C<FS_DIRSYNC_FL>, C<FS_TOPDIR_FL>,
1355 C<FS_FL_USER_MODIFIABLE>.
1356
1357 C<FS_XFLAG_REALTIME>, C<FS_XFLAG_PREALLOC>, C<FS_XFLAG_IMMUTABLE>, C<FS_XFLAG_APPEND>,
1358 C<FS_XFLAG_SYNC>, C<FS_XFLAG_NOATIME>, C<FS_XFLAG_NODUMP>, C<FS_XFLAG_RTINHERIT>,
1359 C<FS_XFLAG_PROJINHERIT>, C<FS_XFLAG_NOSYMLINKS>, C<FS_XFLAG_EXTSIZE>, C<FS_XFLAG_EXTSZINHERIT>,
1360 C<FS_XFLAG_NODEFRAG>, C<FS_XFLAG_FILESTREAM>, C<FS_XFLAG_DAX>, C<FS_XFLAG_HASATTR>,
1361
1362 C<BLKROSET>, C<BLKROGET>, C<BLKRRPART>, C<BLKGETSIZE>, C<BLKFLSBUF>, C<BLKRASET>,
1363 C<BLKRAGET>, C<BLKFRASET>, C<BLKFRAGET>, C<BLKSECTSET>, C<BLKSECTGET>, C<BLKSSZGET>,
1364 C<BLKBSZGET>, C<BLKBSZSET>, C<BLKGETSIZE64>,
1365
1366
1367 =item aio_sync $callback->($status)
1368
1369 Asynchronously call sync and call the callback when finished.
1370
1371 =item aio_fsync $fh, $callback->($status)
1372
1373 Asynchronously call fsync on the given filehandle and call the callback
1374 with the fsync result code.
1375
1376 =item aio_fdatasync $fh, $callback->($status)
1377
1378 Asynchronously call fdatasync on the given filehandle and call the
1379 callback with the fdatasync result code.
1380
1381 If this call isn't available because your OS lacks it or it couldn't be
1382 detected, it will be emulated by calling C<fsync> instead.
1383
1384 =item aio_syncfs $fh, $callback->($status)
1385
1386 Asynchronously call the syncfs syscall to sync the filesystem associated
1387 to the given filehandle and call the callback with the syncfs result
1388 code. If syncfs is not available, calls sync(), but returns C<-1> and sets
1389 errno to C<ENOSYS> nevertheless.
1390
1391 =item aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
1392
1393 Sync the data portion of the file specified by C<$offset> and C<$length>
1394 to disk (but NOT the metadata), by calling the Linux-specific
1395 sync_file_range call. If sync_file_range is not available or it returns
1396 ENOSYS, then fdatasync or fsync is being substituted.
1397
1398 C<$flags> can be a combination of C<IO::AIO::SYNC_FILE_RANGE_WAIT_BEFORE>,
1399 C<IO::AIO::SYNC_FILE_RANGE_WRITE> and
1400 C<IO::AIO::SYNC_FILE_RANGE_WAIT_AFTER>: refer to the sync_file_range
1401 manpage for details.
1402
1403 =item aio_pathsync $pathname, $callback->($status)
1404
1405 This request tries to open, fsync and close the given path. This is a
1406 composite request intended to sync directories after directory operations
1407 (E.g. rename). This might not work on all operating systems or have any
1408 specific effect, but usually it makes sure that directory changes get
1409 written to disc. It works for anything that can be opened for read-only,
1410 not just directories.
1411
1412 Future versions of this function might fall back to other methods when
1413 C<fsync> on the directory fails (such as calling C<sync>).
1414
1415 Passes C<0> when everything went ok, and C<-1> on error.
1416
1417 =cut
1418
1419 sub aio_pathsync($;$) {
1420 my ($path, $cb) = @_;
1421
1422 my $pri = aioreq_pri;
1423 my $grp = aio_group $cb;
1424
1425 aioreq_pri $pri;
1426 add $grp aio_open $path, O_RDONLY, 0, sub {
1427 my ($fh) = @_;
1428 if ($fh) {
1429 aioreq_pri $pri;
1430 add $grp aio_fsync $fh, sub {
1431 $grp->result ($_[0]);
1432
1433 aioreq_pri $pri;
1434 add $grp aio_close $fh;
1435 };
1436 } else {
1437 $grp->result (-1);
1438 }
1439 };
1440
1441 $grp
1442 }
1443
1444 =item aio_msync $scalar, $offset = 0, $length = undef, flags = MS_SYNC, $callback->($status)
1445
1446 This is a rather advanced IO::AIO call, which only works on mmap(2)ed
1447 scalars (see the C<IO::AIO::mmap> function, although it also works on data
1448 scalars managed by the L<Sys::Mmap> or L<Mmap> modules, note that the
1449 scalar must only be modified in-place while an aio operation is pending on
1450 it).
1451
1452 It calls the C<msync> function of your OS, if available, with the memory
1453 area starting at C<$offset> in the string and ending C<$length> bytes
1454 later. If C<$length> is negative, counts from the end, and if C<$length>
1455 is C<undef>, then it goes till the end of the string. The flags can be
1456 either C<IO::AIO::MS_ASYNC> or C<IO::AIO::MS_SYNC>, plus an optional
1457 C<IO::AIO::MS_INVALIDATE>.
1458
1459 =item aio_mtouch $scalar, $offset = 0, $length = undef, flags = 0, $callback->($status)
1460
1461 This is a rather advanced IO::AIO call, which works best on mmap(2)ed
1462 scalars.
1463
1464 It touches (reads or writes) all memory pages in the specified
1465 range inside the scalar. All caveats and parameters are the same
1466 as for C<aio_msync>, above, except for flags, which must be either
1467 C<0> (which reads all pages and ensures they are instantiated) or
1468 C<IO::AIO::MT_MODIFY>, which modifies the memory pages (by reading and
1469 writing an octet from it, which dirties the page).
1470
1471 =item aio_mlock $scalar, $offset = 0, $length = undef, $callback->($status)
1472
1473 This is a rather advanced IO::AIO call, which works best on mmap(2)ed
1474 scalars.
1475
1476 It reads in all the pages of the underlying storage into memory (if any)
1477 and locks them, so they are not getting swapped/paged out or removed.
1478
1479 If C<$length> is undefined, then the scalar will be locked till the end.
1480
1481 On systems that do not implement C<mlock>, this function returns C<-1>
1482 and sets errno to C<ENOSYS>.
1483
1484 Note that the corresponding C<munlock> is synchronous and is
1485 documented under L<MISCELLANEOUS FUNCTIONS>.
1486
1487 Example: open a file, mmap and mlock it - both will be undone when
1488 C<$data> gets destroyed.
1489
1490 open my $fh, "<", $path or die "$path: $!";
1491 my $data;
1492 IO::AIO::mmap $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh;
1493 aio_mlock $data; # mlock in background
1494
1495 =item aio_mlockall $flags, $callback->($status)
1496
1497 Calls the C<mlockall> function with the given C<$flags> (a
1498 combination of C<IO::AIO::MCL_CURRENT>, C<IO::AIO::MCL_FUTURE> and
1499 C<IO::AIO::MCL_ONFAULT>).
1500
1501 On systems that do not implement C<mlockall>, this function returns C<-1>
1502 and sets errno to C<ENOSYS>. Similarly, flag combinations not supported
1503 by the system result in a return value of C<-1> with errno being set to
1504 C<EINVAL>.
1505
1506 Note that the corresponding C<munlockall> is synchronous and is
1507 documented under L<MISCELLANEOUS FUNCTIONS>.
1508
1509 Example: asynchronously lock all current and future pages into memory.
1510
1511 aio_mlockall IO::AIO::MCL_FUTURE;
1512
1513 =item aio_fiemap $fh, $start, $length, $flags, $count, $cb->(\@extents)
1514
1515 Queries the extents of the given file (by calling the Linux C<FIEMAP>
1516 ioctl, see L<http://cvs.schmorp.de/IO-AIO/doc/fiemap.txt> for details). If
1517 the ioctl is not available on your OS, then this request will fail with
1518 C<ENOSYS>.
1519
1520 C<$start> is the starting offset to query extents for, C<$length> is the
1521 size of the range to query - if it is C<undef>, then the whole file will
1522 be queried.
1523
1524 C<$flags> is a combination of flags (C<IO::AIO::FIEMAP_FLAG_SYNC> or
1525 C<IO::AIO::FIEMAP_FLAG_XATTR> - C<IO::AIO::FIEMAP_FLAGS_COMPAT> is also
1526 exported), and is normally C<0> or C<IO::AIO::FIEMAP_FLAG_SYNC> to query
1527 the data portion.
1528
1529 C<$count> is the maximum number of extent records to return. If it is
1530 C<undef>, then IO::AIO queries all extents of the range. As a very special
1531 case, if it is C<0>, then the callback receives the number of extents
1532 instead of the extents themselves (which is unreliable, see below).
1533
1534 If an error occurs, the callback receives no arguments. The special
1535 C<errno> value C<IO::AIO::EBADR> is available to test for flag errors.
1536
1537 Otherwise, the callback receives an array reference with extent
1538 structures. Each extent structure is an array reference itself, with the
1539 following members:
1540
1541 [$logical, $physical, $length, $flags]
1542
1543 Flags is any combination of the following flag values (typically either C<0>
1544 or C<IO::AIO::FIEMAP_EXTENT_LAST> (1)):
1545
1546 C<IO::AIO::FIEMAP_EXTENT_LAST>, C<IO::AIO::FIEMAP_EXTENT_UNKNOWN>,
1547 C<IO::AIO::FIEMAP_EXTENT_DELALLOC>, C<IO::AIO::FIEMAP_EXTENT_ENCODED>,
1548 C<IO::AIO::FIEMAP_EXTENT_DATA_ENCRYPTED>, C<IO::AIO::FIEMAP_EXTENT_NOT_ALIGNED>,
1549 C<IO::AIO::FIEMAP_EXTENT_DATA_INLINE>, C<IO::AIO::FIEMAP_EXTENT_DATA_TAIL>,
1550 C<IO::AIO::FIEMAP_EXTENT_UNWRITTEN>, C<IO::AIO::FIEMAP_EXTENT_MERGED> or
1551 C<IO::AIO::FIEMAP_EXTENT_SHARED>.
1552
1553 At the time of this writing (Linux 3.2), this request is unreliable unless
1554 C<$count> is C<undef>, as the kernel has all sorts of bugs preventing
1555 it to return all extents of a range for files with a large number of
1556 extents. The code (only) works around all these issues if C<$count> is
1557 C<undef>.
1558
1559 =item aio_group $callback->(...)
1560
1561 This is a very special aio request: Instead of doing something, it is a
1562 container for other aio requests, which is useful if you want to bundle
1563 many requests into a single, composite, request with a definite callback
1564 and the ability to cancel the whole request with its subrequests.
1565
1566 Returns an object of class L<IO::AIO::GRP>. See its documentation below
1567 for more info.
1568
1569 Example:
1570
1571 my $grp = aio_group sub {
1572 print "all stats done\n";
1573 };
1574
1575 add $grp
1576 (aio_stat ...),
1577 (aio_stat ...),
1578 ...;
1579
1580 =item aio_nop $callback->()
1581
1582 This is a special request - it does nothing in itself and is only used for
1583 side effects, such as when you want to add a dummy request to a group so
1584 that finishing the requests in the group depends on executing the given
1585 code.
1586
1587 While this request does nothing, it still goes through the execution
1588 phase and still requires a worker thread. Thus, the callback will not
1589 be executed immediately but only after other requests in the queue have
1590 entered their execution phase. This can be used to measure request
1591 latency.
1592
1593 =item IO::AIO::aio_busy $fractional_seconds, $callback->() *NOT EXPORTED*
1594
1595 Mainly used for debugging and benchmarking, this aio request puts one of
1596 the request workers to sleep for the given time.
1597
1598 While it is theoretically handy to have simple I/O scheduling requests
1599 like sleep and file handle readable/writable, the overhead this creates is
1600 immense (it blocks a thread for a long time) so do not use this function
1601 except to put your application under artificial I/O pressure.
1602
1603 =back
1604
1605
1606 =head2 IO::AIO::WD - multiple working directories
1607
1608 Your process only has one current working directory, which is used by all
1609 threads. This makes it hard to use relative paths (some other component
1610 could call C<chdir> at any time, and it is hard to control when the path
1611 will be used by IO::AIO).
1612
1613 One solution for this is to always use absolute paths. This usually works,
1614 but can be quite slow (the kernel has to walk the whole path on every
1615 access), and can also be a hassle to implement.
1616
1617 Newer POSIX systems have a number of functions (openat, fdopendir,
1618 futimensat and so on) that make it possible to specify working directories
1619 per operation.
1620
1621 For portability, and because the clowns who "designed", or shall I write,
1622 perpetrated this new interface were obviously half-drunk, this abstraction
1623 cannot be perfect, though.
1624
1625 IO::AIO allows you to convert directory paths into a so-called IO::AIO::WD
1626 object. This object stores the canonicalised, absolute version of the
1627 path, and on systems that allow it, also a directory file descriptor.
1628
1629 Everywhere where a pathname is accepted by IO::AIO (e.g. in C<aio_stat>
1630 or C<aio_unlink>), one can specify an array reference with an IO::AIO::WD
1631 object and a pathname instead (or the IO::AIO::WD object alone, which
1632 gets interpreted as C<[$wd, "."]>). If the pathname is absolute, the
1633 IO::AIO::WD object is ignored, otherwise the pathname is resolved relative
1634 to that IO::AIO::WD object.
1635
1636 For example, to get a wd object for F</etc> and then stat F<passwd>
1637 inside, you would write:
1638
1639 aio_wd "/etc", sub {
1640 my $etcdir = shift;
1641
1642 # although $etcdir can be undef on error, there is generally no reason
1643 # to check for errors here, as aio_stat will fail with ENOENT
1644 # when $etcdir is undef.
1645
1646 aio_stat [$etcdir, "passwd"], sub {
1647 # yay
1648 };
1649 };
1650
1651 The fact that C<aio_wd> is a request and not a normal function shows that
1652 creating an IO::AIO::WD object is itself a potentially blocking operation,
1653 which is why it is done asynchronously.
1654
1655 To stat the directory obtained with C<aio_wd> above, one could write
1656 either of the following three request calls:
1657
1658 aio_lstat "/etc" , sub { ... # pathname as normal string
1659 aio_lstat [$wd, "."], sub { ... # "." relative to $wd (i.e. $wd itself)
1660 aio_lstat $wd , sub { ... # shorthand for the previous
1661
1662 As with normal pathnames, IO::AIO keeps a copy of the working directory
1663 object and the pathname string, so you could write the following without
1664 causing any issues due to C<$path> getting reused:
1665
1666 my $path = [$wd, undef];
1667
1668 for my $name (qw(abc def ghi)) {
1669 $path->[1] = $name;
1670 aio_stat $path, sub {
1671 # ...
1672 };
1673 }
1674
1675 There are some caveats: when directories get renamed (or deleted), the
1676 pathname string doesn't change, so will point to the new directory (or
1677 nowhere at all), while the directory fd, if available on the system,
1678 will still point to the original directory. Most functions accepting a
1679 pathname will use the directory fd on newer systems, and the string on
1680 older systems. Some functions (such as C<aio_realpath>) will always rely on
1681 the string form of the pathname.
1682
1683 So this functionality is mainly useful to get some protection against
1684 C<chdir>, to easily get an absolute path out of a relative path for future
1685 reference, and to speed up doing many operations in the same directory
1686 (e.g. when stat'ing all files in a directory).
1687
1688 The following functions implement this working directory abstraction:
1689
1690 =over 4
1691
1692 =item aio_wd $pathname, $callback->($wd)
1693
1694 Asynchonously canonicalise the given pathname and convert it to an
1695 IO::AIO::WD object representing it. If possible and supported on the
1696 system, also open a directory fd to speed up pathname resolution relative
1697 to this working directory.
1698
1699 If something goes wrong, then C<undef> is passwd to the callback instead
1700 of a working directory object and C<$!> is set appropriately. Since
1701 passing C<undef> as working directory component of a pathname fails the
1702 request with C<ENOENT>, there is often no need for error checking in the
1703 C<aio_wd> callback, as future requests using the value will fail in the
1704 expected way.
1705
1706 =item IO::AIO::CWD
1707
1708 This is a compile time constant (object) that represents the process
1709 current working directory.
1710
1711 Specifying this object as working directory object for a pathname is as if
1712 the pathname would be specified directly, without a directory object. For
1713 example, these calls are functionally identical:
1714
1715 aio_stat "somefile", sub { ... };
1716 aio_stat [IO::AIO::CWD, "somefile"], sub { ... };
1717
1718 =back
1719
1720 To recover the path associated with an IO::AIO::WD object, you can use
1721 C<aio_realpath>:
1722
1723 aio_realpath $wd, sub {
1724 warn "path is $_[0]\n";
1725 };
1726
1727 Currently, C<aio_statvfs> always, and C<aio_rename> and C<aio_rmdir>
1728 sometimes, fall back to using an absolue path.
1729
1730 =head2 IO::AIO::REQ CLASS
1731
1732 All non-aggregate C<aio_*> functions return an object of this class when
1733 called in non-void context.
1734
1735 =over 4
1736
1737 =item cancel $req
1738
1739 Cancels the request, if possible. Has the effect of skipping execution
1740 when entering the B<execute> state and skipping calling the callback when
1741 entering the the B<result> state, but will leave the request otherwise
1742 untouched (with the exception of readdir). That means that requests that
1743 currently execute will not be stopped and resources held by the request
1744 will not be freed prematurely.
1745
1746 =item cb $req $callback->(...)
1747
1748 Replace (or simply set) the callback registered to the request.
1749
1750 =back
1751
1752 =head2 IO::AIO::GRP CLASS
1753
1754 This class is a subclass of L<IO::AIO::REQ>, so all its methods apply to
1755 objects of this class, too.
1756
1757 A IO::AIO::GRP object is a special request that can contain multiple other
1758 aio requests.
1759
1760 You create one by calling the C<aio_group> constructing function with a
1761 callback that will be called when all contained requests have entered the
1762 C<done> state:
1763
1764 my $grp = aio_group sub {
1765 print "all requests are done\n";
1766 };
1767
1768 You add requests by calling the C<add> method with one or more
1769 C<IO::AIO::REQ> objects:
1770
1771 $grp->add (aio_unlink "...");
1772
1773 add $grp aio_stat "...", sub {
1774 $_[0] or return $grp->result ("error");
1775
1776 # add another request dynamically, if first succeeded
1777 add $grp aio_open "...", sub {
1778 $grp->result ("ok");
1779 };
1780 };
1781
1782 This makes it very easy to create composite requests (see the source of
1783 C<aio_move> for an application) that work and feel like simple requests.
1784
1785 =over 4
1786
1787 =item * The IO::AIO::GRP objects will be cleaned up during calls to
1788 C<IO::AIO::poll_cb>, just like any other request.
1789
1790 =item * They can be canceled like any other request. Canceling will cancel not
1791 only the request itself, but also all requests it contains.
1792
1793 =item * They can also can also be added to other IO::AIO::GRP objects.
1794
1795 =item * You must not add requests to a group from within the group callback (or
1796 any later time).
1797
1798 =back
1799
1800 Their lifetime, simplified, looks like this: when they are empty, they
1801 will finish very quickly. If they contain only requests that are in the
1802 C<done> state, they will also finish. Otherwise they will continue to
1803 exist.
1804
1805 That means after creating a group you have some time to add requests
1806 (precisely before the callback has been invoked, which is only done within
1807 the C<poll_cb>). And in the callbacks of those requests, you can add
1808 further requests to the group. And only when all those requests have
1809 finished will the the group itself finish.
1810
1811 =over 4
1812
1813 =item add $grp ...
1814
1815 =item $grp->add (...)
1816
1817 Add one or more requests to the group. Any type of L<IO::AIO::REQ> can
1818 be added, including other groups, as long as you do not create circular
1819 dependencies.
1820
1821 Returns all its arguments.
1822
1823 =item $grp->cancel_subs
1824
1825 Cancel all subrequests and clears any feeder, but not the group request
1826 itself. Useful when you queued a lot of events but got a result early.
1827
1828 The group request will finish normally (you cannot add requests to the
1829 group).
1830
1831 =item $grp->result (...)
1832
1833 Set the result value(s) that will be passed to the group callback when all
1834 subrequests have finished and set the groups errno to the current value
1835 of errno (just like calling C<errno> without an error number). By default,
1836 no argument will be passed and errno is zero.
1837
1838 =item $grp->errno ([$errno])
1839
1840 Sets the group errno value to C<$errno>, or the current value of errno
1841 when the argument is missing.
1842
1843 Every aio request has an associated errno value that is restored when
1844 the callback is invoked. This method lets you change this value from its
1845 default (0).
1846
1847 Calling C<result> will also set errno, so make sure you either set C<$!>
1848 before the call to C<result>, or call c<errno> after it.
1849
1850 =item feed $grp $callback->($grp)
1851
1852 Sets a feeder/generator on this group: every group can have an attached
1853 generator that generates requests if idle. The idea behind this is that,
1854 although you could just queue as many requests as you want in a group,
1855 this might starve other requests for a potentially long time. For example,
1856 C<aio_scandir> might generate hundreds of thousands of C<aio_stat>
1857 requests, delaying any later requests for a long time.
1858
1859 To avoid this, and allow incremental generation of requests, you can
1860 instead a group and set a feeder on it that generates those requests. The
1861 feed callback will be called whenever there are few enough (see C<limit>,
1862 below) requests active in the group itself and is expected to queue more
1863 requests.
1864
1865 The feed callback can queue as many requests as it likes (i.e. C<add> does
1866 not impose any limits).
1867
1868 If the feed does not queue more requests when called, it will be
1869 automatically removed from the group.
1870
1871 If the feed limit is C<0> when this method is called, it will be set to
1872 C<2> automatically.
1873
1874 Example:
1875
1876 # stat all files in @files, but only ever use four aio requests concurrently:
1877
1878 my $grp = aio_group sub { print "finished\n" };
1879 limit $grp 4;
1880 feed $grp sub {
1881 my $file = pop @files
1882 or return;
1883
1884 add $grp aio_stat $file, sub { ... };
1885 };
1886
1887 =item limit $grp $num
1888
1889 Sets the feeder limit for the group: The feeder will be called whenever
1890 the group contains less than this many requests.
1891
1892 Setting the limit to C<0> will pause the feeding process.
1893
1894 The default value for the limit is C<0>, but note that setting a feeder
1895 automatically bumps it up to C<2>.
1896
1897 =back
1898
1899
1900 =head2 SUPPORT FUNCTIONS
1901
1902 =head3 EVENT PROCESSING AND EVENT LOOP INTEGRATION
1903
1904 =over 4
1905
1906 =item $fileno = IO::AIO::poll_fileno
1907
1908 Return the I<request result pipe file descriptor>. This filehandle must be
1909 polled for reading by some mechanism outside this module (e.g. EV, Glib,
1910 select and so on, see below or the SYNOPSIS). If the pipe becomes readable
1911 you have to call C<poll_cb> to check the results.
1912
1913 See C<poll_cb> for an example.
1914
1915 =item IO::AIO::poll_cb
1916
1917 Process some requests that have reached the result phase (i.e. they have
1918 been executed but the results are not yet reported). You have to call
1919 this "regularly" to finish outstanding requests.
1920
1921 Returns C<0> if all events could be processed (or there were no
1922 events to process), or C<-1> if it returned earlier for whatever
1923 reason. Returns immediately when no events are outstanding. The amount
1924 of events processed depends on the settings of C<IO::AIO::max_poll_req>,
1925 C<IO::AIO::max_poll_time> and C<IO::AIO::max_outstanding>.
1926
1927 If not all requests were processed for whatever reason, the poll file
1928 descriptor will still be ready when C<poll_cb> returns, so normally you
1929 don't have to do anything special to have it called later.
1930
1931 Apart from calling C<IO::AIO::poll_cb> when the event filehandle becomes
1932 ready, it can be beneficial to call this function from loops which submit
1933 a lot of requests, to make sure the results get processed when they become
1934 available and not just when the loop is finished and the event loop takes
1935 over again. This function returns very fast when there are no outstanding
1936 requests.
1937
1938 Example: Install an Event watcher that automatically calls
1939 IO::AIO::poll_cb with high priority (more examples can be found in the
1940 SYNOPSIS section, at the top of this document):
1941
1942 Event->io (fd => IO::AIO::poll_fileno,
1943 poll => 'r', async => 1,
1944 cb => \&IO::AIO::poll_cb);
1945
1946 =item IO::AIO::poll_wait
1947
1948 Wait until either at least one request is in the result phase or no
1949 requests are outstanding anymore.
1950
1951 This is useful if you want to synchronously wait for some requests to
1952 become ready, without actually handling them.
1953
1954 See C<nreqs> for an example.
1955
1956 =item IO::AIO::poll
1957
1958 Waits until some requests have been handled.
1959
1960 Returns the number of requests processed, but is otherwise strictly
1961 equivalent to:
1962
1963 IO::AIO::poll_wait, IO::AIO::poll_cb
1964
1965 =item IO::AIO::flush
1966
1967 Wait till all outstanding AIO requests have been handled.
1968
1969 Strictly equivalent to:
1970
1971 IO::AIO::poll_wait, IO::AIO::poll_cb
1972 while IO::AIO::nreqs;
1973
1974 This function can be useful at program aborts, to make sure outstanding
1975 I/O has been done (C<IO::AIO> uses an C<END> block which already calls
1976 this function on normal exits), or when you are merely using C<IO::AIO>
1977 for its more advanced functions, rather than for async I/O, e.g.:
1978
1979 my ($dirs, $nondirs);
1980 IO::AIO::aio_scandir "/tmp", 0, sub { ($dirs, $nondirs) = @_ };
1981 IO::AIO::flush;
1982 # $dirs, $nondirs are now set
1983
1984 =item IO::AIO::max_poll_reqs $nreqs
1985
1986 =item IO::AIO::max_poll_time $seconds
1987
1988 These set the maximum number of requests (default C<0>, meaning infinity)
1989 that are being processed by C<IO::AIO::poll_cb> in one call, respectively
1990 the maximum amount of time (default C<0>, meaning infinity) spent in
1991 C<IO::AIO::poll_cb> to process requests (more correctly the mininum amount
1992 of time C<poll_cb> is allowed to use).
1993
1994 Setting C<max_poll_time> to a non-zero value creates an overhead of one
1995 syscall per request processed, which is not normally a problem unless your
1996 callbacks are really really fast or your OS is really really slow (I am
1997 not mentioning Solaris here). Using C<max_poll_reqs> incurs no overhead.
1998
1999 Setting these is useful if you want to ensure some level of
2000 interactiveness when perl is not fast enough to process all requests in
2001 time.
2002
2003 For interactive programs, values such as C<0.01> to C<0.1> should be fine.
2004
2005 Example: Install an Event watcher that automatically calls
2006 IO::AIO::poll_cb with low priority, to ensure that other parts of the
2007 program get the CPU sometimes even under high AIO load.
2008
2009 # try not to spend much more than 0.1s in poll_cb
2010 IO::AIO::max_poll_time 0.1;
2011
2012 # use a low priority so other tasks have priority
2013 Event->io (fd => IO::AIO::poll_fileno,
2014 poll => 'r', nice => 1,
2015 cb => &IO::AIO::poll_cb);
2016
2017 =back
2018
2019
2020 =head3 CONTROLLING THE NUMBER OF THREADS
2021
2022 =over
2023
2024 =item IO::AIO::min_parallel $nthreads
2025
2026 Set the minimum number of AIO threads to C<$nthreads>. The current
2027 default is C<8>, which means eight asynchronous operations can execute
2028 concurrently at any one time (the number of outstanding requests,
2029 however, is unlimited).
2030
2031 IO::AIO starts threads only on demand, when an AIO request is queued and
2032 no free thread exists. Please note that queueing up a hundred requests can
2033 create demand for a hundred threads, even if it turns out that everything
2034 is in the cache and could have been processed faster by a single thread.
2035
2036 It is recommended to keep the number of threads relatively low, as some
2037 Linux kernel versions will scale negatively with the number of threads
2038 (higher parallelity => MUCH higher latency). With current Linux 2.6
2039 versions, 4-32 threads should be fine.
2040
2041 Under most circumstances you don't need to call this function, as the
2042 module selects a default that is suitable for low to moderate load.
2043
2044 =item IO::AIO::max_parallel $nthreads
2045
2046 Sets the maximum number of AIO threads to C<$nthreads>. If more than the
2047 specified number of threads are currently running, this function kills
2048 them. This function blocks until the limit is reached.
2049
2050 While C<$nthreads> are zero, aio requests get queued but not executed
2051 until the number of threads has been increased again.
2052
2053 This module automatically runs C<max_parallel 0> at program end, to ensure
2054 that all threads are killed and that there are no outstanding requests.
2055
2056 Under normal circumstances you don't need to call this function.
2057
2058 =item IO::AIO::max_idle $nthreads
2059
2060 Limit the number of threads (default: 4) that are allowed to idle
2061 (i.e., threads that did not get a request to process within the idle
2062 timeout (default: 10 seconds). That means if a thread becomes idle while
2063 C<$nthreads> other threads are also idle, it will free its resources and
2064 exit.
2065
2066 This is useful when you allow a large number of threads (e.g. 100 or 1000)
2067 to allow for extremely high load situations, but want to free resources
2068 under normal circumstances (1000 threads can easily consume 30MB of RAM).
2069
2070 The default is probably ok in most situations, especially if thread
2071 creation is fast. If thread creation is very slow on your system you might
2072 want to use larger values.
2073
2074 =item IO::AIO::idle_timeout $seconds
2075
2076 Sets the minimum idle timeout (default 10) after which worker threads are
2077 allowed to exit. SEe C<IO::AIO::max_idle>.
2078
2079 =item IO::AIO::max_outstanding $maxreqs
2080
2081 Sets the maximum number of outstanding requests to C<$nreqs>. If
2082 you do queue up more than this number of requests, the next call to
2083 C<IO::AIO::poll_cb> (and other functions calling C<poll_cb>, such as
2084 C<IO::AIO::flush> or C<IO::AIO::poll>) will block until the limit is no
2085 longer exceeded.
2086
2087 In other words, this setting does not enforce a queue limit, but can be
2088 used to make poll functions block if the limit is exceeded.
2089
2090 This is a bad function to use in interactive programs because it blocks,
2091 and a bad way to reduce concurrency because it is inexact. If you need to
2092 issue many requests without being able to call a poll function on demand,
2093 it is better to use an C<aio_group> together with a feed callback.
2094
2095 Its main use is in scripts without an event loop - when you want to stat a
2096 lot of files, you can write something like this:
2097
2098 IO::AIO::max_outstanding 32;
2099
2100 for my $path (...) {
2101 aio_stat $path , ...;
2102 IO::AIO::poll_cb;
2103 }
2104
2105 IO::AIO::flush;
2106
2107 The call to C<poll_cb> inside the loop will normally return instantly,
2108 allowing the loop to progress, but as soon as more than C<32> requests
2109 are in-flight, it will block until some requests have been handled. This
2110 keeps the loop from pushing a large number of C<aio_stat> requests onto
2111 the queue (which, with many paths to stat, can use up a lot of memory).
2112
2113 The default value for C<max_outstanding> is very large, so there is no
2114 practical limit on the number of outstanding requests.
2115
2116 =back
2117
2118
2119 =head3 STATISTICAL INFORMATION
2120
2121 =over
2122
2123 =item IO::AIO::nreqs
2124
2125 Returns the number of requests currently in the ready, execute or pending
2126 states (i.e. for which their callback has not been invoked yet).
2127
2128 Example: wait till there are no outstanding requests anymore:
2129
2130 IO::AIO::poll_wait, IO::AIO::poll_cb
2131 while IO::AIO::nreqs;
2132
2133 =item IO::AIO::nready
2134
2135 Returns the number of requests currently in the ready state (not yet
2136 executed).
2137
2138 =item IO::AIO::npending
2139
2140 Returns the number of requests currently in the pending state (executed,
2141 but not yet processed by poll_cb).
2142
2143 =back
2144
2145
2146 =head3 SUBSECOND STAT TIME ACCESS
2147
2148 Both C<aio_stat>/C<aio_lstat> and perl's C<stat>/C<lstat> functions can
2149 generally find access/modification and change times with subsecond time
2150 accuracy of the system supports it, but perl's built-in functions only
2151 return the integer part.
2152
2153 The following functions return the timestamps of the most recent
2154 stat with subsecond precision on most systems and work both after
2155 C<aio_stat>/C<aio_lstat> and perl's C<stat>/C<lstat> calls. Their return
2156 value is only meaningful after a successful C<stat>/C<lstat> call, or
2157 during/after a successful C<aio_stat>/C<aio_lstat> callback.
2158
2159 This is similar to the L<Time::HiRes> C<stat> functions, but can return
2160 full resolution without rounding and work with standard perl C<stat>,
2161 alleviating the need to call the special C<Time::HiRes> functions, which
2162 do not act like their perl counterparts.
2163
2164 On operating systems or file systems where subsecond time resolution is
2165 not supported or could not be detected, a fractional part of C<0> is
2166 returned, so it is always safe to call these functions.
2167
2168 =over 4
2169
2170 =item $seconds = IO::AIO::st_atime, IO::AIO::st_mtime, IO::AIO::st_ctime, IO::AIO::st_btime
2171
2172 Return the access, modication, change or birth time, respectively,
2173 including fractional part. Due to the limited precision of floating point,
2174 the accuracy on most platforms is only a bit better than milliseconds
2175 for times around now - see the I<nsec> function family, below, for full
2176 accuracy.
2177
2178 File birth time is only available when the OS and perl support it (on
2179 FreeBSD and NetBSD at the time of this writing, although support is
2180 adaptive, so if your OS/perl gains support, IO::AIO can take advantage of
2181 it). On systems where it isn't available, C<0> is currently returned, but
2182 this might change to C<undef> in a future version.
2183
2184 =item $seconds = IO::AIO::st_atime2, IO::AIO::st_mtime2, IO::AIO::st_ctime2, IO::AIO::st_btime2
2185
2186 Like the previous, except they return the timestamp as an array reference
2187 with integer second and nanosecond parts (suitable as input to other
2188 functions in C<IO::AIO>).
2189
2190 stat "." or die;
2191 my $mtime = IO::AIO::st_mtime2;
2192 printf "seconds: %d, nanoseconds: %d\n", $mtime->[0], $mtime->[1];
2193
2194 =item ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime
2195
2196 Returns access, modification, change and birth time all in one go, and
2197 maybe more times in the future version.
2198
2199 =item ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtime2
2200
2201 Like C<IO::AIO::st_xtime>, but rteturns all times as array references, as
2202 in C<st_atime2>.
2203
2204 =item $seconds = IO::AIO::st_atimesec, IO::AIO::st_mtimesec, IO::AIO::st_ctimesec, IO::AIO::st_btimesec
2205
2206 The (integral) seconds part of the file access, modification, change and
2207 birth time.
2208
2209 =item $nanoseconds = IO::AIO::st_atimensec, IO::AIO::st_mtimensec, IO::AIO::st_ctimensec, IO::AIO::st_btimensec
2210
2211 Return the fractional access, modifcation, change or birth time, in nanoseconds,
2212 as an integer in the range C<0> to C<999999999>.
2213
2214 =item ($atime, $mtime, $ctime, $btime, ...) = IO::AIO::st_xtimensec
2215
2216 Like the functions above, but returns all four times in one go (and maybe
2217 more in future versions).
2218
2219 =item $counter = IO::AIO::st_gen
2220
2221 Returns the generation counter (in practice this is just a random number)
2222 of the file. This is only available on platforms which have this member in
2223 their C<struct stat> (most BSDs at the time of this writing) and generally
2224 only to the root user. If unsupported, C<0> is returned, but this might
2225 change to C<undef> in a future version.
2226
2227 =back
2228
2229 Example: print the high resolution modification time of F</etc>, using
2230 C<stat>, and C<IO::AIO::aio_stat>.
2231
2232 if (stat "/etc") {
2233 printf "stat(/etc) mtime: %f\n", IO::AIO::st_mtime;
2234 }
2235
2236 IO::AIO::aio_stat "/etc", sub {
2237 $_[0]
2238 and return;
2239
2240 printf "aio_stat(/etc) mtime: %d.%09d\n", (stat _)[9], IO::AIO::st_mtimensec;
2241 };
2242
2243 IO::AIO::flush;
2244
2245 Output of the awbove on my system, showing reduced and full accuracy:
2246
2247 stat(/etc) mtime: 1534043702.020808
2248 aio_stat(/etc) mtime: 1534043702.020807792
2249
2250
2251 =head3 MISCELLANEOUS FUNCTIONS
2252
2253 IO::AIO implements some functions that are useful when you want to use
2254 some "Advanced I/O" function not available to in Perl, without going the
2255 "Asynchronous I/O" route. Many of these have an asynchronous C<aio_*>
2256 counterpart.
2257
2258 =over 4
2259
2260 =item $retval = IO::AIO::fexecve $fh, $argv, $envp
2261
2262 A more-or-less direct equivalent to the POSIX C<fexecve> functions, which
2263 allows you to specify the program to be executed via a file descriptor (or
2264 handle). Returns C<-1> and sets errno to C<ENOSYS> if not available.
2265
2266 =item $retval = IO::AIO::mount $special, $path, $fstype, $flags = 0, $data = undef
2267
2268 Calls the GNU/Linux mount syscall with the given arguments. All except
2269 C<$flags> are strings, and if C<$data> is C<undef>, a C<NULL> will be
2270 passed.
2271
2272 The following values for C<$flags> are available:
2273
2274 C<IO::AIO::MS_RDONLY>, C<IO::AIO::MS_NOSUID>, C<IO::AIO::MS_NODEV>, C<IO::AIO::MS_NOEXEC>, C<IO::AIO::MS_SYNCHRONOUS>,
2275 C<IO::AIO::MS_REMOUNT>, C<IO::AIO::MS_MANDLOCK>, C<IO::AIO::MS_DIRSYNC>, C<IO::AIO::MS_NOATIME>,
2276 C<IO::AIO::MS_NODIRATIME>, C<IO::AIO::MS_BIND>, C<IO::AIO::MS_MOVE>, C<IO::AIO::MS_REC>, C<IO::AIO::MS_SILENT>,
2277 C<IO::AIO::MS_POSIXACL>, C<IO::AIO::MS_UNBINDABLE>, C<IO::AIO::MS_PRIVATE>, C<IO::AIO::MS_SLAVE>, C<IO::AIO::MS_SHARED>,
2278 C<IO::AIO::MS_RELATIME>, C<IO::AIO::MS_KERNMOUNT>, C<IO::AIO::MS_I_VERSION>, C<IO::AIO::MS_STRICTATIME>,
2279 C<IO::AIO::MS_LAZYTIME>, C<IO::AIO::MS_ACTIVE>, C<IO::AIO::MS_NOUSER>, C<IO::AIO::MS_RMT_MASK>, C<IO::AIO::MS_MGC_VAL> and
2280 C<IO::AIO::MS_MGC_MSK>.
2281
2282 =item $retval = IO::AIO::umount $path, $flags = 0
2283
2284 Invokes the GNU/Linux C<umount> or C<umount2> syscalls. Always calls
2285 C<umount> if C<$flags> is C<0>, otherwqise always tries to call
2286 C<umount2>.
2287
2288 The following C<$flags> are available:
2289
2290 C<IO::AIO::MNT_FORCE>, C<IO::AIO::MNT_DETACH>, C<IO::AIO::MNT_EXPIRE> and C<IO::AIO::UMOUNT_NOFOLLOW>.
2291
2292 =item $numfd = IO::AIO::get_fdlimit
2293
2294 Tries to find the current file descriptor limit and returns it, or
2295 C<undef> and sets C<$!> in case of an error. The limit is one larger than
2296 the highest valid file descriptor number.
2297
2298 =item IO::AIO::min_fdlimit [$numfd]
2299
2300 Try to increase the current file descriptor limit(s) to at least C<$numfd>
2301 by changing the soft or hard file descriptor resource limit. If C<$numfd>
2302 is missing, it will try to set a very high limit, although this is not
2303 recommended when you know the actual minimum that you require.
2304
2305 If the limit cannot be raised enough, the function makes a best-effort
2306 attempt to increase the limit as much as possible, using various
2307 tricks, while still failing. You can query the resulting limit using
2308 C<IO::AIO::get_fdlimit>.
2309
2310 If an error occurs, returns C<undef> and sets C<$!>, otherwise returns
2311 true.
2312
2313 =item IO::AIO::sendfile $ofh, $ifh, $offset, $count
2314
2315 Calls the C<eio_sendfile_sync> function, which is like C<aio_sendfile>,
2316 but is blocking (this makes most sense if you know the input data is
2317 likely cached already and the output filehandle is set to non-blocking
2318 operations).
2319
2320 Returns the number of bytes copied, or C<-1> on error.
2321
2322 =item IO::AIO::fadvise $fh, $offset, $len, $advice
2323
2324 Simply calls the C<posix_fadvise> function (see its
2325 manpage for details). The following advice constants are
2326 available: C<IO::AIO::FADV_NORMAL>, C<IO::AIO::FADV_SEQUENTIAL>,
2327 C<IO::AIO::FADV_RANDOM>, C<IO::AIO::FADV_NOREUSE>,
2328 C<IO::AIO::FADV_WILLNEED>, C<IO::AIO::FADV_DONTNEED>.
2329
2330 On systems that do not implement C<posix_fadvise>, this function returns
2331 ENOSYS, otherwise the return value of C<posix_fadvise>.
2332
2333 =item IO::AIO::madvise $scalar, $offset, $len, $advice
2334
2335 Simply calls the C<posix_madvise> function (see its
2336 manpage for details). The following advice constants are
2337 available:
2338 C<IO::AIO::MADV_NORMAL>,
2339 C<IO::AIO::MADV_SEQUENTIAL>,
2340 C<IO::AIO::MADV_RANDOM>,
2341 C<IO::AIO::MADV_WILLNEED> and
2342 C<IO::AIO::MADV_DONTNEED>,
2343 as well as the following system-specific constants (when not available, the are C<0>):
2344 C<IO::AIO::MADV_FREE>,
2345 C<IO::AIO::MADV_REMOVE>,
2346 C<IO::AIO::MADV_DONTFORK>,
2347 C<IO::AIO::MADV_DOFORK>,
2348 C<IO::AIO::MADV_MERGEABLE>,
2349 C<IO::AIO::MADV_UNMERGEABLE>,
2350 C<IO::AIO::MADV_HUGEPAGE>,
2351 C<IO::AIO::MADV_NOHUGEPAGE>,
2352 C<IO::AIO::MADV_DONTDUMP>,
2353 C<IO::AIO::MADV_DODUMP>,
2354 C<IO::AIO::MADV_WIPEONFORK>,
2355 C<IO::AIO::MADV_KEEPONFORK>,
2356 C<IO::AIO::MADV_COLD>,
2357 C<IO::AIO::MADV_PAGEOUT>,
2358 C<IO::AIO::MADV_POPULATE_READ>,
2359 C<IO::AIO::MADV_POPULATE_WRITE>,
2360 C<IO::AIO::MADV_DONTNEED_LOCKED>,
2361 C<IO::AIO::MADV_HWPOISON> and
2362 C<IO::AIO::MADV_SOFT_OFFLINE>.
2363
2364 If C<$offset> is negative, counts from the end. If C<$length> is negative,
2365 the remaining length of the C<$scalar> is used. If possible, C<$length>
2366 will be reduced to fit into the C<$scalar>.
2367
2368 On systems that do not implement C<posix_madvise>, this function returns
2369 ENOSYS, otherwise the return value of C<posix_madvise>.
2370
2371 =item IO::AIO::mprotect $scalar, $offset, $len, $protect
2372
2373 Simply calls the C<mprotect> function on the preferably AIO::mmap'ed
2374 $scalar (see its manpage for details). The following protect
2375 constants are available: C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_READ>,
2376 C<IO::AIO::PROT_WRITE>, C<IO::AIO::PROT_EXEC>.
2377
2378 If C<$offset> is negative, counts from the end. If C<$length> is negative,
2379 the remaining length of the C<$scalar> is used. If possible, C<$length>
2380 will be reduced to fit into the C<$scalar>.
2381
2382 On systems that do not implement C<mprotect>, this function returns
2383 ENOSYS, otherwise the return value of C<mprotect>.
2384
2385 =item IO::AIO::mmap $scalar, $length, $prot, $flags, $fh[, $offset]
2386
2387 Memory-maps a file (or anonymous memory range) and attaches it to the
2388 given C<$scalar>, which will act like a string scalar. Returns true on
2389 success, and false otherwise.
2390
2391 The scalar must exist, but its contents do not matter - this means you
2392 cannot use a nonexistant array or hash element. When in doubt, C<undef>
2393 the scalar first.
2394
2395 The only operations allowed on the mmapped scalar are C<substr>/C<vec>,
2396 which don't change the string length, and most read-only operations such
2397 as copying it or searching it with regexes and so on.
2398
2399 Anything else is unsafe and will, at best, result in memory leaks.
2400
2401 The memory map associated with the C<$scalar> is automatically removed
2402 when the C<$scalar> is undef'd or destroyed, or when the C<IO::AIO::mmap>
2403 or C<IO::AIO::munmap> functions are called on it.
2404
2405 This calls the C<mmap>(2) function internally. See your system's manual
2406 page for details on the C<$length>, C<$prot> and C<$flags> parameters.
2407
2408 The C<$length> must be larger than zero and smaller than the actual
2409 filesize.
2410
2411 C<$prot> is a combination of C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_EXEC>,
2412 C<IO::AIO::PROT_READ> and/or C<IO::AIO::PROT_WRITE>,
2413
2414 C<$flags> can be a combination of
2415 C<IO::AIO::MAP_SHARED> or
2416 C<IO::AIO::MAP_PRIVATE>,
2417 or a number of system-specific flags (when not available, the are C<0>):
2418 C<IO::AIO::MAP_ANONYMOUS> (which is set to C<MAP_ANON> if your system only provides this constant),
2419 C<IO::AIO::MAP_LOCKED>,
2420 C<IO::AIO::MAP_NORESERVE>,
2421 C<IO::AIO::MAP_POPULATE>,
2422 C<IO::AIO::MAP_NONBLOCK>,
2423 C<IO::AIO::MAP_FIXED>,
2424 C<IO::AIO::MAP_GROWSDOWN>,
2425 C<IO::AIO::MAP_32BIT>,
2426 C<IO::AIO::MAP_HUGETLB>,
2427 C<IO::AIO::MAP_STACK>,
2428 C<IO::AIO::MAP_FIXED_NOREPLACE>,
2429 C<IO::AIO::MAP_SHARED_VALIDATE>,
2430 C<IO::AIO::MAP_SYNC>,
2431 C<IO::AIO::MAP_UNINITIALIZED>,
2432 C<IO::AIO::MAP_DENYWRITE>,
2433 C<IO::AIO::MAP_EXECUTABLE>,
2434 C<IO::AIO::MAP_LOCKED> and
2435 C<IO::AIO::MAP_DROPPABLE>.
2436
2437 If C<$fh> is C<undef>, then a file descriptor of C<-1> is passed.
2438
2439 C<$offset> is the offset from the start of the file - it generally must be
2440 a multiple of C<IO::AIO::PAGESIZE> and defaults to C<0>.
2441
2442 Example:
2443
2444 use Digest::MD5;
2445 use IO::AIO;
2446
2447 open my $fh, "<verybigfile"
2448 or die "$!";
2449
2450 IO::AIO::mmap my $data, -s $fh, IO::AIO::PROT_READ, IO::AIO::MAP_SHARED, $fh
2451 or die "verybigfile: $!";
2452
2453 my $fast_md5 = md5 $data;
2454
2455 =item IO::AIO::munmap $scalar
2456
2457 Removes a previous mmap and undefines the C<$scalar>.
2458
2459 =item IO::AIO::mremap $scalar, $new_length, $flags = MREMAP_MAYMOVE[, $new_address = 0]
2460
2461 Calls the Linux-specific mremap(2) system call. The C<$scalar> must have
2462 been mapped by C<IO::AIO::mmap>, and C<$flags> must currently either be
2463 C<0> or C<IO::AIO::MREMAP_MAYMOVE>.
2464
2465 Returns true if successful, and false otherwise. If the underlying mmapped
2466 region has changed address, then the true value has the numerical value
2467 C<1>, otherwise it has the numerical value C<0>:
2468
2469 my $success = IO::AIO::mremap $mmapped, 8192, IO::AIO::MREMAP_MAYMOVE
2470 or die "mremap: $!";
2471
2472 if ($success*1) {
2473 warn "scalar has chanegd address in memory\n";
2474 }
2475
2476 C<IO::AIO::MREMAP_FIXED> and the C<$new_address> argument are currently
2477 implemented, but not supported and might go away in a future version.
2478
2479 On systems where this call is not supported or is not emulated, this call
2480 returns falls and sets C<$!> to C<ENOSYS>.
2481
2482 =item IO::AIO::mlockall $flags
2483
2484 Calls the C<eio_mlockall_sync> function, which is like C<aio_mlockall>,
2485 but is blocking.
2486
2487 =item IO::AIO::munlock $scalar, $offset = 0, $length = undef
2488
2489 Calls the C<munlock> function, undoing the effects of a previous
2490 C<aio_mlock> call (see its description for details).
2491
2492 =item IO::AIO::munlockall
2493
2494 Calls the C<munlockall> function.
2495
2496 On systems that do not implement C<munlockall>, this function returns
2497 ENOSYS, otherwise the return value of C<munlockall>.
2498
2499 =item $fh = IO::AIO::accept4 $r_fh, $sockaddr, $sockaddr_maxlen, $flags
2500
2501 Uses the GNU/Linux C<accept4(2)> syscall, if available, to accept a socket
2502 and return the new file handle on success, or sets C<$!> and returns
2503 C<undef> on error.
2504
2505 The remote name of the new socket will be stored in C<$sockaddr>, which
2506 will be extended to allow for at least C<$sockaddr_maxlen> octets. If the
2507 socket name does not fit into C<$sockaddr_maxlen> octets, this is signaled
2508 by returning a longer string in C<$sockaddr>, which might or might not be
2509 truncated.
2510
2511 To accept name-less sockets, use C<undef> for C<$sockaddr> and C<0> for
2512 C<$sockaddr_maxlen>.
2513
2514 The main reasons to use this syscall rather than portable C<accept(2)>
2515 are that you can specify C<SOCK_NONBLOCK> and/or C<SOCK_CLOEXEC>
2516 flags and you can accept name-less sockets by specifying C<0> for
2517 C<$sockaddr_maxlen>, which is sadly not possible with perl's interface to
2518 C<accept>.
2519
2520 =item IO::AIO::splice $r_fh, $r_off, $w_fh, $w_off, $length, $flags
2521
2522 Calls the GNU/Linux C<splice(2)> syscall, if available. If C<$r_off> or
2523 C<$w_off> are C<undef>, then C<NULL> is passed for these, otherwise they
2524 should be the file offset.
2525
2526 C<$r_fh> and C<$w_fh> should not refer to the same file, as splice might
2527 silently corrupt the data in this case.
2528
2529 The following symbol flag values are available: C<IO::AIO::SPLICE_F_MOVE>,
2530 C<IO::AIO::SPLICE_F_NONBLOCK>, C<IO::AIO::SPLICE_F_MORE> and
2531 C<IO::AIO::SPLICE_F_GIFT>.
2532
2533 See the C<splice(2)> manpage for details.
2534
2535 =item IO::AIO::tee $r_fh, $w_fh, $length, $flags
2536
2537 Calls the GNU/Linux C<tee(2)> syscall, see its manpage and the
2538 description for C<IO::AIO::splice> above for details.
2539
2540 =item $actual_size = IO::AIO::pipesize $r_fh[, $new_size]
2541
2542 Attempts to query or change the pipe buffer size. Obviously works only
2543 on pipes, and currently works only on GNU/Linux systems, and fails with
2544 C<-1>/C<ENOSYS> everywhere else. If anybody knows how to influence pipe buffer
2545 size on other systems, drop me a note.
2546
2547 =item ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
2548
2549 This is a direct interface to the Linux L<pipe2(2)> system call. If
2550 C<$flags> is missing or C<0>, then this should be the same as a call to
2551 perl's built-in C<pipe> function and create a new pipe, and works on
2552 systems that lack the pipe2 syscall. On win32, this case invokes C<_pipe
2553 (..., 4096, O_BINARY)>.
2554
2555 If C<$flags> is non-zero, it tries to invoke the pipe2 system call with
2556 the given flags (Linux 2.6.27, glibc 2.9).
2557
2558 On success, the read and write file handles are returned.
2559
2560 On error, nothing will be returned. If the pipe2 syscall is missing and
2561 C<$flags> is non-zero, fails with C<ENOSYS>.
2562
2563 Please refer to L<pipe2(2)> for more info on the C<$flags>, but at the
2564 time of this writing, C<IO::AIO::O_CLOEXEC>, C<IO::AIO::O_NONBLOCK> and
2565 C<IO::AIO::O_DIRECT> (Linux 3.4, for packet-based pipes) were supported.
2566
2567 Example: create a pipe race-free w.r.t. threads and fork:
2568
2569 my ($rfh, $wfh) = IO::AIO::pipe2 IO::AIO::O_CLOEXEC
2570 or die "pipe2: $!\n";
2571
2572 =item $fh = IO::AIO::memfd_create $pathname[, $flags]
2573
2574 This is a direct interface to the Linux L<memfd_create(2)> system
2575 call. The (unhelpful) default for C<$flags> is C<0>, but your default
2576 should be C<IO::AIO::MFD_CLOEXEC>.
2577
2578 On success, the new memfd filehandle is returned, otherwise returns
2579 C<undef>. If the memfd_create syscall is missing, fails with C<ENOSYS>.
2580
2581 Please refer to L<memfd_create(2)> for more info on this call.
2582
2583 The following C<$flags> values are available: C<IO::AIO::MFD_CLOEXEC>,
2584 C<IO::AIO::MFD_ALLOW_SEALING>, C<IO::AIO::MFD_HUGETLB>,
2585 C<IO::AIO::MFD_HUGETLB_2MB> and C<IO::AIO::MFD_HUGETLB_1GB>.
2586
2587 Example: create a new memfd.
2588
2589 my $fh = IO::AIO::memfd_create "somenameforprocfd", IO::AIO::MFD_CLOEXEC
2590 or die "memfd_create: $!\n";
2591
2592 =item $fh = IO::AIO::pidfd_open $pid[, $flags]
2593
2594 This is an interface to the Linux L<pidfd_open(2)> system call. The
2595 default for C<$flags> is C<0>.
2596
2597 On success, a new pidfd filehandle is returned (that is already set to
2598 close-on-exec), otherwise returns C<undef>. If the syscall is missing,
2599 fails with C<ENOSYS>.
2600
2601 Example: open pid 6341 as pidfd.
2602
2603 my $fh = IO::AIO::pidfd_open 6341
2604 or die "pidfd_open: $!\n";
2605
2606 =item $status = IO::AIO::pidfd_send_signal $pidfh, $signal[, $siginfo[, $flags]]
2607
2608 This is an interface to the Linux L<pidfd_send_signal> system call. The
2609 default for C<$siginfo> is C<undef> and the default for C<$flags> is C<0>.
2610
2611 Returns the system call status. If the syscall is missing, fails with
2612 C<ENOSYS>.
2613
2614 When specified, C<$siginfo> must be a reference to a hash with one or more
2615 of the following members:
2616
2617 =over
2618
2619 =item code - the C<si_code> member
2620
2621 =item pid - the C<si_pid> member
2622
2623 =item uid - the C<si_uid> member
2624
2625 =item value_int - the C<si_value.sival_int> member
2626
2627 =item value_ptr - the C<si_value.sival_ptr> member, specified as an integer
2628
2629 =back
2630
2631 Example: send a SIGKILL to the specified process.
2632
2633 my $status = IO::AIO::pidfd_send_signal $pidfh, 9, undef
2634 and die "pidfd_send_signal: $!\n";
2635
2636 Example: send a SIGKILL to the specified process with extra data.
2637
2638 my $status = IO::AIO::pidfd_send_signal $pidfh, 9, { code => -1, value_int => 7 }
2639 and die "pidfd_send_signal: $!\n";
2640
2641 =item $fh = IO::AIO::pidfd_getfd $pidfh, $targetfd[, $flags]
2642
2643 This is an interface to the Linux L<pidfd_getfd> system call. The default
2644 for C<$flags> is C<0>.
2645
2646 On success, returns a dup'ed copy of the target file descriptor (specified
2647 as an integer) returned (that is already set to close-on-exec), otherwise
2648 returns C<undef>. If the syscall is missing, fails with C<ENOSYS>.
2649
2650 Example: get a copy of standard error of another process and print soemthing to it.
2651
2652 my $errfh = IO::AIO::pidfd_getfd $pidfh, 2
2653 or die "pidfd_getfd: $!\n";
2654 print $errfh "stderr\n";
2655
2656 =item $fh = IO::AIO::eventfd [$initval, [$flags]]
2657
2658 This is a direct interface to the Linux L<eventfd(2)> system call. The
2659 (unhelpful) defaults for C<$initval> and C<$flags> are C<0> for both.
2660
2661 On success, the new eventfd filehandle is returned, otherwise returns
2662 C<undef>. If the eventfd syscall is missing, fails with C<ENOSYS>.
2663
2664 Please refer to L<eventfd(2)> for more info on this call.
2665
2666 The following symbol flag values are available: C<IO::AIO::EFD_CLOEXEC>,
2667 C<IO::AIO::EFD_NONBLOCK> and C<IO::AIO::EFD_SEMAPHORE> (Linux 2.6.30).
2668
2669 Example: create a new eventfd filehandle:
2670
2671 $fh = IO::AIO::eventfd 0, IO::AIO::EFD_CLOEXEC
2672 or die "eventfd: $!\n";
2673
2674 =item $fh = IO::AIO::timerfd_create $clockid[, $flags]
2675
2676 This is a direct interface to the Linux L<timerfd_create(2)> system
2677 call. The (unhelpful) default for C<$flags> is C<0>, but your default
2678 should be C<IO::AIO::TFD_CLOEXEC>.
2679
2680 On success, the new timerfd filehandle is returned, otherwise returns
2681 C<undef>. If the timerfd_create syscall is missing, fails with C<ENOSYS>.
2682
2683 Please refer to L<timerfd_create(2)> for more info on this call.
2684
2685 The following C<$clockid> values are
2686 available: C<IO::AIO::CLOCK_REALTIME>, C<IO::AIO::CLOCK_MONOTONIC>
2687 C<IO::AIO::CLOCK_CLOCK_BOOTTIME> (Linux 3.15)
2688 C<IO::AIO::CLOCK_CLOCK_REALTIME_ALARM> (Linux 3.11) and
2689 C<IO::AIO::CLOCK_CLOCK_BOOTTIME_ALARM> (Linux 3.11).
2690
2691 The following C<$flags> values are available (Linux
2692 2.6.27): C<IO::AIO::TFD_NONBLOCK> and C<IO::AIO::TFD_CLOEXEC>.
2693
2694 Example: create a new timerfd and set it to one-second repeated alarms,
2695 then wait for two alarms:
2696
2697 my $fh = IO::AIO::timerfd_create IO::AIO::CLOCK_BOOTTIME, IO::AIO::TFD_CLOEXEC
2698 or die "timerfd_create: $!\n";
2699
2700 defined IO::AIO::timerfd_settime $fh, 0, 1, 1
2701 or die "timerfd_settime: $!\n";
2702
2703 for (1..2) {
2704 8 == sysread $fh, my $buf, 8
2705 or die "timerfd read failure\n";
2706
2707 printf "number of expirations (likely 1): %d\n",
2708 unpack "Q", $buf;
2709 }
2710
2711 =item ($cur_interval, $cur_value) = IO::AIO::timerfd_settime $fh, $flags, $new_interval, $nbw_value
2712
2713 This is a direct interface to the Linux L<timerfd_settime(2)> system
2714 call. Please refer to its manpage for more info on this call.
2715
2716 The new itimerspec is specified using two (possibly fractional) second
2717 values, C<$new_interval> and C<$new_value>).
2718
2719 On success, the current interval and value are returned (as per
2720 C<timerfd_gettime>). On failure, the empty list is returned.
2721
2722 The following C<$flags> values are
2723 available: C<IO::AIO::TFD_TIMER_ABSTIME> and
2724 C<IO::AIO::TFD_TIMER_CANCEL_ON_SET>.
2725
2726 See C<IO::AIO::timerfd_create> for a full example.
2727
2728 =item ($cur_interval, $cur_value) = IO::AIO::timerfd_gettime $fh
2729
2730 This is a direct interface to the Linux L<timerfd_gettime(2)> system
2731 call. Please refer to its manpage for more info on this call.
2732
2733 On success, returns the current values of interval and value for the given
2734 timerfd (as potentially fractional second values). On failure, the empty
2735 list is returned.
2736
2737 =back
2738
2739 =cut
2740
2741 min_parallel 8;
2742
2743 END { flush }
2744
2745 1;
2746
2747 =head1 EVENT LOOP INTEGRATION
2748
2749 It is recommended to use L<AnyEvent::AIO> to integrate IO::AIO
2750 automatically into many event loops:
2751
2752 # AnyEvent integration (EV, Event, Glib, Tk, POE, urxvt, pureperl...)
2753 use AnyEvent::AIO;
2754
2755 You can also integrate IO::AIO manually into many event loops, here are
2756 some examples of how to do this:
2757
2758 # EV integration
2759 my $aio_w = EV::io IO::AIO::poll_fileno, EV::READ, \&IO::AIO::poll_cb;
2760
2761 # Event integration
2762 Event->io (fd => IO::AIO::poll_fileno,
2763 poll => 'r',
2764 cb => \&IO::AIO::poll_cb);
2765
2766 # Glib/Gtk2 integration
2767 add_watch Glib::IO IO::AIO::poll_fileno,
2768 in => sub { IO::AIO::poll_cb; 1 };
2769
2770 # Tk integration
2771 Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
2772 readable => \&IO::AIO::poll_cb);
2773
2774 # Danga::Socket integration
2775 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
2776 \&IO::AIO::poll_cb);
2777
2778 =head2 FORK BEHAVIOUR
2779
2780 Usage of pthreads in a program changes the semantics of fork
2781 considerably. Specifically, only async-safe functions can be called after
2782 fork. Perl doesn't know about this, so in general, you cannot call fork
2783 with defined behaviour in perl if pthreads are involved. IO::AIO uses
2784 pthreads, so this applies, but many other extensions and (for inexplicable
2785 reasons) perl itself often is linked against pthreads, so this limitation
2786 applies to quite a lot of perls.
2787
2788 This module no longer tries to fight your OS, or POSIX. That means IO::AIO
2789 only works in the process that loaded it. Forking is fully supported, but
2790 using IO::AIO in the child is not.
2791
2792 You might get around by not I<using> IO::AIO before (or after)
2793 forking. You could also try to call the L<IO::AIO::reinit> function in the
2794 child:
2795
2796 =over 4
2797
2798 =item IO::AIO::reinit
2799
2800 Abandons all current requests and I/O threads and simply reinitialises all
2801 data structures. This is not an operation supported by any standards, but
2802 happens to work on GNU/Linux and some newer BSD systems.
2803
2804 The only reasonable use for this function is to call it after forking, if
2805 C<IO::AIO> was used in the parent. Calling it while IO::AIO is active in
2806 the process will result in undefined behaviour. Calling it at any time
2807 will also result in any undefined (by POSIX) behaviour.
2808
2809 =back
2810
2811 =head2 LINUX-SPECIFIC CALLS
2812
2813 When a call is documented as "linux-specific" then this means it
2814 originated on GNU/Linux. C<IO::AIO> will usually try to autodetect the
2815 availability and compatibility of such calls regardless of the platform
2816 it is compiled on, so platforms such as FreeBSD which often implement
2817 these calls will work. When in doubt, call them and see if they fail wth
2818 C<ENOSYS>.
2819
2820 =head2 MEMORY USAGE
2821
2822 Per-request usage:
2823
2824 Each aio request uses - depending on your architecture - around 100-200
2825 bytes of memory. In addition, stat requests need a stat buffer (possibly
2826 a few hundred bytes), readdir requires a result buffer and so on. Perl
2827 scalars and other data passed into aio requests will also be locked and
2828 will consume memory till the request has entered the done state.
2829
2830 This is not awfully much, so queuing lots of requests is not usually a
2831 problem.
2832
2833 Per-thread usage:
2834
2835 In the execution phase, some aio requests require more memory for
2836 temporary buffers, and each thread requires a stack and other data
2837 structures (usually around 16k-128k, depending on the OS).
2838
2839 =head1 KNOWN BUGS
2840
2841 Known bugs will be fixed in the next release :)
2842
2843 =head1 KNOWN ISSUES
2844
2845 Calls that try to "import" foreign memory areas (such as C<IO::AIO::mmap>
2846 or C<IO::AIO::aio_slurp>) do not work with generic lvalues, such as
2847 non-created hash slots or other scalars I didn't think of. It's best to
2848 avoid such and either use scalar variables or making sure that the scalar
2849 exists (e.g. by storing C<undef>) and isn't "funny" (e.g. tied).
2850
2851 I am not sure anything can be done about this, so this is considered a
2852 known issue, rather than a bug.
2853
2854 =head1 SECURITY CONSIDERATIONS
2855
2856 All the functions in this module usually do exactly what they say, and not
2857 more. None of them are hardened against race conditions - for example,
2858 C<IO::AIO::aio_rmtree> will simply iterate over all files and remove
2859 them. For security-hardened versions, use L<File::Path>, C<system "rm
2860 -rf"> or other s.
2861
2862 =head1 SEE ALSO
2863
2864 L<AnyEvent::AIO> for easy integration into event loops, L<Coro::AIO> for a
2865 more natural syntax and L<IO::FDPass> for file descriptor passing.
2866
2867 =head1 AUTHOR
2868
2869 Marc Lehmann <schmorp@schmorp.de>
2870 http://home.schmorp.de/
2871
2872 =cut
2873