ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.250 by root, Mon Aug 18 04:26:02 2014 UTC vs.
Revision 1.267 by root, Tue Aug 9 11:39:17 2016 UTC

1=head1 NAME 1=head1 NAME
2 2
3IO::AIO - Asynchronous Input/Output 3IO::AIO - Asynchronous/Advanced Input/Output
4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use IO::AIO; 7 use IO::AIO;
8 8
58not well-supported or restricted (GNU/Linux doesn't allow them on normal 58not well-supported or restricted (GNU/Linux doesn't allow them on normal
59files currently, for example), and they would only support aio_read and 59files currently, for example), and they would only support aio_read and
60aio_write, so the remaining functionality would have to be implemented 60aio_write, so the remaining functionality would have to be implemented
61using threads anyway. 61using threads anyway.
62 62
63In addition to asynchronous I/O, this module also exports some rather
64arcane interfaces, such as C<madvise> or linux's C<splice> system call,
65which is why the C<A> in C<AIO> can also mean I<advanced>.
66
63Although the module will work in the presence of other (Perl-) threads, 67Although the module will work in the presence of other (Perl-) threads,
64it is currently not reentrant in any way, so use appropriate locking 68it is currently not reentrant in any way, so use appropriate locking
65yourself, always call C<poll_cb> from within the same thread, or never 69yourself, always call C<poll_cb> from within the same thread, or never
66call C<poll_cb> (or other C<aio_> functions) recursively. 70call C<poll_cb> (or other C<aio_> functions) recursively.
67 71
94 98
95 # file contents now in $contents 99 # file contents now in $contents
96 print $contents; 100 print $contents;
97 101
98 # exit event loop and program 102 # exit event loop and program
99 EV::unloop; 103 EV::break;
100 }; 104 };
101 }; 105 };
102 106
103 # possibly queue up other requests, or open GUI windows, 107 # possibly queue up other requests, or open GUI windows,
104 # check for sockets etc. etc. 108 # check for sockets etc. etc.
105 109
106 # process events as long as there are some: 110 # process events as long as there are some:
107 EV::loop; 111 EV::run;
108 112
109=head1 REQUEST ANATOMY AND LIFETIME 113=head1 REQUEST ANATOMY AND LIFETIME
110 114
111Every C<aio_*> function creates a request. which is a C data structure not 115Every C<aio_*> function creates a request. which is a C data structure not
112directly visible to Perl. 116directly visible to Perl.
167use common::sense; 171use common::sense;
168 172
169use base 'Exporter'; 173use base 'Exporter';
170 174
171BEGIN { 175BEGIN {
172 our $VERSION = 4.31; 176 our $VERSION = 4.34;
173 177
174 our @AIO_REQ = qw(aio_sendfile aio_seek aio_read aio_write aio_open aio_close 178 our @AIO_REQ = qw(aio_sendfile aio_seek aio_read aio_write aio_open aio_close
175 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx 179 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
176 aio_scandir aio_symlink aio_readlink aio_realpath aio_sync 180 aio_scandir aio_symlink aio_readlink aio_realpath aio_fcntl aio_ioctl
177 aio_fsync aio_syncfs aio_fdatasync aio_sync_file_range aio_allocate 181 aio_sync aio_fsync aio_syncfs aio_fdatasync aio_sync_file_range
178 aio_pathsync aio_readahead aio_fiemap 182 aio_pathsync aio_readahead aio_fiemap aio_allocate
179 aio_rename aio_link aio_move aio_copy aio_group 183 aio_rename aio_link aio_move aio_copy aio_group
180 aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown 184 aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
181 aio_chmod aio_utime aio_truncate 185 aio_chmod aio_utime aio_truncate
182 aio_msync aio_mtouch aio_mlock aio_mlockall 186 aio_msync aio_mtouch aio_mlock aio_mlockall
183 aio_statvfs 187 aio_statvfs
240 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs) 244 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
241 aio_load $pathname, $data, $callback->($status) 245 aio_load $pathname, $data, $callback->($status)
242 aio_copy $srcpath, $dstpath, $callback->($status) 246 aio_copy $srcpath, $dstpath, $callback->($status)
243 aio_move $srcpath, $dstpath, $callback->($status) 247 aio_move $srcpath, $dstpath, $callback->($status)
244 aio_rmtree $pathname, $callback->($status) 248 aio_rmtree $pathname, $callback->($status)
249 aio_fcntl $fh, $cmd, $arg, $callback->($status)
250 aio_ioctl $fh, $request, $buf, $callback->($status)
245 aio_sync $callback->($status) 251 aio_sync $callback->($status)
246 aio_syncfs $fh, $callback->($status) 252 aio_syncfs $fh, $callback->($status)
247 aio_fsync $fh, $callback->($status) 253 aio_fsync $fh, $callback->($status)
248 aio_fdatasync $fh, $callback->($status) 254 aio_fdatasync $fh, $callback->($status)
249 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status) 255 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
394following POSIX and non-POSIX constants are available (missing ones on 400following POSIX and non-POSIX constants are available (missing ones on
395your system are, as usual, C<0>): 401your system are, as usual, C<0>):
396 402
397C<O_ASYNC>, C<O_DIRECT>, C<O_NOATIME>, C<O_CLOEXEC>, C<O_NOCTTY>, C<O_NOFOLLOW>, 403C<O_ASYNC>, C<O_DIRECT>, C<O_NOATIME>, C<O_CLOEXEC>, C<O_NOCTTY>, C<O_NOFOLLOW>,
398C<O_NONBLOCK>, C<O_EXEC>, C<O_SEARCH>, C<O_DIRECTORY>, C<O_DSYNC>, 404C<O_NONBLOCK>, C<O_EXEC>, C<O_SEARCH>, C<O_DIRECTORY>, C<O_DSYNC>,
399C<O_RSYNC>, C<O_SYNC> and C<O_TTY_INIT>. 405C<O_RSYNC>, C<O_SYNC>, C<O_PATH>, C<O_TMPFILE>, and C<O_TTY_INIT>.
400 406
401 407
402=item aio_close $fh, $callback->($status) 408=item aio_close $fh, $callback->($status)
403 409
404Asynchronously close a file and call the callback with the result 410Asynchronously close a file and call the callback with the result
439=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 445=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
440 446
441=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 447=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
442 448
443Reads or writes C<$length> bytes from or to the specified C<$fh> and 449Reads or writes C<$length> bytes from or to the specified C<$fh> and
444C<$offset> into the scalar given by C<$data> and offset C<$dataoffset> 450C<$offset> into the scalar given by C<$data> and offset C<$dataoffset> and
445and calls the callback without the actual number of bytes read (or -1 on 451calls the callback with the actual number of bytes transferred (or -1 on
446error, just like the syscall). 452error, just like the syscall).
447 453
448C<aio_read> will, like C<sysread>, shrink or grow the C<$data> scalar to 454C<aio_read> will, like C<sysread>, shrink or grow the C<$data> scalar to
449offset plus the actual number of bytes read. 455offset plus the actual number of bytes read.
450 456
508As native sendfile syscalls (as practically any non-POSIX interface hacked 514As native sendfile syscalls (as practically any non-POSIX interface hacked
509together in a hurry to improve benchmark numbers) tend to be rather buggy 515together in a hurry to improve benchmark numbers) tend to be rather buggy
510on many systems, this implementation tries to work around some known bugs 516on many systems, this implementation tries to work around some known bugs
511in Linux and FreeBSD kernels (probably others, too), but that might fail, 517in Linux and FreeBSD kernels (probably others, too), but that might fail,
512so you really really should check the return value of C<aio_sendfile> - 518so you really really should check the return value of C<aio_sendfile> -
513fewre bytes than expected might have been transferred. 519fewer bytes than expected might have been transferred.
514 520
515 521
516=item aio_readahead $fh,$offset,$length, $callback->($retval) 522=item aio_readahead $fh,$offset,$length, $callback->($retval)
517 523
518C<aio_readahead> populates the page cache with data from a file so that 524C<aio_readahead> populates the page cache with data from a file so that
522whole pages, so that offset is effectively rounded down to a page boundary 528whole pages, so that offset is effectively rounded down to a page boundary
523and bytes are read up to the next page boundary greater than or equal to 529and bytes are read up to the next page boundary greater than or equal to
524(off-set+length). C<aio_readahead> does not read beyond the end of the 530(off-set+length). C<aio_readahead> does not read beyond the end of the
525file. The current file offset of the file is left unchanged. 531file. The current file offset of the file is left unchanged.
526 532
527If that syscall doesn't exist (likely if your OS isn't Linux) it will be 533If that syscall doesn't exist (likely if your kernel isn't Linux) it will
528emulated by simply reading the data, which would have a similar effect. 534be emulated by simply reading the data, which would have a similar effect.
529 535
530 536
531=item aio_stat $fh_or_path, $callback->($status) 537=item aio_stat $fh_or_path, $callback->($status)
532 538
533=item aio_lstat $fh, $callback->($status) 539=item aio_lstat $fh, $callback->($status)
626 0x00001373 devfs 632 0x00001373 devfs
627 0x00001cd1 devpts 633 0x00001cd1 devpts
628 0x0000f15f ecryptfs 634 0x0000f15f ecryptfs
629 0x00414a53 efs 635 0x00414a53 efs
630 0x0000137d ext 636 0x0000137d ext
631 0x0000ef53 ext2/ext3 637 0x0000ef53 ext2/ext3/ext4
632 0x0000ef51 ext2 638 0x0000ef51 ext2
639 0xf2f52010 f2fs
633 0x00004006 fat 640 0x00004006 fat
634 0x65735546 fuseblk 641 0x65735546 fuseblk
635 0x65735543 fusectl 642 0x65735543 fusectl
636 0x0bad1dea futexfs 643 0x0bad1dea futexfs
637 0x01161970 gfs2 644 0x01161970 gfs2
638 0x47504653 gpfs 645 0x47504653 gpfs
639 0x00004244 hfs 646 0x00004244 hfs
640 0xf995e849 hpfs 647 0xf995e849 hpfs
648 0x00c0ffee hostfs
641 0x958458f6 hugetlbfs 649 0x958458f6 hugetlbfs
642 0x2bad1dea inotifyfs 650 0x2bad1dea inotifyfs
643 0x00009660 isofs 651 0x00009660 isofs
644 0x000072b6 jffs2 652 0x000072b6 jffs2
645 0x3153464a jfs 653 0x3153464a jfs
660 0x00009fa1 openprom 668 0x00009fa1 openprom
661 0x7461636F ocfs2 669 0x7461636F ocfs2
662 0x00009fa0 proc 670 0x00009fa0 proc
663 0x6165676c pstorefs 671 0x6165676c pstorefs
664 0x0000002f qnx4 672 0x0000002f qnx4
673 0x68191122 qnx6
665 0x858458f6 ramfs 674 0x858458f6 ramfs
666 0x52654973 reiserfs 675 0x52654973 reiserfs
667 0x00007275 romfs 676 0x00007275 romfs
668 0x67596969 rpc_pipefs 677 0x67596969 rpc_pipefs
669 0x73636673 securityfs 678 0x73636673 securityfs
725=item aio_allocate $fh, $mode, $offset, $len, $callback->($status) 734=item aio_allocate $fh, $mode, $offset, $len, $callback->($status)
726 735
727Allocates or frees disk space according to the C<$mode> argument. See the 736Allocates or frees disk space according to the C<$mode> argument. See the
728linux C<fallocate> documentation for details. 737linux C<fallocate> documentation for details.
729 738
730C<$mode> can currently be C<0> or C<IO::AIO::FALLOC_FL_KEEP_SIZE> 739C<$mode> is usually C<0> or C<IO::AIO::FALLOC_FL_KEEP_SIZE> to allocate
731to allocate space, or C<IO::AIO::FALLOC_FL_PUNCH_HOLE | 740space, or C<IO::AIO::FALLOC_FL_PUNCH_HOLE | IO::AIO::FALLOC_FL_KEEP_SIZE>,
732IO::AIO::FALLOC_FL_KEEP_SIZE>, to deallocate a file range. 741to deallocate a file range.
742
743IO::AIO also supports C<FALLOC_FL_COLLAPSE_RANGE>, to remove a range
744(without leaving a hole) and C<FALLOC_FL_ZERO_RANGE>, to zero a range (see
745your L<fallocate(2)> manpage).
733 746
734The file system block size used by C<fallocate> is presumably the 747The file system block size used by C<fallocate> is presumably the
735C<f_bsize> returned by C<statvfs>. 748C<f_bsize> returned by C<statvfs>.
736 749
737If C<fallocate> isn't available or cannot be emulated (currently no 750If C<fallocate> isn't available or cannot be emulated (currently no
1220 }; 1233 };
1221 1234
1222 $grp 1235 $grp
1223} 1236}
1224 1237
1238=item aio_fcntl $fh, $cmd, $arg, $callback->($status)
1239
1240=item aio_ioctl $fh, $request, $buf, $callback->($status)
1241
1242These work just like the C<fcntl> and C<ioctl> built-in functions, except
1243they execute asynchronously and pass the return value to the callback.
1244
1245Both calls can be used for a lot of things, some of which make more sense
1246to run asynchronously in their own thread, while some others make less
1247sense. For example, calls that block waiting for external events, such
1248as locking, will also lock down an I/O thread while it is waiting, which
1249can deadlock the whole I/O system. At the same time, there might be no
1250alternative to using a thread to wait.
1251
1252So in general, you should only use these calls for things that do
1253(filesystem) I/O, not for things that wait for other events (network,
1254other processes), although if you are careful and know what you are doing,
1255you still can.
1256
1257The following constants are available (missing ones are, as usual C<0>):
1258
1259C<FIFREEZE>, C<FITHAW>, C<FITRIM>, C<FICLONE>, C<FICLONERANGE>, C<FIDEDUPERANGE>.
1260
1261C<FS_IOC_GETFLAGS>, C<FS_IOC_SETFLAGS>, C<FS_IOC_GETVERSION>, C<FS_IOC_SETVERSION>,
1262C<FS_IOC_FIEMAP>.
1263
1264C<FS_IOC_FSGETXATTR>, C<FS_IOC_FSSETXATTR>, C<FS_IOC_SET_ENCRYPTION_POLICY>,
1265C<FS_IOC_GET_ENCRYPTION_PWSALT>, C<FS_IOC_GET_ENCRYPTION_POLICY>, C<FS_KEY_DESCRIPTOR_SIZE>.
1266
1267C<FS_SECRM_FL>, C<FS_UNRM_FL>, C<FS_COMPR_FL>, C<FS_SYNC_FL>, C<FS_IMMUTABLE_FL>,
1268C<FS_APPEND_FL>, C<FS_NODUMP_FL>, C<FS_NOATIME_FL>, C<FS_DIRTY_FL>,
1269C<FS_COMPRBLK_FL>, C<FS_NOCOMP_FL>, C<FS_ENCRYPT_FL>, C<FS_BTREE_FL>,
1270C<FS_INDEX_FL>, C<FS_JOURNAL_DATA_FL>, C<FS_NOTAIL_FL>, C<FS_DIRSYNC_FL>, C<FS_TOPDIR_FL>,
1271C<FS_FL_USER_MODIFIABLE>.
1272
1273C<FS_XFLAG_REALTIME>, C<FS_XFLAG_PREALLOC>, C<FS_XFLAG_IMMUTABLE>, C<FS_XFLAG_APPEND>,
1274C<FS_XFLAG_SYNC>, C<FS_XFLAG_NOATIME>, C<FS_XFLAG_NODUMP>, C<FS_XFLAG_RTINHERIT>,
1275C<FS_XFLAG_PROJINHERIT>, C<FS_XFLAG_NOSYMLINKS>, C<FS_XFLAG_EXTSIZE>, C<FS_XFLAG_EXTSZINHERIT>,
1276C<FS_XFLAG_NODEFRAG>, C<FS_XFLAG_FILESTREAM>, C<FS_XFLAG_DAX>, C<FS_XFLAG_HASATTR>,
1277
1225=item aio_sync $callback->($status) 1278=item aio_sync $callback->($status)
1226 1279
1227Asynchronously call sync and call the callback when finished. 1280Asynchronously call sync and call the callback when finished.
1228 1281
1229=item aio_fsync $fh, $callback->($status) 1282=item aio_fsync $fh, $callback->($status)
2013=item IO::AIO::madvise $scalar, $offset, $len, $advice 2066=item IO::AIO::madvise $scalar, $offset, $len, $advice
2014 2067
2015Simply calls the C<posix_madvise> function (see its 2068Simply calls the C<posix_madvise> function (see its
2016manpage for details). The following advice constants are 2069manpage for details). The following advice constants are
2017available: C<IO::AIO::MADV_NORMAL>, C<IO::AIO::MADV_SEQUENTIAL>, 2070available: C<IO::AIO::MADV_NORMAL>, C<IO::AIO::MADV_SEQUENTIAL>,
2018C<IO::AIO::MADV_RANDOM>, C<IO::AIO::MADV_WILLNEED>, C<IO::AIO::MADV_DONTNEED>. 2071C<IO::AIO::MADV_RANDOM>, C<IO::AIO::MADV_WILLNEED>, C<IO::AIO::MADV_DONTNEED>,
2072C<IO::AIO::MADV_FREE>.
2019 2073
2020On systems that do not implement C<posix_madvise>, this function returns 2074On systems that do not implement C<posix_madvise>, this function returns
2021ENOSYS, otherwise the return value of C<posix_madvise>. 2075ENOSYS, otherwise the return value of C<posix_madvise>.
2022 2076
2023=item IO::AIO::mprotect $scalar, $offset, $len, $protect 2077=item IO::AIO::mprotect $scalar, $offset, $len, $protect
2053filesize. 2107filesize.
2054 2108
2055C<$prot> is a combination of C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_EXEC>, 2109C<$prot> is a combination of C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_EXEC>,
2056C<IO::AIO::PROT_READ> and/or C<IO::AIO::PROT_WRITE>, 2110C<IO::AIO::PROT_READ> and/or C<IO::AIO::PROT_WRITE>,
2057 2111
2058C<$flags> can be a combination of C<IO::AIO::MAP_SHARED> or 2112C<$flags> can be a combination of
2059C<IO::AIO::MAP_PRIVATE>, or a number of system-specific flags (when 2113C<IO::AIO::MAP_SHARED> or
2060not available, the are defined as 0): C<IO::AIO::MAP_ANONYMOUS> 2114C<IO::AIO::MAP_PRIVATE>,
2115or a number of system-specific flags (when not available, the are C<0>):
2061(which is set to C<MAP_ANON> if your system only provides this 2116C<IO::AIO::MAP_ANONYMOUS> (which is set to C<MAP_ANON> if your system only provides this constant),
2062constant), C<IO::AIO::MAP_HUGETLB>, C<IO::AIO::MAP_LOCKED>, 2117C<IO::AIO::MAP_LOCKED>,
2063C<IO::AIO::MAP_NORESERVE>, C<IO::AIO::MAP_POPULATE> or 2118C<IO::AIO::MAP_NORESERVE>,
2119C<IO::AIO::MAP_POPULATE>,
2064C<IO::AIO::MAP_NONBLOCK> 2120C<IO::AIO::MAP_NONBLOCK>,
2121C<IO::AIO::MAP_FIXED>,
2122C<IO::AIO::MAP_GROWSDOWN>,
2123C<IO::AIO::MAP_32BIT>,
2124C<IO::AIO::MAP_HUGETLB> or
2125C<IO::AIO::MAP_STACK>.
2065 2126
2066If C<$fh> is C<undef>, then a file descriptor of C<-1> is passed. 2127If C<$fh> is C<undef>, then a file descriptor of C<-1> is passed.
2067 2128
2068C<$offset> is the offset from the start of the file - it generally must be 2129C<$offset> is the offset from the start of the file - it generally must be
2069a multiple of C<IO::AIO::PAGESIZE> and defaults to C<0>. 2130a multiple of C<IO::AIO::PAGESIZE> and defaults to C<0>.
2121 2182
2122Attempts to query or change the pipe buffer size. Obviously works only 2183Attempts to query or change the pipe buffer size. Obviously works only
2123on pipes, and currently works only on GNU/Linux systems, and fails with 2184on pipes, and currently works only on GNU/Linux systems, and fails with
2124C<-1>/C<ENOSYS> everywhere else. If anybody knows how to influence pipe buffer 2185C<-1>/C<ENOSYS> everywhere else. If anybody knows how to influence pipe buffer
2125size on other systems, drop me a note. 2186size on other systems, drop me a note.
2187
2188=item ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
2189
2190This is a direct interface to the Linux L<pipe2(2)> system call. If
2191C<$flags> is missing or C<0>, then this should be the same as a call to
2192perl's built-in C<pipe> function and create a new pipe, and works on
2193systems that lack the pipe2 syscall. On win32, this case invokes C<_pipe
2194(..., 4096, O_BINARY)>.
2195
2196If C<$flags> is non-zero, it tries to invoke the pipe2 system call with
2197the given flags (Linux 2.6.27, glibc 2.9).
2198
2199On success, the read and write file handles are returned.
2200
2201On error, nothing will be returned. If the pipe2 syscall is missing and
2202C<$flags> is non-zero, fails with C<ENOSYS>.
2203
2204Please refer to L<pipe2(2)> for more info on the C<$flags>, but at the
2205time of this writing, C<IO::AIO::O_CLOEXEC>, C<IO::AIO::O_NONBLOCK> and
2206C<IO::AIO::O_DIRECT> (Linux 3.4, for packet-based pipes) were supported.
2126 2207
2127=back 2208=back
2128 2209
2129=cut 2210=cut
2130 2211

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines