ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.253 by root, Thu Jun 25 13:34:28 2015 UTC vs.
Revision 1.267 by root, Tue Aug 9 11:39:17 2016 UTC

1=head1 NAME 1=head1 NAME
2 2
3IO::AIO - Asynchronous Input/Output 3IO::AIO - Asynchronous/Advanced Input/Output
4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use IO::AIO; 7 use IO::AIO;
8 8
58not well-supported or restricted (GNU/Linux doesn't allow them on normal 58not well-supported or restricted (GNU/Linux doesn't allow them on normal
59files currently, for example), and they would only support aio_read and 59files currently, for example), and they would only support aio_read and
60aio_write, so the remaining functionality would have to be implemented 60aio_write, so the remaining functionality would have to be implemented
61using threads anyway. 61using threads anyway.
62 62
63In addition to asynchronous I/O, this module also exports some rather
64arcane interfaces, such as C<madvise> or linux's C<splice> system call,
65which is why the C<A> in C<AIO> can also mean I<advanced>.
66
63Although the module will work in the presence of other (Perl-) threads, 67Although the module will work in the presence of other (Perl-) threads,
64it is currently not reentrant in any way, so use appropriate locking 68it is currently not reentrant in any way, so use appropriate locking
65yourself, always call C<poll_cb> from within the same thread, or never 69yourself, always call C<poll_cb> from within the same thread, or never
66call C<poll_cb> (or other C<aio_> functions) recursively. 70call C<poll_cb> (or other C<aio_> functions) recursively.
67 71
94 98
95 # file contents now in $contents 99 # file contents now in $contents
96 print $contents; 100 print $contents;
97 101
98 # exit event loop and program 102 # exit event loop and program
99 EV::unloop; 103 EV::break;
100 }; 104 };
101 }; 105 };
102 106
103 # possibly queue up other requests, or open GUI windows, 107 # possibly queue up other requests, or open GUI windows,
104 # check for sockets etc. etc. 108 # check for sockets etc. etc.
105 109
106 # process events as long as there are some: 110 # process events as long as there are some:
107 EV::loop; 111 EV::run;
108 112
109=head1 REQUEST ANATOMY AND LIFETIME 113=head1 REQUEST ANATOMY AND LIFETIME
110 114
111Every C<aio_*> function creates a request. which is a C data structure not 115Every C<aio_*> function creates a request. which is a C data structure not
112directly visible to Perl. 116directly visible to Perl.
167use common::sense; 171use common::sense;
168 172
169use base 'Exporter'; 173use base 'Exporter';
170 174
171BEGIN { 175BEGIN {
172 our $VERSION = 4.32; 176 our $VERSION = 4.34;
173 177
174 our @AIO_REQ = qw(aio_sendfile aio_seek aio_read aio_write aio_open aio_close 178 our @AIO_REQ = qw(aio_sendfile aio_seek aio_read aio_write aio_open aio_close
175 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx 179 aio_stat aio_lstat aio_unlink aio_rmdir aio_readdir aio_readdirx
176 aio_scandir aio_symlink aio_readlink aio_realpath aio_sync 180 aio_scandir aio_symlink aio_readlink aio_realpath aio_fcntl aio_ioctl
177 aio_fsync aio_syncfs aio_fdatasync aio_sync_file_range aio_allocate 181 aio_sync aio_fsync aio_syncfs aio_fdatasync aio_sync_file_range
178 aio_pathsync aio_readahead aio_fiemap 182 aio_pathsync aio_readahead aio_fiemap aio_allocate
179 aio_rename aio_link aio_move aio_copy aio_group 183 aio_rename aio_link aio_move aio_copy aio_group
180 aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown 184 aio_nop aio_mknod aio_load aio_rmtree aio_mkdir aio_chown
181 aio_chmod aio_utime aio_truncate 185 aio_chmod aio_utime aio_truncate
182 aio_msync aio_mtouch aio_mlock aio_mlockall 186 aio_msync aio_mtouch aio_mlock aio_mlockall
183 aio_statvfs 187 aio_statvfs
240 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs) 244 aio_scandir $pathname, $maxreq, $callback->($dirs, $nondirs)
241 aio_load $pathname, $data, $callback->($status) 245 aio_load $pathname, $data, $callback->($status)
242 aio_copy $srcpath, $dstpath, $callback->($status) 246 aio_copy $srcpath, $dstpath, $callback->($status)
243 aio_move $srcpath, $dstpath, $callback->($status) 247 aio_move $srcpath, $dstpath, $callback->($status)
244 aio_rmtree $pathname, $callback->($status) 248 aio_rmtree $pathname, $callback->($status)
249 aio_fcntl $fh, $cmd, $arg, $callback->($status)
250 aio_ioctl $fh, $request, $buf, $callback->($status)
245 aio_sync $callback->($status) 251 aio_sync $callback->($status)
246 aio_syncfs $fh, $callback->($status) 252 aio_syncfs $fh, $callback->($status)
247 aio_fsync $fh, $callback->($status) 253 aio_fsync $fh, $callback->($status)
248 aio_fdatasync $fh, $callback->($status) 254 aio_fdatasync $fh, $callback->($status)
249 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status) 255 aio_sync_file_range $fh, $offset, $nbytes, $flags, $callback->($status)
394following POSIX and non-POSIX constants are available (missing ones on 400following POSIX and non-POSIX constants are available (missing ones on
395your system are, as usual, C<0>): 401your system are, as usual, C<0>):
396 402
397C<O_ASYNC>, C<O_DIRECT>, C<O_NOATIME>, C<O_CLOEXEC>, C<O_NOCTTY>, C<O_NOFOLLOW>, 403C<O_ASYNC>, C<O_DIRECT>, C<O_NOATIME>, C<O_CLOEXEC>, C<O_NOCTTY>, C<O_NOFOLLOW>,
398C<O_NONBLOCK>, C<O_EXEC>, C<O_SEARCH>, C<O_DIRECTORY>, C<O_DSYNC>, 404C<O_NONBLOCK>, C<O_EXEC>, C<O_SEARCH>, C<O_DIRECTORY>, C<O_DSYNC>,
399C<O_RSYNC>, C<O_SYNC> and C<O_TTY_INIT>. 405C<O_RSYNC>, C<O_SYNC>, C<O_PATH>, C<O_TMPFILE>, and C<O_TTY_INIT>.
400 406
401 407
402=item aio_close $fh, $callback->($status) 408=item aio_close $fh, $callback->($status)
403 409
404Asynchronously close a file and call the callback with the result 410Asynchronously close a file and call the callback with the result
439=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 445=item aio_read $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
440 446
441=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval) 447=item aio_write $fh,$offset,$length, $data,$dataoffset, $callback->($retval)
442 448
443Reads or writes C<$length> bytes from or to the specified C<$fh> and 449Reads or writes C<$length> bytes from or to the specified C<$fh> and
444C<$offset> into the scalar given by C<$data> and offset C<$dataoffset> 450C<$offset> into the scalar given by C<$data> and offset C<$dataoffset> and
445and calls the callback without the actual number of bytes read (or -1 on 451calls the callback with the actual number of bytes transferred (or -1 on
446error, just like the syscall). 452error, just like the syscall).
447 453
448C<aio_read> will, like C<sysread>, shrink or grow the C<$data> scalar to 454C<aio_read> will, like C<sysread>, shrink or grow the C<$data> scalar to
449offset plus the actual number of bytes read. 455offset plus the actual number of bytes read.
450 456
508As native sendfile syscalls (as practically any non-POSIX interface hacked 514As native sendfile syscalls (as practically any non-POSIX interface hacked
509together in a hurry to improve benchmark numbers) tend to be rather buggy 515together in a hurry to improve benchmark numbers) tend to be rather buggy
510on many systems, this implementation tries to work around some known bugs 516on many systems, this implementation tries to work around some known bugs
511in Linux and FreeBSD kernels (probably others, too), but that might fail, 517in Linux and FreeBSD kernels (probably others, too), but that might fail,
512so you really really should check the return value of C<aio_sendfile> - 518so you really really should check the return value of C<aio_sendfile> -
513fewre bytes than expected might have been transferred. 519fewer bytes than expected might have been transferred.
514 520
515 521
516=item aio_readahead $fh,$offset,$length, $callback->($retval) 522=item aio_readahead $fh,$offset,$length, $callback->($retval)
517 523
518C<aio_readahead> populates the page cache with data from a file so that 524C<aio_readahead> populates the page cache with data from a file so that
522whole pages, so that offset is effectively rounded down to a page boundary 528whole pages, so that offset is effectively rounded down to a page boundary
523and bytes are read up to the next page boundary greater than or equal to 529and bytes are read up to the next page boundary greater than or equal to
524(off-set+length). C<aio_readahead> does not read beyond the end of the 530(off-set+length). C<aio_readahead> does not read beyond the end of the
525file. The current file offset of the file is left unchanged. 531file. The current file offset of the file is left unchanged.
526 532
527If that syscall doesn't exist (likely if your OS isn't Linux) it will be 533If that syscall doesn't exist (likely if your kernel isn't Linux) it will
528emulated by simply reading the data, which would have a similar effect. 534be emulated by simply reading the data, which would have a similar effect.
529 535
530 536
531=item aio_stat $fh_or_path, $callback->($status) 537=item aio_stat $fh_or_path, $callback->($status)
532 538
533=item aio_lstat $fh, $callback->($status) 539=item aio_lstat $fh, $callback->($status)
626 0x00001373 devfs 632 0x00001373 devfs
627 0x00001cd1 devpts 633 0x00001cd1 devpts
628 0x0000f15f ecryptfs 634 0x0000f15f ecryptfs
629 0x00414a53 efs 635 0x00414a53 efs
630 0x0000137d ext 636 0x0000137d ext
631 0x0000ef53 ext2/ext3 637 0x0000ef53 ext2/ext3/ext4
632 0x0000ef51 ext2 638 0x0000ef51 ext2
639 0xf2f52010 f2fs
633 0x00004006 fat 640 0x00004006 fat
634 0x65735546 fuseblk 641 0x65735546 fuseblk
635 0x65735543 fusectl 642 0x65735543 fusectl
636 0x0bad1dea futexfs 643 0x0bad1dea futexfs
637 0x01161970 gfs2 644 0x01161970 gfs2
638 0x47504653 gpfs 645 0x47504653 gpfs
639 0x00004244 hfs 646 0x00004244 hfs
640 0xf995e849 hpfs 647 0xf995e849 hpfs
648 0x00c0ffee hostfs
641 0x958458f6 hugetlbfs 649 0x958458f6 hugetlbfs
642 0x2bad1dea inotifyfs 650 0x2bad1dea inotifyfs
643 0x00009660 isofs 651 0x00009660 isofs
644 0x000072b6 jffs2 652 0x000072b6 jffs2
645 0x3153464a jfs 653 0x3153464a jfs
660 0x00009fa1 openprom 668 0x00009fa1 openprom
661 0x7461636F ocfs2 669 0x7461636F ocfs2
662 0x00009fa0 proc 670 0x00009fa0 proc
663 0x6165676c pstorefs 671 0x6165676c pstorefs
664 0x0000002f qnx4 672 0x0000002f qnx4
673 0x68191122 qnx6
665 0x858458f6 ramfs 674 0x858458f6 ramfs
666 0x52654973 reiserfs 675 0x52654973 reiserfs
667 0x00007275 romfs 676 0x00007275 romfs
668 0x67596969 rpc_pipefs 677 0x67596969 rpc_pipefs
669 0x73636673 securityfs 678 0x73636673 securityfs
1224 }; 1233 };
1225 1234
1226 $grp 1235 $grp
1227} 1236}
1228 1237
1238=item aio_fcntl $fh, $cmd, $arg, $callback->($status)
1239
1240=item aio_ioctl $fh, $request, $buf, $callback->($status)
1241
1242These work just like the C<fcntl> and C<ioctl> built-in functions, except
1243they execute asynchronously and pass the return value to the callback.
1244
1245Both calls can be used for a lot of things, some of which make more sense
1246to run asynchronously in their own thread, while some others make less
1247sense. For example, calls that block waiting for external events, such
1248as locking, will also lock down an I/O thread while it is waiting, which
1249can deadlock the whole I/O system. At the same time, there might be no
1250alternative to using a thread to wait.
1251
1252So in general, you should only use these calls for things that do
1253(filesystem) I/O, not for things that wait for other events (network,
1254other processes), although if you are careful and know what you are doing,
1255you still can.
1256
1257The following constants are available (missing ones are, as usual C<0>):
1258
1259C<FIFREEZE>, C<FITHAW>, C<FITRIM>, C<FICLONE>, C<FICLONERANGE>, C<FIDEDUPERANGE>.
1260
1261C<FS_IOC_GETFLAGS>, C<FS_IOC_SETFLAGS>, C<FS_IOC_GETVERSION>, C<FS_IOC_SETVERSION>,
1262C<FS_IOC_FIEMAP>.
1263
1264C<FS_IOC_FSGETXATTR>, C<FS_IOC_FSSETXATTR>, C<FS_IOC_SET_ENCRYPTION_POLICY>,
1265C<FS_IOC_GET_ENCRYPTION_PWSALT>, C<FS_IOC_GET_ENCRYPTION_POLICY>, C<FS_KEY_DESCRIPTOR_SIZE>.
1266
1267C<FS_SECRM_FL>, C<FS_UNRM_FL>, C<FS_COMPR_FL>, C<FS_SYNC_FL>, C<FS_IMMUTABLE_FL>,
1268C<FS_APPEND_FL>, C<FS_NODUMP_FL>, C<FS_NOATIME_FL>, C<FS_DIRTY_FL>,
1269C<FS_COMPRBLK_FL>, C<FS_NOCOMP_FL>, C<FS_ENCRYPT_FL>, C<FS_BTREE_FL>,
1270C<FS_INDEX_FL>, C<FS_JOURNAL_DATA_FL>, C<FS_NOTAIL_FL>, C<FS_DIRSYNC_FL>, C<FS_TOPDIR_FL>,
1271C<FS_FL_USER_MODIFIABLE>.
1272
1273C<FS_XFLAG_REALTIME>, C<FS_XFLAG_PREALLOC>, C<FS_XFLAG_IMMUTABLE>, C<FS_XFLAG_APPEND>,
1274C<FS_XFLAG_SYNC>, C<FS_XFLAG_NOATIME>, C<FS_XFLAG_NODUMP>, C<FS_XFLAG_RTINHERIT>,
1275C<FS_XFLAG_PROJINHERIT>, C<FS_XFLAG_NOSYMLINKS>, C<FS_XFLAG_EXTSIZE>, C<FS_XFLAG_EXTSZINHERIT>,
1276C<FS_XFLAG_NODEFRAG>, C<FS_XFLAG_FILESTREAM>, C<FS_XFLAG_DAX>, C<FS_XFLAG_HASATTR>,
1277
1229=item aio_sync $callback->($status) 1278=item aio_sync $callback->($status)
1230 1279
1231Asynchronously call sync and call the callback when finished. 1280Asynchronously call sync and call the callback when finished.
1232 1281
1233=item aio_fsync $fh, $callback->($status) 1282=item aio_fsync $fh, $callback->($status)
2017=item IO::AIO::madvise $scalar, $offset, $len, $advice 2066=item IO::AIO::madvise $scalar, $offset, $len, $advice
2018 2067
2019Simply calls the C<posix_madvise> function (see its 2068Simply calls the C<posix_madvise> function (see its
2020manpage for details). The following advice constants are 2069manpage for details). The following advice constants are
2021available: C<IO::AIO::MADV_NORMAL>, C<IO::AIO::MADV_SEQUENTIAL>, 2070available: C<IO::AIO::MADV_NORMAL>, C<IO::AIO::MADV_SEQUENTIAL>,
2022C<IO::AIO::MADV_RANDOM>, C<IO::AIO::MADV_WILLNEED>, C<IO::AIO::MADV_DONTNEED>. 2071C<IO::AIO::MADV_RANDOM>, C<IO::AIO::MADV_WILLNEED>, C<IO::AIO::MADV_DONTNEED>,
2072C<IO::AIO::MADV_FREE>.
2023 2073
2024On systems that do not implement C<posix_madvise>, this function returns 2074On systems that do not implement C<posix_madvise>, this function returns
2025ENOSYS, otherwise the return value of C<posix_madvise>. 2075ENOSYS, otherwise the return value of C<posix_madvise>.
2026 2076
2027=item IO::AIO::mprotect $scalar, $offset, $len, $protect 2077=item IO::AIO::mprotect $scalar, $offset, $len, $protect
2057filesize. 2107filesize.
2058 2108
2059C<$prot> is a combination of C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_EXEC>, 2109C<$prot> is a combination of C<IO::AIO::PROT_NONE>, C<IO::AIO::PROT_EXEC>,
2060C<IO::AIO::PROT_READ> and/or C<IO::AIO::PROT_WRITE>, 2110C<IO::AIO::PROT_READ> and/or C<IO::AIO::PROT_WRITE>,
2061 2111
2062C<$flags> can be a combination of C<IO::AIO::MAP_SHARED> or 2112C<$flags> can be a combination of
2063C<IO::AIO::MAP_PRIVATE>, or a number of system-specific flags (when 2113C<IO::AIO::MAP_SHARED> or
2064not available, the are defined as 0): C<IO::AIO::MAP_ANONYMOUS> 2114C<IO::AIO::MAP_PRIVATE>,
2115or a number of system-specific flags (when not available, the are C<0>):
2065(which is set to C<MAP_ANON> if your system only provides this 2116C<IO::AIO::MAP_ANONYMOUS> (which is set to C<MAP_ANON> if your system only provides this constant),
2066constant), C<IO::AIO::MAP_HUGETLB>, C<IO::AIO::MAP_LOCKED>, 2117C<IO::AIO::MAP_LOCKED>,
2067C<IO::AIO::MAP_NORESERVE>, C<IO::AIO::MAP_POPULATE> or 2118C<IO::AIO::MAP_NORESERVE>,
2119C<IO::AIO::MAP_POPULATE>,
2068C<IO::AIO::MAP_NONBLOCK> 2120C<IO::AIO::MAP_NONBLOCK>,
2121C<IO::AIO::MAP_FIXED>,
2122C<IO::AIO::MAP_GROWSDOWN>,
2123C<IO::AIO::MAP_32BIT>,
2124C<IO::AIO::MAP_HUGETLB> or
2125C<IO::AIO::MAP_STACK>.
2069 2126
2070If C<$fh> is C<undef>, then a file descriptor of C<-1> is passed. 2127If C<$fh> is C<undef>, then a file descriptor of C<-1> is passed.
2071 2128
2072C<$offset> is the offset from the start of the file - it generally must be 2129C<$offset> is the offset from the start of the file - it generally must be
2073a multiple of C<IO::AIO::PAGESIZE> and defaults to C<0>. 2130a multiple of C<IO::AIO::PAGESIZE> and defaults to C<0>.
2130 2187
2131=item ($rfh, $wfh) = IO::AIO::pipe2 [$flags] 2188=item ($rfh, $wfh) = IO::AIO::pipe2 [$flags]
2132 2189
2133This is a direct interface to the Linux L<pipe2(2)> system call. If 2190This is a direct interface to the Linux L<pipe2(2)> system call. If
2134C<$flags> is missing or C<0>, then this should be the same as a call to 2191C<$flags> is missing or C<0>, then this should be the same as a call to
2135perl's built-in C<pipe> function and create a new pipe, and should work 2192perl's built-in C<pipe> function and create a new pipe, and works on
2136everywhere. 2193systems that lack the pipe2 syscall. On win32, this case invokes C<_pipe
2194(..., 4096, O_BINARY)>.
2137 2195
2138If C<$flags> is non-zero, it tries to invoke the pipe2 system call with 2196If C<$flags> is non-zero, it tries to invoke the pipe2 system call with
2139the given flags (Linux 2.6.27, glibc 2.9). 2197the given flags (Linux 2.6.27, glibc 2.9).
2140 2198
2141On success, the read and write file handles are returned. 2199On success, the read and write file handles are returned.

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines