ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.1 by root, Sun Jul 10 17:07:44 2005 UTC vs.
Revision 1.10 by root, Sun Jul 10 23:45:16 2005 UTC

… …
4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use IO::AIO; 7 use IO::AIO;
8 8
9 aio_open "/etc/passwd", O_RDONLY, 0, sub {
10 my ($fh) = @_;
11 ...
12 };
13
14 aio_unlink "/tmp/file", sub { };
15
16 aio_read $fh, 30000, 1024, $buffer, 0, sub {
17 $_[0] > 0 or die "read error: $!";
18 };
19
20 # Event
21 Event->io (fd => IO::AIO::poll_fileno,
22 poll => 'r',
23 cb => \&IO::AIO::poll_cb);
24
25 # Glib/Gtk2
26 add_watch Glib::IO IO::AIO::poll_fileno,
27 \&IO::AIO::poll_cb;
28
29 # Tk
30 Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
31 readable => \&IO::AIO::poll_cb);
32
9=head1 DESCRIPTION 33=head1 DESCRIPTION
10 34
11This module implements asynchronous I/O using whatever means your 35This module implements asynchronous I/O using whatever means your
12operating system supports. Currently, it falls back to Linux::AIO if that 36operating system supports.
13module is available, or uses pthreads to emulato aio functionality.
14 37
15Currently, in this module a number of threads are started that execute 38Currently, a number of threads are started that execute your read/writes
16your read/writes and signal their completion. You don't need thread 39and signal their completion. You don't need thread support in your libc or
17support in your libc or perl, and the threads created by this module will 40perl, and the threads created by this module will not be visible to the
18not be visible to the pthreads library. 41pthreads library. In the future, this module might make use of the native
42aio functions available on many operating systems. However, they are often
43not well-supported (Linux doesn't allow them on normal files currently,
44for example), and they would only support aio_read and aio_write, so the
45remaining functionality would have to be implemented using threads anyway.
19 46
20Although the module will work with in the presence of other threads, it is 47Although the module will work with in the presence of other threads, it is
21not reentrant, so use appropriate locking yourself. 48currently not reentrant, so use appropriate locking yourself.
22 49
23=head2 API NOTES 50=cut
51
52package IO::AIO;
53
54use base 'Exporter';
55
56use Fcntl ();
57
58BEGIN {
59 $VERSION = 0.3;
60
61 @EXPORT = qw(aio_read aio_write aio_open aio_close aio_stat aio_lstat aio_unlink
62 aio_fsync aio_fdatasync aio_readahead);
63 @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel max_outstanding nreqs);
64
65 require XSLoader;
66 XSLoader::load IO::AIO, $VERSION;
67}
68
69=head1 FUNCTIONS
70
71=head2 AIO FUNCTIONS
24 72
25All the C<aio_*> calls are more or less thin wrappers around the syscall 73All the C<aio_*> calls are more or less thin wrappers around the syscall
26with the same name (sans C<aio_>). The arguments are similar or identical, 74with the same name (sans C<aio_>). The arguments are similar or identical,
27and they all accept an additional C<$callback> argument which must be 75and they all accept an additional C<$callback> argument which must be
28a code reference. This code reference will get called with the syscall 76a code reference. This code reference will get called with the syscall
… …
37directory could have changed. Alternatively, you can make sure that you 85directory could have changed. Alternatively, you can make sure that you
38never change the current working directory. 86never change the current working directory.
39 87
40=over 4 88=over 4
41 89
42=cut
43
44package IO::AIO;
45
46use base 'Exporter';
47
48BEGIN {
49 $VERSION = 0.1;
50
51 @EXPORT = qw(aio_read aio_write aio_open aio_close aio_stat aio_lstat aio_unlink
52 aio_fsync aio_fdatasync aio_readahead);
53 @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel nreqs);
54
55 require XSLoader;
56 XSLoader::load IO::AIO, $VERSION;
57}
58
59=item IO::AIO::min_parallel $nthreads
60
61Set the minimum number of AIO threads to C<$nthreads>. The default is
62C<1>, which means a single asynchronous operation can be done at one time
63(the number of outstanding operations, however, is unlimited).
64
65It is recommended to keep the number of threads low, as some linux
66kernel versions will scale negatively with the number of threads (higher
67parallelity => MUCH higher latency).
68
69Under normal circumstances you don't need to call this function, as this
70module automatically starts a single async thread.
71
72=item IO::AIO::max_parallel $nthreads
73
74Sets the maximum number of AIO threads to C<$nthreads>. If more than
75the specified number of threads are currently running, kill them. This
76function blocks until the limit is reached.
77
78This module automatically runs C<max_parallel 0> at program end, to ensure
79that all threads are killed and that there are no outstanding requests.
80
81Under normal circumstances you don't need to call this function.
82
83=item $fileno = IO::AIO::poll_fileno
84
85Return the I<request result pipe filehandle>. This filehandle must be
86polled for reading by some mechanism outside this module (e.g. Event
87or select, see below). If the pipe becomes readable you have to call
88C<poll_cb> to check the results.
89
90See C<poll_cb> for an example.
91
92=item IO::AIO::poll_cb
93
94Process all outstanding events on the result pipe. You have to call this
95regularly. Returns the number of events processed. Returns immediately
96when no events are outstanding.
97
98You can use Event to multiplex, e.g.:
99
100 Event->io (fd => IO::AIO::poll_fileno,
101 poll => 'r', async => 1,
102 cb => \&IO::AIO::poll_cb);
103
104=item IO::AIO::poll_wait
105
106Wait till the result filehandle becomes ready for reading (simply does a
107select on the filehandle. This is useful if you want to synchronously wait
108for some requests to finish).
109
110See C<nreqs> for an example.
111
112=item IO::AIO::nreqs
113
114Returns the number of requests currently outstanding.
115
116Example: wait till there are no outstanding requests anymore:
117
118 IO::AIO::poll_wait, IO::AIO::poll_cb
119 while IO::AIO::nreqs;
120
121=item aio_open $pathname, $flags, $mode, $callback 90=item aio_open $pathname, $flags, $mode, $callback
122 91
123Asynchronously open or create a file and call the callback with the 92Asynchronously open or create a file and call the callback with a newly
124filedescriptor (NOT a perl filehandle, sorry for that, but watch out, this 93created filehandle for the file.
125might change in the future).
126 94
127The pathname passed to C<aio_open> must be absolute. See API NOTES, above, 95The pathname passed to C<aio_open> must be absolute. See API NOTES, above,
128for an explanation. 96for an explanation.
129 97
130The C<$mode> argument is a bitmask. See the C<Fcntl> module for a 98The C<$mode> argument is a bitmask. See the C<Fcntl> module for a
131list. They are the same as used in C<sysopen>. 99list. They are the same as used in C<sysopen>.
132 100
133Example: 101Example:
134 102
135 aio_open "/etc/passwd", O_RDONLY, 0, sub { 103 aio_open "/etc/passwd", O_RDONLY, 0, sub {
136 if ($_[0] >= 0) { 104 if ($_[0]) {
137 open my $fh, "<&$_[0]"; # create a copy for perl
138 aio_close $_[0], sub { }; # close the aio handle
139 print "open successful, fh is $fh\n"; 105 print "open successful, fh is $_[0]\n";
140 ... 106 ...
141 } else { 107 } else {
142 die "open failed: $!\n"; 108 die "open failed: $!\n";
143 } 109 }
144 }; 110 };
145 111
146=item aio_close $fh, $callback 112=item aio_close $fh, $callback
147 113
148Asynchronously close a file and call the callback with the result code. 114Asynchronously close a file and call the callback with the result
115code. I<WARNING:> although accepted, you should not pass in a perl
116filehandle here, as perl will likely close the file descriptor itself when
117the filehandle is destroyed. Normally, you can safely call perls C<close>
118or just let filehandles go out of scope.
149 119
150=item aio_read $fh,$offset,$length, $data,$dataoffset,$callback 120=item aio_read $fh,$offset,$length, $data,$dataoffset,$callback
151 121
152=item aio_write $fh,$offset,$length, $data,$dataoffset,$callback 122=item aio_write $fh,$offset,$length, $data,$dataoffset,$callback
153 123
… …
158 128
159Example: Read 15 bytes at offset 7 into scalar C<$buffer>, strating at 129Example: Read 15 bytes at offset 7 into scalar C<$buffer>, strating at
160offset C<0> within the scalar: 130offset C<0> within the scalar:
161 131
162 aio_read $fh, 7, 15, $buffer, 0, sub { 132 aio_read $fh, 7, 15, $buffer, 0, sub {
163 $_[0] >= 0 or die "read error: $!"; 133 $_[0] > 0 or die "read error: $!";
164 print "read <$buffer>\n"; 134 print "read $_[0] bytes: <$buffer>\n";
165 }; 135 };
166 136
167=item aio_readahead $fh,$offset,$length, $callback 137=item aio_readahead $fh,$offset,$length, $callback
168 138
169Asynchronously reads the specified byte range into the page cache, using 139Asynchronously reads the specified byte range into the page cache, using
… …
214=item aio_fdatasync $fh, $callback 184=item aio_fdatasync $fh, $callback
215 185
216Asynchronously call fdatasync on the given filehandle and call the 186Asynchronously call fdatasync on the given filehandle and call the
217callback with the fdatasync result code. 187callback with the fdatasync result code.
218 188
189=back
190
191=head2 SUPPORT FUNCTIONS
192
193=over 4
194
195=item $fileno = IO::AIO::poll_fileno
196
197Return the I<request result pipe filehandle>. This filehandle must be
198polled for reading by some mechanism outside this module (e.g. Event
199or select, see below). If the pipe becomes readable you have to call
200C<poll_cb> to check the results.
201
202See C<poll_cb> for an example.
203
204=item IO::AIO::poll_cb
205
206Process all outstanding events on the result pipe. You have to call this
207regularly. Returns the number of events processed. Returns immediately
208when no events are outstanding.
209
210You can use Event to multiplex, e.g.:
211
212 Event->io (fd => IO::AIO::poll_fileno,
213 poll => 'r', async => 1,
214 cb => \&IO::AIO::poll_cb);
215
216=item IO::AIO::poll_wait
217
218Wait till the result filehandle becomes ready for reading (simply does a
219select on the filehandle. This is useful if you want to synchronously wait
220for some requests to finish).
221
222See C<nreqs> for an example.
223
224=item IO::AIO::nreqs
225
226Returns the number of requests currently outstanding.
227
228Example: wait till there are no outstanding requests anymore:
229
230 IO::AIO::poll_wait, IO::AIO::poll_cb
231 while IO::AIO::nreqs;
232
233=item IO::AIO::min_parallel $nthreads
234
235Set the minimum number of AIO threads to C<$nthreads>. The default is
236C<1>, which means a single asynchronous operation can be done at one time
237(the number of outstanding operations, however, is unlimited).
238
239It is recommended to keep the number of threads low, as some Linux
240kernel versions will scale negatively with the number of threads (higher
241parallelity => MUCH higher latency). With current Linux 2.6 versions, 4-32
242threads should be fine.
243
244Under normal circumstances you don't need to call this function, as this
245module automatically starts some threads (the exact number might change,
246and is currently 4).
247
248=item IO::AIO::max_parallel $nthreads
249
250Sets the maximum number of AIO threads to C<$nthreads>. If more than
251the specified number of threads are currently running, kill them. This
252function blocks until the limit is reached.
253
254This module automatically runs C<max_parallel 0> at program end, to ensure
255that all threads are killed and that there are no outstanding requests.
256
257Under normal circumstances you don't need to call this function.
258
259=item $oldnreqs = IO::AIO::max_outstanding $nreqs
260
261Sets the maximum number of outstanding requests to C<$nreqs>. If you
262try to queue up more than this number of requests, the caller will block until
263some requests have been handled.
264
265The default is very large, so normally there is no practical limit. If you
266queue up many requests in a loop it it often improves speed if you set
267this to a relatively low number, such as C<100>.
268
269Under normal circumstances you don't need to call this function.
270
271=back
272
219=cut 273=cut
274
275# support function to convert a fd into a perl filehandle
276sub _fd2fh {
277 return undef if $_[0] < 0;
278
279 # try to be perl5.6-compatible
280 local *AIO_FH;
281 open AIO_FH, "+<&=$_[0]"
282 or return undef;
283
284 *AIO_FH
285}
220 286
221min_parallel 4; 287min_parallel 4;
222 288
223END { 289END {
224 max_parallel 0; 290 max_parallel 0;
225} 291}
226 292
2271; 2931;
228 294
229=back
230
231=head1 BUGS
232
233 - aio_open gives a fd, but all other functions expect a perl filehandle.
234
235=head1 SEE ALSO 295=head1 SEE ALSO
236 296
237L<Coro>, L<Linux::AIO>. 297L<Coro>, L<Linux::AIO>.
238 298
239=head1 AUTHOR 299=head1 AUTHOR

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines