ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.1 by root, Sun Jul 10 17:07:44 2005 UTC vs.
Revision 1.16 by root, Mon Jul 11 03:10:08 2005 UTC

… …
4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use IO::AIO; 7 use IO::AIO;
8 8
9 aio_open "/etc/passwd", O_RDONLY, 0, sub {
10 my ($fh) = @_;
11 ...
12 };
13
14 aio_unlink "/tmp/file", sub { };
15
16 aio_read $fh, 30000, 1024, $buffer, 0, sub {
17 $_[0] > 0 or die "read error: $!";
18 };
19
20 # Event
21 Event->io (fd => IO::AIO::poll_fileno,
22 poll => 'r',
23 cb => \&IO::AIO::poll_cb);
24
25 # Glib/Gtk2
26 add_watch Glib::IO IO::AIO::poll_fileno,
27 in => sub { IO::AIO::poll_cb, 1 };
28
29 # Tk
30 Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
31 readable => \&IO::AIO::poll_cb);
32
33 # Danga::Socket
34 Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
35 \&IO::AIO::poll_cb);
36
37
9=head1 DESCRIPTION 38=head1 DESCRIPTION
10 39
11This module implements asynchronous I/O using whatever means your 40This module implements asynchronous I/O using whatever means your
12operating system supports. Currently, it falls back to Linux::AIO if that 41operating system supports.
13module is available, or uses pthreads to emulato aio functionality.
14 42
15Currently, in this module a number of threads are started that execute 43Currently, a number of threads are started that execute your read/writes
16your read/writes and signal their completion. You don't need thread 44and signal their completion. You don't need thread support in your libc or
17support in your libc or perl, and the threads created by this module will 45perl, and the threads created by this module will not be visible to the
18not be visible to the pthreads library. 46pthreads library. In the future, this module might make use of the native
47aio functions available on many operating systems. However, they are often
48not well-supported (Linux doesn't allow them on normal files currently,
49for example), and they would only support aio_read and aio_write, so the
50remaining functionality would have to be implemented using threads anyway.
19 51
20Although the module will work with in the presence of other threads, it is 52Although the module will work with in the presence of other threads, it is
21not reentrant, so use appropriate locking yourself. 53currently not reentrant, so use appropriate locking yourself.
22 54
23=head2 API NOTES 55=cut
56
57package IO::AIO;
58
59use base 'Exporter';
60
61use Fcntl ();
62
63BEGIN {
64 $VERSION = 0.3;
65
66 @EXPORT = qw(aio_read aio_write aio_open aio_close aio_stat aio_lstat aio_unlink
67 aio_fsync aio_fdatasync aio_readahead);
68 @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel max_outstanding nreqs);
69
70 require XSLoader;
71 XSLoader::load IO::AIO, $VERSION;
72}
73
74=head1 FUNCTIONS
75
76=head2 AIO FUNCTIONS
24 77
25All the C<aio_*> calls are more or less thin wrappers around the syscall 78All the C<aio_*> calls are more or less thin wrappers around the syscall
26with the same name (sans C<aio_>). The arguments are similar or identical, 79with the same name (sans C<aio_>). The arguments are similar or identical,
27and they all accept an additional C<$callback> argument which must be 80and they all accept an additional (and optional) C<$callback> argument
28a code reference. This code reference will get called with the syscall 81which must be a code reference. This code reference will get called with
29return code (e.g. most syscalls return C<-1> on error, unlike perl, which 82the syscall return code (e.g. most syscalls return C<-1> on error, unlike
30usually delivers "false") as it's sole argument when the given syscall has 83perl, which usually delivers "false") as it's sole argument when the given
31been executed asynchronously. 84syscall has been executed asynchronously.
32 85
33All functions that expect a filehandle will also accept a file descriptor. 86All functions that expect a filehandle will also accept a file descriptor.
34 87
35The filenames you pass to these routines I<must> be absolute. The reason 88The filenames you pass to these routines I<must> be absolute. The reason
36is that at the time the request is being executed, the current working 89is that at the time the request is being executed, the current working
37directory could have changed. Alternatively, you can make sure that you 90directory could have changed. Alternatively, you can make sure that you
38never change the current working directory. 91never change the current working directory.
39 92
40=over 4 93=over 4
41 94
42=cut
43
44package IO::AIO;
45
46use base 'Exporter';
47
48BEGIN {
49 $VERSION = 0.1;
50
51 @EXPORT = qw(aio_read aio_write aio_open aio_close aio_stat aio_lstat aio_unlink
52 aio_fsync aio_fdatasync aio_readahead);
53 @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel nreqs);
54
55 require XSLoader;
56 XSLoader::load IO::AIO, $VERSION;
57}
58
59=item IO::AIO::min_parallel $nthreads
60
61Set the minimum number of AIO threads to C<$nthreads>. The default is
62C<1>, which means a single asynchronous operation can be done at one time
63(the number of outstanding operations, however, is unlimited).
64
65It is recommended to keep the number of threads low, as some linux
66kernel versions will scale negatively with the number of threads (higher
67parallelity => MUCH higher latency).
68
69Under normal circumstances you don't need to call this function, as this
70module automatically starts a single async thread.
71
72=item IO::AIO::max_parallel $nthreads
73
74Sets the maximum number of AIO threads to C<$nthreads>. If more than
75the specified number of threads are currently running, kill them. This
76function blocks until the limit is reached.
77
78This module automatically runs C<max_parallel 0> at program end, to ensure
79that all threads are killed and that there are no outstanding requests.
80
81Under normal circumstances you don't need to call this function.
82
83=item $fileno = IO::AIO::poll_fileno
84
85Return the I<request result pipe filehandle>. This filehandle must be
86polled for reading by some mechanism outside this module (e.g. Event
87or select, see below). If the pipe becomes readable you have to call
88C<poll_cb> to check the results.
89
90See C<poll_cb> for an example.
91
92=item IO::AIO::poll_cb
93
94Process all outstanding events on the result pipe. You have to call this
95regularly. Returns the number of events processed. Returns immediately
96when no events are outstanding.
97
98You can use Event to multiplex, e.g.:
99
100 Event->io (fd => IO::AIO::poll_fileno,
101 poll => 'r', async => 1,
102 cb => \&IO::AIO::poll_cb);
103
104=item IO::AIO::poll_wait
105
106Wait till the result filehandle becomes ready for reading (simply does a
107select on the filehandle. This is useful if you want to synchronously wait
108for some requests to finish).
109
110See C<nreqs> for an example.
111
112=item IO::AIO::nreqs
113
114Returns the number of requests currently outstanding.
115
116Example: wait till there are no outstanding requests anymore:
117
118 IO::AIO::poll_wait, IO::AIO::poll_cb
119 while IO::AIO::nreqs;
120
121=item aio_open $pathname, $flags, $mode, $callback 95=item aio_open $pathname, $flags, $mode, $callback
122 96
123Asynchronously open or create a file and call the callback with the 97Asynchronously open or create a file and call the callback with a newly
124filedescriptor (NOT a perl filehandle, sorry for that, but watch out, this 98created filehandle for the file.
125might change in the future).
126 99
127The pathname passed to C<aio_open> must be absolute. See API NOTES, above, 100The pathname passed to C<aio_open> must be absolute. See API NOTES, above,
128for an explanation. 101for an explanation.
129 102
130The C<$mode> argument is a bitmask. See the C<Fcntl> module for a 103The C<$mode> argument is a bitmask. See the C<Fcntl> module for a
131list. They are the same as used in C<sysopen>. 104list. They are the same as used in C<sysopen>.
132 105
133Example: 106Example:
134 107
135 aio_open "/etc/passwd", O_RDONLY, 0, sub { 108 aio_open "/etc/passwd", O_RDONLY, 0, sub {
136 if ($_[0] >= 0) { 109 if ($_[0]) {
137 open my $fh, "<&$_[0]"; # create a copy for perl
138 aio_close $_[0], sub { }; # close the aio handle
139 print "open successful, fh is $fh\n"; 110 print "open successful, fh is $_[0]\n";
140 ... 111 ...
141 } else { 112 } else {
142 die "open failed: $!\n"; 113 die "open failed: $!\n";
143 } 114 }
144 }; 115 };
145 116
146=item aio_close $fh, $callback 117=item aio_close $fh, $callback
147 118
148Asynchronously close a file and call the callback with the result code. 119Asynchronously close a file and call the callback with the result
120code. I<WARNING:> although accepted, you should not pass in a perl
121filehandle here, as perl will likely close the file descriptor itself when
122the filehandle is destroyed. Normally, you can safely call perls C<close>
123or just let filehandles go out of scope.
149 124
150=item aio_read $fh,$offset,$length, $data,$dataoffset,$callback 125=item aio_read $fh,$offset,$length, $data,$dataoffset,$callback
151 126
152=item aio_write $fh,$offset,$length, $data,$dataoffset,$callback 127=item aio_write $fh,$offset,$length, $data,$dataoffset,$callback
153 128
… …
158 133
159Example: Read 15 bytes at offset 7 into scalar C<$buffer>, strating at 134Example: Read 15 bytes at offset 7 into scalar C<$buffer>, strating at
160offset C<0> within the scalar: 135offset C<0> within the scalar:
161 136
162 aio_read $fh, 7, 15, $buffer, 0, sub { 137 aio_read $fh, 7, 15, $buffer, 0, sub {
163 $_[0] >= 0 or die "read error: $!"; 138 $_[0] > 0 or die "read error: $!";
164 print "read <$buffer>\n"; 139 print "read $_[0] bytes: <$buffer>\n";
165 }; 140 };
166 141
167=item aio_readahead $fh,$offset,$length, $callback 142=item aio_readahead $fh,$offset,$length, $callback
168 143
169Asynchronously reads the specified byte range into the page cache, using 144Asynchronously reads the specified byte range into the page cache, using
… …
214=item aio_fdatasync $fh, $callback 189=item aio_fdatasync $fh, $callback
215 190
216Asynchronously call fdatasync on the given filehandle and call the 191Asynchronously call fdatasync on the given filehandle and call the
217callback with the fdatasync result code. 192callback with the fdatasync result code.
218 193
194=back
195
196=head2 SUPPORT FUNCTIONS
197
198=over 4
199
200=item $fileno = IO::AIO::poll_fileno
201
202Return the I<request result pipe filehandle>. This filehandle must be
203polled for reading by some mechanism outside this module (e.g. Event
204or select, see below). If the pipe becomes readable you have to call
205C<poll_cb> to check the results.
206
207See C<poll_cb> for an example.
208
209=item IO::AIO::poll_cb
210
211Process all outstanding events on the result pipe. You have to call this
212regularly. Returns the number of events processed. Returns immediately
213when no events are outstanding.
214
215You can use Event to multiplex, e.g.:
216
217 Event->io (fd => IO::AIO::poll_fileno,
218 poll => 'r', async => 1,
219 cb => \&IO::AIO::poll_cb);
220
221=item IO::AIO::poll_wait
222
223Wait till the result filehandle becomes ready for reading (simply does a
224select on the filehandle. This is useful if you want to synchronously wait
225for some requests to finish).
226
227See C<nreqs> for an example.
228
229=item IO::AIO::nreqs
230
231Returns the number of requests currently outstanding.
232
233Example: wait till there are no outstanding requests anymore:
234
235 IO::AIO::poll_wait, IO::AIO::poll_cb
236 while IO::AIO::nreqs;
237
238=item IO::AIO::flush
239
240Wait till all outstanding AIO requests have been handled.
241
242Strictly equivalent to:
243
244 IO::AIO::poll_wait, IO::AIO::poll_cb
245 while IO::AIO::nreqs;
246
247=item IO::AIO::poll
248
249Waits until some requests have been handled.
250
251Strictly equivalent to:
252
253 IO::AIO::poll_wait, IO::AIO::poll_cb
254 if IO::AIO::nreqs;
255
256=item IO::AIO::min_parallel $nthreads
257
258Set the minimum number of AIO threads to C<$nthreads>. The default is
259C<1>, which means a single asynchronous operation can be done at one time
260(the number of outstanding operations, however, is unlimited).
261
262It is recommended to keep the number of threads low, as some Linux
263kernel versions will scale negatively with the number of threads (higher
264parallelity => MUCH higher latency). With current Linux 2.6 versions, 4-32
265threads should be fine.
266
267Under normal circumstances you don't need to call this function, as this
268module automatically starts some threads (the exact number might change,
269and is currently 4).
270
271=item IO::AIO::max_parallel $nthreads
272
273Sets the maximum number of AIO threads to C<$nthreads>. If more than
274the specified number of threads are currently running, kill them. This
275function blocks until the limit is reached.
276
277This module automatically runs C<max_parallel 0> at program end, to ensure
278that all threads are killed and that there are no outstanding requests.
279
280Under normal circumstances you don't need to call this function.
281
282=item $oldnreqs = IO::AIO::max_outstanding $nreqs
283
284Sets the maximum number of outstanding requests to C<$nreqs>. If you
285try to queue up more than this number of requests, the caller will block until
286some requests have been handled.
287
288The default is very large, so normally there is no practical limit. If you
289queue up many requests in a loop it it often improves speed if you set
290this to a relatively low number, such as C<100>.
291
292Under normal circumstances you don't need to call this function.
293
294=back
295
219=cut 296=cut
297
298# support function to convert a fd into a perl filehandle
299sub _fd2fh {
300 return undef if $_[0] < 0;
301
302 # try to be perl5.6-compatible
303 local *AIO_FH;
304 open AIO_FH, "+<&=$_[0]"
305 or return undef;
306
307 *AIO_FH
308}
220 309
221min_parallel 4; 310min_parallel 4;
222 311
223END { 312END {
224 max_parallel 0; 313 max_parallel 0;
225} 314}
226 315
2271; 3161;
228 317
229=back
230
231=head1 BUGS
232
233 - aio_open gives a fd, but all other functions expect a perl filehandle.
234
235=head1 SEE ALSO 318=head1 SEE ALSO
236 319
237L<Coro>, L<Linux::AIO>. 320L<Coro>, L<Linux::AIO>.
238 321
239=head1 AUTHOR 322=head1 AUTHOR

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines