ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
Revision: 1.29
Committed: Wed Aug 17 04:47:02 2005 UTC (18 years, 9 months ago) by root
Branch: MAIN
Changes since 1.28: +7 -4 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.1 =head1 NAME
2    
3     IO::AIO - Asynchronous Input/Output
4    
5     =head1 SYNOPSIS
6    
7     use IO::AIO;
8    
9 root 1.6 aio_open "/etc/passwd", O_RDONLY, 0, sub {
10     my ($fh) = @_;
11     ...
12     };
13    
14     aio_unlink "/tmp/file", sub { };
15    
16     aio_read $fh, 30000, 1024, $buffer, 0, sub {
17 root 1.8 $_[0] > 0 or die "read error: $!";
18 root 1.6 };
19    
20     # Event
21     Event->io (fd => IO::AIO::poll_fileno,
22 root 1.7 poll => 'r',
23 root 1.6 cb => \&IO::AIO::poll_cb);
24    
25     # Glib/Gtk2
26     add_watch Glib::IO IO::AIO::poll_fileno,
27 root 1.22 in => sub { IO::AIO::poll_cb; 1 };
28 root 1.6
29     # Tk
30     Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
31     readable => \&IO::AIO::poll_cb);
32    
33 root 1.11 # Danga::Socket
34     Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
35     \&IO::AIO::poll_cb);
36    
37    
38 root 1.1 =head1 DESCRIPTION
39    
40     This module implements asynchronous I/O using whatever means your
41 root 1.2 operating system supports.
42 root 1.1
43 root 1.2 Currently, a number of threads are started that execute your read/writes
44     and signal their completion. You don't need thread support in your libc or
45     perl, and the threads created by this module will not be visible to the
46     pthreads library. In the future, this module might make use of the native
47     aio functions available on many operating systems. However, they are often
48     not well-supported (Linux doesn't allow them on normal files currently,
49     for example), and they would only support aio_read and aio_write, so the
50     remaining functionality would have to be implemented using threads anyway.
51 root 1.1
52     Although the module will work with in the presence of other threads, it is
53 root 1.22 currently not reentrant, so use appropriate locking yourself, always call
54     C<poll_cb> from within the same thread, or never call C<poll_cb> (or other
55     C<aio_> functions) recursively.
56 root 1.1
57     =cut
58    
59     package IO::AIO;
60    
61 root 1.23 no warnings;
62    
63 root 1.1 use base 'Exporter';
64    
65 root 1.2 use Fcntl ();
66    
67 root 1.1 BEGIN {
68 root 1.28 $VERSION = 1.2;
69 root 1.1
70     @EXPORT = qw(aio_read aio_write aio_open aio_close aio_stat aio_lstat aio_unlink
71 root 1.27 aio_rmdir aio_symlink aio_fsync aio_fdatasync aio_readahead);
72 root 1.4 @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel max_outstanding nreqs);
73 root 1.1
74     require XSLoader;
75     XSLoader::load IO::AIO, $VERSION;
76     }
77    
78 root 1.5 =head1 FUNCTIONS
79 root 1.1
80 root 1.5 =head2 AIO FUNCTIONS
81 root 1.1
82 root 1.5 All the C<aio_*> calls are more or less thin wrappers around the syscall
83     with the same name (sans C<aio_>). The arguments are similar or identical,
84 root 1.14 and they all accept an additional (and optional) C<$callback> argument
85     which must be a code reference. This code reference will get called with
86     the syscall return code (e.g. most syscalls return C<-1> on error, unlike
87     perl, which usually delivers "false") as it's sole argument when the given
88     syscall has been executed asynchronously.
89 root 1.1
90 root 1.23 All functions expecting a filehandle keep a copy of the filehandle
91     internally until the request has finished.
92 root 1.1
93 root 1.28 The pathnames you pass to these routines I<must> be absolute and
94     encoded in byte form. The reason for the former is that at the time the
95     request is being executed, the current working directory could have
96     changed. Alternatively, you can make sure that you never change the
97     current working directory.
98    
99     To encode pathnames to byte form, either make sure you either: a)
100     always pass in filenames you got from outside (command line, readdir
101     etc.), b) are ASCII or ISO 8859-1, c) use the Encode module and encode
102     your pathnames to the locale (or other) encoding in effect in the user
103     environment, d) use Glib::filename_from_unicode on unicode filenames or e)
104     use something else.
105 root 1.1
106 root 1.5 =over 4
107 root 1.1
108     =item aio_open $pathname, $flags, $mode, $callback
109    
110 root 1.2 Asynchronously open or create a file and call the callback with a newly
111     created filehandle for the file.
112 root 1.1
113     The pathname passed to C<aio_open> must be absolute. See API NOTES, above,
114     for an explanation.
115    
116 root 1.20 The C<$flags> argument is a bitmask. See the C<Fcntl> module for a
117     list. They are the same as used by C<sysopen>.
118    
119     Likewise, C<$mode> specifies the mode of the newly created file, if it
120     didn't exist and C<O_CREAT> has been given, just like perl's C<sysopen>,
121     except that it is mandatory (i.e. use C<0> if you don't create new files,
122     and C<0666> or C<0777> if you do).
123 root 1.1
124     Example:
125    
126     aio_open "/etc/passwd", O_RDONLY, 0, sub {
127 root 1.2 if ($_[0]) {
128     print "open successful, fh is $_[0]\n";
129 root 1.1 ...
130     } else {
131     die "open failed: $!\n";
132     }
133     };
134    
135     =item aio_close $fh, $callback
136    
137 root 1.2 Asynchronously close a file and call the callback with the result
138     code. I<WARNING:> although accepted, you should not pass in a perl
139 root 1.20 filehandle here, as perl will likely close the file descriptor another
140     time when the filehandle is destroyed. Normally, you can safely call perls
141     C<close> or just let filehandles go out of scope.
142    
143     This is supposed to be a bug in the API, so that might change. It's
144     therefore best to avoid this function.
145 root 1.1
146     =item aio_read $fh,$offset,$length, $data,$dataoffset,$callback
147    
148     =item aio_write $fh,$offset,$length, $data,$dataoffset,$callback
149    
150     Reads or writes C<length> bytes from the specified C<fh> and C<offset>
151     into the scalar given by C<data> and offset C<dataoffset> and calls the
152     callback without the actual number of bytes read (or -1 on error, just
153     like the syscall).
154    
155 root 1.17 Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at
156 root 1.1 offset C<0> within the scalar:
157    
158     aio_read $fh, 7, 15, $buffer, 0, sub {
159 root 1.9 $_[0] > 0 or die "read error: $!";
160     print "read $_[0] bytes: <$buffer>\n";
161 root 1.1 };
162    
163     =item aio_readahead $fh,$offset,$length, $callback
164    
165 root 1.20 C<aio_readahead> populates the page cache with data from a file so that
166 root 1.1 subsequent reads from that file will not block on disk I/O. The C<$offset>
167     argument specifies the starting point from which data is to be read and
168     C<$length> specifies the number of bytes to be read. I/O is performed in
169     whole pages, so that offset is effectively rounded down to a page boundary
170     and bytes are read up to the next page boundary greater than or equal to
171 root 1.20 (off-set+length). C<aio_readahead> does not read beyond the end of the
172 root 1.1 file. The current file offset of the file is left unchanged.
173    
174 root 1.26 If that syscall doesn't exist (likely if your OS isn't Linux) it will be
175     emulated by simply reading the data, which would have a similar effect.
176    
177 root 1.1 =item aio_stat $fh_or_path, $callback
178    
179     =item aio_lstat $fh, $callback
180    
181     Works like perl's C<stat> or C<lstat> in void context. The callback will
182     be called after the stat and the results will be available using C<stat _>
183     or C<-s _> etc...
184    
185     The pathname passed to C<aio_stat> must be absolute. See API NOTES, above,
186     for an explanation.
187    
188     Currently, the stats are always 64-bit-stats, i.e. instead of returning an
189     error when stat'ing a large file, the results will be silently truncated
190     unless perl itself is compiled with large file support.
191    
192     Example: Print the length of F</etc/passwd>:
193    
194     aio_stat "/etc/passwd", sub {
195     $_[0] and die "stat failed: $!";
196     print "size is ", -s _, "\n";
197     };
198    
199     =item aio_unlink $pathname, $callback
200    
201     Asynchronously unlink (delete) a file and call the callback with the
202     result code.
203    
204 root 1.27 =item aio_rmdir $pathname, $callback
205    
206     Asynchronously rmdir (delete) a directory and call the callback with the
207     result code.
208    
209 root 1.1 =item aio_fsync $fh, $callback
210    
211     Asynchronously call fsync on the given filehandle and call the callback
212     with the fsync result code.
213    
214     =item aio_fdatasync $fh, $callback
215    
216     Asynchronously call fdatasync on the given filehandle and call the
217 root 1.26 callback with the fdatasync result code.
218    
219     If this call isn't available because your OS lacks it or it couldn't be
220     detected, it will be emulated by calling C<fsync> instead.
221 root 1.1
222 root 1.5 =back
223    
224     =head2 SUPPORT FUNCTIONS
225    
226     =over 4
227    
228     =item $fileno = IO::AIO::poll_fileno
229    
230 root 1.20 Return the I<request result pipe file descriptor>. This filehandle must be
231     polled for reading by some mechanism outside this module (e.g. Event or
232     select, see below or the SYNOPSIS). If the pipe becomes readable you have
233     to call C<poll_cb> to check the results.
234 root 1.5
235     See C<poll_cb> for an example.
236    
237     =item IO::AIO::poll_cb
238    
239     Process all outstanding events on the result pipe. You have to call this
240     regularly. Returns the number of events processed. Returns immediately
241     when no events are outstanding.
242    
243 root 1.20 Example: Install an Event watcher that automatically calls
244     IO::AIO::poll_cb with high priority:
245 root 1.5
246     Event->io (fd => IO::AIO::poll_fileno,
247     poll => 'r', async => 1,
248     cb => \&IO::AIO::poll_cb);
249    
250     =item IO::AIO::poll_wait
251    
252     Wait till the result filehandle becomes ready for reading (simply does a
253 root 1.20 C<select> on the filehandle. This is useful if you want to synchronously wait
254 root 1.5 for some requests to finish).
255    
256     See C<nreqs> for an example.
257    
258     =item IO::AIO::nreqs
259    
260 root 1.20 Returns the number of requests currently outstanding (i.e. for which their
261     callback has not been invoked yet).
262 root 1.5
263     Example: wait till there are no outstanding requests anymore:
264    
265     IO::AIO::poll_wait, IO::AIO::poll_cb
266     while IO::AIO::nreqs;
267    
268 root 1.12 =item IO::AIO::flush
269    
270     Wait till all outstanding AIO requests have been handled.
271    
272 root 1.13 Strictly equivalent to:
273    
274     IO::AIO::poll_wait, IO::AIO::poll_cb
275     while IO::AIO::nreqs;
276    
277     =item IO::AIO::poll
278    
279     Waits until some requests have been handled.
280    
281     Strictly equivalent to:
282    
283     IO::AIO::poll_wait, IO::AIO::poll_cb
284     if IO::AIO::nreqs;
285    
286 root 1.5 =item IO::AIO::min_parallel $nthreads
287    
288     Set the minimum number of AIO threads to C<$nthreads>. The default is
289     C<1>, which means a single asynchronous operation can be done at one time
290     (the number of outstanding operations, however, is unlimited).
291    
292     It is recommended to keep the number of threads low, as some Linux
293     kernel versions will scale negatively with the number of threads (higher
294     parallelity => MUCH higher latency). With current Linux 2.6 versions, 4-32
295     threads should be fine.
296    
297     Under normal circumstances you don't need to call this function, as this
298     module automatically starts some threads (the exact number might change,
299     and is currently 4).
300    
301     =item IO::AIO::max_parallel $nthreads
302    
303     Sets the maximum number of AIO threads to C<$nthreads>. If more than
304     the specified number of threads are currently running, kill them. This
305     function blocks until the limit is reached.
306    
307     This module automatically runs C<max_parallel 0> at program end, to ensure
308     that all threads are killed and that there are no outstanding requests.
309    
310     Under normal circumstances you don't need to call this function.
311    
312     =item $oldnreqs = IO::AIO::max_outstanding $nreqs
313    
314     Sets the maximum number of outstanding requests to C<$nreqs>. If you
315     try to queue up more than this number of requests, the caller will block until
316     some requests have been handled.
317    
318     The default is very large, so normally there is no practical limit. If you
319     queue up many requests in a loop it it often improves speed if you set
320     this to a relatively low number, such as C<100>.
321    
322     Under normal circumstances you don't need to call this function.
323    
324     =back
325    
326 root 1.1 =cut
327    
328 root 1.2 # support function to convert a fd into a perl filehandle
329     sub _fd2fh {
330     return undef if $_[0] < 0;
331    
332 root 1.23 # try to generate nice filehandles
333     my $sym = "IO::AIO::fd#$_[0]";
334     local *$sym;
335 root 1.25
336 root 1.27 open *$sym, "+<&=$_[0]" # usually works under any unix
337     or open *$sym, "<&=$_[0]" # cygwin needs this
338     or open *$sym, ">&=$_[0]" # or this
339 root 1.2 or return undef;
340    
341 root 1.23 *$sym
342 root 1.2 }
343    
344 root 1.1 min_parallel 4;
345    
346     END {
347     max_parallel 0;
348     }
349    
350     1;
351    
352 root 1.27 =head2 FORK BEHAVIOUR
353    
354 root 1.29 Before the fork IO::AIO first handles all outstanding requests - if other
355     threads add requests during this period, this time is prolonged. It then
356     enters a quiescent state where no requests can be added in other threads
357     and no results will be processed. After the fork the parent simply leaves
358     the quiescent state and continues request processing, while the child will
359     free the request and result queue and start the same number of threads as
360     were in use by the parent.
361 root 1.27
362 root 1.1 =head1 SEE ALSO
363    
364     L<Coro>, L<Linux::AIO>.
365    
366     =head1 AUTHOR
367    
368     Marc Lehmann <schmorp@schmorp.de>
369     http://home.schmorp.de/
370    
371     =cut
372