ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
Revision: 1.32
Committed: Wed Aug 17 05:26:20 2005 UTC (18 years, 9 months ago) by root
Branch: MAIN
CVS Tags: rel-1_3
Changes since 1.31: +2 -1 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.1 =head1 NAME
2    
3     IO::AIO - Asynchronous Input/Output
4    
5     =head1 SYNOPSIS
6    
7     use IO::AIO;
8    
9 root 1.6 aio_open "/etc/passwd", O_RDONLY, 0, sub {
10     my ($fh) = @_;
11     ...
12     };
13    
14     aio_unlink "/tmp/file", sub { };
15    
16     aio_read $fh, 30000, 1024, $buffer, 0, sub {
17 root 1.8 $_[0] > 0 or die "read error: $!";
18 root 1.6 };
19    
20     # Event
21     Event->io (fd => IO::AIO::poll_fileno,
22 root 1.7 poll => 'r',
23 root 1.6 cb => \&IO::AIO::poll_cb);
24    
25     # Glib/Gtk2
26     add_watch Glib::IO IO::AIO::poll_fileno,
27 root 1.22 in => sub { IO::AIO::poll_cb; 1 };
28 root 1.6
29     # Tk
30     Tk::Event::IO->fileevent (IO::AIO::poll_fileno, "",
31     readable => \&IO::AIO::poll_cb);
32    
33 root 1.11 # Danga::Socket
34     Danga::Socket->AddOtherFds (IO::AIO::poll_fileno =>
35     \&IO::AIO::poll_cb);
36    
37    
38 root 1.1 =head1 DESCRIPTION
39    
40     This module implements asynchronous I/O using whatever means your
41 root 1.2 operating system supports.
42 root 1.1
43 root 1.2 Currently, a number of threads are started that execute your read/writes
44     and signal their completion. You don't need thread support in your libc or
45     perl, and the threads created by this module will not be visible to the
46     pthreads library. In the future, this module might make use of the native
47     aio functions available on many operating systems. However, they are often
48     not well-supported (Linux doesn't allow them on normal files currently,
49     for example), and they would only support aio_read and aio_write, so the
50     remaining functionality would have to be implemented using threads anyway.
51 root 1.1
52     Although the module will work with in the presence of other threads, it is
53 root 1.22 currently not reentrant, so use appropriate locking yourself, always call
54     C<poll_cb> from within the same thread, or never call C<poll_cb> (or other
55     C<aio_> functions) recursively.
56 root 1.1
57     =cut
58    
59     package IO::AIO;
60    
61 root 1.23 no warnings;
62    
63 root 1.1 use base 'Exporter';
64    
65 root 1.2 use Fcntl ();
66    
67 root 1.1 BEGIN {
68 root 1.32 $VERSION = 1.3;
69    
70 root 1.1
71     @EXPORT = qw(aio_read aio_write aio_open aio_close aio_stat aio_lstat aio_unlink
72 root 1.27 aio_rmdir aio_symlink aio_fsync aio_fdatasync aio_readahead);
73 root 1.4 @EXPORT_OK = qw(poll_fileno poll_cb min_parallel max_parallel max_outstanding nreqs);
74 root 1.1
75     require XSLoader;
76     XSLoader::load IO::AIO, $VERSION;
77     }
78    
79 root 1.5 =head1 FUNCTIONS
80 root 1.1
81 root 1.5 =head2 AIO FUNCTIONS
82 root 1.1
83 root 1.5 All the C<aio_*> calls are more or less thin wrappers around the syscall
84     with the same name (sans C<aio_>). The arguments are similar or identical,
85 root 1.14 and they all accept an additional (and optional) C<$callback> argument
86     which must be a code reference. This code reference will get called with
87     the syscall return code (e.g. most syscalls return C<-1> on error, unlike
88     perl, which usually delivers "false") as it's sole argument when the given
89     syscall has been executed asynchronously.
90 root 1.1
91 root 1.23 All functions expecting a filehandle keep a copy of the filehandle
92     internally until the request has finished.
93 root 1.1
94 root 1.28 The pathnames you pass to these routines I<must> be absolute and
95     encoded in byte form. The reason for the former is that at the time the
96     request is being executed, the current working directory could have
97     changed. Alternatively, you can make sure that you never change the
98     current working directory.
99    
100     To encode pathnames to byte form, either make sure you either: a)
101     always pass in filenames you got from outside (command line, readdir
102     etc.), b) are ASCII or ISO 8859-1, c) use the Encode module and encode
103     your pathnames to the locale (or other) encoding in effect in the user
104     environment, d) use Glib::filename_from_unicode on unicode filenames or e)
105     use something else.
106 root 1.1
107 root 1.5 =over 4
108 root 1.1
109     =item aio_open $pathname, $flags, $mode, $callback
110    
111 root 1.2 Asynchronously open or create a file and call the callback with a newly
112     created filehandle for the file.
113 root 1.1
114     The pathname passed to C<aio_open> must be absolute. See API NOTES, above,
115     for an explanation.
116    
117 root 1.20 The C<$flags> argument is a bitmask. See the C<Fcntl> module for a
118     list. They are the same as used by C<sysopen>.
119    
120     Likewise, C<$mode> specifies the mode of the newly created file, if it
121     didn't exist and C<O_CREAT> has been given, just like perl's C<sysopen>,
122     except that it is mandatory (i.e. use C<0> if you don't create new files,
123     and C<0666> or C<0777> if you do).
124 root 1.1
125     Example:
126    
127     aio_open "/etc/passwd", O_RDONLY, 0, sub {
128 root 1.2 if ($_[0]) {
129     print "open successful, fh is $_[0]\n";
130 root 1.1 ...
131     } else {
132     die "open failed: $!\n";
133     }
134     };
135    
136     =item aio_close $fh, $callback
137    
138 root 1.2 Asynchronously close a file and call the callback with the result
139     code. I<WARNING:> although accepted, you should not pass in a perl
140 root 1.20 filehandle here, as perl will likely close the file descriptor another
141     time when the filehandle is destroyed. Normally, you can safely call perls
142     C<close> or just let filehandles go out of scope.
143    
144     This is supposed to be a bug in the API, so that might change. It's
145     therefore best to avoid this function.
146 root 1.1
147     =item aio_read $fh,$offset,$length, $data,$dataoffset,$callback
148    
149     =item aio_write $fh,$offset,$length, $data,$dataoffset,$callback
150    
151     Reads or writes C<length> bytes from the specified C<fh> and C<offset>
152     into the scalar given by C<data> and offset C<dataoffset> and calls the
153     callback without the actual number of bytes read (or -1 on error, just
154     like the syscall).
155    
156 root 1.31 The C<$data> scalar I<MUST NOT> be modified in any way while the request
157     is outstanding. Modifying it can result in segfaults or WW3 (if the
158     necessary/optional hardware is installed).
159    
160 root 1.17 Example: Read 15 bytes at offset 7 into scalar C<$buffer>, starting at
161 root 1.1 offset C<0> within the scalar:
162    
163     aio_read $fh, 7, 15, $buffer, 0, sub {
164 root 1.9 $_[0] > 0 or die "read error: $!";
165     print "read $_[0] bytes: <$buffer>\n";
166 root 1.1 };
167    
168     =item aio_readahead $fh,$offset,$length, $callback
169    
170 root 1.20 C<aio_readahead> populates the page cache with data from a file so that
171 root 1.1 subsequent reads from that file will not block on disk I/O. The C<$offset>
172     argument specifies the starting point from which data is to be read and
173     C<$length> specifies the number of bytes to be read. I/O is performed in
174     whole pages, so that offset is effectively rounded down to a page boundary
175     and bytes are read up to the next page boundary greater than or equal to
176 root 1.20 (off-set+length). C<aio_readahead> does not read beyond the end of the
177 root 1.1 file. The current file offset of the file is left unchanged.
178    
179 root 1.26 If that syscall doesn't exist (likely if your OS isn't Linux) it will be
180     emulated by simply reading the data, which would have a similar effect.
181    
182 root 1.1 =item aio_stat $fh_or_path, $callback
183    
184     =item aio_lstat $fh, $callback
185    
186     Works like perl's C<stat> or C<lstat> in void context. The callback will
187     be called after the stat and the results will be available using C<stat _>
188     or C<-s _> etc...
189    
190     The pathname passed to C<aio_stat> must be absolute. See API NOTES, above,
191     for an explanation.
192    
193     Currently, the stats are always 64-bit-stats, i.e. instead of returning an
194     error when stat'ing a large file, the results will be silently truncated
195     unless perl itself is compiled with large file support.
196    
197     Example: Print the length of F</etc/passwd>:
198    
199     aio_stat "/etc/passwd", sub {
200     $_[0] and die "stat failed: $!";
201     print "size is ", -s _, "\n";
202     };
203    
204     =item aio_unlink $pathname, $callback
205    
206     Asynchronously unlink (delete) a file and call the callback with the
207     result code.
208    
209 root 1.27 =item aio_rmdir $pathname, $callback
210    
211     Asynchronously rmdir (delete) a directory and call the callback with the
212     result code.
213    
214 root 1.1 =item aio_fsync $fh, $callback
215    
216     Asynchronously call fsync on the given filehandle and call the callback
217     with the fsync result code.
218    
219     =item aio_fdatasync $fh, $callback
220    
221     Asynchronously call fdatasync on the given filehandle and call the
222 root 1.26 callback with the fdatasync result code.
223    
224     If this call isn't available because your OS lacks it or it couldn't be
225     detected, it will be emulated by calling C<fsync> instead.
226 root 1.1
227 root 1.5 =back
228    
229     =head2 SUPPORT FUNCTIONS
230    
231     =over 4
232    
233     =item $fileno = IO::AIO::poll_fileno
234    
235 root 1.20 Return the I<request result pipe file descriptor>. This filehandle must be
236     polled for reading by some mechanism outside this module (e.g. Event or
237     select, see below or the SYNOPSIS). If the pipe becomes readable you have
238     to call C<poll_cb> to check the results.
239 root 1.5
240     See C<poll_cb> for an example.
241    
242     =item IO::AIO::poll_cb
243    
244     Process all outstanding events on the result pipe. You have to call this
245     regularly. Returns the number of events processed. Returns immediately
246     when no events are outstanding.
247    
248 root 1.20 Example: Install an Event watcher that automatically calls
249     IO::AIO::poll_cb with high priority:
250 root 1.5
251     Event->io (fd => IO::AIO::poll_fileno,
252     poll => 'r', async => 1,
253     cb => \&IO::AIO::poll_cb);
254    
255     =item IO::AIO::poll_wait
256    
257     Wait till the result filehandle becomes ready for reading (simply does a
258 root 1.20 C<select> on the filehandle. This is useful if you want to synchronously wait
259 root 1.5 for some requests to finish).
260    
261     See C<nreqs> for an example.
262    
263     =item IO::AIO::nreqs
264    
265 root 1.20 Returns the number of requests currently outstanding (i.e. for which their
266     callback has not been invoked yet).
267 root 1.5
268     Example: wait till there are no outstanding requests anymore:
269    
270     IO::AIO::poll_wait, IO::AIO::poll_cb
271     while IO::AIO::nreqs;
272    
273 root 1.12 =item IO::AIO::flush
274    
275     Wait till all outstanding AIO requests have been handled.
276    
277 root 1.13 Strictly equivalent to:
278    
279     IO::AIO::poll_wait, IO::AIO::poll_cb
280     while IO::AIO::nreqs;
281    
282     =item IO::AIO::poll
283    
284     Waits until some requests have been handled.
285    
286     Strictly equivalent to:
287    
288     IO::AIO::poll_wait, IO::AIO::poll_cb
289     if IO::AIO::nreqs;
290    
291 root 1.5 =item IO::AIO::min_parallel $nthreads
292    
293     Set the minimum number of AIO threads to C<$nthreads>. The default is
294     C<1>, which means a single asynchronous operation can be done at one time
295     (the number of outstanding operations, however, is unlimited).
296    
297     It is recommended to keep the number of threads low, as some Linux
298     kernel versions will scale negatively with the number of threads (higher
299     parallelity => MUCH higher latency). With current Linux 2.6 versions, 4-32
300     threads should be fine.
301    
302     Under normal circumstances you don't need to call this function, as this
303     module automatically starts some threads (the exact number might change,
304     and is currently 4).
305    
306     =item IO::AIO::max_parallel $nthreads
307    
308     Sets the maximum number of AIO threads to C<$nthreads>. If more than
309     the specified number of threads are currently running, kill them. This
310     function blocks until the limit is reached.
311    
312     This module automatically runs C<max_parallel 0> at program end, to ensure
313     that all threads are killed and that there are no outstanding requests.
314    
315     Under normal circumstances you don't need to call this function.
316    
317     =item $oldnreqs = IO::AIO::max_outstanding $nreqs
318    
319     Sets the maximum number of outstanding requests to C<$nreqs>. If you
320     try to queue up more than this number of requests, the caller will block until
321     some requests have been handled.
322    
323     The default is very large, so normally there is no practical limit. If you
324     queue up many requests in a loop it it often improves speed if you set
325     this to a relatively low number, such as C<100>.
326    
327     Under normal circumstances you don't need to call this function.
328    
329     =back
330    
331 root 1.1 =cut
332    
333 root 1.2 # support function to convert a fd into a perl filehandle
334     sub _fd2fh {
335     return undef if $_[0] < 0;
336    
337 root 1.23 # try to generate nice filehandles
338     my $sym = "IO::AIO::fd#$_[0]";
339     local *$sym;
340 root 1.25
341 root 1.27 open *$sym, "+<&=$_[0]" # usually works under any unix
342     or open *$sym, "<&=$_[0]" # cygwin needs this
343     or open *$sym, ">&=$_[0]" # or this
344 root 1.2 or return undef;
345    
346 root 1.23 *$sym
347 root 1.2 }
348    
349 root 1.1 min_parallel 4;
350    
351     END {
352     max_parallel 0;
353     }
354    
355     1;
356    
357 root 1.27 =head2 FORK BEHAVIOUR
358    
359 root 1.31 Before the fork IO::AIO enters a quiescent state where no requests can be
360     added in other threads and no results will be processed. After the fork
361     the parent simply leaves the quiescent state and continues request/result
362     processing, while the child clears the request/result queue and starts the
363     same number of threads as were in use by the parent.
364 root 1.27
365 root 1.1 =head1 SEE ALSO
366    
367     L<Coro>, L<Linux::AIO>.
368    
369     =head1 AUTHOR
370    
371     Marc Lehmann <schmorp@schmorp.de>
372     http://home.schmorp.de/
373    
374     =cut
375