ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/IO-AIO/AIO.pm
(Generate patch)

Comparing IO-AIO/AIO.pm (file contents):
Revision 1.70 by root, Tue Oct 24 03:40:38 2006 UTC vs.
Revision 1.73 by root, Tue Oct 24 16:35:04 2006 UTC

… …
51=head1 DESCRIPTION 51=head1 DESCRIPTION
52 52
53This module implements asynchronous I/O using whatever means your 53This module implements asynchronous I/O using whatever means your
54operating system supports. 54operating system supports.
55 55
56Currently, a number of threads are started that execute your read/writes 56In this version, a number of threads are started that execute your
57and signal their completion. You don't need thread support in perl, and 57requests and signal their completion. You don't need thread support
58the threads created by this module will not be visible to perl. In the 58in perl, and the threads created by this module will not be visible
59future, this module might make use of the native aio functions available 59to perl. In the future, this module might make use of the native aio
60on many operating systems. However, they are often not well-supported 60functions available on many operating systems. However, they are often
61(Linux doesn't allow them on normal files currently, for example), 61not well-supported or restricted (Linux doesn't allow them on normal
62and they would only support aio_read and aio_write, so the remaining 62files currently, for example), and they would only support aio_read and
63functionality would have to be implemented using threads anyway. 63aio_write, so the remaining functionality would have to be implemented
64using threads anyway.
64 65
65Although the module will work with in the presence of other threads, 66Although the module will work with in the presence of other (Perl-)
66it is currently not reentrant in any way, so use appropriate locking 67threads, it is currently not reentrant in any way, so use appropriate
67yourself, always call C<poll_cb> from within the same thread, or never 68locking yourself, always call C<poll_cb> from within the same thread, or
68call C<poll_cb> (or other C<aio_> functions) recursively. 69never call C<poll_cb> (or other C<aio_> functions) recursively.
70
71=head1 REQUEST ANATOMY AND LIFETIME
72
73Every C<aio_*> function creates a request. which is a C data structure not
74directly visible to Perl.
75
76If called in non-void context, every request function returns a Perl
77object representing the request. In void context, nothing is returned,
78which saves a bit of memory.
79
80The perl object is a fairly standard ref-to-hash object. The hash contents
81are not used by IO::AIO so you are free to store anything you like in it.
82
83During their existance, aio requests travel through the following states,
84in order:
85
86=over 4
87
88=item ready
89
90Immediately after a request is created it is put into the ready state,
91waiting for a thread to execute it.
92
93=item execute
94
95A thread has accepted the request for processing and is currently
96executing it (e.g. blocking in read).
97
98=item pending
99
100The request has been executed and is waiting for result processing.
101
102While request submission and execution is fully asynchronous, result
103processing is not and relies on the perl interpreter calling C<poll_cb>
104(or another function with the same effect).
105
106=item result
107
108The request results are processed synchronously by C<poll_cb>.
109
110The C<poll_cb> function will process all outstanding aio requests by
111calling their callbacks, freeing memory associated with them and managing
112any groups they are contained in.
113
114=item done
115
116Request has reached the end of its lifetime and holds no resources anymore
117(except possibly for the Perl object, but its connection to the actual
118aio request is severed and calling its methods will either do nothing or
119result in a runtime error).
69 120
70=cut 121=cut
71 122
72package IO::AIO; 123package IO::AIO;
73 124
… …
212 $_[0] > 0 or die "read error: $!"; 263 $_[0] > 0 or die "read error: $!";
213 print "read $_[0] bytes: <$buffer>\n"; 264 print "read $_[0] bytes: <$buffer>\n";
214 }; 265 };
215 266
216=item aio_move $srcpath, $dstpath, $callback->($status) 267=item aio_move $srcpath, $dstpath, $callback->($status)
217
218[EXPERIMENTAL due to internal aio_group use]
219 268
220Try to move the I<file> (directories not supported as either source or 269Try to move the I<file> (directories not supported as either source or
221destination) from C<$srcpath> to C<$dstpath> and call the callback with 270destination) from C<$srcpath> to C<$dstpath> and call the callback with
222the C<0> (error) or C<-1> ok. 271the C<0> (error) or C<-1> ok.
223 272
… …
375 424
376The callback a single argument which is either C<undef> or an array-ref 425The callback a single argument which is either C<undef> or an array-ref
377with the filenames. 426with the filenames.
378 427
379=item aio_scandir $path, $maxreq, $callback->($dirs, $nondirs) 428=item aio_scandir $path, $maxreq, $callback->($dirs, $nondirs)
380
381[EXPERIMENTAL due to internal aio_group use]
382 429
383Scans a directory (similar to C<aio_readdir>) but additionally tries to 430Scans a directory (similar to C<aio_readdir>) but additionally tries to
384separate the entries of directory C<$path> into two sets of names, ones 431separate the entries of directory C<$path> into two sets of names, ones
385you can recurse into (directories or links to them), and ones you cannot 432you can recurse into (directories or links to them), and ones you cannot
386recurse into (everything else). 433recurse into (everything else).
… …
548 595
549[EXPERIMENTAL] 596[EXPERIMENTAL]
550 597
551This is a very special aio request: Instead of doing something, it is a 598This is a very special aio request: Instead of doing something, it is a
552container for other aio requests, which is useful if you want to bundle 599container for other aio requests, which is useful if you want to bundle
553many requests into a single, composite, request. 600many requests into a single, composite, request with a definite callback
601and the ability to cancel the whole request with its subrequests.
554 602
555Returns an object of class L<IO::AIO::GRP>. See its documentation below 603Returns an object of class L<IO::AIO::GRP>. See its documentation below
556for more info. 604for more info.
557 605
558Example: 606Example:
… …
577phase and still requires a worker thread. Thus, the callback will not 625phase and still requires a worker thread. Thus, the callback will not
578be executed immediately but only after other requests in the queue have 626be executed immediately but only after other requests in the queue have
579entered their execution phase. This can be used to measure request 627entered their execution phase. This can be used to measure request
580latency. 628latency.
581 629
582=item IO::AIO::aio_sleep $fractional_seconds, $callback->() *NOT EXPORTED* 630=item IO::AIO::aio_busy $fractional_seconds, $callback->() *NOT EXPORTED*
583 631
584Mainly used for debugging and benchmarking, this aio request puts one of 632Mainly used for debugging and benchmarking, this aio request puts one of
585the request workers to sleep for the given time. 633the request workers to sleep for the given time.
586 634
587While it is theoretically handy to have simple I/O scheduling requests 635While it is theoretically handy to have simple I/O scheduling requests
588like sleep and file handle readable/writable, the overhead this creates 636like sleep and file handle readable/writable, the overhead this creates is
589is immense, so do not use this function except to put your application 637immense (it blocks a thread for a long time) so do not use this function
590under artificial I/O pressure. 638except to put your application under artificial I/O pressure.
591 639
592=back 640=back
593 641
594=head2 IO::AIO::REQ CLASS 642=head2 IO::AIO::REQ CLASS
595 643
596All non-aggregate C<aio_*> functions return an object of this class when 644All non-aggregate C<aio_*> functions return an object of this class when
597called in non-void context. 645called in non-void context.
598
599A request always moves through the following five states in its lifetime,
600in order: B<ready> (request has been created, but has not been executed
601yet), B<execute> (request is currently being executed), B<pending>
602(request has been executed but callback has not been called yet),
603B<result> (results are being processed synchronously, includes calling the
604callback) and B<done> (request has reached the end of its lifetime and
605holds no resources anymore).
606 646
607=over 4 647=over 4
608 648
609=item cancel $req 649=item cancel $req
610 650
… …
891This module should do "the right thing" when the process using it forks: 931This module should do "the right thing" when the process using it forks:
892 932
893Before the fork, IO::AIO enters a quiescent state where no requests 933Before the fork, IO::AIO enters a quiescent state where no requests
894can be added in other threads and no results will be processed. After 934can be added in other threads and no results will be processed. After
895the fork the parent simply leaves the quiescent state and continues 935the fork the parent simply leaves the quiescent state and continues
896request/result processing, while the child clears the request/result 936request/result processing, while the child frees the request/result queue
897queue (so the requests started before the fork will only be handled in 937(so that the requests started before the fork will only be handled in the
898the parent). Threads will be started on demand until the limit ste in the 938parent). Threads will be started on demand until the limit set in the
899parent process has been reached again. 939parent process has been reached again.
900 940
901In short: the parent will, after a short pause, continue as if fork had 941In short: the parent will, after a short pause, continue as if fork had
902not been called, while the child will act as if IO::AIO has not been used 942not been called, while the child will act as if IO::AIO has not been used
903yet. 943yet.
904 944
905=head2 MEMORY USAGE 945=head2 MEMORY USAGE
906 946
947Per-request usage:
948
907Each aio request uses - depending on your architecture - around 128 bytes 949Each aio request uses - depending on your architecture - around 100-200
908of memory. In addition, stat requests need a stat buffer (possibly a few 950bytes of memory. In addition, stat requests need a stat buffer (possibly
909hundred bytes). Perl scalars and other data passed into aio requests will 951a few hundred bytes), readdir requires a result buffer and so on. Perl
910also be locked. 952scalars and other data passed into aio requests will also be locked and
953will consume memory till the request has entered the done state.
911 954
912This is now awfully much, so queuing lots of requests is not usually a 955This is now awfully much, so queuing lots of requests is not usually a
913problem. 956problem.
914 957
915Each thread needs a stack area which is usually around 16k, sometimes much 958Per-thread usage:
916larger, depending on the OS. 959
960In the execution phase, some aio requests require more memory for
961temporary buffers, and each thread requires a stack and other data
962structures (usually around 16k-128k, depending on the OS).
963
964=head1 KNOWN BUGS
965
966Known bugs will be fixed in the next release.
917 967
918=head1 SEE ALSO 968=head1 SEE ALSO
919 969
920L<Coro::AIO>. 970L<Coro::AIO>.
921 971

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines