--- AnyEvent/lib/AnyEvent.pm 2008/04/23 11:25:42 1.55 +++ AnyEvent/lib/AnyEvent.pm 2008/04/25 13:48:42 1.84 @@ -2,7 +2,7 @@ AnyEvent - provide framework for multiple event loops -EV, Event, Coro::EV, Coro::Event, Glib, Tk, Perl, Event::Lib - various supported event loops +EV, Event, Coro::EV, Coro::Event, Glib, Tk, Perl, Event::Lib, Qt, POE - various supported event loops =head1 SYNOPSIS @@ -82,11 +82,13 @@ During the first call of any watcher-creation method, the module tries to detect the currently loaded event loop by probing whether one of the following modules is already loaded: L, L, L, -L, L, L. The first one found is used. If none are found, -the module tries to load these modules in the stated order. The first one -that can be successfully loaded will be used. If, after this, still none -could be found, AnyEvent will fall back to a pure-perl event loop, which -is not very efficient, but should work everywhere. +L, L, L, L, L, L, +L. The first one found is used. If none are found, the module tries +to load these modules (excluding Tk, Event::Lib, Qt and POE as the pure perl +adaptor should always succeed) in the order given. The first one that can +be successfully loaded will be used. If, after this, still none could be +found, AnyEvent will fall back to a pure-perl event loop, which is not +very efficient, but should work everywhere. Because AnyEvent first checks for modules that are already loaded, loading an event model explicitly before first using AnyEvent will likely make @@ -136,7 +138,7 @@ my variables are only visible after the statement in which they are declared. -=head2 IO WATCHERS +=head2 I/O WATCHERS You can create an I/O watcher by calling the C<< AnyEvent->io >> method with the following mandatory key-value pairs as arguments: @@ -147,16 +149,18 @@ respectively. C is the callback to invoke each time the file handle becomes ready. -File handles will be kept alive, so as long as the watcher exists, the -file handle exists, too. - -It is not allowed to close a file handle as long as any watcher is active -on the underlying file descriptor. +The I/O watcher might use the underlying file descriptor or a copy of it. +You must not close a file handle as long as any watcher is active on the +underlying file descriptor. Some event loops issue spurious readyness notifications, so you should always use non-blocking calls when reading/writing from/to your file handles. +Although the callback might get passed parameters, their value and +presence is undefined and you cannot rely on them. Portable AnyEvent +callbacks cannot use arguments passed to I/O watcher callbacks. + Example: # wait for readability of STDIN, then read a line and disable the watcher @@ -179,6 +183,10 @@ timer you have to create a new watcher (this is a limitation by both Tk and Glib). +Although the callback might get passed parameters, their value and +presence is undefined and you cannot rely on them. Portable AnyEvent +callbacks cannot use arguments passed to time watcher callbacks. + Example: # fire an event after 7.7 seconds @@ -208,15 +216,16 @@ in 10 seconds") and based on wallclock time (absolute, "fire at 12 o'clock"). -While most event loops expect timers to specified in a relative way, they use -absolute time internally. This makes a difference when your clock "jumps", -for example, when ntp decides to set your clock backwards from the wrong 2014-01-01 to -2008-01-01, a watcher that you created to fire "after" a second might actually take -six years to finally fire. +While most event loops expect timers to specified in a relative way, they +use absolute time internally. This makes a difference when your clock +"jumps", for example, when ntp decides to set your clock backwards from +the wrong date of 2014-01-01 to 2008-01-01, a watcher that is supposed to +fire "after" a second might actually take six years to finally fire. AnyEvent cannot compensate for this. The only event loop that is conscious -about these issues is L, which offers both relative (ev_timer) and -absolute (ev_periodic) timers. +about these issues is L, which offers both relative (ev_timer, based +on true relative time) and absolute (ev_periodic, based on wallclock time) +timers. AnyEvent always prefers relative timers, if available, matching the AnyEvent API. @@ -227,7 +236,7 @@ I without any C prefix, C is the Perl callback to be invoked whenever a signal occurs. -Multiple signals occurances can be clumped together into one callback +Multiple signal occurances can be clumped together into one callback invocation, and callback invocation will be synchronous. synchronous means that it might take a while until the signal gets handled by the process, but it is guarenteed not to interrupt any other callbacks. @@ -252,16 +261,38 @@ signal handler for C. The callback will be called with the pid and exit status (as returned by waitpid). -Example: wait for pid 1333 +There is a slight catch to child watchers, however: you usually start them +I the child process was created, and this means the process could +have exited already (and no SIGCHLD will be sent anymore). + +Not all event models handle this correctly (POE doesn't), but even for +event models that I handle this correctly, they usually need to be +loaded before the process exits (i.e. before you fork in the first place). + +This means you cannot create a child watcher as the very first thing in an +AnyEvent program, you I to create at least one watcher before you +C the child (alternatively, you can call C). + +Example: fork a process and wait for it + + my $done = AnyEvent->condvar; + + AnyEvent::detect; # force event module to be initialised + + my $pid = fork or exit 5; my $w = AnyEvent->child ( - pid => 1333, + pid => $pid, cb => sub { my ($pid, $status) = @_; warn "pid $pid exited with status $status"; + $done->broadcast; }, ); + # do something else, then wait for process exit + $done->wait; + =head2 CONDITION VARIABLES Condition variables can be created by calling the C<< AnyEvent->condvar >> @@ -355,12 +386,24 @@ AnyEvent::Impl::CoroEV based on Coro::EV, best choice. AnyEvent::Impl::CoroEvent based on Coro::Event, second best choice. - AnyEvent::Impl::EV based on EV (an interface to libev, also best choice). - AnyEvent::Impl::Event based on Event, also second best choice :) + AnyEvent::Impl::EV based on EV (an interface to libev, best choice). + AnyEvent::Impl::Event based on Event, second best choice. AnyEvent::Impl::Glib based on Glib, third-best choice. - AnyEvent::Impl::Tk based on Tk, very bad choice. AnyEvent::Impl::Perl pure-perl implementation, inefficient but portable. + AnyEvent::Impl::Tk based on Tk, very bad choice. + AnyEvent::Impl::Qt based on Qt, cannot be autoprobed (see its docs). AnyEvent::Impl::EventLib based on Event::Lib, leaks memory and worse. + AnyEvent::Impl::POE based on POE, not generic enough for full support. + +There is no support for WxWidgets, as WxWidgets has no support for +watching file handles. However, you can use WxWidgets through the +POE Adaptor, as POE has a Wx backend that simply polls 20 times per +second, which was considered to be too horrible to even consider for +AnyEvent. Likewise, other POE backends can be used by AnyEvent by using +it's adaptor. + +AnyEvent knows about L and L and will try to use L when +autodetecting them. =item AnyEvent::detect @@ -421,7 +464,7 @@ use Carp; -our $VERSION = '3.12'; +our $VERSION = '3.3'; our $MODEL; our $AUTOLOAD; @@ -438,11 +481,16 @@ [Event:: => AnyEvent::Impl::Event::], [Glib:: => AnyEvent::Impl::Glib::], [Tk:: => AnyEvent::Impl::Tk::], + [Wx:: => AnyEvent::Impl::POE::], + [Prima:: => AnyEvent::Impl::POE::], [AnyEvent::Impl::Perl:: => AnyEvent::Impl::Perl::], - [Event::Lib:: => AnyEvent::Impl::EventLib::], + # everything below here will not be autoprobed as the pureperl backend should work everywhere + [Event::Lib:: => AnyEvent::Impl::EventLib::], # too buggy + [Qt:: => AnyEvent::Impl::Qt::], # requires special main program + [POE::Kernel:: => AnyEvent::Impl::POE::], # lasciate ogni speranza ); -our %method = map +($_ => 1), qw(io timer condvar broadcast wait signal one_event DESTROY); +our %method = map +($_ => 1), qw(io timer signal child condvar broadcast wait one_event DESTROY); sub detect() { unless ($MODEL) { @@ -453,6 +501,8 @@ if (eval "require $model") { $MODEL = $model; warn "AnyEvent: loaded model '$model' (forced by \$PERL_ANYEVENT_MODEL), using it.\n" if $verbose > 1; + } else { + warn "AnyEvent: unable to load model '$model' (from \$PERL_ANYEVENT_MODEL):\n$@" if $verbose; } } @@ -655,6 +705,14 @@ =item C +By default, AnyEvent will be completely silent except in fatal +conditions. You can set this environment variable to make AnyEvent more +talkative. + +When set to C<1> or higher, causes AnyEvent to warn about unexpected +conditions, such as not being able to load the event model specified by +C. + When set to C<2> or higher, cause AnyEvent to report to STDERR which event model it chooses. @@ -678,7 +736,7 @@ =head1 EXAMPLE PROGRAM -The following program uses an IO watcher to read data from STDIN, a timer +The following program uses an I/O watcher to read data from STDIN, a timer to display a message once per second, and a condition variable to quit the program when the user enters quit: @@ -832,6 +890,126 @@ $quit->wait; + +=head1 BENCHMARK + +To give you an idea of the performance and overheads that AnyEvent adds +over the event loops themselves (and to give you an impression of the +speed of various event loops), here is a benchmark of various supported +event models natively and with anyevent. The benchmark creates a lot of +timers (with a zero timeout) and I/O watchers (watching STDOUT, a pty, to +become writable, which it is), lets them fire exactly once and destroys +them again. + +Rewriting the benchmark to use many different sockets instead of using +the same filehandle for all I/O watchers results in a much longer runtime +(socket creation is expensive), but qualitatively the same figures, so it +was not used. + +=head2 Explanation of the columns + +I is the number of event watchers created/destroyed. Since +different event models feature vastly different performances, each event +loop was given a number of watchers so that overall runtime is acceptable +and similar between tested event loop (and keep them from crashing): Glib +would probably take thousands of years if asked to process the same number +of watchers as EV in this benchmark. + +I is the number of bytes (as measured by the resident set size, +RSS) consumed by each watcher. This method of measuring captures both C +and Perl-based overheads. + +I is the time, in microseconds (millionths of seconds), that it +takes to create a single watcher. The callback is a closure shared between +all watchers, to avoid adding memory overhead. That means closure creation +and memory usage is not included in the figures. + +I is the time, in microseconds, used to invoke a simple +callback. The callback simply counts down a Perl variable and after it was +invoked "watcher" times, it would C<< ->broadcast >> a condvar once to +signal the end of this phase. + +I is the time, in microseconds, that it takes to destroy a single +watcher. + +=head2 Results + + name watchers bytes create invoke destroy comment + EV/EV 400000 244 0.56 0.46 0.31 EV native interface + EV/Any 100000 244 2.50 0.46 0.29 EV + AnyEvent watchers + CoroEV/Any 100000 244 2.49 0.44 0.29 coroutines + Coro::Signal + Perl/Any 100000 513 4.92 0.87 1.12 pure perl implementation + Event/Event 16000 516 31.88 31.30 0.85 Event native interface + Event/Any 16000 936 39.17 33.63 1.43 Event + AnyEvent watchers + Glib/Any 16000 1357 98.22 12.41 54.00 quadratic behaviour + Tk/Any 2000 1860 26.97 67.98 14.00 SEGV with >> 2000 watchers + POE/Event 2000 6644 108.64 736.02 14.73 via POE::Loop::Event + POE/Select 2000 6343 94.13 809.12 565.96 via POE::Loop::Select + +=head2 Discussion + +The benchmark does I measure scalability of the event loop very +well. For example, a select-based event loop (such as the pure perl one) +can never compete with an event loop that uses epoll when the number of +file descriptors grows high. In this benchmark, all events become ready at +the same time, so select/poll-based implementations get an unnatural speed +boost. + +C is the sole leader regarding speed and memory use, which are both +maximal/minimal, respectively. Even when going through AnyEvent, it uses +far less memory than any other event loop and is still faster than Event +natively. + +The pure perl implementation is hit in a few sweet spots (both the +zero timeout and the use of a single fd hit optimisations in the perl +interpreter and the backend itself, and all watchers become ready at the +same time). Nevertheless this shows that it adds very little overhead in +itself. Like any select-based backend its performance becomes really bad +with lots of file descriptors (and few of them active), of course, but +this was not subject of this benchmark. + +The C module has a relatively high setup and callback invocation cost, +but overall scores on the third place. + +C's memory usage is quite a bit bit higher, but it features a +faster callback invocation and overall ends up in the same class as +C. However, Glib scales extremely badly, doubling the number of +watchers increases the processing time by more than a factor of four, +making it completely unusable when using larger numbers of watchers +(note that only a single file descriptor was used in the benchmark, so +inefficiencies of C do not account for this). + +The C adaptor works relatively well. The fact that it crashes with +more than 2000 watchers is a big setback, however, as correctness takes +precedence over speed. Nevertheless, its performance is surprising, as the +file descriptor is dup()ed for each watcher. This shows that the dup() +employed by some adaptors is not a big performance issue (it does incur a +hidden memory cost inside the kernel, though, that is not reflected in the +figures above). + +C, regardless of underlying event loop (wether using its pure perl +select-based backend or the Event module) shows abysmal performance and +memory usage: Watchers use almost 30 times as much memory as EV watchers, +and 10 times as much memory as both Event or EV via AnyEvent. Watcher +invocation is almost 900 times slower than with AnyEvent's pure perl +implementation. The design of the POE adaptor class in AnyEvent can not +really account for this, as session creation overhead is small compared +to execution of the state machine, which is coded pretty optimally within +L. POE simply seems to be abysmally slow. + +=head2 Summary + +Using EV through AnyEvent is faster than any other event loop, but most +event loops have acceptable performance with or without AnyEvent. + +The overhead AnyEvent adds is usually much smaller than the overhead of +the actual event loop, only with extremely fast event loops such as the EV +adds AnyEvent significant overhead. + +And you should simply avoid POE like the plague if you want performance or +reasonable memory usage. + + =head1 FORK Most event libraries are not fork-safe. The ones who are usually are @@ -840,6 +1018,7 @@ If you have to fork, you must either do so I creating your first watcher OR you must not use AnyEvent at all in the child. + =head1 SECURITY CONSIDERATIONS AnyEvent can be forced to load any event model via @@ -856,18 +1035,21 @@ use AnyEvent; + =head1 SEE ALSO Event modules: L, L, L, L, L, L, L, L, L, L, -L. +L, L, L. Implementations: L, L, L, L, L, -L, L, L. +L, L, L, +L, L. Nontrivial usage examples: L, L. + =head1 AUTHOR Marc Lehmann