--- AnyEvent/README 2009/03/27 10:49:50 1.36 +++ AnyEvent/README 2009/06/29 21:00:32 1.42 @@ -1,27 +1,34 @@ NAME AnyEvent - provide framework for multiple event loops - EV, Event, Glib, Tk, Perl, Event::Lib, Qt, POE - various supported event - loops + EV, Event, Glib, Tk, Perl, Event::Lib, Qt and POE are various supported + event loops. SYNOPSIS use AnyEvent; - my $w = AnyEvent->io (fh => $fh, poll => "r|w", cb => sub { ... }); + # file descriptor readable + my $w = AnyEvent->io (fh => $fh, poll => "r", cb => sub { ... }); + # one-shot or repeating timers my $w = AnyEvent->timer (after => $seconds, cb => sub { ... }); my $w = AnyEvent->timer (after => $seconds, interval => $seconds, cb => ... print AnyEvent->now; # prints current event loop time print AnyEvent->time; # think Time::HiRes::time or simply CORE::time. + # POSIX signal my $w = AnyEvent->signal (signal => "TERM", cb => sub { ... }); + # child process exit my $w = AnyEvent->child (pid => $pid, cb => sub { my ($pid, $status) = @_; ... }); + # called when event loop idle (if applicable) + my $w = AnyEvent->idle (cb => sub { ... }); + my $w = AnyEvent->condvar; # stores whether a condition was flagged $w->send; # wake up current and all future recv's $w->recv; # enters "main loop" till $condvar gets ->send @@ -316,6 +323,20 @@ the difference between "AnyEvent->time" and "AnyEvent->now" into account. + AnyEvent->now_update + Some event loops (such as EV or AnyEvent::Impl::Perl) cache the + current time for each loop iteration (see the discussion of + AnyEvent->now, above). + + When a callback runs for a long time (or when the process sleeps), + then this "current" time will differ substantially from the real + time, which might affect timers and time-outs. + + When this is the case, you can call this method, which will update + the event loop's idea of "current time". + + Note that updating the time *might* cause some events to be handled. + SIGNAL WATCHERS You can watch for signals using a signal watcher, "signal" is the signal *name* in uppercase and without any "SIG" prefix, "cb" is the Perl @@ -361,10 +382,12 @@ them *after* the child process was created, and this means the process could have exited already (and no SIGCHLD will be sent anymore). - Not all event models handle this correctly (POE doesn't), but even for - event models that *do* handle this correctly, they usually need to be - loaded before the process exits (i.e. before you fork in the first - place). + Not all event models handle this correctly (neither POE nor IO::Async + do, see their AnyEvent::Impl manpages for details), but even for event + models that *do* handle this correctly, they usually need to be loaded + before the process exits (i.e. before you fork in the first place). + AnyEvent's pure perl event loop handles all cases correctly regardless + of when you start the watcher. This means you cannot create a child watcher as the very first thing in an AnyEvent program, you *have* to create at least one watcher before @@ -373,10 +396,10 @@ Example: fork a process and wait for it my $done = AnyEvent->condvar; - - my $pid = fork or exit 5; - - my $w = AnyEvent->child ( + + my $pid = fork or exit 5; + + my $w = AnyEvent->child ( pid => $pid, cb => sub { my ($pid, $status) = @_; @@ -384,10 +407,44 @@ $done->send; }, ); - - # do something else, then wait for process exit + + # do something else, then wait for process exit $done->recv; + IDLE WATCHERS + Sometimes there is a need to do something, but it is not so important to + do it instantly, but only when there is nothing better to do. This + "nothing better to do" is usually defined to be "no other events need + attention by the event loop". + + Idle watchers ideally get invoked when the event loop has nothing better + to do, just before it would block the process to wait for new events. + Instead of blocking, the idle watcher is invoked. + + Most event loops unfortunately do not really support idle watchers (only + EV, Event and Glib do it in a usable fashion) - for the rest, AnyEvent + will simply call the callback "from time to time". + + Example: read lines from STDIN, but only process them when the program + is otherwise idle: + + my @lines; # read data + my $idle_w; + my $io_w = AnyEvent->io (fh => \*STDIN, poll => 'r', cb => sub { + push @lines, scalar ; + + # start an idle watcher, if not already done + $idle_w ||= AnyEvent->idle (cb => sub { + # handle only one line, when there are lines left + if (my $line = shift @lines) { + print "handled when idle: $line"; + } else { + # otherwise disable the idle watcher again + undef $idle_w; + } + }); + }); + CONDITION VARIABLES If you are familiar with some event loops you will know that all of them require you to run some blocking "loop", "run" or similar function that @@ -522,8 +579,6 @@ $cv->begin ([group callback]) $cv->end - These two methods are EXPERIMENTAL and MIGHT CHANGE. - These two methods can be used to combine many transactions/events into one. For example, a function that pings many hosts in parallel might want to use a condition variable for the whole process. @@ -534,7 +589,38 @@ *supposed* to call "->send", but that is not required. If no callback was set, "send" will be called without any arguments. - Let's clarify this with the ping example: + You can think of "$cv->send" giving you an OR condition (one call + sends), while "$cv->begin" and "$cv->end" giving you an AND + condition (all "begin" calls must be "end"'ed before the condvar + sends). + + Let's start with a simple example: you have two I/O watchers (for + example, STDOUT and STDERR for a program), and you want to wait for + both streams to close before activating a condvar: + + my $cv = AnyEvent->condvar; + + $cv->begin; # first watcher + my $w1 = AnyEvent->io (fh => $fh1, cb => sub { + defined sysread $fh1, my $buf, 4096 + or $cv->end; + }); + + $cv->begin; # second watcher + my $w2 = AnyEvent->io (fh => $fh2, cb => sub { + defined sysread $fh2, my $buf, 4096 + or $cv->end; + }); + + $cv->recv; + + This works because for every event source (EOF on file handle), + there is one call to "begin", so the condvar waits for all calls to + "end" before sending. + + The ping example mentioned above is slightly more complicated, as + the there are results to be passwd back, and the number of tasks + that are begung can potentially be zero: my $cv = AnyEvent->condvar; @@ -564,11 +650,11 @@ ensures that "send" is called even when "no" hosts are being pinged (the loop doesn't execute once). - This is the general pattern when you "fan out" into multiple - subrequests: use an outer "begin"/"end" pair to set the callback and - ensure "end" is called at least once, and then, for each subrequest - you start, call "begin" and for each subrequest you finish, call - "end". + This is the general pattern when you "fan out" into multiple (but + potentially none) subrequests: use an outer "begin"/"end" pair to + set the callback and ensure "end" is called at least once, and then, + for each subrequest you start, call "begin" and for each subrequest + you finish, call "end". METHODS FOR CONSUMERS These methods should only be used by the consuming side, i.e. the code @@ -644,6 +730,10 @@ AnyEvent::Impl::EventLib based on Event::Lib, leaks memory and worse. AnyEvent::Impl::POE based on POE, not generic enough for full support. + # warning, support for IO::Async is only partial, as it is too broken + # and limited toe ven support the AnyEvent API. See AnyEvent::Impl::Async. + AnyEvent::Impl::IOAsync based on IO::Async, cannot be autoprobed (see its docs). + There is no support for WxWidgets, as WxWidgets has no support for watching file handles. However, you can use WxWidgets through the POE Adaptor, as POE has a Wx backend that simply polls 20 times per @@ -830,7 +920,11 @@ ENVIRONMENT VARIABLES The following environment variables are used by this module or its - submodules: + submodules. + + Note that AnyEvent will remove *all* environment variables starting with + "PERL_ANYEVENT_" from %ENV when it is loaded while taint mode is + enabled. "PERL_ANYEVENT_VERBOSE" By default, AnyEvent will be completely silent except in fatal @@ -849,11 +943,11 @@ argument checking is very costly. Setting this variable to a true value will cause AnyEvent to load "AnyEvent::Strict" and then to thoroughly check the arguments passed to most method calls. If it - finds any problems it will croak. + finds any problems, it will croak. In other words, enables "strict" mode. - Unlike "use strict", it is definitely recommended ot keep it off in + Unlike "use strict", it is definitely recommended to keep it off in production. Keeping "PERL_ANYEVENT_STRICT=1" in your environment while developing programs can be very useful, however. @@ -1151,6 +1245,8 @@ Perl/Any 100000 452 4.13 0.73 0.95 pure perl implementation Event/Event 16000 517 32.20 31.80 0.81 Event native interface Event/Any 16000 590 35.85 31.55 1.06 Event + AnyEvent watchers + IOAsync/Any 16000 989 38.10 32.77 11.13 via IO::Async::Loop::IO_Poll + IOAsync/Any 16000 990 37.59 29.50 10.61 via IO::Async::Loop::Epoll Glib/Any 16000 1357 102.33 12.31 51.00 quadratic behaviour Tk/Any 2000 1860 27.20 66.31 14.00 SEGV with >> 2000 watchers POE/Event 2000 6328 109.99 751.67 14.02 via POE::Loop::Event @@ -1190,6 +1286,9 @@ The "Event" module has a relatively high setup and callback invocation cost, but overall scores in on the third place. + "IO::Async" performs admirably well, about on par with "Event", even + when using its pure perl backend. + "Glib"'s memory usage is quite a bit higher, but it features a faster callback invocation and overall ends up in the same class as "Event". However, Glib scales extremely badly, doubling the number of watchers @@ -1269,12 +1368,14 @@ and creating a new one that moves the timeout into the future. Results - name sockets create request - EV 20000 69.01 11.16 - Perl 20000 73.32 35.87 - Event 20000 212.62 257.32 - Glib 20000 651.16 1896.30 - POE 20000 349.67 12317.24 uses POE::Loop::Event + name sockets create request + EV 20000 69.01 11.16 + Perl 20000 73.32 35.87 + IOAsync 20000 157.00 98.14 epoll + IOAsync 20000 159.31 616.06 poll + Event 20000 212.62 257.32 + Glib 20000 651.16 1896.30 + POE 20000 349.67 12317.24 uses POE::Loop::Event Discussion This benchmark *does* measure scalability and overall performance of the @@ -1286,6 +1387,9 @@ Perl surprisingly comes second. It is much faster than the C-based event loops Event and Glib. + IO::Async performs very well when using its epoll backend, and still + quite good compared to Glib when using its pure perl backend. + Event suffers from high setup time as well (look at its code and you will understand why). Callback invocation also has a high overhead compared to the "$_->() for .."-style loop that the Perl event loop @@ -1345,6 +1449,63 @@ * C-based event loops perform very well with small number of watchers, as the management overhead dominates. + THE IO::Lambda BENCHMARK + Recently I was told about the benchmark in the IO::Lambda manpage, which + could be misinterpreted to make AnyEvent look bad. In fact, the + benchmark simply compares IO::Lambda with POE, and IO::Lambda looks + better (which shouldn't come as a surprise to anybody). As such, the + benchmark is fine, and mostly shows that the AnyEvent backend from + IO::Lambda isn't very optimal. But how would AnyEvent compare when used + without the extra baggage? To explore this, I wrote the equivalent + benchmark for AnyEvent. + + The benchmark itself creates an echo-server, and then, for 500 times, + connects to the echo server, sends a line, waits for the reply, and then + creates the next connection. This is a rather bad benchmark, as it + doesn't test the efficiency of the framework or much non-blocking I/O, + but it is a benchmark nevertheless. + + name runtime + Lambda/select 0.330 sec + + optimized 0.122 sec + Lambda/AnyEvent 0.327 sec + + optimized 0.138 sec + Raw sockets/select 0.077 sec + POE/select, components 0.662 sec + POE/select, raw sockets 0.226 sec + POE/select, optimized 0.404 sec + + AnyEvent/select/nb 0.085 sec + AnyEvent/EV/nb 0.068 sec + +state machine 0.134 sec + + The benchmark is also a bit unfair (my fault): the IO::Lambda/POE + benchmarks actually make blocking connects and use 100% blocking I/O, + defeating the purpose of an event-based solution. All of the newly + written AnyEvent benchmarks use 100% non-blocking connects (using + AnyEvent::Socket::tcp_connect and the asynchronous pure perl DNS + resolver), so AnyEvent is at a disadvantage here, as non-blocking + connects generally require a lot more bookkeeping and event handling + than blocking connects (which involve a single syscall only). + + The last AnyEvent benchmark additionally uses AnyEvent::Handle, which + offers similar expressive power as POE and IO::Lambda, using + conventional Perl syntax. This means that both the echo server and the + client are 100% non-blocking, further placing it at a disadvantage. + + As you can see, the AnyEvent + EV combination even beats the + hand-optimised "raw sockets benchmark", while AnyEvent + its pure perl + backend easily beats IO::Lambda and POE. + + And even the 100% non-blocking version written using the high-level (and + slow :) AnyEvent::Handle abstraction beats both POE and IO::Lambda by a + large margin, even though it does all of DNS, tcp-connect and socket I/O + in a non-blocking way. + + The two AnyEvent benchmarks programs can be found as eg/ae0.pl and + eg/ae2.pl in the AnyEvent distribution, the remaining benchmarks are + part of the IO::lambda distribution and were used without any changes. + SIGNALS AnyEvent currently installs handlers for these signals: @@ -1353,6 +1514,9 @@ emulation for event loops that do not support them natively. Also, some event loops install a similar handler. + If, when AnyEvent is loaded, SIGCHLD is set to IGNORE, then AnyEvent + will reset it to default, to avoid losing child exit statuses. + SIGPIPE A no-op handler is installed for "SIGPIPE" when $SIG{PIPE} is "undef" when AnyEvent gets loaded. @@ -1389,13 +1553,17 @@ before the first watcher gets created, e.g. with a "BEGIN" block: BEGIN { delete $ENV{PERL_ANYEVENT_MODEL} } - - use AnyEvent; + + use AnyEvent; Similar considerations apply to $ENV{PERL_ANYEVENT_VERBOSE}, as that can be used to probe what backend is used and gain other information (which is probably even less useful to an attacker than PERL_ANYEVENT_MODEL), - and $ENV{PERL_ANYEGENT_STRICT}. + and $ENV{PERL_ANYEVENT_STRICT}. + + Note that AnyEvent will remove *all* environment variables starting with + "PERL_ANYEVENT_" from %ENV when it is loaded while taint mode is + enabled. BUGS Perl 5.8 has numerous memleaks that sometimes hit this module and are