--- AnyEvent/lib/AnyEvent.pm 2009/04/16 08:15:25 1.204 +++ AnyEvent/lib/AnyEvent.pm 2009/06/29 21:00:32 1.223 @@ -1,28 +1,36 @@ =head1 NAME -AnyEvent - provide framework for multiple event loops +AnyEvent - provide framework for multiple event loops -EV, Event, Glib, Tk, Perl, Event::Lib, Qt, POE - various supported event loops +EV, Event, Glib, Tk, Perl, Event::Lib, Qt and POE are various supported +event loops. =head1 SYNOPSIS use AnyEvent; - my $w = AnyEvent->io (fh => $fh, poll => "r|w", cb => sub { ... }); + # file descriptor readable + my $w = AnyEvent->io (fh => $fh, poll => "r", cb => sub { ... }); + # one-shot or repeating timers my $w = AnyEvent->timer (after => $seconds, cb => sub { ... }); my $w = AnyEvent->timer (after => $seconds, interval => $seconds, cb => ... print AnyEvent->now; # prints current event loop time print AnyEvent->time; # think Time::HiRes::time or simply CORE::time. + # POSIX signal my $w = AnyEvent->signal (signal => "TERM", cb => sub { ... }); + # child process exit my $w = AnyEvent->child (pid => $pid, cb => sub { my ($pid, $status) = @_; ... }); + # called when event loop idle (if applicable) + my $w = AnyEvent->idle (cb => sub { ... }); + my $w = AnyEvent->condvar; # stores whether a condition was flagged $w->send; # wake up current and all future recv's $w->recv; # enters "main loop" till $condvar gets ->send @@ -322,6 +330,21 @@ difference between C<< AnyEvent->time >> and C<< AnyEvent->now >> into account. +=item AnyEvent->now_update + +Some event loops (such as L or L) cache +the current time for each loop iteration (see the discussion of L<< +AnyEvent->now >>, above). + +When a callback runs for a long time (or when the process sleeps), then +this "current" time will differ substantially from the real time, which +might affect timers and time-outs. + +When this is the case, you can call this method, which will update the +event loop's idea of "current time". + +Note that updating the time I cause some events to be handled. + =back =head2 SIGNAL WATCHERS @@ -371,13 +394,17 @@ I the child process was created, and this means the process could have exited already (and no SIGCHLD will be sent anymore). -Not all event models handle this correctly (POE doesn't), but even for -event models that I handle this correctly, they usually need to be -loaded before the process exits (i.e. before you fork in the first place). - -This means you cannot create a child watcher as the very first thing in an -AnyEvent program, you I to create at least one watcher before you -C the child (alternatively, you can call C). +Not all event models handle this correctly (neither POE nor IO::Async do, +see their AnyEvent::Impl manpages for details), but even for event models +that I handle this correctly, they usually need to be loaded before +the process exits (i.e. before you fork in the first place). AnyEvent's +pure perl event loop handles all cases correctly regardless of when you +start the watcher. + +This means you cannot create a child watcher as the very first +thing in an AnyEvent program, you I to create at least one +watcher before you C the child (alternatively, you can call +C). Example: fork a process and wait for it @@ -397,6 +424,41 @@ # do something else, then wait for process exit $done->recv; +=head2 IDLE WATCHERS + +Sometimes there is a need to do something, but it is not so important +to do it instantly, but only when there is nothing better to do. This +"nothing better to do" is usually defined to be "no other events need +attention by the event loop". + +Idle watchers ideally get invoked when the event loop has nothing +better to do, just before it would block the process to wait for new +events. Instead of blocking, the idle watcher is invoked. + +Most event loops unfortunately do not really support idle watchers (only +EV, Event and Glib do it in a usable fashion) - for the rest, AnyEvent +will simply call the callback "from time to time". + +Example: read lines from STDIN, but only process them when the +program is otherwise idle: + + my @lines; # read data + my $idle_w; + my $io_w = AnyEvent->io (fh => \*STDIN, poll => 'r', cb => sub { + push @lines, scalar ; + + # start an idle watcher, if not already done + $idle_w ||= AnyEvent->idle (cb => sub { + # handle only one line, when there are lines left + if (my $line = shift @lines) { + print "handled when idle: $line"; + } else { + # otherwise disable the idle watcher again + undef $idle_w; + } + }); + }); + =head2 CONDITION VARIABLES If you are familiar with some event loops you will know that all of them @@ -539,8 +601,6 @@ =item $cv->end -These two methods are EXPERIMENTAL and MIGHT CHANGE. - These two methods can be used to combine many transactions/events into one. For example, a function that pings many hosts in parallel might want to use a condition variable for the whole process. @@ -551,7 +611,37 @@ is I to call C<< ->send >>, but that is not required. If no callback was set, C will be called without any arguments. -Let's clarify this with the ping example: +You can think of C<< $cv->send >> giving you an OR condition (one call +sends), while C<< $cv->begin >> and C<< $cv->end >> giving you an AND +condition (all C calls must be C'ed before the condvar sends). + +Let's start with a simple example: you have two I/O watchers (for example, +STDOUT and STDERR for a program), and you want to wait for both streams to +close before activating a condvar: + + my $cv = AnyEvent->condvar; + + $cv->begin; # first watcher + my $w1 = AnyEvent->io (fh => $fh1, cb => sub { + defined sysread $fh1, my $buf, 4096 + or $cv->end; + }); + + $cv->begin; # second watcher + my $w2 = AnyEvent->io (fh => $fh2, cb => sub { + defined sysread $fh2, my $buf, 4096 + or $cv->end; + }); + + $cv->recv; + +This works because for every event source (EOF on file handle), there is +one call to C, so the condvar waits for all calls to C before +sending. + +The ping example mentioned above is slightly more complicated, as the +there are results to be passwd back, and the number of tasks that are +begung can potentially be zero: my $cv = AnyEvent->condvar; @@ -581,10 +671,11 @@ C is called even when C hosts are being pinged (the loop doesn't execute once). -This is the general pattern when you "fan out" into multiple subrequests: -use an outer C/C pair to set the callback and ensure C -is called at least once, and then, for each subrequest you start, call -C and for each subrequest you finish, call C. +This is the general pattern when you "fan out" into multiple (but +potentially none) subrequests: use an outer C/C pair to set +the callback and ensure C is called at least once, and then, for each +subrequest you start, call C and for each subrequest you finish, +call C. =back @@ -674,6 +765,10 @@ AnyEvent::Impl::EventLib based on Event::Lib, leaks memory and worse. AnyEvent::Impl::POE based on POE, not generic enough for full support. + # warning, support for IO::Async is only partial, as it is too broken + # and limited toe ven support the AnyEvent API. See AnyEvent::Impl::Async. + AnyEvent::Impl::IOAsync based on IO::Async, cannot be autoprobed (see its docs). + There is no support for WxWidgets, as WxWidgets has no support for watching file handles. However, you can use WxWidgets through the POE Adaptor, as POE has a Wx backend that simply polls 20 times per @@ -875,7 +970,7 @@ use Carp; -our $VERSION = 4.351; +our $VERSION = 4.45; our $MODEL; our $AUTOLOAD; @@ -886,8 +981,11 @@ our $WIN32; BEGIN { - my $win32 = ! ! ($^O =~ /mswin32/i); - eval "sub WIN32(){ $win32 }"; + eval "sub WIN32(){ " . (($^O =~ /mswin32/i)*1) ." }"; + eval "sub TAINT(){ " . (${^TAINT}*1) . " }"; + + delete @ENV{grep /^PERL_ANYEVENT_/, keys %ENV} + if ${^TAINT}; } our $verbose = $ENV{PERL_ANYEVENT_VERBOSE}*1; @@ -915,9 +1013,17 @@ [POE::Kernel:: => AnyEvent::Impl::POE::], # lasciate ogni speranza [Wx:: => AnyEvent::Impl::POE::], [Prima:: => AnyEvent::Impl::POE::], + # IO::Async is just too broken - we would need workaorunds for its + # byzantine signal and broken child handling, among others. + # IO::Async is rather hard to detect, as it doesn't have any + # obvious default class. +# [IO::Async:: => AnyEvent::Impl::IOAsync::], # requires special main program +# [IO::Async::Loop:: => AnyEvent::Impl::IOAsync::], # requires special main program +# [IO::Async::Notifier:: => AnyEvent::Impl::IOAsync::], # requires special main program ); -our %method = map +($_ => 1), qw(io timer time now signal child condvar one_event DESTROY); +our %method = map +($_ => 1), + qw(io timer time now now_update signal child idle condvar one_event DESTROY); our @post_detect; @@ -932,12 +1038,12 @@ push @post_detect, $cb; defined wantarray - ? bless \$cb, "AnyEvent::Util::PostDetect" + ? bless \$cb, "AnyEvent::Util::postdetect" : () } } -sub AnyEvent::Util::PostDetect::DESTROY { +sub AnyEvent::Util::postdetect::DESTROY { @post_detect = grep $_ != ${$_[0]}, @post_detect; } @@ -1015,7 +1121,7 @@ # utility function to dup a filehandle. this is used by many backends # to support binding more than one watcher per filehandle (they usually # allow only one watcher per fd, so we dup it to get a different one). -sub _dupfh($$$$) { +sub _dupfh($$;$$) { my ($poll, $fh, $r, $w) = @_; # cygwin requires the fh mode to be matching, unix doesn't @@ -1033,10 +1139,10 @@ package AnyEvent::Base; -# default implementation for now and time +# default implementations for many methods BEGIN { - if (eval "use Time::HiRes (); time (); 1") { + if (eval "use Time::HiRes (); Time::HiRes::time (); 1") { *_time = \&Time::HiRes::time; # if (eval "use POSIX (); (POSIX::times())... } else { @@ -1046,11 +1152,12 @@ sub time { _time } sub now { _time } +sub now_update { } # default implementation for ->condvar sub condvar { - bless { @_ == 3 ? (_ae_cb => $_[2]) : () }, AnyEvent::CondVar:: + bless { @_ == 3 ? (_ae_cb => $_[2]) : () }, "AnyEvent::CondVar" } # default implementation for ->signal @@ -1084,15 +1191,15 @@ pipe $SIGPIPE_R, $SIGPIPE_W; fcntl $SIGPIPE_R, &Fcntl::F_SETFL, &Fcntl::O_NONBLOCK if $SIGPIPE_R; fcntl $SIGPIPE_W, &Fcntl::F_SETFL, &Fcntl::O_NONBLOCK if $SIGPIPE_W; # just in case + + # not strictly required, as $^F is normally 2, but let's make sure... + fcntl $SIGPIPE_R, &Fcntl::F_SETFD, &Fcntl::FD_CLOEXEC; + fcntl $SIGPIPE_W, &Fcntl::F_SETFD, &Fcntl::FD_CLOEXEC; } $SIGPIPE_R or Carp::croak "AnyEvent: unable to create a signal reporting pipe: $!\n"; - # not strictly required, as $^F is normally 2, but let's make sure... - fcntl $SIGPIPE_R, &Fcntl::F_SETFD, &Fcntl::FD_CLOEXEC; - fcntl $SIGPIPE_W, &Fcntl::F_SETFD, &Fcntl::FD_CLOEXEC; - $SIG_IO = AnyEvent->io (fh => $SIGPIPE_R, poll => "r", cb => \&_signal_exec); } @@ -1106,15 +1213,18 @@ undef $SIG_EV{$signal}; }; - bless [$signal, $arg{cb}], "AnyEvent::Base::Signal" + bless [$signal, $arg{cb}], "AnyEvent::Base::signal" } -sub AnyEvent::Base::Signal::DESTROY { +sub AnyEvent::Base::signal::DESTROY { my ($signal, $cb) = @{$_[0]}; delete $SIG_CB{$signal}{$cb}; - delete $SIG{$signal} unless keys %{ $SIG_CB{$signal} }; + # delete doesn't work with older perls - they then + # print weird messages, or just unconditionally exit + # instead of getting the default action. + undef $SIG{$signal} unless keys %{ $SIG_CB{$signal} }; } # default implementation for ->child @@ -1122,24 +1232,13 @@ our %PID_CB; our $CHLD_W; our $CHLD_DELAY_W; -our $PID_IDLE; our $WNOHANG; -sub _child_wait { +sub _sigchld { while (0 < (my $pid = waitpid -1, $WNOHANG)) { $_->($pid, $?) for (values %{ $PID_CB{$pid} || {} }), (values %{ $PID_CB{0} || {} }); } - - undef $PID_IDLE; -} - -sub _sigchld { - # make sure we deliver these changes "synchronous" with the event loop. - $CHLD_DELAY_W ||= AnyEvent->timer (after => 0, cb => sub { - undef $CHLD_DELAY_W; - &_child_wait; - }); } sub child { @@ -1150,9 +1249,7 @@ $PID_CB{$pid}{$arg{cb}} = $arg{cb}; - unless ($WNOHANG) { - $WNOHANG = eval { local $SIG{__DIE__}; require POSIX; &POSIX::WNOHANG } || 1; - } + $WNOHANG ||= eval { local $SIG{__DIE__}; require POSIX; &POSIX::WNOHANG } || 1; unless ($CHLD_W) { $CHLD_W = AnyEvent->signal (signal => 'CHLD', cb => \&_sigchld); @@ -1160,10 +1257,10 @@ &_sigchld; } - bless [$pid, $arg{cb}], "AnyEvent::Base::Child" + bless [$pid, $arg{cb}], "AnyEvent::Base::child" } -sub AnyEvent::Base::Child::DESTROY { +sub AnyEvent::Base::child::DESTROY { my ($pid, $cb) = @{$_[0]}; delete $PID_CB{$pid}{$cb}; @@ -1172,6 +1269,42 @@ undef $CHLD_W unless keys %PID_CB; } +# idle emulation is done by simply using a timer, regardless +# of whether the process is idle or not, and not letting +# the callback use more than 50% of the time. +sub idle { + my (undef, %arg) = @_; + + my ($cb, $w, $rcb) = $arg{cb}; + + $rcb = sub { + if ($cb) { + $w = _time; + &$cb; + $w = _time - $w; + + # never use more then 50% of the time for the idle watcher, + # within some limits + $w = 0.0001 if $w < 0.0001; + $w = 5 if $w > 5; + + $w = AnyEvent->timer (after => $w, cb => $rcb); + } else { + # clean up... + undef $w; + undef $rcb; + } + }; + + $w = AnyEvent->timer (after => 0.05, cb => $rcb); + + bless \\$cb, "AnyEvent::Base::idle" +} + +sub AnyEvent::Base::idle::DESTROY { + undef $${$_[0]}; +} + package AnyEvent::CondVar; our @ISA = AnyEvent::CondVar::Base::; @@ -1253,7 +1386,11 @@ =head1 ENVIRONMENT VARIABLES The following environment variables are used by this module or its -submodules: +submodules. + +Note that AnyEvent will remove I environment variables starting with +C from C<%ENV> when it is loaded while taint mode is +enabled. =over 4 @@ -1275,12 +1412,12 @@ AnyEvent does not do much argument checking by default, as thorough argument checking is very costly. Setting this variable to a true value will cause AnyEvent to load C and then to thoroughly -check the arguments passed to most method calls. If it finds any problems +check the arguments passed to most method calls. If it finds any problems, it will croak. In other words, enables "strict" mode. -Unlike C, it is definitely recommended ot keep it off in +Unlike C, it is definitely recommended to keep it off in production. Keeping C in your environment while developing programs can be very useful, however. @@ -1589,6 +1726,8 @@ Perl/Any 100000 452 4.13 0.73 0.95 pure perl implementation Event/Event 16000 517 32.20 31.80 0.81 Event native interface Event/Any 16000 590 35.85 31.55 1.06 Event + AnyEvent watchers + IOAsync/Any 16000 989 38.10 32.77 11.13 via IO::Async::Loop::IO_Poll + IOAsync/Any 16000 990 37.59 29.50 10.61 via IO::Async::Loop::Epoll Glib/Any 16000 1357 102.33 12.31 51.00 quadratic behaviour Tk/Any 2000 1860 27.20 66.31 14.00 SEGV with >> 2000 watchers POE/Event 2000 6328 109.99 751.67 14.02 via POE::Loop::Event @@ -1628,6 +1767,9 @@ The C module has a relatively high setup and callback invocation cost, but overall scores in on the third place. +C performs admirably well, about on par with C, even +when using its pure perl backend. + C's memory usage is quite a bit higher, but it features a faster callback invocation and overall ends up in the same class as C. However, Glib scales extremely badly, doubling the number of @@ -1714,12 +1856,14 @@ =head3 Results - name sockets create request - EV 20000 69.01 11.16 - Perl 20000 73.32 35.87 - Event 20000 212.62 257.32 - Glib 20000 651.16 1896.30 - POE 20000 349.67 12317.24 uses POE::Loop::Event + name sockets create request + EV 20000 69.01 11.16 + Perl 20000 73.32 35.87 + IOAsync 20000 157.00 98.14 epoll + IOAsync 20000 159.31 616.06 poll + Event 20000 212.62 257.32 + Glib 20000 651.16 1896.30 + POE 20000 349.67 12317.24 uses POE::Loop::Event =head3 Discussion @@ -1732,6 +1876,9 @@ Perl surprisingly comes second. It is much faster than the C-based event loops Event and Glib. +IO::Async performs very well when using its epoll backend, and still quite +good compared to Glib when using its pure perl backend. + Event suffers from high setup time as well (look at its code and you will understand why). Callback invocation also has a high overhead compared to the C<< $_->() for .. >>-style loop that the Perl event loop uses. Event @@ -1803,6 +1950,63 @@ =back +=head2 THE IO::Lambda BENCHMARK + +Recently I was told about the benchmark in the IO::Lambda manpage, which +could be misinterpreted to make AnyEvent look bad. In fact, the benchmark +simply compares IO::Lambda with POE, and IO::Lambda looks better (which +shouldn't come as a surprise to anybody). As such, the benchmark is +fine, and mostly shows that the AnyEvent backend from IO::Lambda isn't +very optimal. But how would AnyEvent compare when used without the extra +baggage? To explore this, I wrote the equivalent benchmark for AnyEvent. + +The benchmark itself creates an echo-server, and then, for 500 times, +connects to the echo server, sends a line, waits for the reply, and then +creates the next connection. This is a rather bad benchmark, as it doesn't +test the efficiency of the framework or much non-blocking I/O, but it is a +benchmark nevertheless. + + name runtime + Lambda/select 0.330 sec + + optimized 0.122 sec + Lambda/AnyEvent 0.327 sec + + optimized 0.138 sec + Raw sockets/select 0.077 sec + POE/select, components 0.662 sec + POE/select, raw sockets 0.226 sec + POE/select, optimized 0.404 sec + + AnyEvent/select/nb 0.085 sec + AnyEvent/EV/nb 0.068 sec + +state machine 0.134 sec + +The benchmark is also a bit unfair (my fault): the IO::Lambda/POE +benchmarks actually make blocking connects and use 100% blocking I/O, +defeating the purpose of an event-based solution. All of the newly +written AnyEvent benchmarks use 100% non-blocking connects (using +AnyEvent::Socket::tcp_connect and the asynchronous pure perl DNS +resolver), so AnyEvent is at a disadvantage here, as non-blocking connects +generally require a lot more bookkeeping and event handling than blocking +connects (which involve a single syscall only). + +The last AnyEvent benchmark additionally uses L, which +offers similar expressive power as POE and IO::Lambda, using conventional +Perl syntax. This means that both the echo server and the client are 100% +non-blocking, further placing it at a disadvantage. + +As you can see, the AnyEvent + EV combination even beats the +hand-optimised "raw sockets benchmark", while AnyEvent + its pure perl +backend easily beats IO::Lambda and POE. + +And even the 100% non-blocking version written using the high-level (and +slow :) L abstraction beats both POE and IO::Lambda by a +large margin, even though it does all of DNS, tcp-connect and socket I/O +in a non-blocking way. + +The two AnyEvent benchmarks programs can be found as F and +F in the AnyEvent distribution, the remaining benchmarks are +part of the IO::lambda distribution and were used without any changes. + =head1 SIGNALS @@ -1816,6 +2020,9 @@ emulation for event loops that do not support them natively. Also, some event loops install a similar handler. +If, when AnyEvent is loaded, SIGCHLD is set to IGNORE, then AnyEvent will +reset it to default, to avoid losing child exit statuses. + =item SIGPIPE A no-op handler is installed for C when C<$SIG{PIPE}> is C @@ -1836,10 +2043,12 @@ =cut +undef $SIG{CHLD} + if $SIG{CHLD} eq 'IGNORE'; + $SIG{PIPE} = sub { } unless defined $SIG{PIPE}; - =head1 FORK Most event libraries are not fork-safe. The ones who are usually are @@ -1869,7 +2078,11 @@ Similar considerations apply to $ENV{PERL_ANYEVENT_VERBOSE}, as that can be used to probe what backend is used and gain other information (which is probably even less useful to an attacker than PERL_ANYEVENT_MODEL), and -$ENV{PERL_ANYEGENT_STRICT}. +$ENV{PERL_ANYEVENT_STRICT}. + +Note that AnyEvent will remove I environment variables starting with +C from C<%ENV> when it is loaded while taint mode is +enabled. =head1 BUGS