[ViewVC] Diff of: cvs/AnyEvent/lib/AnyEvent.pm

Comparing AnyEvent/lib/AnyEvent.pm (file contents):
Revision 1.208 by root, Sun Apr 26 18:12:53 2009 UTC vs.
Revision 1.228 by root, Wed Jul 8 01:11:12 2009 UTC

 =head1 NAME
 AnyEvent - provide framework for multiple event loops
-EV, Event, Glib, Tk, Perl, Event::Lib, Qt, POE - various supported event loops
+EV, Event, Glib, Tk, Perl, Event::Lib, Qt and POE are various supported
+event loops.
 =head1 SYNOPSIS
    use AnyEvent;
 =head2 I/O WATCHERS
 You can create an I/O watcher by calling the C<< AnyEvent->io >> method
 with the following mandatory key-value pairs as arguments:
-C<fh> is the Perl I<file handle> (I<not> file descriptor) to watch
+C<fh> is the Perl I<file handle> (I<not> file descriptor, see below) to
-for events (AnyEvent might or might not keep a reference to this file
+watch for events (AnyEvent might or might not keep a reference to this
-handle). Note that only file handles pointing to things for which
+file handle). Note that only file handles pointing to things for which
 non-blocking operation makes sense are allowed. This includes sockets,
 most character devices, pipes, fifos and so on, but not for example files
 or block devices.
 C<poll> must be a string that is either C<r> or C<w>, which creates a
       chomp (my $input = <STDIN>);
       warn "read: $input\n";
       undef $w;
    });
+=head3 GETTING A FILE HANDLE FROM A FILE DESCRIPTOR
+It is not uncommon to only have a file descriptor, while AnyEvent requires
+a Perl file handle.
+There are basically two methods to convert a file descriptor into a file handle. If you own
+the file descriptor, you can open it with C<&=>, as in:
+   open my $fh, "<&=$fileno" or die "xxx: §!";
+This will "own" the file descriptor, meaning that when C<$fh> is
+destroyed, it will automatically close the C<$fileno>. Also, note that
+the open mode (read, write, read/write) must correspond with how the
+underlying file descriptor was opened.
+In many cases, taking over the file descriptor is now what you want, in
+which case the only alternative is to dup the file descriptor:
+   open my $fh, "<&$fileno" or die "xxx: $!";
+This has the advantage of not closing the file descriptor and the
+disadvantage of making a slow copy.
 =head2 TIME WATCHERS
 You can create a time watcher by calling the C<< AnyEvent->timer >>
 method with the following mandatory arguments:
 There is a slight catch to child watchers, however: you usually start them
 I<after> the child process was created, and this means the process could
 have exited already (and no SIGCHLD will be sent anymore).
-Not all event models handle this correctly (POE doesn't), but even for
+Not all event models handle this correctly (neither POE nor IO::Async do,
+see their AnyEvent::Impl manpages for details), but even for event models
-event models that I<do> handle this correctly, they usually need to be
+that I<do> handle this correctly, they usually need to be loaded before
-loaded before the process exits (i.e. before you fork in the first place).
+the process exits (i.e. before you fork in the first place). AnyEvent's
+pure perl event loop handles all cases correctly regardless of when you
+start the watcher.
-This means you cannot create a child watcher as the very first thing in an
+This means you cannot create a child watcher as the very first
-AnyEvent program, you I<have> to create at least one watcher before you
+thing in an AnyEvent program, you I<have> to create at least one
-C<fork> the child (alternatively, you can call C<AnyEvent::detect>).
+watcher before you C<fork> the child (alternatively, you can call
+C<AnyEvent::detect>).
 Example: fork a process and wait for it
    my $done = AnyEvent->condvar;
 =item $cv->begin ([group callback])
 =item $cv->end
-These two methods are EXPERIMENTAL and MIGHT CHANGE.
 These two methods can be used to combine many transactions/events into
 one. For example, a function that pings many hosts in parallel might want
 to use a condition variable for the whole process.
 Every call to C<< ->begin >> will increment a counter, and every call to
 C<< ->end >> will decrement it.  If the counter reaches C<0> in C<< ->end
 >>, the (last) callback passed to C<begin> will be executed. That callback
 is I<supposed> to call C<< ->send >>, but that is not required. If no
 callback was set, C<send> will be called without any arguments.
-Let's clarify this with the ping example:
+You can think of C<< $cv->send >> giving you an OR condition (one call
+sends), while C<< $cv->begin >> and C<< $cv->end >> giving you an AND
+condition (all C<begin> calls must be C<end>'ed before the condvar sends).
+Let's start with a simple example: you have two I/O watchers (for example,
+STDOUT and STDERR for a program), and you want to wait for both streams to
+close before activating a condvar:
+   my $cv = AnyEvent->condvar;
+   $cv->begin; # first watcher
+   my $w1 = AnyEvent->io (fh => $fh1, cb => sub {
+      defined sysread $fh1, my $buf, 4096
+         or $cv->end;
+   });
+   $cv->begin; # second watcher
+   my $w2 = AnyEvent->io (fh => $fh2, cb => sub {
+      defined sysread $fh2, my $buf, 4096
+         or $cv->end;
+   });
+   $cv->recv;
+This works because for every event source (EOF on file handle), there is
+one call to C<begin>, so the condvar waits for all calls to C<end> before
+sending.
+The ping example mentioned above is slightly more complicated, as the
+there are results to be passwd back, and the number of tasks that are
+begung can potentially be zero:
    my $cv = AnyEvent->condvar;
    my %result;
    $cv->begin (sub { $cv->send (\%result) });
 loop, which serves two important purposes: first, it sets the callback
 to be called once the counter reaches C<0>, and second, it ensures that
 C<send> is called even when C<no> hosts are being pinged (the loop
 doesn't execute once).
-This is the general pattern when you "fan out" into multiple subrequests:
+This is the general pattern when you "fan out" into multiple (but
-use an outer C<begin>/C<end> pair to set the callback and ensure C<end>
+potentially none) subrequests: use an outer C<begin>/C<end> pair to set
-is called at least once, and then, for each subrequest you start, call
+the callback and ensure C<end> is called at least once, and then, for each
-C<begin> and for each subrequest you finish, call C<end>.
+subrequest you start, call C<begin> and for each subrequest you finish,
+call C<end>.
 =back
 =head3 METHODS FOR CONSUMERS
    AnyEvent::Impl::Tk        based on Tk, very bad choice.
    AnyEvent::Impl::Qt        based on Qt, cannot be autoprobed (see its docs).
    AnyEvent::Impl::EventLib  based on Event::Lib, leaks memory and worse.
    AnyEvent::Impl::POE       based on POE, not generic enough for full support.
+   # warning, support for IO::Async is only partial, as it is too broken
+   # and limited toe ven support the AnyEvent API. See AnyEvent::Impl::Async.
+   AnyEvent::Impl::IOAsync   based on IO::Async, cannot be autoprobed (see its docs).
 There is no support for WxWidgets, as WxWidgets has no support for
 watching file handles. However, you can use WxWidgets through the
 POE Adaptor, as POE has a Wx backend that simply polls 20 times per
 second, which was considered to be too horrible to even consider for
 AnyEvent. Likewise, other POE backends can be used by AnyEvent by using
 no warnings;
 use strict qw(vars subs);
 use Carp;
-our $VERSION = 4.4;
+our $VERSION = 4.8;
 our $MODEL;
 our $AUTOLOAD;
 our @ISA;
 our @REGISTRY;
 our $WIN32;
 BEGIN {
-   my $win32 = ! ! ($^O =~ /mswin32/i);
+   eval "sub WIN32(){ " . (($^O =~ /mswin32/i)*1) ." }";
-   eval "sub WIN32(){ $win32 }";
+   eval "sub TAINT(){ " . (${^TAINT}*1) . " }";
+   delete @ENV{grep /^PERL_ANYEVENT_/, keys %ENV}
+      if ${^TAINT};
 }
 our $verbose = $ENV{PERL_ANYEVENT_VERBOSE}*1;
 our %PROTOCOL; # (ipv4|ipv6) => (1|2), higher numbers are preferred
    [Event::Lib::           => AnyEvent::Impl::EventLib::], # too buggy
    [Qt::                   => AnyEvent::Impl::Qt::],       # requires special main program
    [POE::Kernel::          => AnyEvent::Impl::POE::],      # lasciate ogni speranza
    [Wx::                   => AnyEvent::Impl::POE::],
    [Prima::                => AnyEvent::Impl::POE::],
+   # IO::Async is just too broken - we would need workaorunds for its
+   # byzantine signal and broken child handling, among others.
+   # IO::Async is rather hard to detect, as it doesn't have any
+   # obvious default class.
+#   [IO::Async::            => AnyEvent::Impl::IOAsync::],  # requires special main program
+#   [IO::Async::Loop::      => AnyEvent::Impl::IOAsync::],  # requires special main program
+#   [IO::Async::Notifier::  => AnyEvent::Impl::IOAsync::],  # requires special main program
 );
 our %method = map +($_ => 1),
    qw(io timer time now now_update signal child idle condvar one_event DESTROY);
 }
 # utility function to dup a filehandle. this is used by many backends
 # to support binding more than one watcher per filehandle (they usually
 # allow only one watcher per fd, so we dup it to get a different one).
-sub _dupfh($$$$) {
+sub _dupfh($$;$$) {
    my ($poll, $fh, $r, $w) = @_;
    # cygwin requires the fh mode to be matching, unix doesn't
    my ($rw, $mode) = $poll eq "r" ? ($r, "<")
                    : $poll eq "w" ? ($w, ">")
          AnyEvent::Util::fh_nonblocking ($SIGPIPE_W) if $SIGPIPE_W; # just in case
       } else {
          pipe $SIGPIPE_R, $SIGPIPE_W;
          fcntl $SIGPIPE_R, &Fcntl::F_SETFL, &Fcntl::O_NONBLOCK if $SIGPIPE_R;
          fcntl $SIGPIPE_W, &Fcntl::F_SETFL, &Fcntl::O_NONBLOCK if $SIGPIPE_W; # just in case
+         # not strictly required, as $^F is normally 2, but let's make sure...
+         fcntl $SIGPIPE_R, &Fcntl::F_SETFD, &Fcntl::FD_CLOEXEC;
+         fcntl $SIGPIPE_W, &Fcntl::F_SETFD, &Fcntl::FD_CLOEXEC;
       }
       $SIGPIPE_R
          or Carp::croak "AnyEvent: unable to create a signal reporting pipe: $!\n";
-      # not strictly required, as $^F is normally 2, but let's make sure...
-      fcntl $SIGPIPE_R, &Fcntl::F_SETFD, &Fcntl::FD_CLOEXEC;
-      fcntl $SIGPIPE_W, &Fcntl::F_SETFD, &Fcntl::FD_CLOEXEC;
       $SIG_IO = AnyEvent->io (fh => $SIGPIPE_R, poll => "r", cb => \&_signal_exec);
    }
    my $signal = uc $arg{signal}
 sub AnyEvent::Base::signal::DESTROY {
    my ($signal, $cb) = @{$_[0]};
    delete $SIG_CB{$signal}{$cb};
+   # delete doesn't work with older perls - they then
+   # print weird messages, or just unconditionally exit
+   # instead of getting the default action.
-   delete $SIG{$signal} unless keys %{ $SIG_CB{$signal} };
+   undef $SIG{$signal} unless keys %{ $SIG_CB{$signal} };
 }
 # default implementation for ->child
 our %PID_CB;
 our $CHLD_W;
 our $CHLD_DELAY_W;
-our $PID_IDLE;
 our $WNOHANG;
-sub _child_wait {
+sub _sigchld {
    while (0 < (my $pid = waitpid -1, $WNOHANG)) {
       $_->($pid, $?) for (values %{ $PID_CB{$pid} || {} }),
                          (values %{ $PID_CB{0}    || {} });
    }
-   undef $PID_IDLE;
-}
-sub _sigchld {
-   # make sure we deliver these changes "synchronous" with the event loop.
-   $CHLD_DELAY_W ||= AnyEvent->timer (after => 0, cb => sub {
-      undef $CHLD_DELAY_W;
-      &_child_wait;
-   });
 }
 sub child {
    my (undef, %arg) = @_;
    defined (my $pid = $arg{pid} + 0)
       or Carp::croak "required option 'pid' is missing";
    $PID_CB{$pid}{$arg{cb}} = $arg{cb};
-   unless ($WNOHANG) {
-      $WNOHANG = eval { local $SIG{__DIE__}; require POSIX; &POSIX::WNOHANG } || 1;
+   $WNOHANG ||= eval { local $SIG{__DIE__}; require POSIX; &POSIX::WNOHANG } || 1;
-   }
    unless ($CHLD_W) {
       $CHLD_W = AnyEvent->signal (signal => 'CHLD', cb => \&_sigchld);
       # child could be a zombie already, so make at least one round
       &_sigchld;
    undef $CHLD_W unless keys %PID_CB;
 }
 # idle emulation is done by simply using a timer, regardless
-# of whether the proces sis idle or not, and not letting
+# of whether the process is idle or not, and not letting
 # the callback use more than 50% of the time.
 sub idle {
    my (undef, %arg) = @_;
    my ($cb, $w, $rcb) = $arg{cb};
 so on.
 =head1 ENVIRONMENT VARIABLES
 The following environment variables are used by this module or its
-submodules:
+submodules.
+Note that AnyEvent will remove I<all> environment variables starting with
+C<PERL_ANYEVENT_> from C<%ENV> when it is loaded while taint mode is
+enabled.
 =over 4
 =item C<PERL_ANYEVENT_VERBOSE>
 =item C<PERL_ANYEVENT_STRICT>
 AnyEvent does not do much argument checking by default, as thorough
 argument checking is very costly. Setting this variable to a true value
 will cause AnyEvent to load C<AnyEvent::Strict> and then to thoroughly
-check the arguments passed to most method calls. If it finds any problems
+check the arguments passed to most method calls. If it finds any problems,
 it will croak.
 In other words, enables "strict" mode.
-Unlike C<use strict>, it is definitely recommended ot keep it off in
+Unlike C<use strict>, it is definitely recommended to keep it off in
 production. Keeping C<PERL_ANYEVENT_STRICT=1> in your environment while
 developing programs can be very useful, however.
 =item C<PERL_ANYEVENT_MODEL>
 =item C<PERL_ANYEVENT_MAX_FORKS>
 The maximum number of child processes that C<AnyEvent::Util::fork_call>
 will create in parallel.
+=item C<PERL_ANYEVENT_MAX_OUTSTANDING_DNS>
+The default value for the C<max_outstanding> parameter for the default DNS
+resolver - this is the maximum number of parallel DNS requests that are
+sent to the DNS server.
+=item C<PERL_ANYEVENT_RESOLV_CONF>
+The file to use instead of F</etc/resolv.conf> (or OS-specific
+configuration) in the default resolver. When set to the empty string, no
+default config will be used.
+=item C<PERL_ANYEVENT_CA_FILE>, C<PERL_ANYEVENT_CA_PATH>.
+When neither C<ca_file> nor C<ca_path> was specified during
+L<AnyEvent::TLS> context creation, and either of these environment
+variables exist, they will be used to specify CA certificate locations
+instead of a system-dependent default.
 =back
 =head1 SUPPLYING YOUR OWN EVENT MODEL INTERFACE
         EV/Any   100000   224   2.88   0.34    0.27 EV + AnyEvent watchers
     CoroEV/Any   100000   224   2.85   0.35    0.28 coroutines + Coro::Signal
       Perl/Any   100000   452   4.13   0.73    0.95 pure perl implementation
    Event/Event    16000   517  32.20  31.80    0.81 Event native interface
      Event/Any    16000   590  35.85  31.55    1.06 Event + AnyEvent watchers
+   IOAsync/Any    16000   989  38.10  32.77   11.13 via IO::Async::Loop::IO_Poll
+   IOAsync/Any    16000   990  37.59  29.50   10.61 via IO::Async::Loop::Epoll
       Glib/Any    16000  1357 102.33  12.31   51.00 quadratic behaviour
         Tk/Any     2000  1860  27.20  66.31   14.00 SEGV with >> 2000 watchers
      POE/Event     2000  6328 109.99 751.67   14.02 via POE::Loop::Event
     POE/Select     2000  6027  94.54 809.13  579.80 via POE::Loop::Select
 performance becomes really bad with lots of file descriptors (and few of
 them active), of course, but this was not subject of this benchmark.
 The C<Event> module has a relatively high setup and callback invocation
 cost, but overall scores in on the third place.
+C<IO::Async> performs admirably well, about on par with C<Event>, even
+when using its pure perl backend.
 C<Glib>'s memory usage is quite a bit higher, but it features a
 faster callback invocation and overall ends up in the same class as
 C<Event>. However, Glib scales extremely badly, doubling the number of
 watchers increases the processing time by more than a factor of four,
 it to another server. This includes deleting the old timeout and creating
 a new one that moves the timeout into the future.
 =head3 Results
-    name sockets create  request
+     name sockets create  request
-      EV   20000  69.01    11.16
+       EV   20000  69.01    11.16
-    Perl   20000  73.32    35.87
+     Perl   20000  73.32    35.87
+  IOAsync   20000 157.00    98.14 epoll
+  IOAsync   20000 159.31   616.06 poll
-   Event   20000 212.62   257.32
+    Event   20000 212.62   257.32
-    Glib   20000 651.16  1896.30
+     Glib   20000 651.16  1896.30
-     POE   20000 349.67 12317.24 uses POE::Loop::Event
+      POE   20000 349.67 12317.24 uses POE::Loop::Event
 =head3 Discussion
 This benchmark I<does> measure scalability and overall performance of the
 particular event loop.
 EV is again fastest. Since it is using epoll on my system, the setup time
 is relatively high, though.
 Perl surprisingly comes second. It is much faster than the C-based event
 loops Event and Glib.
+IO::Async performs very well when using its epoll backend, and still quite
+good compared to Glib when using its pure perl backend.
 Event suffers from high setup time as well (look at its code and you will
 understand why). Callback invocation also has a high overhead compared to
 the C<< $_->() for .. >>-style loop that the Perl event loop uses. Event
 uses select or poll in basically all documented configurations.
 =item * C-based event loops perform very well with small number of
 watchers, as the management overhead dominates.
 =back
+=head2 THE IO::Lambda BENCHMARK
+Recently I was told about the benchmark in the IO::Lambda manpage, which
+could be misinterpreted to make AnyEvent look bad. In fact, the benchmark
+simply compares IO::Lambda with POE, and IO::Lambda looks better (which
+shouldn't come as a surprise to anybody). As such, the benchmark is
+fine, and mostly shows that the AnyEvent backend from IO::Lambda isn't
+very optimal. But how would AnyEvent compare when used without the extra
+baggage? To explore this, I wrote the equivalent benchmark for AnyEvent.
+The benchmark itself creates an echo-server, and then, for 500 times,
+connects to the echo server, sends a line, waits for the reply, and then
+creates the next connection. This is a rather bad benchmark, as it doesn't
+test the efficiency of the framework or much non-blocking I/O, but it is a
+benchmark nevertheless.
+   name                    runtime
+   Lambda/select           0.330 sec
+      + optimized          0.122 sec
+   Lambda/AnyEvent         0.327 sec
+      + optimized          0.138 sec
+   Raw sockets/select      0.077 sec
+   POE/select, components  0.662 sec
+   POE/select, raw sockets 0.226 sec
+   POE/select, optimized   0.404 sec
+   AnyEvent/select/nb      0.085 sec
+   AnyEvent/EV/nb          0.068 sec
+      +state machine       0.134 sec
+The benchmark is also a bit unfair (my fault): the IO::Lambda/POE
+benchmarks actually make blocking connects and use 100% blocking I/O,
+defeating the purpose of an event-based solution. All of the newly
+written AnyEvent benchmarks use 100% non-blocking connects (using
+AnyEvent::Socket::tcp_connect and the asynchronous pure perl DNS
+resolver), so AnyEvent is at a disadvantage here, as non-blocking connects
+generally require a lot more bookkeeping and event handling than blocking
+connects (which involve a single syscall only).
+The last AnyEvent benchmark additionally uses L<AnyEvent::Handle>, which
+offers similar expressive power as POE and IO::Lambda, using conventional
+Perl syntax. This means that both the echo server and the client are 100%
+non-blocking, further placing it at a disadvantage.
+As you can see, the AnyEvent + EV combination even beats the
+hand-optimised "raw sockets benchmark", while AnyEvent + its pure perl
+backend easily beats IO::Lambda and POE.
+And even the 100% non-blocking version written using the high-level (and
+slow :) L<AnyEvent::Handle> abstraction beats both POE and IO::Lambda by a
+large margin, even though it does all of DNS, tcp-connect and socket I/O
+in a non-blocking way.
+The two AnyEvent benchmarks programs can be found as F<eg/ae0.pl> and
+F<eg/ae2.pl> in the AnyEvent distribution, the remaining benchmarks are
+part of the IO::lambda distribution and were used without any changes.
 =head1 SIGNALS
 AnyEvent currently installs handlers for these signals:
 =item SIGCHLD
 A handler for C<SIGCHLD> is installed by AnyEvent's child watcher
 emulation for event loops that do not support them natively. Also, some
 event loops install a similar handler.
+If, when AnyEvent is loaded, SIGCHLD is set to IGNORE, then AnyEvent will
+reset it to default, to avoid losing child exit statuses.
 =item SIGPIPE
 A no-op handler is installed for C<SIGPIPE> when C<$SIG{PIPE}> is C<undef>
 when AnyEvent gets loaded.
 =back
 =cut
+undef $SIG{CHLD}
+   if $SIG{CHLD} eq 'IGNORE';
 $SIG{PIPE} = sub { }
    unless defined $SIG{PIPE};
 =head1 FORK
 Most event libraries are not fork-safe. The ones who are usually are
 because they rely on inefficient but fork-safe C<select> or C<poll>
    use AnyEvent;
 Similar considerations apply to $ENV{PERL_ANYEVENT_VERBOSE}, as that can
 be used to probe what backend is used and gain other information (which is
 probably even less useful to an attacker than PERL_ANYEVENT_MODEL), and
-$ENV{PERL_ANYEGENT_STRICT}.
+$ENV{PERL_ANYEVENT_STRICT}.
+Note that AnyEvent will remove I<all> environment variables starting with
+C<PERL_ANYEVENT_> from C<%ENV> when it is loaded while taint mode is
+enabled.
 =head1 BUGS
 Perl 5.8 has numerous memleaks that sometimes hit this module and are hard

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent/lib/AnyEvent.pm (file contents): Revision 1.208 by root, Sun Apr 26 18:12:53 2009 UTC vs. Revision 1.228 by root, Wed Jul 8 01:11:12 2009 UTC

Diff Legend

Comparing AnyEvent/lib/AnyEvent.pm (file contents):
Revision 1.208 by root, Sun Apr 26 18:12:53 2009 UTC vs.
Revision 1.228 by root, Wed Jul 8 01:11:12 2009 UTC