[ViewVC] Diff of: cvs/AnyEvent-Fork/Fork.pm

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.9 by root, Thu Apr 4 03:45:12 2013 UTC vs.
Revision 1.23 by root, Sat Apr 6 08:29:43 2013 UTC

 =head1 NAME
 AnyEvent::Fork - everything you wanted to use fork() for, but couldn't
-ATTENTION, this is a very early release, and very untested. Consider it a
-technology preview.
 =head1 SYNOPSIS
    use AnyEvent::Fork;
       while (my $socket = $listener->accept) {
          # do sth. with new socket
       }
    }
+   ##################################################################
+   # use AnyEvent::Fork as a faster fork+exec
+   # this runs /bin/echo hi, with stdout redirected to /tmp/log
+   # and stderr to the communications socket. it is usually faster
+   # than fork+exec, but still let's you prepare the environment.
+   open my $output, ">/tmp/log" or die "$!";
+   AnyEvent::Fork
+      ->new
+      ->eval ('
+           sub run {
+              my ($fh, $output, @cmd) = @_;
+              # perl will clear close-on-exec on STDOUT/STDERR
+              open STDOUT, ">&", $output or die;
+              open STDERR, ">&", $fh or die;
+              exec @cmd;
+           }
+        ')
+      ->send_fh ($output)
+      ->send_arg ("/bin/echo", "hi")
+      ->run ("run", my $cv = AE::cv);
+   my $stderr = $cv->recv;
 =head1 DESCRIPTION
 This module allows you to create new processes, without actually forking
 them from your current process (avoiding the problems of forking), but
 preserving most of the advantages of fork.
 It can be used to create new worker processes or new independent
 subprocesses for short- and long-running jobs, process pools (e.g. for use
 in pre-forked servers) but also to spawn new external processes (such as
-CGI scripts from a webserver), which can be faster (and more well behaved)
+CGI scripts from a web server), which can be faster (and more well behaved)
 than using fork+exec in big processes.
 Special care has been taken to make this module useful from other modules,
 while still supporting specialised environments such as L<App::Staticperl>
 or L<PAR::Packer>.
+=head1 WHAT THIS MODULE IS NOT
+This module only creates processes and lets you pass file handles and
+strings to it, and run perl code. It does not implement any kind of RPC -
+there is no back channel from the process back to you, and there is no RPC
+or message passing going on.
+If you need some form of RPC, you can either implement it yourself
+in whatever way you like, use some message-passing module such
+as L<AnyEvent::MP>, some pipe such as L<AnyEvent::ZeroMQ>, use
+L<AnyEvent::Handle> on both sides to send e.g. JSON or Storable messages,
+and so on.
 =head1 PROBLEM STATEMENT
 There are two ways to implement parallel processing on UNIX like operating
 systems - fork and process, and fork+exec and process. They have different
 or fork+exec instead.
 =item Forking usually creates a copy-on-write copy of the parent
 process. Memory (for example, modules or data files that have been
 will not take additional memory). When exec'ing a new process, modules
-and data files might need to be loaded again, at extra cpu and memory
+and data files might need to be loaded again, at extra CPU and memory
 cost. Likewise when forking, all data structures are copied as well - if
 the program frees them and replaces them by new data, the child processes
 will retain the memory even if it isn't used.
 This module allows the main program to do a controlled fork, and allows
 as template, and also tries hard to identify the correct path to the perl
 interpreter. With a cooperative main program, exec'ing the interpreter
 might not even be necessary.
 =item Forking might be impossible when a program is running. For example,
-POSIX makes it almost impossible to fork from a multithreaded program and
+POSIX makes it almost impossible to fork from a multi-threaded program and
 do anything useful in the child - strictly speaking, if your perl program
 uses posix threads (even indirectly via e.g. L<IO::AIO> or L<threads>),
 you cannot call fork on the perl level anymore, at all.
-This module can safely fork helper processes at any time, by caling
+This module can safely fork helper processes at any time, by calling
 fork+exec in C, in a POSIX-compatible way.
 =item Parallel processing with fork might be inconvenient or difficult
 to implement. For example, when a program uses an event loop and creates
 watchers it becomes very hard to use the event loop from a child
 needed the first time. Forking from this process shares the memory used
 for the perl interpreter with the new process, but loading modules takes
 time, and the memory is not shared with anything else.
 This is ideal for when you only need one extra process of a kind, with the
-option of starting and stipping it on demand.
+option of starting and stopping it on demand.
 Example:
    AnyEvent::Fork
       ->new
 modules you loaded) is shared between the processes, and each new process
 consumes relatively little memory of its own.
 The disadvantage of this approach is that you need to create a template
 process for the sole purpose of forking new processes from it, but if you
-only need a fixed number of proceses you can create them, and then destroy
+only need a fixed number of processes you can create them, and then destroy
 the template process.
 Example:
    my $template = AnyEvent::Fork->new->require ("Some::Module");
 package AnyEvent::Fork;
 use common::sense;
-use Socket ();
+use Errno ();
 use AnyEvent;
-use AnyEvent::Fork::Util;
 use AnyEvent::Util ();
+use IO::FDPass;
+our $VERSION = 0.5;
 our $PERL; # the path to the perl interpreter, deduces with various forms of magic
 =item my $pool = new AnyEvent::Fork key => value...
 our $TEMPLATE;
 sub _cmd {
    my $self = shift;
-   #TODO: maybe append the packet to any existing string command already in the queue
-   # ideally, we would want to use "a (w/a)*" as format string, but perl versions
+   # ideally, we would want to use "a (w/a)*" as format string, but perl
-   # from at least 5.8.9 to 5.16.3 are all buggy and can't unpack it.
+   # versions from at least 5.8.9 to 5.16.3 are all buggy and can't unpack
-   push @{ $self->[2] }, pack "N/a", pack "(w/a)*", @_;
+   # it.
+   push @{ $self->[2] }, pack "a L/a*", $_[0], $_[1];
    $self->[3] ||= AE::io $self->[1], 1, sub {
+      do {
-      # send the next "thing" in the queue - either a reference to an fh,
+         # send the next "thing" in the queue - either a reference to an fh,
-      # or a plain string.
+         # or a plain string.
-      if (ref $self->[2][0]) {
+         if (ref $self->[2][0]) {
-         # send fh
+            # send fh
-         AnyEvent::Fork::Util::fd_send fileno $self->[1], fileno ${ $self->[2][0] }
+            unless (IO::FDPass::send fileno $self->[1], fileno ${ $self->[2][0] }) {
+               return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
+               undef $self->[3];
+               die "AnyEvent::Fork: file descriptor send failure: $!";
+            }
-            and shift @{ $self->[2] };
+            shift @{ $self->[2] };
-      } else {
+         } else {
-         # send string
+            # send string
-         my $len = syswrite $self->[1], $self->[2][0]
+            my $len = syswrite $self->[1], $self->[2][0];
+            unless ($len) {
+               return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
+               undef $self->[3];
-            or do { undef $self->[3]; die "AnyEvent::Fork: command write failure: $!" };
+               die "AnyEvent::Fork: command write failure: $!";
+            }
-         substr $self->[2][0], 0, $len, "";
+            substr $self->[2][0], 0, $len, "";
-         shift @{ $self->[2] } unless length $self->[2][0];
+            shift @{ $self->[2] } unless length $self->[2][0];
-      }
+         }
+      } while @{ $self->[2] };
-      unless (@{ $self->[2] }) {
+      # everything written
-         undef $self->[3];
+      undef $self->[3];
-         # invoke run callback
+      # invoke run callback, if any
-         $self->[0]->($self->[1]) if $self->[0];
+      $self->[4]->($self->[1]) if $self->[4];
-      }
    };
+   () # make sure we don't leak the watcher
 }
 sub _new {
-   my ($self, $fh) = @_;
+   my ($self, $fh, $pid) = @_;
    AnyEvent::Util::fh_nonblocking $fh, 1;
    $self = bless [
-      undef, # run callback
+      $pid,
       $fh,
       [],    # write queue - strings or fd's
       undef, # AE watcher
    ], $self;
    if ($pid eq 0) {
       require AnyEvent::Fork::Serve;
       $AnyEvent::Fork::Serve::OWNER = $parent;
       close $fh;
       $0 = "$_[1] of $parent";
+      $SIG{CHLD} = 'IGNORE';
       AnyEvent::Fork::Serve::serve ($slave);
-      AnyEvent::Fork::Util::_exit 0;
+      exit 0;
    } elsif (!$pid) {
       die "AnyEvent::Fork::Early/Template: unable to fork template process: $!";
    }
-   AnyEvent::Fork->_new ($fh)
+   AnyEvent::Fork->_new ($fh, $pid)
 }
 =item my $proc = new AnyEvent::Fork
 Create a new "empty" perl interpreter process and returns its process
 reduces the amount of memory sharing that is possible, and is also slower.
 You should use C<new> whenever possible, except when having a template
 process around is unacceptable.
-The path to the perl interpreter is divined usign various methods - first
+The path to the perl interpreter is divined using various methods - first
 C<$^X> is investigated to see if the path ends with something that sounds
 as if it were the perl interpreter. Failing this, the module falls back to
 using C<$Config::Config{perlpath}>.
 =cut
    my $perl = $;
    # first we try $^X, but the path must be absolute (always on win32), and end in sth.
    # that looks like perl. this obviously only works for posix and win32
    unless (
-      (AnyEvent::Fork::Util::WIN32 || $perl =~ m%^/%)
+      ($^O eq "MSWin32" || $perl =~ m%^/%)
       && $perl =~ m%[/\\]perl(?:[0-9]+(\.[0-9]+)+)?(\.exe)?$%i
    ) {
       # if it doesn't look perlish enough, try Config
       require Config;
       $perl = $Config::Config{perlpath};
    require Proc::FastSpawn;
    my ($fh, $slave) = AnyEvent::Util::portable_socketpair;
    Proc::FastSpawn::fd_inherit (fileno $slave);
+   # new fh's should always be set cloexec (due to $^F),
+   # but hey, not on win32, so we always clear the inherit flag.
+   Proc::FastSpawn::fd_inherit (fileno $fh, 0);
    # quick. also doesn't work in win32. of course. what did you expect
    #local $ENV{PERL5LIB} = join ":", grep !ref, @INC;
    my %env = %ENV;
-   $env{PERL5LIB} = join +(AnyEvent::Fork::Util::WIN32 ? ";" : ":"), grep !ref, @INC;
+   $env{PERL5LIB} = join +($^O eq "MSWin32" ? ";" : ":"), grep !ref, @INC;
-   Proc::FastSpawn::spawn (
+   my $pid = Proc::FastSpawn::spawn (
       $perl,
       ["perl", "-MAnyEvent::Fork::Serve", "-e", "AnyEvent::Fork::Serve::me", fileno $slave, $$],
       [map "$_=$env{$_}", keys %env],
    ) or die "unable to spawn AnyEvent::Fork server: $!";
-   $self->_new ($fh)
+   $self->_new ($fh, $pid)
+}
+=item $pid = $proc->pid
+Returns the process id of the process I<iff it is a direct child of the
+process> running AnyEvent::Fork, and C<undef> otherwise.
+Normally, only processes created via C<< AnyEvent::Fork->new_exec >> and
+L<AnyEvent::Fork::Template> are direct children, and you are responsible
+to clean up their zombies when they die.
+All other processes are not direct children, and will be cleaned up by
+AnyEvent::Fork.
+=cut
+sub pid {
+   $_[0][0]
 }
 =item $proc = $proc->eval ($perlcode, @args)
 Evaluates the given C<$perlcode> as ... perl code, while setting C<@_> to
-the strings specified by C<@args>.
+the strings specified by C<@args>, in the "main" package.
 This call is meant to do any custom initialisation that might be required
 (for example, the C<require> method uses it). It's not supposed to be used
 to completely take over the process, use C<run> for that.
 The code will usually be executed after this call returns, and there is no
 way to pass anything back to the calling process. Any evaluation errors
 will be reported to stderr and cause the process to exit.
+If you want to execute some code to take over the process (see the
+"fork+exec" example in the SYNOPSIS), you should compile a function via
+C<eval> first, and then call it via C<run>. This also gives you access to
+any arguments passed via the C<send_xxx> methods, such as file handles.
 Returns the process object for easy chaining of method calls.
 =cut
 sub eval {
    my ($self, $code, @args) = @_;
-   $self->_cmd (e => $code, @args);
+   $self->_cmd (e => pack "(w/a*)*", $code, @args);
    $self
 }
 =item $proc = $proc->require ($module, ...)
 accomplished by simply not storing the file handles anywhere after passing
 them to this method.
 Returns the process object for easy chaining of method calls.
-Example: pass an fh to a process, and release it without closing. it will
+Example: pass a file handle to a process, and release it without
-be closed automatically when it is no longer used.
+closing. It will be closed automatically when it is no longer used.
    $proc->send_fh ($my_fh);
    undef $my_fh; # free the reference if you want, but DO NOT CLOSE IT
 =cut
 =item $proc = $proc->send_arg ($string, ...)
 Send one or more argument strings to the process, to prepare a call to
 C<run>. The strings can be any octet string.
+The protocol is optimised to pass a moderate number of relatively short
+strings - while you can pass up to 4GB of data in one go, this is more
+meant to pass some ID information or other startup info, not big chunks of
+data.
-Returns the process object for easy chaining of emthod calls.
+Returns the process object for easy chaining of method calls.
 =cut
 sub send_arg {
    my ($self, @arg) = @_;
-   $self->_cmd (a => @arg);
+   $self->_cmd (a => pack "(w/a*)*", @arg);
    $self
 }
 =item $proc->run ($func, $cb->($fh))
-Enter the function specified by the fully qualified name in C<$func> in
+Enter the function specified by the function name in C<$func> in the
-the process. The function is called with the communication socket as first
+process. The function is called with the communication socket as first
 argument, followed by all file handles and string arguments sent earlier
 via C<send_fh> and C<send_arg> methods, in the order they were called.
-If the called function returns, the process exits.
+The function name should be fully qualified, but if it isn't, it will be
+looked up in the main package.
-Preparing the process can take time - when the process is ready, the
+If the called function returns, doesn't exist, or any error occurs, the
+process exits.
+Preparing the process is done in the background - when all commands have
-callback is invoked with the local communications socket as argument.
+been sent, the callback is invoked with the local communications socket
+as argument. At this point you can start using the socket in any way you
+like.
-The process object becomes unusable on return from this function.
+The process object becomes unusable on return from this function - any
+further method calls result in undefined behaviour.
 If the communication socket isn't used, it should be closed on both sides,
 to save on kernel memory.
 The socket is non-blocking in the parent, and blocking in the newly
-created process. The close-on-exec flag is set on both. Even if not used
+created process. The close-on-exec flag is set in both.
-otherwise, the socket can be a good indicator for the existance of the
+Even if not used otherwise, the socket can be a good indicator for the
-process - if the other process exits, you get a readable event on it,
+existence of the process - if the other process exits, you get a readable
-because exiting the process closes the socket (if it didn't create any
+event on it, because exiting the process closes the socket (if it didn't
-children using fork).
+create any children using fork).
 Example: create a template for a process pool, pass a few strings, some
 file handles, then fork, pass one more string, and run some code.
    my $pool = AnyEvent::Fork
          ->send_arg ("str3")
          ->run ("Some::function", sub {
             my ($fh) = @_;
             # fh is nonblocking, but we trust that the OS can accept these
-            # extra 3 octets anyway.
+            # few octets anyway.
             syswrite $fh, "hi #$_\n";
             # $fh is being closed here, as we don't store it anywhere
          });
    }
    # Some::function might look like this - all parameters passed before fork
    # and after will be passed, in order, after the communications socket.
    sub Some::function {
       my ($fh, $str1, $str2, $fh1, $fh2, $str3) = @_;
-      print scalar <$fh>; # prints "hi 1\n" and "hi 2\n"
+      print scalar <$fh>; # prints "hi #1\n" and "hi #2\n" in any order
    }
 =cut
 sub run {
    my ($self, $func, $cb) = @_;
-   $self->[0] = $cb;
+   $self->[4] = $cb;
    $self->_cmd (r => $func);
 }
 =back
+=head1 PERFORMANCE
+Now for some unscientific benchmark numbers (all done on an amd64
+GNU/Linux box). These are intended to give you an idea of the relative
+performance you can expect, they are not meant to be absolute performance
+numbers.
+OK, so, I ran a simple benchmark that creates a socket pair, forks, calls
+exit in the child and waits for the socket to close in the parent. I did
+load AnyEvent, EV and AnyEvent::Fork, for a total process size of 5100kB.
+   2079 new processes per second, using manual socketpair + fork
+Then I did the same thing, but instead of calling fork, I called
+AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the
+socket form the child to close on exit. This does the same thing as manual
+socket pair + fork, except that what is forked is the template process
+(2440kB), and the socket needs to be passed to the server at the other end
+of the socket first.
+   2307 new processes per second, using AnyEvent::Fork->new
+And finally, using C<new_exec> instead C<new>, using vforks+execs to exec
+a new perl interpreter and compile the small server each time, I get:
+    479 vfork+execs per second, using AnyEvent::Fork->new_exec
+So how can C<< AnyEvent->new >> be faster than a standard fork, even
+though it uses the same operations, but adds a lot of overhead?
+The difference is simply the process size: forking the 6MB process takes
+so much longer than forking the 2.5MB template process that the overhead
+introduced is canceled out.
+If the benchmark process grows, the normal fork becomes even slower:
+   1340 new processes, manual fork in a 20MB process
+    731 new processes, manual fork in a 200MB process
+    235 new processes, manual fork in a 2000MB process
+What that means (to me) is that I can use this module without having a
+very bad conscience because of the extra overhead required to start new
+processes.
+=head1 TYPICAL PROBLEMS
+This section lists typical problems that remain. I hope by recognising
+them, most can be avoided.
+=over 4
+=item "leaked" file descriptors for exec'ed processes
+POSIX systems inherit file descriptors by default when exec'ing a new
+process. While perl itself laudably sets the close-on-exec flags on new
+file handles, most C libraries don't care, and even if all cared, it's
+often not possible to set the flag in a race-free manner.
+That means some file descriptors can leak through. And since it isn't
+possible to know which file descriptors are "good" and "necessary" (or
+even to know which file descriptors are open), there is no good way to
+close the ones that might harm.
+As an example of what "harm" can be done consider a web server that
+accepts connections and afterwards some module uses AnyEvent::Fork for the
+first time, causing it to fork and exec a new process, which might inherit
+the network socket. When the server closes the socket, it is still open
+in the child (which doesn't even know that) and the client might conclude
+that the connection is still fine.
+For the main program, there are multiple remedies available -
+L<AnyEvent::Fork::Early> is one, creating a process early and not using
+C<new_exec> is another, as in both cases, the first process can be exec'ed
+well before many random file descriptors are open.
+In general, the solution for these kind of problems is to fix the
+libraries or the code that leaks those file descriptors.
+Fortunately, most of these leaked descriptors do no harm, other than
+sitting on some resources.
+=item "leaked" file descriptors for fork'ed processes
+Normally, L<AnyEvent::Fork> does start new processes by exec'ing them,
+which closes file descriptors not marked for being inherited.
+However, L<AnyEvent::Fork::Early> and L<AnyEvent::Fork::Template> offer
+a way to create these processes by forking, and this leaks more file
+descriptors than exec'ing them, as there is no way to mark descriptors as
+"close on fork".
+An example would be modules like L<EV>, L<IO::AIO> or L<Gtk2>. Both create
+pipes for internal uses, and L<Gtk2> might open a connection to the X
+server. L<EV> and L<IO::AIO> can deal with fork, but Gtk2 might have
+trouble with a fork.
+The solution is to either not load these modules before use'ing
+L<AnyEvent::Fork::Early> or L<AnyEvent::Fork::Template>, or to delay
+initialising them, for example, by calling C<init Gtk2> manually.
+=item exit runs destructors
+This only applies to users of Lc<AnyEvent::Fork:Early> and
+L<AnyEvent::Fork::Template>.
+When a process created by AnyEvent::Fork exits, it might do so by calling
+exit, or simply letting perl reach the end of the program. At which point
+Perl runs all destructors.
+Not all destructors are fork-safe - for example, an object that represents
+the connection to an X display might tell the X server to free resources,
+which is inconvenient when the "real" object in the parent still needs to
+use them.
+This is obviously not a problem for L<AnyEvent::Fork::Early>, as you used
+it as the very first thing, right?
+It is a problem for L<AnyEvent::Fork::Template> though - and the solution
+is to not create objects with nontrivial destructors that might have an
+effect outside of Perl.
+=back
 =head1 PORTABILITY NOTES
-Win32 is a loser - code has been written for this platform, pain has been
+Native win32 perls are somewhat supported (AnyEvent::Fork::Early is a nop,
-felt, but in the end, this platform is just too broken - maybe a later
+and ::Template is not going to work), and it cost a lot of blood and sweat
-version can do it.
+to make it so, mostly due to the bloody broken perl that nobody seems to
+care about. The fork emulation is a bad joke - I have yet to see something
+useful that you can do with it without running into memory corruption
+issues or other braindamage. Hrrrr.
+Cygwin perl is not supported at the moment, as it should implement fd
+passing, but doesn't, and rolling my own is hard, as cygwin doesn't
+support enough functionality to do it.
+=head1 SEE ALSO
+L<AnyEvent::Fork::Early> (to avoid executing a perl interpreter),
+L<AnyEvent::Fork::Template> (to create a process by forking the main
+program at a convenient time).
 =head1 AUTHOR
  Marc Lehmann <schmorp@schmorp.de>
  http://home.schmorp.de/

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork/Fork.pm (file contents): Revision 1.9 by root, Thu Apr 4 03:45:12 2013 UTC vs. Revision 1.23 by root, Sat Apr 6 08:29:43 2013 UTC

Diff Legend

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.9 by root, Thu Apr 4 03:45:12 2013 UTC vs.
Revision 1.23 by root, Sat Apr 6 08:29:43 2013 UTC