[ViewVC] Diff of: cvs/AnyEvent-Fork/Fork.pm

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.37 by root, Sat Apr 6 20:06:39 2013 UTC vs.
Revision 1.47 by root, Thu Apr 18 20:17:34 2013 UTC

 Special care has been taken to make this module useful from other modules,
 while still supporting specialised environments such as L<App::Staticperl>
 or L<PAR::Packer>.
-=head1 WHAT THIS MODULE IS NOT
+=head2 WHAT THIS MODULE IS NOT
 This module only creates processes and lets you pass file handles and
 strings to it, and run perl code. It does not implement any kind of RPC -
 there is no back channel from the process back to you, and there is no RPC
 or message passing going on.
-If you need some form of RPC, you can either implement it yourself
+If you need some form of RPC, you could use the L<AnyEvent::Fork::RPC>
-in whatever way you like, use some message-passing module such
+companion module, which adds simple RPC/job queueing to a process created
-as L<AnyEvent::MP>, some pipe such as L<AnyEvent::ZeroMQ>, use
+by this module.
-L<AnyEvent::Handle> on both sides to send e.g. JSON or Storable messages,
-and so on.
+Or you can implement it yourself in whatever way you like, use some
+message-passing module such as L<AnyEvent::MP>, some pipe such as
+L<AnyEvent::ZeroMQ>, use L<AnyEvent::Handle> on both sides to send
+e.g. JSON or Storable messages, and so on.
+=head2 COMPARISON TO OTHER MODULES
+There is an abundance of modules on CPAN that do "something fork", such as
+L<Parallel::ForkManager>, L<AnyEvent::ForkManager>, L<AnyEvent::Worker>
+or L<AnyEvent::Subprocess>. There are modules that implement their own
+process management, such as L<AnyEvent::DBI>.
+The problems that all these modules try to solve are real, however, none
+of them (from what I have seen) tackle the very real problems of unwanted
+memory sharing, efficiency, not being able to use event processing or
+similar modules in the processes they create.
+This module doesn't try to replace any of them - instead it tries to solve
+the problem of creating processes with a minimum of fuss and overhead (and
+also luxury). Ideally, most of these would use AnyEvent::Fork internally,
+except they were written before AnyEvent:Fork was available, so obviously
+had to roll their own.
-=head1 PROBLEM STATEMENT
+=head2 PROBLEM STATEMENT
 There are two traditional ways to implement parallel processing on UNIX
 like operating systems - fork and process, and fork+exec and process. They
 have different advantages and disadvantages that I describe below,
 together with how this module tries to mitigate the disadvantages.
       }
    }
 =head2 use AnyEvent::Fork as a faster fork+exec
-This runs C</bin/echo hi>, with stdandard output redirected to /tmp/log
+This runs C</bin/echo hi>, with standard output redirected to F</tmp/log>
 and standard error redirected to the communications socket. It is usually
 faster than fork+exec, but still lets you prepare the environment.
    open my $output, ">/tmp/log" or die "$!";
    AnyEvent::Fork
       ->new
       ->eval ('
+           # compile a helper function for later use
            sub run {
               my ($fh, $output, @cmd) = @_;
               # perl will clear close-on-exec on STDOUT/STDERR
               open STDOUT, ">&", $output or die;
 =head1 CONCEPTS
 This module can create new processes either by executing a new perl
 process, or by forking from an existing "template" process.
+All these processes are called "child processes" (whether they are direct
+children or not), while the process that manages them is called the
+"parent process".
 Each such process comes with its own file handle that can be used to
 communicate with it (it's actually a socket - one end in the new process,
 one end in the main process), and among the things you can do in it are
 load modules, fork new processes, send file handles to it, and execute
 use AnyEvent;
 use AnyEvent::Util ();
 use IO::FDPass;
-our $VERSION = 0.5;
+our $VERSION = 0.7;
-our $PERL; # the path to the perl interpreter, deduces with various forms of magic
-=over 4
-=back
-=cut
 # the early fork template process
 our $EARLY;
 # the empty template process
 our $TEMPLATE;
+sub QUEUE() { 0 }
+sub FH()    { 1 }
+sub WW()    { 2 }
+sub PID()   { 3 }
+sub CB()    { 4 }
+sub _new {
+   my ($self, $fh, $pid) = @_;
+   AnyEvent::Util::fh_nonblocking $fh, 1;
+   $self = bless [
+      [],    # write queue - strings or fd's
+      $fh,
+      undef, # AE watcher
+      $pid,
+   ], $self;
+   $self
+}
 sub _cmd {
    my $self = shift;
    # ideally, we would want to use "a (w/a)*" as format string, but perl
    # versions from at least 5.8.9 to 5.16.3 are all buggy and can't unpack
    # it.
-   push @{ $self->[2] }, pack "a L/a*", $_[0], $_[1];
+   push @{ $self->[QUEUE] }, pack "a L/a*", $_[0], $_[1];
-   $self->[3] ||= AE::io $self->[1], 1, sub {
+   $self->[WW] ||= AE::io $self->[FH], 1, sub {
       do {
          # send the next "thing" in the queue - either a reference to an fh,
          # or a plain string.
-         if (ref $self->[2][0]) {
+         if (ref $self->[QUEUE][0]) {
             # send fh
-            unless (IO::FDPass::send fileno $self->[1], fileno ${ $self->[2][0] }) {
+            unless (IO::FDPass::send fileno $self->[FH], fileno ${ $self->[QUEUE][0] }) {
                return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
-               undef $self->[3];
+               undef $self->[WW];
                die "AnyEvent::Fork: file descriptor send failure: $!";
             }
-            shift @{ $self->[2] };
+            shift @{ $self->[QUEUE] };
          } else {
             # send string
-            my $len = syswrite $self->[1], $self->[2][0];
+            my $len = syswrite $self->[FH], $self->[QUEUE][0];
             unless ($len) {
                return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
                undef $self->[3];
                die "AnyEvent::Fork: command write failure: $!";
             }
-            substr $self->[2][0], 0, $len, "";
+            substr $self->[QUEUE][0], 0, $len, "";
-            shift @{ $self->[2] } unless length $self->[2][0];
+            shift @{ $self->[QUEUE] } unless length $self->[QUEUE][0];
          }
-      } while @{ $self->[2] };
+      } while @{ $self->[QUEUE] };
       # everything written
-      undef $self->[3];
+      undef $self->[WW];
       # invoke run callback, if any
-      $self->[4]->($self->[1]) if $self->[4];
+      $self->[CB]->($self->[FH]) if $self->[CB];
    };
    () # make sure we don't leak the watcher
-}
-sub _new {
-   my ($self, $fh, $pid) = @_;
-   AnyEvent::Util::fh_nonblocking $fh, 1;
-   $self = bless [
-      $pid,
-      $fh,
-      [],    # write queue - strings or fd's
-      undef, # AE watcher
-   ], $self;
-   $self
 }
 # fork template from current process, used by AnyEvent::Fork::Early/Template
 sub _new_fork {
    my ($fh, $slave) = AnyEvent::Util::portable_socketpair;
    if ($pid eq 0) {
       require AnyEvent::Fork::Serve;
       $AnyEvent::Fork::Serve::OWNER = $parent;
       close $fh;
       $0 = "$_[1] of $parent";
-      $SIG{CHLD} = 'IGNORE';
       AnyEvent::Fork::Serve::serve ($slave);
       exit 0;
    } elsif (!$pid) {
       die "AnyEvent::Fork::Early/Template: unable to fork template process: $!";
    }
 AnyEvent::Fork itself.
 =cut
 sub pid {
-   $_[0][0]
+   $_[0][PID]
 }
 =item $proc = $proc->eval ($perlcode, @args)
-Evaluates the given C<$perlcode> as ... perl code, while setting C<@_> to
+Evaluates the given C<$perlcode> as ... Perl code, while setting C<@_> to
 the strings specified by C<@args>, in the "main" package.
 This call is meant to do any custom initialisation that might be required
 (for example, the C<require> method uses it). It's not supposed to be used
 to completely take over the process, use C<run> for that.
 sub send_fh {
    my ($self, @fh) = @_;
    for my $fh (@fh) {
       $self->_cmd ("h");
-      push @{ $self->[2] }, \$fh;
+      push @{ $self->[QUEUE] }, \$fh;
    }
    $self
 }
 =cut
 sub run {
    my ($self, $func, $cb) = @_;
-   $self->[4] = $cb;
+   $self->[CB] = $cb;
    $self->_cmd (r => $func);
 }
 =back
 So how can C<< AnyEvent->new >> be faster than a standard fork, even
 though it uses the same operations, but adds a lot of overhead?
 The difference is simply the process size: forking the 5MB process takes
 so much longer than forking the 2.5MB template process that the extra
-overhead introduced is canceled out.
+overhead is canceled out.
 If the benchmark process grows, the normal fork becomes even slower:
    1340 new processes, manual fork of a 20MB process
     731 new processes, manual fork of a 200MB process
 L<AnyEvent::Fork::Early> or L<AnyEvent::Fork::Template>, or to delay
 initialising them, for example, by calling C<init Gtk2> manually.
 =item exiting calls object destructors
-This only applies to users of Lc<AnyEvent::Fork:Early> and
+This only applies to users of L<AnyEvent::Fork:Early> and
-L<AnyEvent::Fork::Template>.
+L<AnyEvent::Fork::Template>, or when initialising code creates objects
+that reference external resources.
 When a process created by AnyEvent::Fork exits, it might do so by calling
 exit, or simply letting perl reach the end of the program. At which point
 Perl runs all destructors.
 Cygwin perl is not supported at the moment due to some hilarious
 shortcomings of its API - see L<IO::FDPoll> for more details.
 =head1 SEE ALSO
-L<AnyEvent::Fork::Early> (to avoid executing a perl interpreter),
+L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all
+(part of this distribution).
-L<AnyEvent::Fork::Template> (to create a process by forking the main
+L<AnyEvent::Fork::Template>, to create a process by forking the main
-program at a convenient time).
+program at a convenient time (part of this distribution).
-=head1 AUTHOR
+L<AnyEvent::Fork::RPC>, for simple RPC to child processes (on CPAN).
+=head1 AUTHOR AND CONTACT INFORMATION
  Marc Lehmann <schmorp@schmorp.de>
- http://home.schmorp.de/
+ http://software.schmorp.de/pkg/AnyEvent-Fork
 =cut
 1

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork/Fork.pm (file contents): Revision 1.37 by root, Sat Apr 6 20:06:39 2013 UTC vs. Revision 1.47 by root, Thu Apr 18 20:17:34 2013 UTC

Diff Legend

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.37 by root, Sat Apr 6 20:06:39 2013 UTC vs.
Revision 1.47 by root, Thu Apr 18 20:17:34 2013 UTC