[ViewVC] Annotation of: cvs/AnyEvent-Fork/README

NAME
    AnyEvent::Fork - everything you wanted to use fork() for, but couldn't

SYNOPSIS
       use AnyEvent::Fork;

       ##################################################################
       # create a single new process, tell it to run your worker function

       AnyEvent::Fork
          ->new
          ->require ("MyModule")
          ->run ("MyModule::worker, sub {
             my ($master_filehandle) = @_;

             # now $master_filehandle is connected to the
             # $slave_filehandle in the new process.
          });

       # MyModule::worker might look like this
       sub MyModule::worker {
          my ($slave_filehandle) = @_;

          # now $slave_filehandle is connected to the $master_filehandle
          # in the original prorcess. have fun!
       }

       ##################################################################
       # create a pool of server processes all accepting on the same socket

       # create listener socket
       my $listener = ...;

       # create a pool template, initialise it and give it the socket
       my $pool = AnyEvent::Fork
                     ->new
                     ->require ("Some::Stuff", "My::Server")
                     ->send_fh ($listener);

       # now create 10 identical workers
       for my $id (1..10) {
          $pool
             ->fork
             ->send_arg ($id)
             ->run ("My::Server::run");
       }

       # now do other things - maybe use the filehandle provided by run
       # to wait for the processes to die. or whatever.

       # My::Server::run might look like this
       sub My::Server::run {
          my ($slave, $listener, $id) = @_;

          close $slave; # we do not use the socket, so close it to save resources

          # we could go ballistic and use e.g. AnyEvent here, or IO::AIO,
          # or anything we usually couldn't do in a process forked normally.
          while (my $socket = $listener->accept) {
             # do sth. with new socket
          }
       }

DESCRIPTION
    This module allows you to create new processes, without actually forking
    them from your current process (avoiding the problems of forking), but
    preserving most of the advantages of fork.

    It can be used to create new worker processes or new independent
    subprocesses for short- and long-running jobs, process pools (e.g. for
    use in pre-forked servers) but also to spawn new external processes
    (such as CGI scripts from a web server), which can be faster (and more
    well behaved) than using fork+exec in big processes.

    Special care has been taken to make this module useful from other
    modules, while still supporting specialised environments such as
    App::Staticperl or PAR::Packer.

WHAT THIS MODULE IS NOT
    This module only creates processes and lets you pass file handles and
    strings to it, and run perl code. It does not implement any kind of RPC
    - there is no back channel from the process back to you, and there is no
    RPC or message passing going on.

    If you need some form of RPC, you can either implement it yourself in
    whatever way you like, use some message-passing module such as
    AnyEvent::MP, some pipe such as AnyEvent::ZeroMQ, use AnyEvent::Handle
    on both sides to send e.g. JSON or Storable messages, and so on.

PROBLEM STATEMENT
    There are two ways to implement parallel processing on UNIX like
    operating systems - fork and process, and fork+exec and process. They
    have different advantages and disadvantages that I describe below,
    together with how this module tries to mitigate the disadvantages.

    Forking from a big process can be very slow (a 5GB process needs 0.05s
    to fork on my 3.6GHz amd64 GNU/Linux box for example). This overhead is
    often shared with exec (because you have to fork first), but in some
    circumstances (e.g. when vfork is used), fork+exec can be much faster.
        This module can help here by telling a small(er) helper process to
        fork, or fork+exec instead.

    Forking usually creates a copy-on-write copy of the parent process.
    Memory (for example, modules or data files that have been will not take
    additional memory). When exec'ing a new process, modules and data files
    might need to be loaded again, at extra CPU and memory cost. Likewise
    when forking, all data structures are copied as well - if the program
    frees them and replaces them by new data, the child processes will
    retain the memory even if it isn't used.
        This module allows the main program to do a controlled fork, and
        allows modules to exec processes safely at any time. When creating a
        custom process pool you can take advantage of data sharing via fork
        without risking to share large dynamic data structures that will
        blow up child memory usage.

    Exec'ing a new perl process might be difficult and slow. For example, it
    is not easy to find the correct path to the perl interpreter, and all
    modules have to be loaded from disk again. Long running processes might
    run into problems when perl is upgraded for example.
        This module supports creating pre-initialised perl processes to be
        used as template, and also tries hard to identify the correct path
        to the perl interpreter. With a cooperative main program, exec'ing
        the interpreter might not even be necessary.

    Forking might be impossible when a program is running. For example,
    POSIX makes it almost impossible to fork from a multi-threaded program
    and do anything useful in the child - strictly speaking, if your perl
    program uses posix threads (even indirectly via e.g. IO::AIO or
    threads), you cannot call fork on the perl level anymore, at all.
        This module can safely fork helper processes at any time, by calling
        fork+exec in C, in a POSIX-compatible way.

    Parallel processing with fork might be inconvenient or difficult to
    implement. For example, when a program uses an event loop and creates
    watchers it becomes very hard to use the event loop from a child
    program, as the watchers already exist but are only meaningful in the
    parent. Worse, a module might want to use such a system, not knowing
    whether another module or the main program also does, leading to
    problems.
        This module only lets the main program create pools by forking
        (because only the main program can know when it is still safe to do
        so) - all other pools are created by fork+exec, after which such
        modules can again be loaded.

CONCEPTS
    This module can create new processes either by executing a new perl
    process, or by forking from an existing "template" process.

    Each such process comes with its own file handle that can be used to
    communicate with it (it's actually a socket - one end in the new
    process, one end in the main process), and among the things you can do
    in it are load modules, fork new processes, send file handles to it, and
    execute functions.

    There are multiple ways to create additional processes to execute some
    jobs:

    fork a new process from the "default" template process, load code, run
    it
        This module has a "default" template process which it executes when
        it is needed the first time. Forking from this process shares the
        memory used for the perl interpreter with the new process, but
        loading modules takes time, and the memory is not shared with
        anything else.

        This is ideal for when you only need one extra process of a kind,
        with the option of starting and stopping it on demand.

        Example:

           AnyEvent::Fork
              ->new
              ->require ("Some::Module")
              ->run ("Some::Module::run", sub {
                 my ($fork_fh) = @_;
              });

    fork a new template process, load code, then fork processes off of it
    and run the code
        When you need to have a bunch of processes that all execute the same
        (or very similar) tasks, then a good way is to create a new template
        process for them, loading all the modules you need, and then create
        your worker processes from this new template process.

        This way, all code (and data structures) that can be shared (e.g.
        the modules you loaded) is shared between the processes, and each
        new process consumes relatively little memory of its own.

        The disadvantage of this approach is that you need to create a
        template process for the sole purpose of forking new processes from
        it, but if you only need a fixed number of processes you can create
        them, and then destroy the template process.

        Example:

           my $template = AnyEvent::Fork->new->require ("Some::Module");
   
           for (1..10) {
              $template->fork->run ("Some::Module::run", sub {
                 my ($fork_fh) = @_;
              });
           }

           # at this point, you can keep $template around to fork new processes
           # later, or you can destroy it, which causes it to vanish.

    execute a new perl interpreter, load some code, run it
        This is relatively slow, and doesn't allow you to share memory
        between multiple processes.

        The only advantage is that you don't have to have a template process
        hanging around all the time to fork off some new processes, which
        might be an advantage when there are long time spans where no extra
        processes are needed.

        Example:

           AnyEvent::Fork
              ->new_exec
              ->require ("Some::Module")
              ->run ("Some::Module::run", sub {
                 my ($fork_fh) = @_;
              });

FUNCTIONS
    my $pool = new AnyEvent::Fork key => value...
        Create a new process pool. The following named parameters are
        supported:

    my $proc = new AnyEvent::Fork
        Create a new "empty" perl interpreter process and returns its
        process object for further manipulation.

        The new process is forked from a template process that is kept
        around for this purpose. When it doesn't exist yet, it is created by
        a call to "new_exec" and kept around for future calls.

        When the process object is destroyed, it will release the file
        handle that connects it with the new process. When the new process
        has not yet called "run", then the process will exit. Otherwise,
        what happens depends entirely on the code that is executed.

    $new_proc = $proc->fork
        Forks $proc, creating a new process, and returns the process object
        of the new process.

        If any of the "send_" functions have been called before fork, then
        they will be cloned in the child. For example, in a pre-forked
        server, you might "send_fh" the listening socket into the template
        process, and then keep calling "fork" and "run".

    my $proc = new_exec AnyEvent::Fork
        Create a new "empty" perl interpreter process and returns its
        process object for further manipulation.

        Unlike the "new" method, this method *always* spawns a new perl
        process (except in some cases, see AnyEvent::Fork::Early for
        details). This reduces the amount of memory sharing that is
        possible, and is also slower.

        You should use "new" whenever possible, except when having a
        template process around is unacceptable.

        The path to the perl interpreter is divined using various methods -
        first $^X is investigated to see if the path ends with something
        that sounds as if it were the perl interpreter. Failing this, the
        module falls back to using $Config::Config{perlpath}.

    $pid = $proc->pid
        Returns the process id of the process *iff it is a direct child of
        the process* running AnyEvent::Fork, and "undef" otherwise.

        Normally, only processes created via "AnyEvent::Fork->new_exec" and
        AnyEvent::Fork::Template are direct children, and you are
        responsible to clean up their zombies when they die.

        All other processes are not direct children, and will be cleaned up
        by AnyEvent::Fork.

    $proc = $proc->eval ($perlcode, @args)
        Evaluates the given $perlcode as ... perl code, while setting @_ to
        the strings specified by @args.

        This call is meant to do any custom initialisation that might be
        required (for example, the "require" method uses it). It's not
        supposed to be used to completely take over the process, use "run"
        for that.

        The code will usually be executed after this call returns, and there
        is no way to pass anything back to the calling process. Any
        evaluation errors will be reported to stderr and cause the process
        to exit.

        Returns the process object for easy chaining of method calls.

    $proc = $proc->require ($module, ...)
        Tries to load the given module(s) into the process

        Returns the process object for easy chaining of method calls.

    $proc = $proc->send_fh ($handle, ...)
        Send one or more file handles (*not* file descriptors) to the
        process, to prepare a call to "run".

        The process object keeps a reference to the handles until this is
        done, so you must not explicitly close the handles. This is most
        easily accomplished by simply not storing the file handles anywhere
        after passing them to this method.

        Returns the process object for easy chaining of method calls.

        Example: pass a file handle to a process, and release it without
        closing. It will be closed automatically when it is no longer used.

           $proc->send_fh ($my_fh);
           undef $my_fh; # free the reference if you want, but DO NOT CLOSE IT

    $proc = $proc->send_arg ($string, ...)
        Send one or more argument strings to the process, to prepare a call
        to "run". The strings can be any octet string.

        The protocol is optimised to pass a moderate number of relatively
        short strings - while you can pass up to 4GB of data in one go, this
        is more meant to pass some ID information or other startup info, not
        big chunks of data.

        Returns the process object for easy chaining of method calls.

    $proc->run ($func, $cb->($fh))
        Enter the function specified by the fully qualified name in $func in
        the process. The function is called with the communication socket as
        first argument, followed by all file handles and string arguments
        sent earlier via "send_fh" and "send_arg" methods, in the order they
        were called.

        If the called function returns, the process exits.

        Preparing the process can take time - when the process is ready, the
        callback is invoked with the local communications socket as
        argument.

        The process object becomes unusable on return from this function.

        If the communication socket isn't used, it should be closed on both
        sides, to save on kernel memory.

        The socket is non-blocking in the parent, and blocking in the newly
        created process. The close-on-exec flag is set on both. Even if not
        used otherwise, the socket can be a good indicator for the existence
        of the process - if the other process exits, you get a readable
        event on it, because exiting the process closes the socket (if it
        didn't create any children using fork).

        Example: create a template for a process pool, pass a few strings,
        some file handles, then fork, pass one more string, and run some
        code.

           my $pool = AnyEvent::Fork
                         ->new
                         ->send_arg ("str1", "str2")
                         ->send_fh ($fh1, $fh2);

           for (1..2) {
              $pool
                 ->fork
                 ->send_arg ("str3")
                 ->run ("Some::function", sub {
                    my ($fh) = @_;

                    # fh is nonblocking, but we trust that the OS can accept these
                    # extra 3 octets anyway.
                    syswrite $fh, "hi #$_\n";

                    # $fh is being closed here, as we don't store it anywhere
                 });
           }

           # Some::function might look like this - all parameters passed before fork
           # and after will be passed, in order, after the communications socket.
           sub Some::function {
              my ($fh, $str1, $str2, $fh1, $fh2, $str3) = @_;

              print scalar <$fh>; # prints "hi 1\n" and "hi 2\n"
           }

PERFORMANCE
    Now for some unscientific benchmark numbers (all done on an amd64
    GNU/Linux box). These are intended to give you an idea of the relative
    performance you can expect, they are not meant to be absolute
    performance numbers.

    OK, so, I ran a simple benchmark that creates a socket pair, forks,
    calls exit in the child and waits for the socket to close in the parent.
    I did load AnyEvent, EV and AnyEvent::Fork, for a total process size of
    5100kB.

       2079 new processes per second, using manual socketpair + fork

    Then I did the same thing, but instead of calling fork, I called
    AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the
    socket form the child to close on exit. This does the same thing as
    manual socket pair + fork, except that what is forked is the template
    process (2440kB), and the socket needs to be passed to the server at the
    other end of the socket first.

       2307 new processes per second, using AnyEvent::Fork->new

    And finally, using "new_exec" instead "new", using vforks+execs to exec
    a new perl interpreter and compile the small server each time, I get:

        479 vfork+execs per second, using AnyEvent::Fork->new_exec

    So how can "AnyEvent->new" be faster than a standard fork, even though
    it uses the same operations, but adds a lot of overhead?

    The difference is simply the process size: forking the 6MB process takes
    so much longer than forking the 2.5MB template process that the overhead
    introduced is canceled out.

    If the benchmark process grows, the normal fork becomes even slower:

       1340 new processes, manual fork in a 20MB process
        731 new processes, manual fork in a 200MB process
        235 new processes, manual fork in a 2000MB process

    What that means (to me) is that I can use this module without having a
    very bad conscience because of the extra overhead required to start new
    processes.

TYPICAL PROBLEMS
    This section lists typical problems that remain. I hope by recognising
    them, most can be avoided.

    "leaked" file descriptors for exec'ed processes
        POSIX systems inherit file descriptors by default when exec'ing a
        new process. While perl itself laudably sets the close-on-exec flags
        on new file handles, most C libraries don't care, and even if all
        cared, it's often not possible to set the flag in a race-free
        manner.

        That means some file descriptors can leak through. And since it
        isn't possible to know which file descriptors are "good" and
        "necessary" (or even to know which file descriptors are open), there
        is no good way to close the ones that might harm.

        As an example of what "harm" can be done consider a web server that
        accepts connections and afterwards some module uses AnyEvent::Fork
        for the first time, causing it to fork and exec a new process, which
        might inherit the network socket. When the server closes the socket,
        it is still open in the child (which doesn't even know that) and the
        client might conclude that the connection is still fine.

        For the main program, there are multiple remedies available -
        AnyEvent::Fork::Early is one, creating a process early and not using
        "new_exec" is another, as in both cases, the first process can be
        exec'ed well before many random file descriptors are open.

        In general, the solution for these kind of problems is to fix the
        libraries or the code that leaks those file descriptors.

        Fortunately, most of these leaked descriptors do no harm, other than
        sitting on some resources.

    "leaked" file descriptors for fork'ed processes
        Normally, AnyEvent::Fork does start new processes by exec'ing them,
        which closes file descriptors not marked for being inherited.

        However, AnyEvent::Fork::Early and AnyEvent::Fork::Template offer a
        way to create these processes by forking, and this leaks more file
        descriptors than exec'ing them, as there is no way to mark
        descriptors as "close on fork".

        An example would be modules like EV, IO::AIO or Gtk2. Both create
        pipes for internal uses, and Gtk2 might open a connection to the X
        server. EV and IO::AIO can deal with fork, but Gtk2 might have
        trouble with a fork.

        The solution is to either not load these modules before use'ing
        AnyEvent::Fork::Early or AnyEvent::Fork::Template, or to delay
        initialising them, for example, by calling "init Gtk2" manually.

    exit runs destructors
        This only applies to users of Lc<AnyEvent::Fork:Early> and
        AnyEvent::Fork::Template.

        When a process created by AnyEvent::Fork exits, it might do so by
        calling exit, or simply letting perl reach the end of the program.
        At which point Perl runs all destructors.

        Not all destructors are fork-safe - for example, an object that
        represents the connection to an X display might tell the X server to
        free resources, which is inconvenient when the "real" object in the
        parent still needs to use them.

        This is obviously not a problem for AnyEvent::Fork::Early, as you
        used it as the very first thing, right?

        It is a problem for AnyEvent::Fork::Template though - and the
        solution is to not create objects with nontrivial destructors that
        might have an effect outside of Perl.

PORTABILITY NOTES
    Native win32 perls are somewhat supported (AnyEvent::Fork::Early is a
    nop, and ::Template is not going to work), and it cost a lot of blood
    and sweat to make it so, mostly due to the bloody broken perl that
    nobody seems to care about. The fork emulation is a bad joke - I have
    yet to see something useful that you can do with it without running into
    memory corruption issues or other braindamage. Hrrrr.

    Cygwin perl is not supported at the moment, as it should implement fd
    passing, but doesn't, and rolling my own is hard, as cygwin doesn't
    support enough functionality to do it.

SEE ALSO
    AnyEvent::Fork::Early (to avoid executing a perl interpreter),
    AnyEvent::Fork::Template (to create a process by forking the main
    program at a convenient time).

AUTHOR
     Marc Lehmann <schmorp@schmorp.de>
     http://home.schmorp.de/

Revision:	1.4
Committed:	Sat Apr 6 03:42:26 2013 UTC (11 years, 1 month ago) by root
Branch:	MAIN
CVS Tags:	rel-0_5
Changes since 1.3:	+106 -15 lines
Log Message:	0.5