[ViewVC] Diff of: cvs/AnyEvent-Fork-RPC/README

Comparing AnyEvent-Fork-RPC/README (file contents):
Revision 1.2 by root, Thu Apr 18 20:04:51 2013 UTC vs.
Revision 1.5 by root, Thu May 12 16:54:43 2016 UTC

 NAME
     AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork
-    THE API IS NOT FINISHED, CONSIDER THIS A TECHNOLOGY DEMO
 SYNOPSIS
+       use AnyEvent::Fork;
        use AnyEvent::Fork::RPC;
-       # use AnyEvent::Fork is not needed
        my $rpc = AnyEvent::Fork
           ->new
           ->require ("MyModule")
           ->AnyEvent::Fork::RPC::run (
        $cv->recv;
 DESCRIPTION
     This module implements a simple RPC protocol and backend for processes
-    created via AnyEvent::Fork, allowing you to call a function in the child
+    created via AnyEvent::Fork or AnyEvent::Fork::Remote, allowing you to
-    process and receive its return values (up to 4GB serialised).
+    call a function in the child process and receive its return values (up
+    to 4GB serialised).
     It implements two different backends: a synchronous one that works like
     a normal function call, and an asynchronous one that can run multiple
     jobs concurrently in the child, using AnyEvent.
     It also implements an asynchronous event mechanism from the child to the
     parent, that could be used for progress indications or other
     information.
-    Loading this module also always loads AnyEvent::Fork, so you can make a
-    separate "use AnyEvent::Fork" if you wish, but you don't have to.
 EXAMPLES
   Example 1: Synchronous Backend
     Here is a simple example that implements a backend that executes
     "unlink" and "rmdir" calls, and reports their status back. It also
     which is clearly silly, but illustrates the use of events.
     First the parent process:
        use AnyEvent;
+       use AnyEvent::Fork;
        use AnyEvent::Fork::RPC;
        my $done = AE::cv;
        my $rpc = AnyEvent::Fork
           ->new
           ->require ("MyWorker")
           ->AnyEvent::Fork::RPC::run ("MyWorker::run",
-             on_error   => sub { warn "FATAL: $_[0]"; exit 1 },
+             on_error   => sub { warn "ERROR: $_[0]"; exit 1 },
              on_event   => sub { warn "$_[0] requests handled\n" },
              on_destroy => $done,
           );
        for my $id (1..6) {
     child process may exit at any time, so you should call $done only when
     you really *are* done.
   Example 2: Asynchronous Backend
     This example implements multiple count-downs in the child, using
-    AnyEvent timers. While this is a bit silly (one could use timers in te
+    AnyEvent timers. While this is a bit silly (one could use timers in the
     parent just as well), it illustrates the ability to use AnyEvent in the
     child and the fact that responses can arrive in a different order then
     the requests.
     It also shows how to embed the actual child code into a "__DATA__"
     look so silly anymore.
     Without further ado, here is the code:
        use AnyEvent;
+       use AnyEvent::Fork;
        use AnyEvent::Fork::RPC;
        my $done = AE::cv;
        my $rpc = AnyEvent::Fork
           ->new
           ->require ("AnyEvent::Fork::RPC::Async")
           ->eval (do { local $/; <DATA> })
           ->AnyEvent::Fork::RPC::run ("run",
              async      => 1,
-             on_error   => sub { warn "FATAL: $_[0]"; exit 1 },
+             on_error   => sub { warn "ERROR: $_[0]"; exit 1 },
              on_event   => sub { print $_[0] },
              on_destroy => $done,
           );
        for my $count (3, 2, 1) {
     This concludes the async example. Since AnyEvent::Fork does not actually
     fork, you are free to use about any module in the child, not just
     AnyEvent, but also IO::AIO, or Tk for example.
+  Example 3: Asynchronous backend with Coro
+    With Coro you can create a nice asynchronous backend implementation by
+    defining an rpc server function that creates a new Coro thread for every
+    request that calls a function "normally", i.e. the parameters from the
+    parent process are passed to it, and any return values are returned to
+    the parent process, e.g.:
+       package My::Arith;
+       sub add {
+          return $_[0] + $_[1];
+       }
+       sub mul {
+          return $_[0] * $_[1];
+       }
+       sub run {
+          my ($done, $func, @arg) = @_;
+          Coro::async_pool {
+             $done->($func->(@arg));
+          };
+       }
+    The "run" function creates a new thread for every invocation, using the
+    first argument as function name, and calls the $done callback on it's
+    return values. This makes it quite natural to define the "add" and "mul"
+    functions to add or multiply two numbers and return the result.
+    Since this is the asynchronous backend, it's quite possible to define
+    RPC function that do I/O or wait for external events - their execution
+    will overlap as needed.
+    The above could be used like this:
+       my $rpc = AnyEvent::Fork
+          ->new
+          ->require ("MyWorker")
+          ->AnyEvent::Fork::RPC::run ("My::Arith::run",
+             on_error => ..., on_event => ..., on_destroy => ...,
+          );
+       $rpc->(add => 1, 3, Coro::rouse_cb); say Coro::rouse_wait;
+       $rpc->(mul => 3, 2, Coro::rouse_cb); say Coro::rouse_wait;
+    The "say"'s will print 4 and 6.
+  Example 4: Forward AnyEvent::Log messages using "on_event"
+    This partial example shows how to use the "event" function to forward
+    AnyEvent::Log messages to the parent.
+    For this, the parent needs to provide a suitable "on_event":
+       ->AnyEvent::Fork::RPC::run (
+          on_event => sub {
+             if ($_[0] eq "ae_log") {
+                my (undef, $level, $message) = @_;
+                AE::log $level, $message;
+             } else {
+                # other event types
+             }
+          },
+       )
+    In the child, as early as possible, the following code should
+    reconfigure AnyEvent::Log to log via "AnyEvent::Fork::RPC::event":
+       $AnyEvent::Log::LOG->log_cb (sub {
+          my ($timestamp, $orig_ctx, $level, $message) = @{+shift};
+          if (defined &AnyEvent::Fork::RPC::event) {
+             AnyEvent::Fork::RPC::event (ae_log => $level, $message);
+          } else {
+             warn "[$$ before init] $message\n";
+          }
+       });
+    There is an important twist - the "AnyEvent::Fork::RPC::event" function
+    is only defined when the child is fully initialised. If you redirect the
+    log messages in your "init" function for example, then the "event"
+    function might not yet be available. This is why the log callback checks
+    whether the fucntion is there using "defined", and only then uses it to
+    log the message.
 PARENT PROCESS USAGE
     This module exports nothing, and only implements a single function:
     my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]
         The traditional way to call it. But it is way cooler to call it in
             Called on (fatal) errors, with a descriptive (hopefully)
             message. If this callback is not provided, but "on_event" is,
             then the "on_event" callback is called with the first argument
             being the string "error", followed by the error message.
-            If neither handler is provided it prints the error to STDERR and
+            If neither handler is provided, then the error is reported with
-            will start failing badly.
+            loglevel "error" via "AE::log".
         on_event => $cb->(...)
             Called for every call to the "AnyEvent::Fork::RPC::event"
             function in the child, with the arguments of that function
             passed to the callback.
             It is called very early - before the serialisers are created or
             the $function name is resolved into a function reference, so it
             could be used to load any modules that provide the serialiser or
             function. It can not, however, create events.
+        done => $function (default "CORE::exit")
+            The function to call when the asynchronous backend detects an
+            end of file condition when reading from the communications
+            socket *and* there are no outstanding requests. It's ignored by
+            the synchronous backend.
+            By overriding this you can prolong the life of a RPC process
+            after e.g. the parent has exited by running the event loop in
+            the provided function (or simply calling it, for example, when
+            your child process uses EV you could provide EV::run as "done"
+            function).
+            Of course, in that case you are responsible for exiting at the
+            appropriate time and not returning from
         async => $boolean (default: 0)
             The default server used in the child does all I/O blockingly,
             and only allows a single RPC call to execute concurrently.
             Setting "async" to a true value switches to another
             to be transferred between the processes. For this, they have to
             be frozen and thawed in both parent and child processes.
             By default, only octet strings can be passed between the
             processes, which is reasonably fast and efficient and requires
-            no extra modules.
+            no extra modules (the "AnyEvent::Fork::RPC" distribution does
+            not provide these extra serialiser modules).
             For more complicated use cases, you can provide your own freeze
             and thaw functions, by specifying a string with perl source
             code. It's supposed to return two code references when
             evaluated: the first receives a list of perl values and must
             If you need an external module for serialisation, then you can
             either pre-load it into your AnyEvent::Fork process, or you can
             add a "use" or "require" statement into the serialiser string.
             Or both.
-            Here are some examples - some of them are also available as
+            Here are some examples - all of them are also available as
             global variables that make them easier to use.
-            octet strings - $AnyEvent::Fork::RPC::STRING_SERIALISER
+            $AnyEvent::Fork::RPC::STRING_SERIALISER - octet strings only
-                This serialiser concatenates length-prefixes octet strings,
+                This serialiser (currently the default) concatenates
-                and is the default.
+                length-prefixes octet strings, and is the default. That
+                means you can only pass (and return) strings containing
+                character codes 0-255.
+                The main advantages of this serialiser are the high speed
+                and that it doesn't need another module. The main
+                disadvantage is that you are very limited in what you can
+                pass - only octet strings.
                 Implementation:
                    (
                       sub { pack   "(w/a*)*", @_ },
                       sub { unpack "(w/a*)*", shift }
                    )
+            $AnyEvent::Fork::RPC::CBOR_XS_SERIALISER - uses CBOR::XS
+                This serialiser creates CBOR::XS arrays - you have to make
+                sure the CBOR::XS module is installed for this serialiser to
+                work. It can be beneficial for sharing when you preload the
+                CBOR::XS module in a template process.
+                CBOR::XS is about as fast as the octet string serialiser,
+                but supports complex data structures (similar to JSON) and
+                is faster than any of the other serialisers. If you have the
+                CBOR::XS module available, it's the best choice.
+                The encoder enables "allow_sharing" (so this serialisation
+                method can encode cyclic and self-referencing data
+                structures).
+                Implementation:
+                   use CBOR::XS ();
+                   (
+                      sub {    CBOR::XS::encode_cbor_sharing \@_ },
+                      sub { @{ CBOR::XS::decode_cbor shift } }
+                   )
-            json - $AnyEvent::Fork::RPC::JSON_SERIALISER
+            $AnyEvent::Fork::RPC::JSON_SERIALISER - uses JSON::XS or JSON
                 This serialiser creates JSON arrays - you have to make sure
                 the JSON module is installed for this serialiser to work. It
                 can be beneficial for sharing when you preload the JSON
                 module in a template process.
                    (
                       sub {    JSON::encode_json \@_ },
                       sub { @{ JSON::decode_json shift } }
                    )
-            storable - $AnyEvent::Fork::RPC::STORABLE_SERIALISER
+            $AnyEvent::Fork::RPC::STORABLE_SERIALISER - Storable
                 This serialiser uses Storable, which means it has high
                 chance of serialising just about anything you throw at it,
                 at the cost of having very high overhead per operation. It
-                also comes with perl.
+                also comes with perl. It should be used when you need to
+                serialise complex data structures.
                 Implementation:
                    use Storable ();
                    (
                       sub {    Storable::freeze \@_ },
+                      sub { @{ Storable::thaw shift } }
+                   )
+            $AnyEvent::Fork::RPC::NSTORABLE_SERIALISER - portable Storable
+                This serialiser also uses Storable, but uses it's "network"
+                format to serialise data, which makes it possible to talk to
+                different perl binaries (for example, when talking to a
+                process created with AnyEvent::Fork::Remote).
+                Implementation:
+                   use Storable ();
+                   (
+                      sub {    Storable::nfreeze \@_ },
                       sub { @{ Storable::thaw shift } }
                    )
         See the examples section earlier in this document for some actual
         examples.
         the child process to the parent, except that there is no notion of
         return values.
         See the examples section earlier in this document for some actual
         examples.
+  PROCESS EXIT
+    If and when the child process exits depends on the backend and
+    configuration. Apart from explicit exits (e.g. by calling "exit") or
+    runtime conditions (uncaught exceptions, signals etc.), the backends
+    exit under these conditions:
+    Synchronous Backend
+        The synchronous backend is very simple: when the process waits for
+        another request to arrive and the writing side (usually in the
+        parent) is closed, it will exit normally, i.e. as if your main
+        program reached the end of the file.
+        That means that if your parent process exits, the RPC process will
+        usually exit as well, either because it is idle anyway, or because
+        it executes a request. In the latter case, you will likely get an
+        error when the RPc process tries to send the results to the parent
+        (because agruably, you shouldn't exit your parent while there are
+        still outstanding requests).
+        The process is usually quiescent when it happens, so it should
+        rarely be a problem, and "END" handlers can be used to clean up.
+    Asynchronous Backend
+        For the asynchronous backend, things are more complicated: Whenever
+        it listens for another request by the parent, it might detect that
+        the socket was closed (e.g. because the parent exited). It will sotp
+        listening for new requests and instead try to write out any
+        remaining data (if any) or simply check whether the socket can be
+        written to. After this, the RPC process is effectively done - no new
+        requests are incoming, no outstanding request data can be written
+        back.
+        Since chances are high that there are event watchers that the RPC
+        server knows nothing about (why else would one use the async backend
+        if not for the ability to register watchers?), the event loop would
+        often happily continue.
+        This is why the asynchronous backend explicitly calls "CORE::exit"
+        when it is done (under other circumstances, such as when there is an
+        I/O error and there is outstanding data to write, it will log a
+        fatal message via AnyEvent::Log, also causing the program to exit).
+        You can override this by specifying a function name to call via the
+        "done" parameter instead.
 ADVANCED TOPICS
   Choosing a backend
     So how do you decide which backend to use? Well, that's your problem to
     solve, but here are some thoughts on the matter:
         1000 jobs are queued and the jobs are slow, they will all run
         concurrently. The child must implement some queueing/limiting
         mechanism if this causes problems. Alternatively, the parent could
         limit the amount of rpc calls that are outstanding.
-        Blocking use of condvars is not supported.
+        Blocking use of condvars is not supported (in the main thread,
+        outside of e.g. Coro threads).
         Using event-based modules such as IO::AIO, Gtk2, Tk and so on is
         easy.
   Passing file descriptors
     the half it has passed earlier.
     Here is some (untested) pseudocode to that effect:
        use AnyEvent::Util;
+       use AnyEvent::Fork;
        use AnyEvent::Fork::RPC;
        use IO::FDPass;
        my ($s1, $s2) = AnyEvent::Util::portable_socketpair;
     Of course, this might be blocking if you pass a lot of file descriptors,
     so you might want to look into AnyEvent::FDpasser which can handle the
     gory details.
+EXCEPTIONS
+    There are no provisions whatsoever for catching exceptions at this time
+    - in the child, exceptions might kill the process, causing calls to be
+    lost and the parent encountering a fatal error. In the parent,
+    exceptions in the result callback will not be caught and cause undefined
+    behaviour.
 SEE ALSO
     AnyEvent::Fork, to create the processes in the first place.
+    AnyEvent::Fork::Remote, likewise, but helpful for remote processes.
     AnyEvent::Fork::Pool, to manage whole pools of processes.
 AUTHOR AND CONTACT INFORMATION
      Marc Lehmann <schmorp@schmorp.de>

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork-RPC/README (file contents): Revision 1.2 by root, Thu Apr 18 20:04:51 2013 UTC vs. Revision 1.5 by root, Thu May 12 16:54:43 2016 UTC

Diff Legend

Comparing AnyEvent-Fork-RPC/README (file contents):
Revision 1.2 by root, Thu Apr 18 20:04:51 2013 UTC vs.
Revision 1.5 by root, Thu May 12 16:54:43 2016 UTC