ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork-RPC/README
Revision: 1.7
Committed: Sun Sep 15 20:18:14 2019 UTC (5 years, 1 month ago) by root
Branch: MAIN
CVS Tags: HEAD
Changes since 1.6: +83 -6 lines
Log Message:
2.0

File Contents

# User Rev Content
1 root 1.2 NAME
2     AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork
3    
4     SYNOPSIS
5 root 1.4 use AnyEvent::Fork;
6 root 1.2 use AnyEvent::Fork::RPC;
7    
8     my $rpc = AnyEvent::Fork
9     ->new
10     ->require ("MyModule")
11     ->AnyEvent::Fork::RPC::run (
12     "MyModule::server",
13     );
14    
15     use AnyEvent;
16    
17     my $cv = AE::cv;
18    
19     $rpc->(1, 2, 3, sub {
20     print "MyModule::server returned @_\n";
21     $cv->send;
22     });
23    
24     $cv->recv;
25    
26     DESCRIPTION
27     This module implements a simple RPC protocol and backend for processes
28 root 1.4 created via AnyEvent::Fork or AnyEvent::Fork::Remote, allowing you to
29     call a function in the child process and receive its return values (up
30     to 4GB serialised).
31 root 1.2
32     It implements two different backends: a synchronous one that works like
33     a normal function call, and an asynchronous one that can run multiple
34     jobs concurrently in the child, using AnyEvent.
35    
36     It also implements an asynchronous event mechanism from the child to the
37     parent, that could be used for progress indications or other
38     information.
39    
40     EXAMPLES
41     Example 1: Synchronous Backend
42     Here is a simple example that implements a backend that executes
43     "unlink" and "rmdir" calls, and reports their status back. It also
44     reports the number of requests it has processed every three requests,
45     which is clearly silly, but illustrates the use of events.
46    
47     First the parent process:
48    
49     use AnyEvent;
50 root 1.4 use AnyEvent::Fork;
51 root 1.2 use AnyEvent::Fork::RPC;
52    
53     my $done = AE::cv;
54    
55     my $rpc = AnyEvent::Fork
56     ->new
57     ->require ("MyWorker")
58     ->AnyEvent::Fork::RPC::run ("MyWorker::run",
59 root 1.4 on_error => sub { warn "ERROR: $_[0]"; exit 1 },
60 root 1.2 on_event => sub { warn "$_[0] requests handled\n" },
61     on_destroy => $done,
62     );
63    
64     for my $id (1..6) {
65     $rpc->(rmdir => "/tmp/somepath/$id", sub {
66     $_[0]
67     or warn "/tmp/somepath/$id: $_[1]\n";
68     });
69     }
70    
71     undef $rpc;
72    
73     $done->recv;
74    
75     The parent creates the process, queues a few rmdir's. It then forgets
76     about the $rpc object, so that the child exits after it has handled the
77     requests, and then it waits till the requests have been handled.
78    
79     The child is implemented using a separate module, "MyWorker", shown
80     here:
81    
82     package MyWorker;
83    
84     my $count;
85    
86     sub run {
87     my ($cmd, $path) = @_;
88    
89     AnyEvent::Fork::RPC::event ($count)
90     unless ++$count % 3;
91    
92     my $status = $cmd eq "rmdir" ? rmdir $path
93     : $cmd eq "unlink" ? unlink $path
94     : die "fatal error, illegal command '$cmd'";
95    
96     $status or (0, "$!")
97     }
98    
99     1
100    
101     The "run" function first sends a "progress" event every three calls, and
102     then executes "rmdir" or "unlink", depending on the first parameter (or
103     dies with a fatal error - obviously, you must never let this happen :).
104    
105     Eventually it returns the status value true if the command was
106     successful, or the status value 0 and the stringified error message.
107    
108     On my system, running the first code fragment with the given MyWorker.pm
109     in the current directory yields:
110    
111     /tmp/somepath/1: No such file or directory
112     /tmp/somepath/2: No such file or directory
113     3 requests handled
114     /tmp/somepath/3: No such file or directory
115     /tmp/somepath/4: No such file or directory
116     /tmp/somepath/5: No such file or directory
117     6 requests handled
118     /tmp/somepath/6: No such file or directory
119    
120     Obviously, none of the directories I am trying to delete even exist.
121     Also, the events and responses are processed in exactly the same order
122     as they were created in the child, which is true for both synchronous
123     and asynchronous backends.
124    
125     Note that the parentheses in the call to "AnyEvent::Fork::RPC::event"
126     are not optional. That is because the function isn't defined when the
127     code is compiled. You can make sure it is visible by pre-loading the
128     correct backend module in the call to "require":
129    
130     ->require ("AnyEvent::Fork::RPC::Sync", "MyWorker")
131    
132     Since the backend module declares the "event" function, loading it first
133     ensures that perl will correctly interpret calls to it.
134    
135     And as a final remark, there is a fine module on CPAN that can
136     asynchronously "rmdir" and "unlink" and a lot more, and more efficiently
137     than this example, namely IO::AIO.
138    
139     Example 1a: the same with the asynchronous backend
140     This example only shows what needs to be changed to use the async
141     backend instead. Doing this is not very useful, the purpose of this
142     example is to show the minimum amount of change that is required to go
143     from the synchronous to the asynchronous backend.
144    
145     To use the async backend in the previous example, you need to add the
146     "async" parameter to the "AnyEvent::Fork::RPC::run" call:
147    
148     ->AnyEvent::Fork::RPC::run ("MyWorker::run",
149     async => 1,
150     ...
151    
152     And since the function call protocol is now changed, you need to adopt
153     "MyWorker::run" to the async API.
154    
155     First, you need to accept the extra initial $done callback:
156    
157     sub run {
158     my ($done, $cmd, $path) = @_;
159    
160     And since a response is now generated when $done is called, as opposed
161     to when the function returns, we need to call the $done function with
162     the status:
163    
164     $done->($status or (0, "$!"));
165    
166     A few remarks are in order. First, it's quite pointless to use the async
167     backend for this example - but it *is* possible. Second, you can call
168     $done before or after returning from the function. Third, having both
169     returned from the function and having called the $done callback, the
170     child process may exit at any time, so you should call $done only when
171     you really *are* done.
172    
173     Example 2: Asynchronous Backend
174     This example implements multiple count-downs in the child, using
175 root 1.5 AnyEvent timers. While this is a bit silly (one could use timers in the
176 root 1.2 parent just as well), it illustrates the ability to use AnyEvent in the
177     child and the fact that responses can arrive in a different order then
178     the requests.
179    
180     It also shows how to embed the actual child code into a "__DATA__"
181     section, so it doesn't need any external files at all.
182    
183     And when your parent process is often busy, and you have stricter timing
184     requirements, then running timers in a child process suddenly doesn't
185     look so silly anymore.
186    
187     Without further ado, here is the code:
188    
189     use AnyEvent;
190 root 1.4 use AnyEvent::Fork;
191 root 1.2 use AnyEvent::Fork::RPC;
192    
193     my $done = AE::cv;
194    
195     my $rpc = AnyEvent::Fork
196     ->new
197     ->require ("AnyEvent::Fork::RPC::Async")
198     ->eval (do { local $/; <DATA> })
199     ->AnyEvent::Fork::RPC::run ("run",
200     async => 1,
201 root 1.4 on_error => sub { warn "ERROR: $_[0]"; exit 1 },
202 root 1.2 on_event => sub { print $_[0] },
203     on_destroy => $done,
204     );
205    
206     for my $count (3, 2, 1) {
207     $rpc->($count, sub {
208     warn "job $count finished\n";
209     });
210     }
211    
212     undef $rpc;
213    
214     $done->recv;
215    
216     __DATA__
217    
218     # this ends up in main, as we don't use a package declaration
219    
220     use AnyEvent;
221    
222     sub run {
223     my ($done, $count) = @_;
224    
225     my $n;
226    
227     AnyEvent::Fork::RPC::event "starting to count up to $count\n";
228    
229     my $w; $w = AE::timer 1, 1, sub {
230     ++$n;
231    
232     AnyEvent::Fork::RPC::event "count $n of $count\n";
233    
234     if ($n == $count) {
235     undef $w;
236     $done->();
237     }
238     };
239     }
240    
241     The parent part (the one before the "__DATA__" section) isn't very
242     different from the earlier examples. It sets async mode, preloads the
243     backend module (so the "AnyEvent::Fork::RPC::event" function is
244     declared), uses a slightly different "on_event" handler (which we use
245     simply for logging purposes) and then, instead of loading a module with
246     the actual worker code, it "eval"'s the code from the data section in
247     the child process.
248    
249     It then starts three countdowns, from 3 to 1 seconds downwards, destroys
250     the rpc object so the example finishes eventually, and then just waits
251     for the stuff to trickle in.
252    
253     The worker code uses the event function to log some progress messages,
254     but mostly just creates a recurring one-second timer.
255    
256     The timer callback increments a counter, logs a message, and eventually,
257     when the count has been reached, calls the finish callback.
258    
259     On my system, this results in the following output. Since all timers
260     fire at roughly the same time, the actual order isn't guaranteed, but
261     the order shown is very likely what you would get, too.
262    
263     starting to count up to 3
264     starting to count up to 2
265     starting to count up to 1
266     count 1 of 3
267     count 1 of 2
268     count 1 of 1
269     job 1 finished
270     count 2 of 2
271     job 2 finished
272     count 2 of 3
273     count 3 of 3
274     job 3 finished
275    
276     While the overall ordering isn't guaranteed, the async backend still
277     guarantees that events and responses are delivered to the parent process
278     in the exact same ordering as they were generated in the child process.
279    
280     And unless your system is *very* busy, it should clearly show that the
281     job started last will finish first, as it has the lowest count.
282    
283     This concludes the async example. Since AnyEvent::Fork does not actually
284     fork, you are free to use about any module in the child, not just
285     AnyEvent, but also IO::AIO, or Tk for example.
286    
287 root 1.4 Example 3: Asynchronous backend with Coro
288     With Coro you can create a nice asynchronous backend implementation by
289     defining an rpc server function that creates a new Coro thread for every
290     request that calls a function "normally", i.e. the parameters from the
291     parent process are passed to it, and any return values are returned to
292     the parent process, e.g.:
293    
294     package My::Arith;
295    
296     sub add {
297     return $_[0] + $_[1];
298     }
299    
300     sub mul {
301     return $_[0] * $_[1];
302     }
303    
304     sub run {
305     my ($done, $func, @arg) = @_;
306    
307     Coro::async_pool {
308     $done->($func->(@arg));
309     };
310     }
311    
312     The "run" function creates a new thread for every invocation, using the
313     first argument as function name, and calls the $done callback on it's
314     return values. This makes it quite natural to define the "add" and "mul"
315     functions to add or multiply two numbers and return the result.
316    
317     Since this is the asynchronous backend, it's quite possible to define
318     RPC function that do I/O or wait for external events - their execution
319     will overlap as needed.
320    
321     The above could be used like this:
322    
323     my $rpc = AnyEvent::Fork
324     ->new
325     ->require ("MyWorker")
326     ->AnyEvent::Fork::RPC::run ("My::Arith::run",
327     on_error => ..., on_event => ..., on_destroy => ...,
328     );
329    
330     $rpc->(add => 1, 3, Coro::rouse_cb); say Coro::rouse_wait;
331     $rpc->(mul => 3, 2, Coro::rouse_cb); say Coro::rouse_wait;
332    
333     The "say"'s will print 4 and 6.
334    
335     Example 4: Forward AnyEvent::Log messages using "on_event"
336     This partial example shows how to use the "event" function to forward
337     AnyEvent::Log messages to the parent.
338    
339     For this, the parent needs to provide a suitable "on_event":
340    
341     ->AnyEvent::Fork::RPC::run (
342     on_event => sub {
343     if ($_[0] eq "ae_log") {
344     my (undef, $level, $message) = @_;
345     AE::log $level, $message;
346     } else {
347     # other event types
348     }
349     },
350     )
351    
352     In the child, as early as possible, the following code should
353     reconfigure AnyEvent::Log to log via "AnyEvent::Fork::RPC::event":
354    
355     $AnyEvent::Log::LOG->log_cb (sub {
356     my ($timestamp, $orig_ctx, $level, $message) = @{+shift};
357    
358     if (defined &AnyEvent::Fork::RPC::event) {
359     AnyEvent::Fork::RPC::event (ae_log => $level, $message);
360     } else {
361     warn "[$$ before init] $message\n";
362     }
363     });
364    
365     There is an important twist - the "AnyEvent::Fork::RPC::event" function
366     is only defined when the child is fully initialised. If you redirect the
367     log messages in your "init" function for example, then the "event"
368     function might not yet be available. This is why the log callback checks
369 root 1.7 whether the function is there using "defined", and only then uses it to
370 root 1.4 log the message.
371    
372 root 1.2 PARENT PROCESS USAGE
373     This module exports nothing, and only implements a single function:
374    
375     my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]
376     The traditional way to call it. But it is way cooler to call it in
377     the following way:
378    
379     my $rpc = $fork->AnyEvent::Fork::RPC::run ($function, [key => value...])
380     This "run" function/method can be used in place of the
381     AnyEvent::Fork::run method. Just like that method, it takes over the
382     AnyEvent::Fork process, but instead of calling the specified
383     $function directly, it runs a server that accepts RPC calls and
384     handles responses.
385    
386     It returns a function reference that can be used to call the
387     function in the child process, handling serialisation and data
388     transfers.
389    
390     The following key/value pairs are allowed. It is recommended to have
391     at least an "on_error" or "on_event" handler set.
392    
393     on_error => $cb->($msg)
394     Called on (fatal) errors, with a descriptive (hopefully)
395     message. If this callback is not provided, but "on_event" is,
396     then the "on_event" callback is called with the first argument
397     being the string "error", followed by the error message.
398    
399 root 1.4 If neither handler is provided, then the error is reported with
400     loglevel "error" via "AE::log".
401 root 1.2
402     on_event => $cb->(...)
403     Called for every call to the "AnyEvent::Fork::RPC::event"
404     function in the child, with the arguments of that function
405     passed to the callback.
406    
407     Also called on errors when no "on_error" handler is provided.
408    
409     on_destroy => $cb->()
410     Called when the $rpc object has been destroyed and all requests
411     have been successfully handled. This is useful when you queue
412     some requests and want the child to go away after it has handled
413     them. The problem is that the parent must not exit either until
414     all requests have been handled, and this can be accomplished by
415     waiting for this callback.
416    
417 root 1.7 init => $function (default: none)
418 root 1.2 When specified (by name), this function is called in the child
419     as the very first thing when taking over the process, with all
420     the arguments normally passed to the "AnyEvent::Fork::run"
421     function, except the communications socket.
422    
423     It can be used to do one-time things in the child such as
424     storing passed parameters or opening database connections.
425    
426     It is called very early - before the serialisers are created or
427     the $function name is resolved into a function reference, so it
428     could be used to load any modules that provide the serialiser or
429     function. It can not, however, create events.
430    
431 root 1.7 done => $function (default: "CORE::exit")
432 root 1.4 The function to call when the asynchronous backend detects an
433     end of file condition when reading from the communications
434 root 1.7 socket *and* there are no outstanding requests. It is ignored by
435 root 1.4 the synchronous backend.
436    
437     By overriding this you can prolong the life of a RPC process
438     after e.g. the parent has exited by running the event loop in
439     the provided function (or simply calling it, for example, when
440 root 1.5 your child process uses EV you could provide EV::run as "done"
441 root 1.4 function).
442    
443     Of course, in that case you are responsible for exiting at the
444     appropriate time and not returning from
445    
446 root 1.2 async => $boolean (default: 0)
447     The default server used in the child does all I/O blockingly,
448     and only allows a single RPC call to execute concurrently.
449    
450     Setting "async" to a true value switches to another
451     implementation that uses AnyEvent in the child and allows
452     multiple concurrent RPC calls (it does not support recursion in
453     the event loop however, blocking condvar calls will fail).
454    
455     The actual API in the child is documented in the section that
456     describes the calling semantics of the returned $rpc function.
457    
458     If you want to pre-load the actual back-end modules to enable
459     memory sharing, then you should load "AnyEvent::Fork::RPC::Sync"
460     for synchronous, and "AnyEvent::Fork::RPC::Async" for
461     asynchronous mode.
462    
463     If you use a template process and want to fork both sync and
464     async children, then it is permissible to load both modules.
465    
466     serialiser => $string (default:
467     $AnyEvent::Fork::RPC::STRING_SERIALISER)
468     All arguments, result data and event data have to be serialised
469     to be transferred between the processes. For this, they have to
470     be frozen and thawed in both parent and child processes.
471    
472     By default, only octet strings can be passed between the
473     processes, which is reasonably fast and efficient and requires
474 root 1.5 no extra modules (the "AnyEvent::Fork::RPC" distribution does
475     not provide these extra serialiser modules).
476 root 1.2
477     For more complicated use cases, you can provide your own freeze
478     and thaw functions, by specifying a string with perl source
479     code. It's supposed to return two code references when
480     evaluated: the first receives a list of perl values and must
481     return an octet string. The second receives the octet string and
482     must return the original list of values.
483    
484     If you need an external module for serialisation, then you can
485     either pre-load it into your AnyEvent::Fork process, or you can
486     add a "use" or "require" statement into the serialiser string.
487     Or both.
488    
489 root 1.5 Here are some examples - all of them are also available as
490 root 1.2 global variables that make them easier to use.
491    
492 root 1.5 $AnyEvent::Fork::RPC::STRING_SERIALISER - octet strings only
493     This serialiser (currently the default) concatenates
494     length-prefixes octet strings, and is the default. That
495     means you can only pass (and return) strings containing
496     character codes 0-255.
497    
498     The main advantages of this serialiser are the high speed
499     and that it doesn't need another module. The main
500     disadvantage is that you are very limited in what you can
501     pass - only octet strings.
502 root 1.2
503     Implementation:
504    
505     (
506     sub { pack "(w/a*)*", @_ },
507     sub { unpack "(w/a*)*", shift }
508     )
509    
510 root 1.5 $AnyEvent::Fork::RPC::CBOR_XS_SERIALISER - uses CBOR::XS
511     This serialiser creates CBOR::XS arrays - you have to make
512     sure the CBOR::XS module is installed for this serialiser to
513     work. It can be beneficial for sharing when you preload the
514     CBOR::XS module in a template process.
515    
516     CBOR::XS is about as fast as the octet string serialiser,
517     but supports complex data structures (similar to JSON) and
518     is faster than any of the other serialisers. If you have the
519     CBOR::XS module available, it's the best choice.
520    
521     The encoder enables "allow_sharing" (so this serialisation
522     method can encode cyclic and self-referencing data
523     structures).
524    
525     Implementation:
526    
527     use CBOR::XS ();
528     (
529     sub { CBOR::XS::encode_cbor_sharing \@_ },
530     sub { @{ CBOR::XS::decode_cbor shift } }
531     )
532    
533     $AnyEvent::Fork::RPC::JSON_SERIALISER - uses JSON::XS or JSON
534 root 1.2 This serialiser creates JSON arrays - you have to make sure
535     the JSON module is installed for this serialiser to work. It
536     can be beneficial for sharing when you preload the JSON
537     module in a template process.
538    
539     JSON (with JSON::XS installed) is slower than the octet
540     string serialiser, but usually much faster than Storable,
541     unless big chunks of binary data need to be transferred.
542    
543     Implementation:
544    
545     use JSON ();
546     (
547     sub { JSON::encode_json \@_ },
548     sub { @{ JSON::decode_json shift } }
549     )
550    
551 root 1.5 $AnyEvent::Fork::RPC::STORABLE_SERIALISER - Storable
552 root 1.2 This serialiser uses Storable, which means it has high
553     chance of serialising just about anything you throw at it,
554     at the cost of having very high overhead per operation. It
555 root 1.4 also comes with perl. It should be used when you need to
556     serialise complex data structures.
557 root 1.2
558     Implementation:
559    
560     use Storable ();
561     (
562     sub { Storable::freeze \@_ },
563     sub { @{ Storable::thaw shift } }
564     )
565    
566 root 1.5 $AnyEvent::Fork::RPC::NSTORABLE_SERIALISER - portable Storable
567 root 1.4 This serialiser also uses Storable, but uses it's "network"
568     format to serialise data, which makes it possible to talk to
569     different perl binaries (for example, when talking to a
570     process created with AnyEvent::Fork::Remote).
571    
572     Implementation:
573    
574     use Storable ();
575     (
576     sub { Storable::nfreeze \@_ },
577     sub { @{ Storable::thaw shift } }
578     )
579    
580 root 1.7 buflen => $bytes (default: "512 - 16")
581     The starting size of the read buffer for request and response
582     data.
583    
584     "AnyEvent::Fork::RPC" ensures that the buffer for reeading
585     request and response data is large enough for at leats aingle
586     request or response, and will dynamically enlarge the buffer if
587     needed.
588    
589     While this ensures that memory is not overly wasted, it
590     typically leads to having to do one syscall per request, which
591     can be inefficient in some cases. In such cases, it can be
592     beneficient to increase the buffer size to hold more than one
593     request.
594    
595     buflen_req => $bytes (default: same as "buflen")
596     Overrides "buflen" for request data (as read by the forked
597     process).
598    
599     buflen_res => $bytes (default: same as "buflen")
600     Overrides "buflen" for response data (replies read by the parent
601     process).
602    
603 root 1.2 See the examples section earlier in this document for some actual
604     examples.
605    
606     $rpc->(..., $cb->(...))
607     The RPC object returned by "AnyEvent::Fork::RPC::run" is actually a
608     code reference. There are two things you can do with it: call it,
609     and let it go out of scope (let it get destroyed).
610    
611     If "async" was false when $rpc was created (the default), then, if
612     you call $rpc, the $function is invoked with all arguments passed to
613     $rpc except the last one (the callback). When the function returns,
614     the callback will be invoked with all the return values.
615    
616     If "async" was true, then the $function receives an additional
617     initial argument, the result callback. In this case, returning from
618     $function does nothing - the function only counts as "done" when the
619     result callback is called, and any arguments passed to it are
620     considered the return values. This makes it possible to "return"
621     from event handlers or e.g. Coro threads.
622    
623     The other thing that can be done with the RPC object is to destroy
624     it. In this case, the child process will execute all remaining RPC
625     calls, report their results, and then exit.
626    
627     See the examples section earlier in this document for some actual
628     examples.
629    
630     CHILD PROCESS USAGE
631     The following function is not available in this module. They are only
632     available in the namespace of this module when the child is running,
633     without having to load any extra modules. They are part of the
634     child-side API of AnyEvent::Fork::RPC.
635    
636 root 1.7 Note that these functions are typically not yet declared when code is
637     compiled into the child, because the backend module is only loaded when
638     you call "run", which is typically the last method you call on the fork
639     object.
640    
641     Therefore, you either have to explicitly pre-load the right backend
642     module or mark calls to these functions as function calls, e.g.:
643    
644     AnyEvent::Fork::RPC::event (0 => "five");
645     AnyEvent::Fork::RPC::event->(0 => "five");
646     &AnyEvent::Fork::RPC::flush;
647    
648     AnyEvent::Fork::RPC::event (...)
649 root 1.2 Send an event to the parent. Events are a bit like RPC calls made by
650     the child process to the parent, except that there is no notion of
651     return values.
652    
653     See the examples section earlier in this document for some actual
654     examples.
655    
656 root 1.6 Note: the event data, like any data send to the parent, might not be
657     sent immediatelly but queued for later sending, so there is no
658     guarantee that the event has been sent to the parent when the call
659     returns - when you e.g. exit directly after calling this function,
660 root 1.7 the parent might never receive the event. See the next function for
661     a remedy.
662    
663     $success = AnyEvent::Fork::RPC::flush ()
664     Synchronously wait and flush the reply data to the parent. Returns
665     true on success and false otherwise (i.e. when the reply data cannot
666     be written at all). Ignoring the success status is a common and
667     healthy behaviour.
668    
669     Only the "async" backend does something on "flush" - the "sync"
670     backend is not buffering reply data and always returns true from
671     this function.
672    
673     Normally, reply data might or might not be written to the parent
674     immediatelly but is buffered. This can greatly improve performance
675     and efficiency, but sometimes can get in your way: for example. when
676     you want to send an error message just before exiting, or when you
677     want to ensure replies timely reach the parent before starting a
678     long blocking operation.
679    
680     In these cases, you can call this function to flush any outstanding
681     reply data to the parent. This is done blockingly, so no requests
682     will be handled and no event callbacks will be called.
683    
684     For example, you could wrap your request function in a "eval" block
685     and report the exception string back to the caller just before
686     exiting:
687    
688     sub req {
689     ...
690    
691     eval {
692     ...
693     };
694    
695     if ($@) {
696     AnyEvent::RPC::event (throw => "$@");
697     AnyEvent::RPC::flush ();
698     exit;
699     }
700    
701     ...
702     }
703 root 1.6
704 root 1.4 PROCESS EXIT
705     If and when the child process exits depends on the backend and
706     configuration. Apart from explicit exits (e.g. by calling "exit") or
707     runtime conditions (uncaught exceptions, signals etc.), the backends
708     exit under these conditions:
709    
710     Synchronous Backend
711     The synchronous backend is very simple: when the process waits for
712     another request to arrive and the writing side (usually in the
713     parent) is closed, it will exit normally, i.e. as if your main
714     program reached the end of the file.
715    
716     That means that if your parent process exits, the RPC process will
717     usually exit as well, either because it is idle anyway, or because
718     it executes a request. In the latter case, you will likely get an
719     error when the RPc process tries to send the results to the parent
720     (because agruably, you shouldn't exit your parent while there are
721     still outstanding requests).
722    
723     The process is usually quiescent when it happens, so it should
724     rarely be a problem, and "END" handlers can be used to clean up.
725    
726     Asynchronous Backend
727     For the asynchronous backend, things are more complicated: Whenever
728     it listens for another request by the parent, it might detect that
729     the socket was closed (e.g. because the parent exited). It will sotp
730     listening for new requests and instead try to write out any
731 root 1.5 remaining data (if any) or simply check whether the socket can be
732 root 1.4 written to. After this, the RPC process is effectively done - no new
733     requests are incoming, no outstanding request data can be written
734     back.
735    
736     Since chances are high that there are event watchers that the RPC
737     server knows nothing about (why else would one use the async backend
738     if not for the ability to register watchers?), the event loop would
739     often happily continue.
740    
741     This is why the asynchronous backend explicitly calls "CORE::exit"
742     when it is done (under other circumstances, such as when there is an
743     I/O error and there is outstanding data to write, it will log a
744     fatal message via AnyEvent::Log, also causing the program to exit).
745    
746     You can override this by specifying a function name to call via the
747     "done" parameter instead.
748    
749 root 1.2 ADVANCED TOPICS
750     Choosing a backend
751     So how do you decide which backend to use? Well, that's your problem to
752     solve, but here are some thoughts on the matter:
753    
754     Synchronous
755     The synchronous backend does not rely on any external modules (well,
756     except common::sense, which works around a bug in how perl's warning
757     system works). This keeps the process very small, for example, on my
758     system, an empty perl interpreter uses 1492kB RSS, which becomes
759     2020kB after "use warnings; use strict" (for people who grew up with
760     C64s around them this is probably shocking every single time they
761     see it). The worker process in the first example in this document
762     uses 1792kB.
763    
764     Since the calls are done synchronously, slow jobs will keep newer
765     jobs from executing.
766    
767     The synchronous backend also has no overhead due to running an event
768     loop - reading requests is therefore very efficient, while writing
769     responses is less so, as every response results in a write syscall.
770    
771     If the parent process is busy and a bit slow reading responses, the
772     child waits instead of processing further requests. This also limits
773     the amount of memory needed for buffering, as never more than one
774     response has to be buffered.
775    
776     The API in the child is simple - you just have to define a function
777     that does something and returns something.
778    
779     It's hard to use modules or code that relies on an event loop, as
780     the child cannot execute anything while it waits for more input.
781    
782     Asynchronous
783     The asynchronous backend relies on AnyEvent, which tries to be
784     small, but still comes at a price: On my system, the worker from
785     example 1a uses 3420kB RSS (for AnyEvent, which loads EV, which
786     needs XSLoader which in turn loads a lot of other modules such as
787     warnings, strict, vars, Exporter...).
788    
789     It batches requests and responses reasonably efficiently, doing only
790     as few reads and writes as needed, but needs to poll for events via
791     the event loop.
792    
793     Responses are queued when the parent process is busy. This means the
794     child can continue to execute any queued requests. It also means
795     that a child might queue a lot of responses in memory when it
796     generates them and the parent process is slow accepting them.
797    
798     The API is not a straightforward RPC pattern - you have to call a
799     "done" callback to pass return values and signal completion. Also,
800     more importantly, the API starts jobs as fast as possible - when
801     1000 jobs are queued and the jobs are slow, they will all run
802     concurrently. The child must implement some queueing/limiting
803     mechanism if this causes problems. Alternatively, the parent could
804     limit the amount of rpc calls that are outstanding.
805    
806 root 1.5 Blocking use of condvars is not supported (in the main thread,
807     outside of e.g. Coro threads).
808 root 1.2
809     Using event-based modules such as IO::AIO, Gtk2, Tk and so on is
810     easy.
811    
812     Passing file descriptors
813     Unlike AnyEvent::Fork, this module has no in-built file handle or file
814     descriptor passing abilities.
815    
816     The reason is that passing file descriptors is extraordinary tricky
817     business, and conflicts with efficient batching of messages.
818    
819     There still is a method you can use: Create a
820     "AnyEvent::Util::portable_socketpair" and "send_fh" one half of it to
821     the process before you pass control to "AnyEvent::Fork::RPC::run".
822    
823     Whenever you want to pass a file descriptor, send an rpc request to the
824     child process (so it expects the descriptor), then send it over the
825     other half of the socketpair. The child should fetch the descriptor from
826     the half it has passed earlier.
827    
828     Here is some (untested) pseudocode to that effect:
829    
830     use AnyEvent::Util;
831 root 1.4 use AnyEvent::Fork;
832 root 1.2 use AnyEvent::Fork::RPC;
833     use IO::FDPass;
834    
835     my ($s1, $s2) = AnyEvent::Util::portable_socketpair;
836    
837     my $rpc = AnyEvent::Fork
838     ->new
839     ->send_fh ($s2)
840     ->require ("MyWorker")
841     ->AnyEvent::Fork::RPC::run ("MyWorker::run"
842     init => "MyWorker::init",
843     );
844    
845     undef $s2; # no need to keep it around
846    
847     # pass an fd
848     $rpc->("i'll send some fd now, please expect it!", my $cv = AE::cv);
849    
850     IO::FDPass fileno $s1, fileno $handle_to_pass;
851    
852     $cv->recv;
853    
854     The MyWorker module could look like this:
855    
856     package MyWorker;
857    
858     use IO::FDPass;
859    
860     my $s2;
861    
862     sub init {
863     $s2 = $_[0];
864     }
865    
866     sub run {
867     if ($_[0] eq "i'll send some fd now, please expect it!") {
868     my $fd = IO::FDPass::recv fileno $s2;
869     ...
870     }
871     }
872    
873     Of course, this might be blocking if you pass a lot of file descriptors,
874     so you might want to look into AnyEvent::FDpasser which can handle the
875     gory details.
876    
877 root 1.3 EXCEPTIONS
878     There are no provisions whatsoever for catching exceptions at this time
879 root 1.5 - in the child, exceptions might kill the process, causing calls to be
880 root 1.3 lost and the parent encountering a fatal error. In the parent,
881     exceptions in the result callback will not be caught and cause undefined
882     behaviour.
883    
884 root 1.2 SEE ALSO
885     AnyEvent::Fork, to create the processes in the first place.
886    
887 root 1.4 AnyEvent::Fork::Remote, likewise, but helpful for remote processes.
888    
889 root 1.2 AnyEvent::Fork::Pool, to manage whole pools of processes.
890    
891     AUTHOR AND CONTACT INFORMATION
892     Marc Lehmann <schmorp@schmorp.de>
893     http://software.schmorp.de/pkg/AnyEvent-Fork-RPC
894