ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork-RPC/README
(Generate patch)

Comparing AnyEvent-Fork-RPC/README (file contents):
Revision 1.3 by root, Sun Apr 21 12:27:03 2013 UTC vs.
Revision 1.4 by root, Wed Sep 25 11:06:11 2013 UTC

1NAME 1NAME
2 AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork 2 AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork
3 3
4 THE API IS NOT FINISHED, CONSIDER THIS A BETA RELEASE
5
6SYNOPSIS 4SYNOPSIS
5 use AnyEvent::Fork;
7 use AnyEvent::Fork::RPC; 6 use AnyEvent::Fork::RPC;
8 # use AnyEvent::Fork is not needed
9 7
10 my $rpc = AnyEvent::Fork 8 my $rpc = AnyEvent::Fork
11 ->new 9 ->new
12 ->require ("MyModule") 10 ->require ("MyModule")
13 ->AnyEvent::Fork::RPC::run ( 11 ->AnyEvent::Fork::RPC::run (
25 23
26 $cv->recv; 24 $cv->recv;
27 25
28DESCRIPTION 26DESCRIPTION
29 This module implements a simple RPC protocol and backend for processes 27 This module implements a simple RPC protocol and backend for processes
30 created via AnyEvent::Fork, allowing you to call a function in the child 28 created via AnyEvent::Fork or AnyEvent::Fork::Remote, allowing you to
31 process and receive its return values (up to 4GB serialised). 29 call a function in the child process and receive its return values (up
30 to 4GB serialised).
32 31
33 It implements two different backends: a synchronous one that works like 32 It implements two different backends: a synchronous one that works like
34 a normal function call, and an asynchronous one that can run multiple 33 a normal function call, and an asynchronous one that can run multiple
35 jobs concurrently in the child, using AnyEvent. 34 jobs concurrently in the child, using AnyEvent.
36 35
37 It also implements an asynchronous event mechanism from the child to the 36 It also implements an asynchronous event mechanism from the child to the
38 parent, that could be used for progress indications or other 37 parent, that could be used for progress indications or other
39 information. 38 information.
40
41 Loading this module also always loads AnyEvent::Fork, so you can make a
42 separate "use AnyEvent::Fork" if you wish, but you don't have to.
43 39
44EXAMPLES 40EXAMPLES
45 Example 1: Synchronous Backend 41 Example 1: Synchronous Backend
46 Here is a simple example that implements a backend that executes 42 Here is a simple example that implements a backend that executes
47 "unlink" and "rmdir" calls, and reports their status back. It also 43 "unlink" and "rmdir" calls, and reports their status back. It also
49 which is clearly silly, but illustrates the use of events. 45 which is clearly silly, but illustrates the use of events.
50 46
51 First the parent process: 47 First the parent process:
52 48
53 use AnyEvent; 49 use AnyEvent;
50 use AnyEvent::Fork;
54 use AnyEvent::Fork::RPC; 51 use AnyEvent::Fork::RPC;
55 52
56 my $done = AE::cv; 53 my $done = AE::cv;
57 54
58 my $rpc = AnyEvent::Fork 55 my $rpc = AnyEvent::Fork
59 ->new 56 ->new
60 ->require ("MyWorker") 57 ->require ("MyWorker")
61 ->AnyEvent::Fork::RPC::run ("MyWorker::run", 58 ->AnyEvent::Fork::RPC::run ("MyWorker::run",
62 on_error => sub { warn "FATAL: $_[0]"; exit 1 }, 59 on_error => sub { warn "ERROR: $_[0]"; exit 1 },
63 on_event => sub { warn "$_[0] requests handled\n" }, 60 on_event => sub { warn "$_[0] requests handled\n" },
64 on_destroy => $done, 61 on_destroy => $done,
65 ); 62 );
66 63
67 for my $id (1..6) { 64 for my $id (1..6) {
188 look so silly anymore. 185 look so silly anymore.
189 186
190 Without further ado, here is the code: 187 Without further ado, here is the code:
191 188
192 use AnyEvent; 189 use AnyEvent;
190 use AnyEvent::Fork;
193 use AnyEvent::Fork::RPC; 191 use AnyEvent::Fork::RPC;
194 192
195 my $done = AE::cv; 193 my $done = AE::cv;
196 194
197 my $rpc = AnyEvent::Fork 195 my $rpc = AnyEvent::Fork
198 ->new 196 ->new
199 ->require ("AnyEvent::Fork::RPC::Async") 197 ->require ("AnyEvent::Fork::RPC::Async")
200 ->eval (do { local $/; <DATA> }) 198 ->eval (do { local $/; <DATA> })
201 ->AnyEvent::Fork::RPC::run ("run", 199 ->AnyEvent::Fork::RPC::run ("run",
202 async => 1, 200 async => 1,
203 on_error => sub { warn "FATAL: $_[0]"; exit 1 }, 201 on_error => sub { warn "ERROR: $_[0]"; exit 1 },
204 on_event => sub { print $_[0] }, 202 on_event => sub { print $_[0] },
205 on_destroy => $done, 203 on_destroy => $done,
206 ); 204 );
207 205
208 for my $count (3, 2, 1) { 206 for my $count (3, 2, 1) {
284 282
285 This concludes the async example. Since AnyEvent::Fork does not actually 283 This concludes the async example. Since AnyEvent::Fork does not actually
286 fork, you are free to use about any module in the child, not just 284 fork, you are free to use about any module in the child, not just
287 AnyEvent, but also IO::AIO, or Tk for example. 285 AnyEvent, but also IO::AIO, or Tk for example.
288 286
287 Example 3: Asynchronous backend with Coro
288 With Coro you can create a nice asynchronous backend implementation by
289 defining an rpc server function that creates a new Coro thread for every
290 request that calls a function "normally", i.e. the parameters from the
291 parent process are passed to it, and any return values are returned to
292 the parent process, e.g.:
293
294 package My::Arith;
295
296 sub add {
297 return $_[0] + $_[1];
298 }
299
300 sub mul {
301 return $_[0] * $_[1];
302 }
303
304 sub run {
305 my ($done, $func, @arg) = @_;
306
307 Coro::async_pool {
308 $done->($func->(@arg));
309 };
310 }
311
312 The "run" function creates a new thread for every invocation, using the
313 first argument as function name, and calls the $done callback on it's
314 return values. This makes it quite natural to define the "add" and "mul"
315 functions to add or multiply two numbers and return the result.
316
317 Since this is the asynchronous backend, it's quite possible to define
318 RPC function that do I/O or wait for external events - their execution
319 will overlap as needed.
320
321 The above could be used like this:
322
323 my $rpc = AnyEvent::Fork
324 ->new
325 ->require ("MyWorker")
326 ->AnyEvent::Fork::RPC::run ("My::Arith::run",
327 on_error => ..., on_event => ..., on_destroy => ...,
328 );
329
330 $rpc->(add => 1, 3, Coro::rouse_cb); say Coro::rouse_wait;
331 $rpc->(mul => 3, 2, Coro::rouse_cb); say Coro::rouse_wait;
332
333 The "say"'s will print 4 and 6.
334
335 Example 4: Forward AnyEvent::Log messages using "on_event"
336 This partial example shows how to use the "event" function to forward
337 AnyEvent::Log messages to the parent.
338
339 For this, the parent needs to provide a suitable "on_event":
340
341 ->AnyEvent::Fork::RPC::run (
342 on_event => sub {
343 if ($_[0] eq "ae_log") {
344 my (undef, $level, $message) = @_;
345 AE::log $level, $message;
346 } else {
347 # other event types
348 }
349 },
350 )
351
352 In the child, as early as possible, the following code should
353 reconfigure AnyEvent::Log to log via "AnyEvent::Fork::RPC::event":
354
355 $AnyEvent::Log::LOG->log_cb (sub {
356 my ($timestamp, $orig_ctx, $level, $message) = @{+shift};
357
358 if (defined &AnyEvent::Fork::RPC::event) {
359 AnyEvent::Fork::RPC::event (ae_log => $level, $message);
360 } else {
361 warn "[$$ before init] $message\n";
362 }
363 });
364
365 There is an important twist - the "AnyEvent::Fork::RPC::event" function
366 is only defined when the child is fully initialised. If you redirect the
367 log messages in your "init" function for example, then the "event"
368 function might not yet be available. This is why the log callback checks
369 whether the fucntion is there using "defined", and only then uses it to
370 log the message.
371
289PARENT PROCESS USAGE 372PARENT PROCESS USAGE
290 This module exports nothing, and only implements a single function: 373 This module exports nothing, and only implements a single function:
291 374
292 my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...] 375 my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]
293 The traditional way to call it. But it is way cooler to call it in 376 The traditional way to call it. But it is way cooler to call it in
311 Called on (fatal) errors, with a descriptive (hopefully) 394 Called on (fatal) errors, with a descriptive (hopefully)
312 message. If this callback is not provided, but "on_event" is, 395 message. If this callback is not provided, but "on_event" is,
313 then the "on_event" callback is called with the first argument 396 then the "on_event" callback is called with the first argument
314 being the string "error", followed by the error message. 397 being the string "error", followed by the error message.
315 398
316 If neither handler is provided it prints the error to STDERR and 399 If neither handler is provided, then the error is reported with
317 will start failing badly. 400 loglevel "error" via "AE::log".
318 401
319 on_event => $cb->(...) 402 on_event => $cb->(...)
320 Called for every call to the "AnyEvent::Fork::RPC::event" 403 Called for every call to the "AnyEvent::Fork::RPC::event"
321 function in the child, with the arguments of that function 404 function in the child, with the arguments of that function
322 passed to the callback. 405 passed to the callback.
343 It is called very early - before the serialisers are created or 426 It is called very early - before the serialisers are created or
344 the $function name is resolved into a function reference, so it 427 the $function name is resolved into a function reference, so it
345 could be used to load any modules that provide the serialiser or 428 could be used to load any modules that provide the serialiser or
346 function. It can not, however, create events. 429 function. It can not, however, create events.
347 430
431 done => $function (default "CORE::exit")
432 The function to call when the asynchronous backend detects an
433 end of file condition when reading from the communications
434 socket *and* there are no outstanding requests. It's ignored by
435 the synchronous backend.
436
437 By overriding this you can prolong the life of a RPC process
438 after e.g. the parent has exited by running the event loop in
439 the provided function (or simply calling it, for example, when
440 your child process uses EV you could provide EV::loop as "done"
441 function).
442
443 Of course, in that case you are responsible for exiting at the
444 appropriate time and not returning from
445
348 async => $boolean (default: 0) 446 async => $boolean (default: 0)
349 The default server used in the child does all I/O blockingly, 447 The default server used in the child does all I/O blockingly,
350 and only allows a single RPC call to execute concurrently. 448 and only allows a single RPC call to execute concurrently.
351 449
352 Setting "async" to a true value switches to another 450 Setting "async" to a true value switches to another
390 Here are some examples - some of them are also available as 488 Here are some examples - some of them are also available as
391 global variables that make them easier to use. 489 global variables that make them easier to use.
392 490
393 octet strings - $AnyEvent::Fork::RPC::STRING_SERIALISER 491 octet strings - $AnyEvent::Fork::RPC::STRING_SERIALISER
394 This serialiser concatenates length-prefixes octet strings, 492 This serialiser concatenates length-prefixes octet strings,
395 and is the default. 493 and is the default. That means you can only pass (and
494 return) strings containing character codes 0-255.
396 495
397 Implementation: 496 Implementation:
398 497
399 ( 498 (
400 sub { pack "(w/a*)*", @_ }, 499 sub { pack "(w/a*)*", @_ },
421 520
422 storable - $AnyEvent::Fork::RPC::STORABLE_SERIALISER 521 storable - $AnyEvent::Fork::RPC::STORABLE_SERIALISER
423 This serialiser uses Storable, which means it has high 522 This serialiser uses Storable, which means it has high
424 chance of serialising just about anything you throw at it, 523 chance of serialising just about anything you throw at it,
425 at the cost of having very high overhead per operation. It 524 at the cost of having very high overhead per operation. It
426 also comes with perl. 525 also comes with perl. It should be used when you need to
526 serialise complex data structures.
427 527
428 Implementation: 528 Implementation:
429 529
430 use Storable (); 530 use Storable ();
431 ( 531 (
432 sub { Storable::freeze \@_ }, 532 sub { Storable::freeze \@_ },
533 sub { @{ Storable::thaw shift } }
534 )
535
536 portable storable - $AnyEvent::Fork::RPC::NSTORABLE_SERIALISER
537 This serialiser also uses Storable, but uses it's "network"
538 format to serialise data, which makes it possible to talk to
539 different perl binaries (for example, when talking to a
540 process created with AnyEvent::Fork::Remote).
541
542 Implementation:
543
544 use Storable ();
545 (
546 sub { Storable::nfreeze \@_ },
433 sub { @{ Storable::thaw shift } } 547 sub { @{ Storable::thaw shift } }
434 ) 548 )
435 549
436 See the examples section earlier in this document for some actual 550 See the examples section earlier in this document for some actual
437 examples. 551 examples.
471 the child process to the parent, except that there is no notion of 585 the child process to the parent, except that there is no notion of
472 return values. 586 return values.
473 587
474 See the examples section earlier in this document for some actual 588 See the examples section earlier in this document for some actual
475 examples. 589 examples.
590
591 PROCESS EXIT
592 If and when the child process exits depends on the backend and
593 configuration. Apart from explicit exits (e.g. by calling "exit") or
594 runtime conditions (uncaught exceptions, signals etc.), the backends
595 exit under these conditions:
596
597 Synchronous Backend
598 The synchronous backend is very simple: when the process waits for
599 another request to arrive and the writing side (usually in the
600 parent) is closed, it will exit normally, i.e. as if your main
601 program reached the end of the file.
602
603 That means that if your parent process exits, the RPC process will
604 usually exit as well, either because it is idle anyway, or because
605 it executes a request. In the latter case, you will likely get an
606 error when the RPc process tries to send the results to the parent
607 (because agruably, you shouldn't exit your parent while there are
608 still outstanding requests).
609
610 The process is usually quiescent when it happens, so it should
611 rarely be a problem, and "END" handlers can be used to clean up.
612
613 Asynchronous Backend
614 For the asynchronous backend, things are more complicated: Whenever
615 it listens for another request by the parent, it might detect that
616 the socket was closed (e.g. because the parent exited). It will sotp
617 listening for new requests and instead try to write out any
618 remaining data (if any) or simply check whether the socket cna be
619 written to. After this, the RPC process is effectively done - no new
620 requests are incoming, no outstanding request data can be written
621 back.
622
623 Since chances are high that there are event watchers that the RPC
624 server knows nothing about (why else would one use the async backend
625 if not for the ability to register watchers?), the event loop would
626 often happily continue.
627
628 This is why the asynchronous backend explicitly calls "CORE::exit"
629 when it is done (under other circumstances, such as when there is an
630 I/O error and there is outstanding data to write, it will log a
631 fatal message via AnyEvent::Log, also causing the program to exit).
632
633 You can override this by specifying a function name to call via the
634 "done" parameter instead.
476 635
477ADVANCED TOPICS 636ADVANCED TOPICS
478 Choosing a backend 637 Choosing a backend
479 So how do you decide which backend to use? Well, that's your problem to 638 So how do you decide which backend to use? Well, that's your problem to
480 solve, but here are some thoughts on the matter: 639 solve, but here are some thoughts on the matter:
553 the half it has passed earlier. 712 the half it has passed earlier.
554 713
555 Here is some (untested) pseudocode to that effect: 714 Here is some (untested) pseudocode to that effect:
556 715
557 use AnyEvent::Util; 716 use AnyEvent::Util;
717 use AnyEvent::Fork;
558 use AnyEvent::Fork::RPC; 718 use AnyEvent::Fork::RPC;
559 use IO::FDPass; 719 use IO::FDPass;
560 720
561 my ($s1, $s2) = AnyEvent::Util::portable_socketpair; 721 my ($s1, $s2) = AnyEvent::Util::portable_socketpair;
562 722
608 behaviour. 768 behaviour.
609 769
610SEE ALSO 770SEE ALSO
611 AnyEvent::Fork, to create the processes in the first place. 771 AnyEvent::Fork, to create the processes in the first place.
612 772
773 AnyEvent::Fork::Remote, likewise, but helpful for remote processes.
774
613 AnyEvent::Fork::Pool, to manage whole pools of processes. 775 AnyEvent::Fork::Pool, to manage whole pools of processes.
614 776
615AUTHOR AND CONTACT INFORMATION 777AUTHOR AND CONTACT INFORMATION
616 Marc Lehmann <schmorp@schmorp.de> 778 Marc Lehmann <schmorp@schmorp.de>
617 http://software.schmorp.de/pkg/AnyEvent-Fork-RPC 779 http://software.schmorp.de/pkg/AnyEvent-Fork-RPC

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines