ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork-RPC/RPC.pm
(Generate patch)

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.30 by root, Sun Aug 25 22:21:15 2013 UTC vs.
Revision 1.46 by root, Sun Sep 15 20:18:14 2019 UTC

… …
175you really I<are> done. 175you really I<are> done.
176 176
177=head2 Example 2: Asynchronous Backend 177=head2 Example 2: Asynchronous Backend
178 178
179This example implements multiple count-downs in the child, using 179This example implements multiple count-downs in the child, using
180L<AnyEvent> timers. While this is a bit silly (one could use timers in te 180L<AnyEvent> timers. While this is a bit silly (one could use timers in the
181parent just as well), it illustrates the ability to use AnyEvent in the 181parent just as well), it illustrates the ability to use AnyEvent in the
182child and the fact that responses can arrive in a different order then the 182child and the fact that responses can arrive in a different order then the
183requests. 183requests.
184 184
185It also shows how to embed the actual child code into a C<__DATA__> 185It also shows how to embed the actual child code into a C<__DATA__>
… …
371 371
372There is an important twist - the C<AnyEvent::Fork::RPC::event> function 372There is an important twist - the C<AnyEvent::Fork::RPC::event> function
373is only defined when the child is fully initialised. If you redirect the 373is only defined when the child is fully initialised. If you redirect the
374log messages in your C<init> function for example, then the C<event> 374log messages in your C<init> function for example, then the C<event>
375function might not yet be available. This is why the log callback checks 375function might not yet be available. This is why the log callback checks
376whether the fucntion is there using C<defined>, and only then uses it to 376whether the function is there using C<defined>, and only then uses it to
377log the message. 377log the message.
378 378
379=head1 PARENT PROCESS USAGE 379=head1 PARENT PROCESS USAGE
380 380
381This module exports nothing, and only implements a single function: 381This module exports nothing, and only implements a single function:
… …
391use Errno (); 391use Errno ();
392use Guard (); 392use Guard ();
393 393
394use AnyEvent; 394use AnyEvent;
395 395
396our $VERSION = 1.1; 396our $VERSION = '2.0';
397 397
398=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...] 398=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]
399 399
400The traditional way to call it. But it is way cooler to call it in the 400The traditional way to call it. But it is way cooler to call it in the
401following way: 401following way:
… …
439been successfully handled. This is useful when you queue some requests and 439been successfully handled. This is useful when you queue some requests and
440want the child to go away after it has handled them. The problem is that 440want the child to go away after it has handled them. The problem is that
441the parent must not exit either until all requests have been handled, and 441the parent must not exit either until all requests have been handled, and
442this can be accomplished by waiting for this callback. 442this can be accomplished by waiting for this callback.
443 443
444=item init => $function (default none) 444=item init => $function (default: none)
445 445
446When specified (by name), this function is called in the child as the very 446When specified (by name), this function is called in the child as the very
447first thing when taking over the process, with all the arguments normally 447first thing when taking over the process, with all the arguments normally
448passed to the C<AnyEvent::Fork::run> function, except the communications 448passed to the C<AnyEvent::Fork::run> function, except the communications
449socket. 449socket.
… …
454It is called very early - before the serialisers are created or the 454It is called very early - before the serialisers are created or the
455C<$function> name is resolved into a function reference, so it could be 455C<$function> name is resolved into a function reference, so it could be
456used to load any modules that provide the serialiser or function. It can 456used to load any modules that provide the serialiser or function. It can
457not, however, create events. 457not, however, create events.
458 458
459=item done => $function (default: C<CORE::exit>)
460
461The function to call when the asynchronous backend detects an end of file
462condition when reading from the communications socket I<and> there are no
463outstanding requests. It is ignored by the synchronous backend.
464
465By overriding this you can prolong the life of a RPC process after e.g.
466the parent has exited by running the event loop in the provided function
467(or simply calling it, for example, when your child process uses L<EV> you
468could provide L<EV::run> as C<done> function).
469
470Of course, in that case you are responsible for exiting at the appropriate
471time and not returning from
472
459=item async => $boolean (default: 0) 473=item async => $boolean (default: C<0>)
460 474
461The default server used in the child does all I/O blockingly, and only 475The default server used in the child does all I/O blockingly, and only
462allows a single RPC call to execute concurrently. 476allows a single RPC call to execute concurrently.
463 477
464Setting C<async> to a true value switches to another implementation that 478Setting C<async> to a true value switches to another implementation that
… …
474synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode. 488synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode.
475 489
476If you use a template process and want to fork both sync and async 490If you use a template process and want to fork both sync and async
477children, then it is permissible to load both modules. 491children, then it is permissible to load both modules.
478 492
479=item serialiser => $string (default: $AnyEvent::Fork::RPC::STRING_SERIALISER) 493=item serialiser => $string (default: C<$AnyEvent::Fork::RPC::STRING_SERIALISER>)
480 494
481All arguments, result data and event data have to be serialised to be 495All arguments, result data and event data have to be serialised to be
482transferred between the processes. For this, they have to be frozen and 496transferred between the processes. For this, they have to be frozen and
483thawed in both parent and child processes. 497thawed in both parent and child processes.
484 498
485By default, only octet strings can be passed between the processes, which 499By default, only octet strings can be passed between the processes,
486is reasonably fast and efficient and requires no extra modules. 500which is reasonably fast and efficient and requires no extra modules
501(the C<AnyEvent::Fork::RPC> distribution does not provide these extra
502serialiser modules).
487 503
488For more complicated use cases, you can provide your own freeze and thaw 504For more complicated use cases, you can provide your own freeze and thaw
489functions, by specifying a string with perl source code. It's supposed to 505functions, by specifying a string with perl source code. It's supposed to
490return two code references when evaluated: the first receives a list of 506return two code references when evaluated: the first receives a list of
491perl values and must return an octet string. The second receives the octet 507perl values and must return an octet string. The second receives the octet
… …
493 509
494If you need an external module for serialisation, then you can either 510If you need an external module for serialisation, then you can either
495pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use> 511pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use>
496or C<require> statement into the serialiser string. Or both. 512or C<require> statement into the serialiser string. Or both.
497 513
498Here are some examples - some of them are also available as global 514Here are some examples - all of them are also available as global
499variables that make them easier to use. 515variables that make them easier to use.
500 516
501=over 4 517=over 4
502 518
503=item octet strings - C<$AnyEvent::Fork::RPC::STRING_SERIALISER> 519=item C<$AnyEvent::Fork::RPC::STRING_SERIALISER> - octet strings only
504 520
505This serialiser concatenates length-prefixes octet strings, and is the 521This serialiser (currently the default) concatenates length-prefixes octet
506default. That means you can only pass (and return) strings containing 522strings, and is the default. That means you can only pass (and return)
507character codes 0-255. 523strings containing character codes 0-255.
524
525The main advantages of this serialiser are the high speed and that it
526doesn't need another module. The main disadvantage is that you are very
527limited in what you can pass - only octet strings.
508 528
509Implementation: 529Implementation:
510 530
511 ( 531 (
512 sub { pack "(w/a*)*", @_ }, 532 sub { pack "(w/a*)*", @_ },
513 sub { unpack "(w/a*)*", shift } 533 sub { unpack "(w/a*)*", shift }
514 ) 534 )
515 535
516=item json - C<$AnyEvent::Fork::RPC::JSON_SERIALISER> 536=item C<$AnyEvent::Fork::RPC::CBOR_XS_SERIALISER> - uses L<CBOR::XS>
537
538This serialiser creates CBOR::XS arrays - you have to make sure the
539L<CBOR::XS> module is installed for this serialiser to work. It can be
540beneficial for sharing when you preload the L<CBOR::XS> module in a template
541process.
542
543L<CBOR::XS> is about as fast as the octet string serialiser, but supports
544complex data structures (similar to JSON) and is faster than any of the
545other serialisers. If you have the L<CBOR::XS> module available, it's the
546best choice.
547
548The encoder enables C<allow_sharing> (so this serialisation method can
549encode cyclic and self-referencing data structures).
550
551Implementation:
552
553 use CBOR::XS ();
554 (
555 sub { CBOR::XS::encode_cbor_sharing \@_ },
556 sub { @{ CBOR::XS::decode_cbor shift } }
557 )
558
559=item C<$AnyEvent::Fork::RPC::JSON_SERIALISER> - uses L<JSON::XS> or L<JSON>
517 560
518This serialiser creates JSON arrays - you have to make sure the L<JSON> 561This serialiser creates JSON arrays - you have to make sure the L<JSON>
519module is installed for this serialiser to work. It can be beneficial for 562module is installed for this serialiser to work. It can be beneficial for
520sharing when you preload the L<JSON> module in a template process. 563sharing when you preload the L<JSON> module in a template process.
521 564
… …
529 ( 572 (
530 sub { JSON::encode_json \@_ }, 573 sub { JSON::encode_json \@_ },
531 sub { @{ JSON::decode_json shift } } 574 sub { @{ JSON::decode_json shift } }
532 ) 575 )
533 576
534=item storable - C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER> 577=item C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER> - L<Storable>
535 578
536This serialiser uses L<Storable>, which means it has high chance of 579This serialiser uses L<Storable>, which means it has high chance of
537serialising just about anything you throw at it, at the cost of having 580serialising just about anything you throw at it, at the cost of having
538very high overhead per operation. It also comes with perl. It should be 581very high overhead per operation. It also comes with perl. It should be
539used when you need to serialise complex data structures. 582used when you need to serialise complex data structures.
… …
544 ( 587 (
545 sub { Storable::freeze \@_ }, 588 sub { Storable::freeze \@_ },
546 sub { @{ Storable::thaw shift } } 589 sub { @{ Storable::thaw shift } }
547 ) 590 )
548 591
549=item portable storable - C<$AnyEvent::Fork::RPC::NSTORABLE_SERIALISER> 592=item C<$AnyEvent::Fork::RPC::NSTORABLE_SERIALISER> - portable Storable
550 593
551This serialiser also uses L<Storable>, but uses it's "network" format 594This serialiser also uses L<Storable>, but uses it's "network" format
552to serialise data, which makes it possible to talk to different 595to serialise data, which makes it possible to talk to different
553perl binaries (for example, when talking to a process created with 596perl binaries (for example, when talking to a process created with
554L<AnyEvent::Fork::Remote>). 597L<AnyEvent::Fork::Remote>).
… …
561 sub { @{ Storable::thaw shift } } 604 sub { @{ Storable::thaw shift } }
562 ) 605 )
563 606
564=back 607=back
565 608
609=item buflen => $bytes (default: C<512 - 16>)
610
611The starting size of the read buffer for request and response data.
612
613C<AnyEvent::Fork::RPC> ensures that the buffer for reeading request and
614response data is large enough for at leats aingle request or response, and
615will dynamically enlarge the buffer if needed.
616
617While this ensures that memory is not overly wasted, it typically leads
618to having to do one syscall per request, which can be inefficient in some
619cases. In such cases, it can be beneficient to increase the buffer size to
620hold more than one request.
621
622=item buflen_req => $bytes (default: same as C<buflen>)
623
624Overrides C<buflen> for request data (as read by the forked process).
625
626=item buflen_res => $bytes (default: same as C<buflen>)
627
628Overrides C<buflen> for response data (replies read by the parent process).
629
566=back 630=back
567 631
568See the examples section earlier in this document for some actual 632See the examples section earlier in this document for some actual
569examples. 633examples.
570 634
571=cut 635=cut
572 636
573our $STRING_SERIALISER = '(sub { pack "(w/a*)*", @_ }, sub { unpack "(w/a*)*", shift })'; 637our $STRING_SERIALISER = '(sub { pack "(w/a*)*", @_ }, sub { unpack "(w/a*)*", shift })';
638our $CBOR_XS_SERIALISER = 'use CBOR::XS (); (sub { CBOR::XS::encode_cbor_sharing \@_ }, sub { @{ CBOR::XS::decode_cbor shift } })';
574our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })'; 639our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })';
575our $STORABLE_SERIALISER = 'use Storable (); (sub { Storable::freeze \@_ }, sub { @{ Storable::thaw shift } })'; 640our $STORABLE_SERIALISER = 'use Storable (); (sub { Storable::freeze \@_ }, sub { @{ Storable::thaw shift } })';
576our $NSTORABLE_SERIALISER = 'use Storable (); (sub { Storable::nfreeze \@_ }, sub { @{ Storable::thaw shift } })'; 641our $NSTORABLE_SERIALISER = 'use Storable (); (sub { Storable::nfreeze \@_ }, sub { @{ Storable::thaw shift } })';
577 642
578sub run { 643sub run {
579 my ($self, $function, %arg) = @_; 644 my ($self, $function, %arg) = @_;
… …
592 $on_event ||= sub { $on_error->("event received, but no on_event handler") }; 657 $on_event ||= sub { $on_error->("event received, but no on_event handler") };
593 658
594 my ($f, $t) = eval $serialiser; die $@ if $@; 659 my ($f, $t) = eval $serialiser; die $@ if $@;
595 660
596 my (@rcb, %rcb, $fh, $shutdown, $wbuf, $ww); 661 my (@rcb, %rcb, $fh, $shutdown, $wbuf, $ww);
597 my ($rlen, $rbuf, $rw) = 512 - 16; 662 my ($rlen, $rbuf, $rw) = $arg{buflen_res} || $arg{buflen} || 512 - 16;
598 663
599 my $wcb = sub { 664 my $wcb = sub {
600 my $len = syswrite $fh, $wbuf; 665 my $len = syswrite $fh, $wbuf;
601 666
602 unless (defined $len) { 667 unless (defined $len) {
… …
614 } 679 }
615 }; 680 };
616 681
617 my $module = "AnyEvent::Fork::RPC::" . ($arg{async} ? "Async" : "Sync"); 682 my $module = "AnyEvent::Fork::RPC::" . ($arg{async} ? "Async" : "Sync");
618 683
619 $self->require ($module) 684 $self->eval ("use $module 2 ()")
620 ->send_arg ($function, $arg{init}, $serialiser) 685 ->send_arg (
686 function => $function,
687 init => $arg{init},
688 serialiser => $serialiser,
689 done => $arg{done} || "$module\::do_exit",
690 rlen => $arg{buflen_req} || $arg{buflen} || 512 - 16,
691 -10 # the above are 10 arguments
692 )
621 ->run ("$module\::run", sub { 693 ->run ("$module\::run", sub {
622 $fh = shift; 694 $fh = shift
695 or return $on_error->("connection failed");
623 696
624 my ($id, $len); 697 my ($id, $len);
625 $rw = AE::io $fh, 0, sub { 698 $rw = AE::io $fh, 0, sub {
626 $rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf; 699 $rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf;
627 $len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf; 700 $len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf;
… …
728The following function is not available in this module. They are only 801The following function is not available in this module. They are only
729available in the namespace of this module when the child is running, 802available in the namespace of this module when the child is running,
730without having to load any extra modules. They are part of the child-side 803without having to load any extra modules. They are part of the child-side
731API of L<AnyEvent::Fork::RPC>. 804API of L<AnyEvent::Fork::RPC>.
732 805
806Note that these functions are typically not yet declared when code is
807compiled into the child, because the backend module is only loaded when
808you call C<run>, which is typically the last method you call on the fork
809object.
810
811Therefore, you either have to explicitly pre-load the right backend module
812or mark calls to these functions as function calls, e.g.:
813
814 AnyEvent::Fork::RPC::event (0 => "five");
815 AnyEvent::Fork::RPC::event->(0 => "five");
816 &AnyEvent::Fork::RPC::flush;
817
733=over 4 818=over 4
734 819
735=item AnyEvent::Fork::RPC::event ... 820=item AnyEvent::Fork::RPC::event (...)
736 821
737Send an event to the parent. Events are a bit like RPC calls made by the 822Send an event to the parent. Events are a bit like RPC calls made by the
738child process to the parent, except that there is no notion of return 823child process to the parent, except that there is no notion of return
739values. 824values.
740 825
741See the examples section earlier in this document for some actual 826See the examples section earlier in this document for some actual
742examples. 827examples.
828
829Note: the event data, like any data send to the parent, might not be sent
830immediatelly but queued for later sending, so there is no guarantee that
831the event has been sent to the parent when the call returns - when you
832e.g. exit directly after calling this function, the parent might never
833receive the event. See the next function for a remedy.
834
835=item $success = AnyEvent::Fork::RPC::flush ()
836
837Synchronously wait and flush the reply data to the parent. Returns true on
838success and false otherwise (i.e. when the reply data cannot be written at
839all). Ignoring the success status is a common and healthy behaviour.
840
841Only the "async" backend does something on C<flush> - the "sync" backend
842is not buffering reply data and always returns true from this function.
843
844Normally, reply data might or might not be written to the parent
845immediatelly but is buffered. This can greatly improve performance and
846efficiency, but sometimes can get in your way: for example. when you want
847to send an error message just before exiting, or when you want to ensure
848replies timely reach the parent before starting a long blocking operation.
849
850In these cases, you can call this function to flush any outstanding reply
851data to the parent. This is done blockingly, so no requests will be
852handled and no event callbacks will be called.
853
854For example, you could wrap your request function in a C<eval> block and
855report the exception string back to the caller just before exiting:
856
857 sub req {
858 ...
859
860 eval {
861 ...
862 };
863
864 if ($@) {
865 AnyEvent::RPC::event (throw => "$@");
866 AnyEvent::RPC::flush ();
867 exit;
868 }
869
870 ...
871 }
872
873=back
874
875=head2 PROCESS EXIT
876
877If and when the child process exits depends on the backend and
878configuration. Apart from explicit exits (e.g. by calling C<exit>) or
879runtime conditions (uncaught exceptions, signals etc.), the backends exit
880under these conditions:
881
882=over 4
883
884=item Synchronous Backend
885
886The synchronous backend is very simple: when the process waits for another
887request to arrive and the writing side (usually in the parent) is closed,
888it will exit normally, i.e. as if your main program reached the end of the
889file.
890
891That means that if your parent process exits, the RPC process will usually
892exit as well, either because it is idle anyway, or because it executes a
893request. In the latter case, you will likely get an error when the RPc
894process tries to send the results to the parent (because agruably, you
895shouldn't exit your parent while there are still outstanding requests).
896
897The process is usually quiescent when it happens, so it should rarely be a
898problem, and C<END> handlers can be used to clean up.
899
900=item Asynchronous Backend
901
902For the asynchronous backend, things are more complicated: Whenever it
903listens for another request by the parent, it might detect that the socket
904was closed (e.g. because the parent exited). It will sotp listening for
905new requests and instead try to write out any remaining data (if any) or
906simply check whether the socket can be written to. After this, the RPC
907process is effectively done - no new requests are incoming, no outstanding
908request data can be written back.
909
910Since chances are high that there are event watchers that the RPC server
911knows nothing about (why else would one use the async backend if not for
912the ability to register watchers?), the event loop would often happily
913continue.
914
915This is why the asynchronous backend explicitly calls C<CORE::exit> when
916it is done (under other circumstances, such as when there is an I/O error
917and there is outstanding data to write, it will log a fatal message via
918L<AnyEvent::Log>, also causing the program to exit).
919
920You can override this by specifying a function name to call via the C<done>
921parameter instead.
743 922
744=back 923=back
745 924
746=head1 ADVANCED TOPICS 925=head1 ADVANCED TOPICS
747 926
… …
803are queued and the jobs are slow, they will all run concurrently. The 982are queued and the jobs are slow, they will all run concurrently. The
804child must implement some queueing/limiting mechanism if this causes 983child must implement some queueing/limiting mechanism if this causes
805problems. Alternatively, the parent could limit the amount of rpc calls 984problems. Alternatively, the parent could limit the amount of rpc calls
806that are outstanding. 985that are outstanding.
807 986
808Blocking use of condvars is not supported. 987Blocking use of condvars is not supported (in the main thread, outside of
988e.g. L<Coro> threads).
809 989
810Using event-based modules such as L<IO::AIO>, L<Gtk2>, L<Tk> and so on is 990Using event-based modules such as L<IO::AIO>, L<Gtk2>, L<Tk> and so on is
811easy. 991easy.
812 992
813=back 993=back
… …
879gory details. 1059gory details.
880 1060
881=head1 EXCEPTIONS 1061=head1 EXCEPTIONS
882 1062
883There are no provisions whatsoever for catching exceptions at this time - 1063There are no provisions whatsoever for catching exceptions at this time -
884in the child, exeptions might kill the process, causing calls to be lost 1064in the child, exceptions might kill the process, causing calls to be lost
885and the parent encountering a fatal error. In the parent, exceptions in 1065and the parent encountering a fatal error. In the parent, exceptions in
886the result callback will not be caught and cause undefined behaviour. 1066the result callback will not be caught and cause undefined behaviour.
887 1067
888=head1 SEE ALSO 1068=head1 SEE ALSO
889 1069

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines