… | |
… | |
175 | you really I<are> done. |
175 | you really I<are> done. |
176 | |
176 | |
177 | =head2 Example 2: Asynchronous Backend |
177 | =head2 Example 2: Asynchronous Backend |
178 | |
178 | |
179 | This example implements multiple count-downs in the child, using |
179 | This example implements multiple count-downs in the child, using |
180 | L<AnyEvent> timers. While this is a bit silly (one could use timers in te |
180 | L<AnyEvent> timers. While this is a bit silly (one could use timers in the |
181 | parent just as well), it illustrates the ability to use AnyEvent in the |
181 | parent just as well), it illustrates the ability to use AnyEvent in the |
182 | child and the fact that responses can arrive in a different order then the |
182 | child and the fact that responses can arrive in a different order then the |
183 | requests. |
183 | requests. |
184 | |
184 | |
185 | It also shows how to embed the actual child code into a C<__DATA__> |
185 | It also shows how to embed the actual child code into a C<__DATA__> |
… | |
… | |
371 | |
371 | |
372 | There is an important twist - the C<AnyEvent::Fork::RPC::event> function |
372 | There is an important twist - the C<AnyEvent::Fork::RPC::event> function |
373 | is only defined when the child is fully initialised. If you redirect the |
373 | is only defined when the child is fully initialised. If you redirect the |
374 | log messages in your C<init> function for example, then the C<event> |
374 | log messages in your C<init> function for example, then the C<event> |
375 | function might not yet be available. This is why the log callback checks |
375 | function might not yet be available. This is why the log callback checks |
376 | whether the fucntion is there using C<defined>, and only then uses it to |
376 | whether the function is there using C<defined>, and only then uses it to |
377 | log the message. |
377 | log the message. |
378 | |
378 | |
379 | =head1 PARENT PROCESS USAGE |
379 | =head1 PARENT PROCESS USAGE |
380 | |
380 | |
381 | This module exports nothing, and only implements a single function: |
381 | This module exports nothing, and only implements a single function: |
… | |
… | |
391 | use Errno (); |
391 | use Errno (); |
392 | use Guard (); |
392 | use Guard (); |
393 | |
393 | |
394 | use AnyEvent; |
394 | use AnyEvent; |
395 | |
395 | |
396 | our $VERSION = 1.21; |
396 | our $VERSION = 1.25; |
397 | |
397 | |
398 | =item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...] |
398 | =item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...] |
399 | |
399 | |
400 | The traditional way to call it. But it is way cooler to call it in the |
400 | The traditional way to call it. But it is way cooler to call it in the |
401 | following way: |
401 | following way: |
… | |
… | |
439 | been successfully handled. This is useful when you queue some requests and |
439 | been successfully handled. This is useful when you queue some requests and |
440 | want the child to go away after it has handled them. The problem is that |
440 | want the child to go away after it has handled them. The problem is that |
441 | the parent must not exit either until all requests have been handled, and |
441 | the parent must not exit either until all requests have been handled, and |
442 | this can be accomplished by waiting for this callback. |
442 | this can be accomplished by waiting for this callback. |
443 | |
443 | |
444 | =item init => $function (default none) |
444 | =item init => $function (default: none) |
445 | |
445 | |
446 | When specified (by name), this function is called in the child as the very |
446 | When specified (by name), this function is called in the child as the very |
447 | first thing when taking over the process, with all the arguments normally |
447 | first thing when taking over the process, with all the arguments normally |
448 | passed to the C<AnyEvent::Fork::run> function, except the communications |
448 | passed to the C<AnyEvent::Fork::run> function, except the communications |
449 | socket. |
449 | socket. |
… | |
… | |
454 | It is called very early - before the serialisers are created or the |
454 | It is called very early - before the serialisers are created or the |
455 | C<$function> name is resolved into a function reference, so it could be |
455 | C<$function> name is resolved into a function reference, so it could be |
456 | used to load any modules that provide the serialiser or function. It can |
456 | used to load any modules that provide the serialiser or function. It can |
457 | not, however, create events. |
457 | not, however, create events. |
458 | |
458 | |
459 | =item done => $function (default C<CORE::exit>) |
459 | =item done => $function (default: C<CORE::exit>) |
460 | |
460 | |
461 | The function to call when the asynchronous backend detects an end of file |
461 | The function to call when the asynchronous backend detects an end of file |
462 | condition when reading from the communications socket I<and> there are no |
462 | condition when reading from the communications socket I<and> there are no |
463 | outstanding requests. It's ignored by the synchronous backend. |
463 | outstanding requests. It's ignored by the synchronous backend. |
464 | |
464 | |
465 | By overriding this you can prolong the life of a RPC process after e.g. |
465 | By overriding this you can prolong the life of a RPC process after e.g. |
466 | the parent has exited by running the event loop in the provided function |
466 | the parent has exited by running the event loop in the provided function |
467 | (or simply calling it, for example, when your child process uses L<EV> you |
467 | (or simply calling it, for example, when your child process uses L<EV> you |
468 | could provide L<EV::loop> as C<done> function). |
468 | could provide L<EV::run> as C<done> function). |
469 | |
469 | |
470 | Of course, in that case you are responsible for exiting at the appropriate |
470 | Of course, in that case you are responsible for exiting at the appropriate |
471 | time and not returning from |
471 | time and not returning from |
472 | |
472 | |
473 | =item async => $boolean (default: 0) |
473 | =item async => $boolean (default: C<0>) |
474 | |
474 | |
475 | The default server used in the child does all I/O blockingly, and only |
475 | The default server used in the child does all I/O blockingly, and only |
476 | allows a single RPC call to execute concurrently. |
476 | allows a single RPC call to execute concurrently. |
477 | |
477 | |
478 | Setting C<async> to a true value switches to another implementation that |
478 | Setting C<async> to a true value switches to another implementation that |
… | |
… | |
488 | synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode. |
488 | synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode. |
489 | |
489 | |
490 | If you use a template process and want to fork both sync and async |
490 | If you use a template process and want to fork both sync and async |
491 | children, then it is permissible to load both modules. |
491 | children, then it is permissible to load both modules. |
492 | |
492 | |
493 | =item serialiser => $string (default: $AnyEvent::Fork::RPC::STRING_SERIALISER) |
493 | =item serialiser => $string (default: C<$AnyEvent::Fork::RPC::STRING_SERIALISER>) |
494 | |
494 | |
495 | All arguments, result data and event data have to be serialised to be |
495 | All arguments, result data and event data have to be serialised to be |
496 | transferred between the processes. For this, they have to be frozen and |
496 | transferred between the processes. For this, they have to be frozen and |
497 | thawed in both parent and child processes. |
497 | thawed in both parent and child processes. |
498 | |
498 | |
499 | By default, only octet strings can be passed between the processes, which |
499 | By default, only octet strings can be passed between the processes, |
500 | is reasonably fast and efficient and requires no extra modules. |
500 | which is reasonably fast and efficient and requires no extra modules |
|
|
501 | (the C<AnyEvent::Fork::RPC> distribution does not provide these extra |
|
|
502 | serialiser modules). |
501 | |
503 | |
502 | For more complicated use cases, you can provide your own freeze and thaw |
504 | For more complicated use cases, you can provide your own freeze and thaw |
503 | functions, by specifying a string with perl source code. It's supposed to |
505 | functions, by specifying a string with perl source code. It's supposed to |
504 | return two code references when evaluated: the first receives a list of |
506 | return two code references when evaluated: the first receives a list of |
505 | perl values and must return an octet string. The second receives the octet |
507 | perl values and must return an octet string. The second receives the octet |
… | |
… | |
507 | |
509 | |
508 | If you need an external module for serialisation, then you can either |
510 | If you need an external module for serialisation, then you can either |
509 | pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use> |
511 | pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use> |
510 | or C<require> statement into the serialiser string. Or both. |
512 | or C<require> statement into the serialiser string. Or both. |
511 | |
513 | |
512 | Here are some examples - some of them are also available as global |
514 | Here are some examples - all of them are also available as global |
513 | variables that make them easier to use. |
515 | variables that make them easier to use. |
514 | |
516 | |
515 | =over 4 |
517 | =over 4 |
516 | |
518 | |
517 | =item octet strings - C<$AnyEvent::Fork::RPC::STRING_SERIALISER> |
519 | =item C<$AnyEvent::Fork::RPC::STRING_SERIALISER> - octet strings only |
518 | |
520 | |
519 | This serialiser concatenates length-prefixes octet strings, and is the |
521 | This serialiser (currently the default) concatenates length-prefixes octet |
520 | default. That means you can only pass (and return) strings containing |
522 | strings, and is the default. That means you can only pass (and return) |
521 | character codes 0-255. |
523 | strings containing character codes 0-255. |
|
|
524 | |
|
|
525 | The main advantages of this serialiser are the high speed and that it |
|
|
526 | doesn't need another module. The main disadvantage is that you are very |
|
|
527 | limited in what you can pass - only octet strings. |
522 | |
528 | |
523 | Implementation: |
529 | Implementation: |
524 | |
530 | |
525 | ( |
531 | ( |
526 | sub { pack "(w/a*)*", @_ }, |
532 | sub { pack "(w/a*)*", @_ }, |
527 | sub { unpack "(w/a*)*", shift } |
533 | sub { unpack "(w/a*)*", shift } |
528 | ) |
534 | ) |
529 | |
535 | |
530 | =item json - C<$AnyEvent::Fork::RPC::JSON_SERIALISER> |
536 | =item C<$AnyEvent::Fork::RPC::CBOR_XS_SERIALISER> - uses L<CBOR::XS> |
|
|
537 | |
|
|
538 | This serialiser creates CBOR::XS arrays - you have to make sure the |
|
|
539 | L<CBOR::XS> module is installed for this serialiser to work. It can be |
|
|
540 | beneficial for sharing when you preload the L<CBOR::XS> module in a template |
|
|
541 | process. |
|
|
542 | |
|
|
543 | L<CBOR::XS> is about as fast as the octet string serialiser, but supports |
|
|
544 | complex data structures (similar to JSON) and is faster than any of the |
|
|
545 | other serialisers. If you have the L<CBOR::XS> module available, it's the |
|
|
546 | best choice. |
|
|
547 | |
|
|
548 | The encoder enables C<allow_sharing> (so this serialisation method can |
|
|
549 | encode cyclic and self-referencing data structures). |
|
|
550 | |
|
|
551 | Implementation: |
|
|
552 | |
|
|
553 | use CBOR::XS (); |
|
|
554 | ( |
|
|
555 | sub { CBOR::XS::encode_cbor_sharing \@_ }, |
|
|
556 | sub { @{ CBOR::XS::decode_cbor shift } } |
|
|
557 | ) |
|
|
558 | |
|
|
559 | =item C<$AnyEvent::Fork::RPC::JSON_SERIALISER> - uses L<JSON::XS> or L<JSON> |
531 | |
560 | |
532 | This serialiser creates JSON arrays - you have to make sure the L<JSON> |
561 | This serialiser creates JSON arrays - you have to make sure the L<JSON> |
533 | module is installed for this serialiser to work. It can be beneficial for |
562 | module is installed for this serialiser to work. It can be beneficial for |
534 | sharing when you preload the L<JSON> module in a template process. |
563 | sharing when you preload the L<JSON> module in a template process. |
535 | |
564 | |
… | |
… | |
543 | ( |
572 | ( |
544 | sub { JSON::encode_json \@_ }, |
573 | sub { JSON::encode_json \@_ }, |
545 | sub { @{ JSON::decode_json shift } } |
574 | sub { @{ JSON::decode_json shift } } |
546 | ) |
575 | ) |
547 | |
576 | |
548 | =item storable - C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER> |
577 | =item C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER> - L<Storable> |
549 | |
578 | |
550 | This serialiser uses L<Storable>, which means it has high chance of |
579 | This serialiser uses L<Storable>, which means it has high chance of |
551 | serialising just about anything you throw at it, at the cost of having |
580 | serialising just about anything you throw at it, at the cost of having |
552 | very high overhead per operation. It also comes with perl. It should be |
581 | very high overhead per operation. It also comes with perl. It should be |
553 | used when you need to serialise complex data structures. |
582 | used when you need to serialise complex data structures. |
… | |
… | |
558 | ( |
587 | ( |
559 | sub { Storable::freeze \@_ }, |
588 | sub { Storable::freeze \@_ }, |
560 | sub { @{ Storable::thaw shift } } |
589 | sub { @{ Storable::thaw shift } } |
561 | ) |
590 | ) |
562 | |
591 | |
563 | =item portable storable - C<$AnyEvent::Fork::RPC::NSTORABLE_SERIALISER> |
592 | =item C<$AnyEvent::Fork::RPC::NSTORABLE_SERIALISER> - portable Storable |
564 | |
593 | |
565 | This serialiser also uses L<Storable>, but uses it's "network" format |
594 | This serialiser also uses L<Storable>, but uses it's "network" format |
566 | to serialise data, which makes it possible to talk to different |
595 | to serialise data, which makes it possible to talk to different |
567 | perl binaries (for example, when talking to a process created with |
596 | perl binaries (for example, when talking to a process created with |
568 | L<AnyEvent::Fork::Remote>). |
597 | L<AnyEvent::Fork::Remote>). |
… | |
… | |
575 | sub { @{ Storable::thaw shift } } |
604 | sub { @{ Storable::thaw shift } } |
576 | ) |
605 | ) |
577 | |
606 | |
578 | =back |
607 | =back |
579 | |
608 | |
|
|
609 | =item buflen => $bytes (default: C<512 - 16>) |
|
|
610 | |
|
|
611 | The starting size of the read buffer for request and response data. |
|
|
612 | |
|
|
613 | C<AnyEvent::Fork::RPC> ensures that the buffer for reeading request and |
|
|
614 | response data is large enough for at leats aingle request or response, and |
|
|
615 | will dynamically enlarge the buffer if needed. |
|
|
616 | |
|
|
617 | While this ensures that memory is not overly wasted, it typically leads |
|
|
618 | to having to do one syscall per request, which can be inefficient in some |
|
|
619 | cases. In such cases, it can be beneficient to increase the buffer size to |
|
|
620 | hold more than one request. |
|
|
621 | |
|
|
622 | =item buflen_req => $bytes (default: same as C<buflen>) |
|
|
623 | |
|
|
624 | Overrides C<buflen> for request data (as read by the forked process). |
|
|
625 | |
|
|
626 | =item buflen_res => $bytes (default: same as C<buflen>) |
|
|
627 | |
|
|
628 | Overrides C<buflen> for response data (replies read by the parent process). |
|
|
629 | |
580 | =back |
630 | =back |
581 | |
631 | |
582 | See the examples section earlier in this document for some actual |
632 | See the examples section earlier in this document for some actual |
583 | examples. |
633 | examples. |
584 | |
634 | |
585 | =cut |
635 | =cut |
586 | |
636 | |
587 | our $STRING_SERIALISER = '(sub { pack "(w/a*)*", @_ }, sub { unpack "(w/a*)*", shift })'; |
637 | our $STRING_SERIALISER = '(sub { pack "(w/a*)*", @_ }, sub { unpack "(w/a*)*", shift })'; |
|
|
638 | our $CBOR_XS_SERIALISER = 'use CBOR::XS (); (sub { CBOR::XS::encode_cbor_sharing \@_ }, sub { @{ CBOR::XS::decode_cbor shift } })'; |
588 | our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })'; |
639 | our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })'; |
589 | our $STORABLE_SERIALISER = 'use Storable (); (sub { Storable::freeze \@_ }, sub { @{ Storable::thaw shift } })'; |
640 | our $STORABLE_SERIALISER = 'use Storable (); (sub { Storable::freeze \@_ }, sub { @{ Storable::thaw shift } })'; |
590 | our $NSTORABLE_SERIALISER = 'use Storable (); (sub { Storable::nfreeze \@_ }, sub { @{ Storable::thaw shift } })'; |
641 | our $NSTORABLE_SERIALISER = 'use Storable (); (sub { Storable::nfreeze \@_ }, sub { @{ Storable::thaw shift } })'; |
591 | |
642 | |
592 | sub run { |
643 | sub run { |
593 | my ($self, $function, %arg) = @_; |
644 | my ($self, $function, %arg) = @_; |
… | |
… | |
606 | $on_event ||= sub { $on_error->("event received, but no on_event handler") }; |
657 | $on_event ||= sub { $on_error->("event received, but no on_event handler") }; |
607 | |
658 | |
608 | my ($f, $t) = eval $serialiser; die $@ if $@; |
659 | my ($f, $t) = eval $serialiser; die $@ if $@; |
609 | |
660 | |
610 | my (@rcb, %rcb, $fh, $shutdown, $wbuf, $ww); |
661 | my (@rcb, %rcb, $fh, $shutdown, $wbuf, $ww); |
611 | my ($rlen, $rbuf, $rw) = 512 - 16; |
662 | my ($rlen, $rbuf, $rw) = $arg{buflen_res} || $arg{buflen} || 512 - 16; |
612 | |
663 | |
613 | my $wcb = sub { |
664 | my $wcb = sub { |
614 | my $len = syswrite $fh, $wbuf; |
665 | my $len = syswrite $fh, $wbuf; |
615 | |
666 | |
616 | unless (defined $len) { |
667 | unless (defined $len) { |
… | |
… | |
628 | } |
679 | } |
629 | }; |
680 | }; |
630 | |
681 | |
631 | my $module = "AnyEvent::Fork::RPC::" . ($arg{async} ? "Async" : "Sync"); |
682 | my $module = "AnyEvent::Fork::RPC::" . ($arg{async} ? "Async" : "Sync"); |
632 | |
683 | |
633 | $self->require ($module) |
684 | $self->eval ("use $module 2 ()") |
634 | ->send_arg ($function, $arg{init}, $serialiser, $arg{done} || "$module\::do_exit") |
685 | ->send_arg ( |
|
|
686 | function => $function, |
|
|
687 | init => $arg{init}, |
|
|
688 | serialiser => $serialiser, |
|
|
689 | done => $arg{done} || "$module\::do_exit", |
|
|
690 | rlen => $arg{buflen_req} || $arg{buflen} || 512 - 16, |
|
|
691 | -10 # the above are 10 arguments |
|
|
692 | ) |
635 | ->run ("$module\::run", sub { |
693 | ->run ("$module\::run", sub { |
636 | $fh = shift; |
694 | $fh = shift |
|
|
695 | or return $on_error->("connection failed"); |
637 | |
696 | |
638 | my ($id, $len); |
697 | my ($id, $len); |
639 | $rw = AE::io $fh, 0, sub { |
698 | $rw = AE::io $fh, 0, sub { |
640 | $rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf; |
699 | $rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf; |
641 | $len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf; |
700 | $len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf; |
… | |
… | |
753 | values. |
812 | values. |
754 | |
813 | |
755 | See the examples section earlier in this document for some actual |
814 | See the examples section earlier in this document for some actual |
756 | examples. |
815 | examples. |
757 | |
816 | |
|
|
817 | Note: the event data, like any data send to the parent, might not be sent |
|
|
818 | immediatelly but queued for later sending, so there is no guarantee that |
|
|
819 | the event has been sent to the parent when the call returns - when you |
|
|
820 | e.g. exit directly after calling this function, the parent might never |
|
|
821 | receive the event. |
|
|
822 | |
758 | =back |
823 | =back |
759 | |
824 | |
760 | =head2 PROCESS EXIT |
825 | =head2 PROCESS EXIT |
761 | |
826 | |
762 | If and when the child process exits depends on the backend and |
827 | If and when the child process exits depends on the backend and |
… | |
… | |
786 | |
851 | |
787 | For the asynchronous backend, things are more complicated: Whenever it |
852 | For the asynchronous backend, things are more complicated: Whenever it |
788 | listens for another request by the parent, it might detect that the socket |
853 | listens for another request by the parent, it might detect that the socket |
789 | was closed (e.g. because the parent exited). It will sotp listening for |
854 | was closed (e.g. because the parent exited). It will sotp listening for |
790 | new requests and instead try to write out any remaining data (if any) or |
855 | new requests and instead try to write out any remaining data (if any) or |
791 | simply check whether the socket cna be written to. After this, the RPC |
856 | simply check whether the socket can be written to. After this, the RPC |
792 | process is effectively done - no new requests are incoming, no outstanding |
857 | process is effectively done - no new requests are incoming, no outstanding |
793 | request data can be written back. |
858 | request data can be written back. |
794 | |
859 | |
795 | Since chances are high that there are event watchers that the RPC server |
860 | Since chances are high that there are event watchers that the RPC server |
796 | knows nothing about (why else would one use the async backend if not for |
861 | knows nothing about (why else would one use the async backend if not for |
… | |
… | |
867 | are queued and the jobs are slow, they will all run concurrently. The |
932 | are queued and the jobs are slow, they will all run concurrently. The |
868 | child must implement some queueing/limiting mechanism if this causes |
933 | child must implement some queueing/limiting mechanism if this causes |
869 | problems. Alternatively, the parent could limit the amount of rpc calls |
934 | problems. Alternatively, the parent could limit the amount of rpc calls |
870 | that are outstanding. |
935 | that are outstanding. |
871 | |
936 | |
872 | Blocking use of condvars is not supported. |
937 | Blocking use of condvars is not supported (in the main thread, outside of |
|
|
938 | e.g. L<Coro> threads). |
873 | |
939 | |
874 | Using event-based modules such as L<IO::AIO>, L<Gtk2>, L<Tk> and so on is |
940 | Using event-based modules such as L<IO::AIO>, L<Gtk2>, L<Tk> and so on is |
875 | easy. |
941 | easy. |
876 | |
942 | |
877 | =back |
943 | =back |
… | |
… | |
943 | gory details. |
1009 | gory details. |
944 | |
1010 | |
945 | =head1 EXCEPTIONS |
1011 | =head1 EXCEPTIONS |
946 | |
1012 | |
947 | There are no provisions whatsoever for catching exceptions at this time - |
1013 | There are no provisions whatsoever for catching exceptions at this time - |
948 | in the child, exeptions might kill the process, causing calls to be lost |
1014 | in the child, exceptions might kill the process, causing calls to be lost |
949 | and the parent encountering a fatal error. In the parent, exceptions in |
1015 | and the parent encountering a fatal error. In the parent, exceptions in |
950 | the result callback will not be caught and cause undefined behaviour. |
1016 | the result callback will not be caught and cause undefined behaviour. |
951 | |
1017 | |
952 | =head1 SEE ALSO |
1018 | =head1 SEE ALSO |
953 | |
1019 | |