--- AnyEvent-Fork-RPC/RPC.pm 2013/08/31 16:35:33 1.31 +++ AnyEvent-Fork-RPC/RPC.pm 2019/09/15 20:18:14 1.46 @@ -177,7 +177,7 @@ =head2 Example 2: Asynchronous Backend This example implements multiple count-downs in the child, using -L timers. While this is a bit silly (one could use timers in te +L timers. While this is a bit silly (one could use timers in the parent just as well), it illustrates the ability to use AnyEvent in the child and the fact that responses can arrive in a different order then the requests. @@ -373,7 +373,7 @@ is only defined when the child is fully initialised. If you redirect the log messages in your C function for example, then the C function might not yet be available. This is why the log callback checks -whether the fucntion is there using C, and only then uses it to +whether the function is there using C, and only then uses it to log the message. =head1 PARENT PROCESS USAGE @@ -393,7 +393,7 @@ use AnyEvent; -our $VERSION = 1.1; +our $VERSION = '2.0'; =item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...] @@ -441,7 +441,7 @@ the parent must not exit either until all requests have been handled, and this can be accomplished by waiting for this callback. -=item init => $function (default none) +=item init => $function (default: none) When specified (by name), this function is called in the child as the very first thing when taking over the process, with all the arguments normally @@ -456,21 +456,21 @@ used to load any modules that provide the serialiser or function. It can not, however, create events. -=item done => $function (default C) +=item done => $function (default: C) The function to call when the asynchronous backend detects an end of file condition when reading from the communications socket I there are no -outstanding requests. It's ignored by the synchronous backend. +outstanding requests. It is ignored by the synchronous backend. By overriding this you can prolong the life of a RPC process after e.g. the parent has exited by running the event loop in the provided function (or simply calling it, for example, when your child process uses L you -could provide L as C function). +could provide L as C function). Of course, in that case you are responsible for exiting at the appropriate time and not returning from -=item async => $boolean (default: 0) +=item async => $boolean (default: C<0>) The default server used in the child does all I/O blockingly, and only allows a single RPC call to execute concurrently. @@ -490,14 +490,16 @@ If you use a template process and want to fork both sync and async children, then it is permissible to load both modules. -=item serialiser => $string (default: $AnyEvent::Fork::RPC::STRING_SERIALISER) +=item serialiser => $string (default: C<$AnyEvent::Fork::RPC::STRING_SERIALISER>) All arguments, result data and event data have to be serialised to be transferred between the processes. For this, they have to be frozen and thawed in both parent and child processes. -By default, only octet strings can be passed between the processes, which -is reasonably fast and efficient and requires no extra modules. +By default, only octet strings can be passed between the processes, +which is reasonably fast and efficient and requires no extra modules +(the C distribution does not provide these extra +serialiser modules). For more complicated use cases, you can provide your own freeze and thaw functions, by specifying a string with perl source code. It's supposed to @@ -509,16 +511,20 @@ pre-load it into your L process, or you can add a C or C statement into the serialiser string. Or both. -Here are some examples - some of them are also available as global +Here are some examples - all of them are also available as global variables that make them easier to use. =over 4 -=item octet strings - C<$AnyEvent::Fork::RPC::STRING_SERIALISER> +=item C<$AnyEvent::Fork::RPC::STRING_SERIALISER> - octet strings only -This serialiser concatenates length-prefixes octet strings, and is the -default. That means you can only pass (and return) strings containing -character codes 0-255. +This serialiser (currently the default) concatenates length-prefixes octet +strings, and is the default. That means you can only pass (and return) +strings containing character codes 0-255. + +The main advantages of this serialiser are the high speed and that it +doesn't need another module. The main disadvantage is that you are very +limited in what you can pass - only octet strings. Implementation: @@ -527,7 +533,30 @@ sub { unpack "(w/a*)*", shift } ) -=item json - C<$AnyEvent::Fork::RPC::JSON_SERIALISER> +=item C<$AnyEvent::Fork::RPC::CBOR_XS_SERIALISER> - uses L + +This serialiser creates CBOR::XS arrays - you have to make sure the +L module is installed for this serialiser to work. It can be +beneficial for sharing when you preload the L module in a template +process. + +L is about as fast as the octet string serialiser, but supports +complex data structures (similar to JSON) and is faster than any of the +other serialisers. If you have the L module available, it's the +best choice. + +The encoder enables C (so this serialisation method can +encode cyclic and self-referencing data structures). + +Implementation: + + use CBOR::XS (); + ( + sub { CBOR::XS::encode_cbor_sharing \@_ }, + sub { @{ CBOR::XS::decode_cbor shift } } + ) + +=item C<$AnyEvent::Fork::RPC::JSON_SERIALISER> - uses L or L This serialiser creates JSON arrays - you have to make sure the L module is installed for this serialiser to work. It can be beneficial for @@ -545,7 +574,7 @@ sub { @{ JSON::decode_json shift } } ) -=item storable - C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER> +=item C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER> - L This serialiser uses L, which means it has high chance of serialising just about anything you throw at it, at the cost of having @@ -560,7 +589,7 @@ sub { @{ Storable::thaw shift } } ) -=item portable storable - C<$AnyEvent::Fork::RPC::NSTORABLE_SERIALISER> +=item C<$AnyEvent::Fork::RPC::NSTORABLE_SERIALISER> - portable Storable This serialiser also uses L, but uses it's "network" format to serialise data, which makes it possible to talk to different @@ -577,6 +606,27 @@ =back +=item buflen => $bytes (default: C<512 - 16>) + +The starting size of the read buffer for request and response data. + +C ensures that the buffer for reeading request and +response data is large enough for at leats aingle request or response, and +will dynamically enlarge the buffer if needed. + +While this ensures that memory is not overly wasted, it typically leads +to having to do one syscall per request, which can be inefficient in some +cases. In such cases, it can be beneficient to increase the buffer size to +hold more than one request. + +=item buflen_req => $bytes (default: same as C) + +Overrides C for request data (as read by the forked process). + +=item buflen_res => $bytes (default: same as C) + +Overrides C for response data (replies read by the parent process). + =back See the examples section earlier in this document for some actual @@ -585,7 +635,8 @@ =cut our $STRING_SERIALISER = '(sub { pack "(w/a*)*", @_ }, sub { unpack "(w/a*)*", shift })'; -our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })'; +our $CBOR_XS_SERIALISER = 'use CBOR::XS (); (sub { CBOR::XS::encode_cbor_sharing \@_ }, sub { @{ CBOR::XS::decode_cbor shift } })'; +our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })'; our $STORABLE_SERIALISER = 'use Storable (); (sub { Storable::freeze \@_ }, sub { @{ Storable::thaw shift } })'; our $NSTORABLE_SERIALISER = 'use Storable (); (sub { Storable::nfreeze \@_ }, sub { @{ Storable::thaw shift } })'; @@ -608,7 +659,7 @@ my ($f, $t) = eval $serialiser; die $@ if $@; my (@rcb, %rcb, $fh, $shutdown, $wbuf, $ww); - my ($rlen, $rbuf, $rw) = 512 - 16; + my ($rlen, $rbuf, $rw) = $arg{buflen_res} || $arg{buflen} || 512 - 16; my $wcb = sub { my $len = syswrite $fh, $wbuf; @@ -630,10 +681,18 @@ my $module = "AnyEvent::Fork::RPC::" . ($arg{async} ? "Async" : "Sync"); - $self->require ($module) - ->send_arg ($function, $arg{init}, $serialiser, $arg{done} || "CORE::exit") + $self->eval ("use $module 2 ()") + ->send_arg ( + function => $function, + init => $arg{init}, + serialiser => $serialiser, + done => $arg{done} || "$module\::do_exit", + rlen => $arg{buflen_req} || $arg{buflen} || 512 - 16, + -10 # the above are 10 arguments + ) ->run ("$module\::run", sub { - $fh = shift; + $fh = shift + or return $on_error->("connection failed"); my ($id, $len); $rw = AE::io $fh, 0, sub { @@ -744,9 +803,21 @@ without having to load any extra modules. They are part of the child-side API of L. +Note that these functions are typically not yet declared when code is +compiled into the child, because the backend module is only loaded when +you call C, which is typically the last method you call on the fork +object. + +Therefore, you either have to explicitly pre-load the right backend module +or mark calls to these functions as function calls, e.g.: + + AnyEvent::Fork::RPC::event (0 => "five"); + AnyEvent::Fork::RPC::event->(0 => "five"); + &AnyEvent::Fork::RPC::flush; + =over 4 -=item AnyEvent::Fork::RPC::event ... +=item AnyEvent::Fork::RPC::event (...) Send an event to the parent. Events are a bit like RPC calls made by the child process to the parent, except that there is no notion of return @@ -755,6 +826,50 @@ See the examples section earlier in this document for some actual examples. +Note: the event data, like any data send to the parent, might not be sent +immediatelly but queued for later sending, so there is no guarantee that +the event has been sent to the parent when the call returns - when you +e.g. exit directly after calling this function, the parent might never +receive the event. See the next function for a remedy. + +=item $success = AnyEvent::Fork::RPC::flush () + +Synchronously wait and flush the reply data to the parent. Returns true on +success and false otherwise (i.e. when the reply data cannot be written at +all). Ignoring the success status is a common and healthy behaviour. + +Only the "async" backend does something on C - the "sync" backend +is not buffering reply data and always returns true from this function. + +Normally, reply data might or might not be written to the parent +immediatelly but is buffered. This can greatly improve performance and +efficiency, but sometimes can get in your way: for example. when you want +to send an error message just before exiting, or when you want to ensure +replies timely reach the parent before starting a long blocking operation. + +In these cases, you can call this function to flush any outstanding reply +data to the parent. This is done blockingly, so no requests will be +handled and no event callbacks will be called. + +For example, you could wrap your request function in a C block and +report the exception string back to the caller just before exiting: + + sub req { + ... + + eval { + ... + }; + + if ($@) { + AnyEvent::RPC::event (throw => "$@"); + AnyEvent::RPC::flush (); + exit; + } + + ... + } + =back =head2 PROCESS EXIT @@ -788,7 +903,7 @@ listens for another request by the parent, it might detect that the socket was closed (e.g. because the parent exited). It will sotp listening for new requests and instead try to write out any remaining data (if any) or -simply check whether the socket cna be written to. After this, the RPC +simply check whether the socket can be written to. After this, the RPC process is effectively done - no new requests are incoming, no outstanding request data can be written back. @@ -798,8 +913,9 @@ continue. This is why the asynchronous backend explicitly calls C when -it is done (it will raise an exception under other circumstances, which -might lead to the process not exiting on it's own). +it is done (under other circumstances, such as when there is an I/O error +and there is outstanding data to write, it will log a fatal message via +L, also causing the program to exit). You can override this by specifying a function name to call via the C parameter instead. @@ -868,7 +984,8 @@ problems. Alternatively, the parent could limit the amount of rpc calls that are outstanding. -Blocking use of condvars is not supported. +Blocking use of condvars is not supported (in the main thread, outside of +e.g. L threads). Using event-based modules such as L, L, L and so on is easy. @@ -944,7 +1061,7 @@ =head1 EXCEPTIONS There are no provisions whatsoever for catching exceptions at this time - -in the child, exeptions might kill the process, causing calls to be lost +in the child, exceptions might kill the process, causing calls to be lost and the parent encountering a fatal error. In the parent, exceptions in the result callback will not be caught and cause undefined behaviour.