[ViewVC] Diff of: cvs/AnyEvent-Fork-RPC/RPC.pm

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.10 by root, Wed Apr 17 22:04:49 2013 UTC vs.
Revision 1.39 by root, Thu May 12 16:54:43 2016 UTC

…		…
2		2
3	AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork	3	AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork
4		4
5	=head1 SYNOPSIS	5	=head1 SYNOPSIS
6		6
		7	use AnyEvent::Fork;
7	use AnyEvent::Fork::RPC;	8	use AnyEvent::Fork::RPC;
8	# use AnyEvent::Fork is not needed
9		9
10	my $rpc = AnyEvent::Fork	10	my $rpc = AnyEvent::Fork
11	->new	11	->new
12	->require ("MyModule")	12	->require ("MyModule")
13	->AnyEvent::Fork::RPC::run (	13	->AnyEvent::Fork::RPC::run (
14	"MyModule::server",	14	"MyModule::server",
15	);	15	);
16		16
		17	use AnyEvent;
		18
17	my $cv = AE::cv;	19	my $cv = AE::cv;
18		20
19	$rpc->(1, 2, 3, sub {	21	$rpc->(1, 2, 3, sub {
20	print "MyModule::server returned @_\n";	22	print "MyModule::server returned @_\n";
21	$cv->send;	23	$cv->send;
…		…
24	$cv->recv;	26	$cv->recv;
25		27
26	=head1 DESCRIPTION	28	=head1 DESCRIPTION
27		29
28	This module implements a simple RPC protocol and backend for processes	30	This module implements a simple RPC protocol and backend for processes
29	created via L<AnyEvent::Fork>, allowing you to call a function in the	31	created via L<AnyEvent::Fork> or L<AnyEvent::Fork::Remote>, allowing you
30	child process and receive its return values (up to 4GB serialised).	32	to call a function in the child process and receive its return values (up
		33	to 4GB serialised).
31		34
32	It implements two different backends: a synchronous one that works like a	35	It implements two different backends: a synchronous one that works like a
33	normal function call, and an asynchronous one that can run multiple jobs	36	normal function call, and an asynchronous one that can run multiple jobs
34	concurrently in the child, using AnyEvent.	37	concurrently in the child, using AnyEvent.
35		38
36	It also implements an asynchronous event mechanism from the child to the	39	It also implements an asynchronous event mechanism from the child to the
37	parent, that could be used for progress indications or other information.	40	parent, that could be used for progress indications or other information.
38
39	Loading this module also always loads L<AnyEvent::Fork>, so you can make a
40	separate C<use AnyEvent::Fork> if you wish, but you don't have to.
41		41
42	=head1 EXAMPLES	42	=head1 EXAMPLES
43		43
44	=head2 Example 1: Synchronous Backend	44	=head2 Example 1: Synchronous Backend
45		45
…		…
58		58
59	my $rpc = AnyEvent::Fork	59	my $rpc = AnyEvent::Fork
60	->new	60	->new
61	->require ("MyWorker")	61	->require ("MyWorker")
62	->AnyEvent::Fork::RPC::run ("MyWorker::run",	62	->AnyEvent::Fork::RPC::run ("MyWorker::run",
63	on_error => sub { warn "FATAL: $_[0]"; exit 1 },	63	on_error => sub { warn "ERROR: $_[0]"; exit 1 },
64	on_event => sub { warn "$_[0] requests handled\n" },	64	on_event => sub { warn "$_[0] requests handled\n" },
65	on_destroy => $done,	65	on_destroy => $done,
66	);	66	);
67		67
68	for my $id (1..6) {	68	for my $id (1..6) {
…		…
174	child process may exit at any time, so you should call C<$done> only when	174	child process may exit at any time, so you should call C<$done> only when
175	you really I<are> done.	175	you really I<are> done.
176		176
177	=head2 Example 2: Asynchronous Backend	177	=head2 Example 2: Asynchronous Backend
178		178
179	#TODO	179	This example implements multiple count-downs in the child, using
		180	L<AnyEvent> timers. While this is a bit silly (one could use timers in the
		181	parent just as well), it illustrates the ability to use AnyEvent in the
		182	child and the fact that responses can arrive in a different order then the
		183	requests.
		184
		185	It also shows how to embed the actual child code into a C<__DATA__>
		186	section, so it doesn't need any external files at all.
		187
		188	And when your parent process is often busy, and you have stricter timing
		189	requirements, then running timers in a child process suddenly doesn't look
		190	so silly anymore.
		191
		192	Without further ado, here is the code:
		193
		194	use AnyEvent;
		195	use AnyEvent::Fork;
		196	use AnyEvent::Fork::RPC;
		197
		198	my $done = AE::cv;
		199
		200	my $rpc = AnyEvent::Fork
		201	->new
		202	->require ("AnyEvent::Fork::RPC::Async")
		203	->eval (do { local $/; <DATA> })
		204	->AnyEvent::Fork::RPC::run ("run",
		205	async => 1,
		206	on_error => sub { warn "ERROR: $_[0]"; exit 1 },
		207	on_event => sub { print $_[0] },
		208	on_destroy => $done,
		209	);
		210
		211	for my $count (3, 2, 1) {
		212	$rpc->($count, sub {
		213	warn "job $count finished\n";
		214	});
		215	}
		216
		217	undef $rpc;
		218
		219	$done->recv;
		220
		221	__DATA__
		222
		223	# this ends up in main, as we don't use a package declaration
		224
		225	use AnyEvent;
		226
		227	sub run {
		228	my ($done, $count) = @_;
		229
		230	my $n;
		231
		232	AnyEvent::Fork::RPC::event "starting to count up to $count\n";
		233
		234	my $w; $w = AE::timer 1, 1, sub {
		235	++$n;
		236
		237	AnyEvent::Fork::RPC::event "count $n of $count\n";
		238
		239	if ($n == $count) {
		240	undef $w;
		241	$done->();
		242	}
		243	};
		244	}
		245
		246	The parent part (the one before the C<__DATA__> section) isn't very
		247	different from the earlier examples. It sets async mode, preloads
		248	the backend module (so the C<AnyEvent::Fork::RPC::event> function is
		249	declared), uses a slightly different C<on_event> handler (which we use
		250	simply for logging purposes) and then, instead of loading a module with
		251	the actual worker code, it C<eval>'s the code from the data section in the
		252	child process.
		253
		254	It then starts three countdowns, from 3 to 1 seconds downwards, destroys
		255	the rpc object so the example finishes eventually, and then just waits for
		256	the stuff to trickle in.
		257
		258	The worker code uses the event function to log some progress messages, but
		259	mostly just creates a recurring one-second timer.
		260
		261	The timer callback increments a counter, logs a message, and eventually,
		262	when the count has been reached, calls the finish callback.
		263
		264	On my system, this results in the following output. Since all timers fire
		265	at roughly the same time, the actual order isn't guaranteed, but the order
		266	shown is very likely what you would get, too.
		267
		268	starting to count up to 3
		269	starting to count up to 2
		270	starting to count up to 1
		271	count 1 of 3
		272	count 1 of 2
		273	count 1 of 1
		274	job 1 finished
		275	count 2 of 2
		276	job 2 finished
		277	count 2 of 3
		278	count 3 of 3
		279	job 3 finished
		280
		281	While the overall ordering isn't guaranteed, the async backend still
		282	guarantees that events and responses are delivered to the parent process
		283	in the exact same ordering as they were generated in the child process.
		284
		285	And unless your system is I<very> busy, it should clearly show that the
		286	job started last will finish first, as it has the lowest count.
		287
		288	This concludes the async example. Since L<AnyEvent::Fork> does not
		289	actually fork, you are free to use about any module in the child, not just
		290	L<AnyEvent>, but also L<IO::AIO>, or L<Tk> for example.
		291
		292	=head2 Example 3: Asynchronous backend with Coro
		293
		294	With L<Coro> you can create a nice asynchronous backend implementation by
		295	defining an rpc server function that creates a new Coro thread for every
		296	request that calls a function "normally", i.e. the parameters from the
		297	parent process are passed to it, and any return values are returned to the
		298	parent process, e.g.:
		299
		300	package My::Arith;
		301
		302	sub add {
		303	return $_[0] + $_[1];
		304	}
		305
		306	sub mul {
		307	return $_[0] * $_[1];
		308	}
		309
		310	sub run {
		311	my ($done, $func, @arg) = @_;
		312
		313	Coro::async_pool {
		314	$done->($func->(@arg));
		315	};
		316	}
		317
		318	The C<run> function creates a new thread for every invocation, using the
		319	first argument as function name, and calls the C<$done> callback on it's
		320	return values. This makes it quite natural to define the C<add> and C<mul>
		321	functions to add or multiply two numbers and return the result.
		322
		323	Since this is the asynchronous backend, it's quite possible to define RPC
		324	function that do I/O or wait for external events - their execution will
		325	overlap as needed.
		326
		327	The above could be used like this:
		328
		329	my $rpc = AnyEvent::Fork
		330	->new
		331	->require ("MyWorker")
		332	->AnyEvent::Fork::RPC::run ("My::Arith::run",
		333	on_error => ..., on_event => ..., on_destroy => ...,
		334	);
		335
		336	$rpc->(add => 1, 3, Coro::rouse_cb); say Coro::rouse_wait;
		337	$rpc->(mul => 3, 2, Coro::rouse_cb); say Coro::rouse_wait;
		338
		339	The C<say>'s will print C<4> and C<6>.
		340
		341	=head2 Example 4: Forward AnyEvent::Log messages using C<on_event>
		342
		343	This partial example shows how to use the C<event> function to forward
		344	L<AnyEvent::Log> messages to the parent.
		345
		346	For this, the parent needs to provide a suitable C<on_event>:
		347
		348	->AnyEvent::Fork::RPC::run (
		349	on_event => sub {
		350	if ($_[0] eq "ae_log") {
		351	my (undef, $level, $message) = @_;
		352	AE::log $level, $message;
		353	} else {
		354	# other event types
		355	}
		356	},
		357	)
		358
		359	In the child, as early as possible, the following code should reconfigure
		360	L<AnyEvent::Log> to log via C<AnyEvent::Fork::RPC::event>:
		361
		362	$AnyEvent::Log::LOG->log_cb (sub {
		363	my ($timestamp, $orig_ctx, $level, $message) = @{+shift};
		364
		365	if (defined &AnyEvent::Fork::RPC::event) {
		366	AnyEvent::Fork::RPC::event (ae_log => $level, $message);
		367	} else {
		368	warn "[$$ before init] $message\n";
		369	}
		370	});
		371
		372	There is an important twist - the C<AnyEvent::Fork::RPC::event> function
		373	is only defined when the child is fully initialised. If you redirect the
		374	log messages in your C<init> function for example, then the C<event>
		375	function might not yet be available. This is why the log callback checks
		376	whether the fucntion is there using C<defined>, and only then uses it to
		377	log the message.
180		378
181	=head1 PARENT PROCESS USAGE	379	=head1 PARENT PROCESS USAGE
182		380
183	This module exports nothing, and only implements a single function:	381	This module exports nothing, and only implements a single function:
184		382
…		…
192		390
193	use Errno ();	391	use Errno ();
194	use Guard ();	392	use Guard ();
195		393
196	use AnyEvent;	394	use AnyEvent;
197	use AnyEvent::Fork; # we don't actually depend on it, this is for convenience
198		395
199	our $VERSION = 0.1;	396	our $VERSION = 1.22;
200		397
201	=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]	398	=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]
202		399
203	The traditional way to call it. But it is way cooler to call it in the	400	The traditional way to call it. But it is way cooler to call it in the
204	following way:	401	following way:
…		…
224	Called on (fatal) errors, with a descriptive (hopefully) message. If	421	Called on (fatal) errors, with a descriptive (hopefully) message. If
225	this callback is not provided, but C<on_event> is, then the C<on_event>	422	this callback is not provided, but C<on_event> is, then the C<on_event>
226	callback is called with the first argument being the string C<error>,	423	callback is called with the first argument being the string C<error>,
227	followed by the error message.	424	followed by the error message.
228		425
229	If neither handler is provided it prints the error to STDERR and will	426	If neither handler is provided, then the error is reported with loglevel
230	start failing badly.	427	C<error> via C<AE::log>.
231		428
232	=item on_event => $cb->(...)	429	=item on_event => $cb->(...)
233		430
234	Called for every call to the C<AnyEvent::Fork::RPC::event> function in the	431	Called for every call to the C<AnyEvent::Fork::RPC::event> function in the
235	child, with the arguments of that function passed to the callback.	432	child, with the arguments of that function passed to the callback.
…		…
257	It is called very early - before the serialisers are created or the	454	It is called very early - before the serialisers are created or the
258	C<$function> name is resolved into a function reference, so it could be	455	C<$function> name is resolved into a function reference, so it could be
259	used to load any modules that provide the serialiser or function. It can	456	used to load any modules that provide the serialiser or function. It can
260	not, however, create events.	457	not, however, create events.
261		458
		459	=item done => $function (default C<CORE::exit>)
		460
		461	The function to call when the asynchronous backend detects an end of file
		462	condition when reading from the communications socket I<and> there are no
		463	outstanding requests. It's ignored by the synchronous backend.
		464
		465	By overriding this you can prolong the life of a RPC process after e.g.
		466	the parent has exited by running the event loop in the provided function
		467	(or simply calling it, for example, when your child process uses L<EV> you
		468	could provide L<EV::run> as C<done> function).
		469
		470	Of course, in that case you are responsible for exiting at the appropriate
		471	time and not returning from
		472
262	=item async => $boolean (default: 0)	473	=item async => $boolean (default: 0)
263		474
264	The default server used in the child does all I/O blockingly, and only	475	The default server used in the child does all I/O blockingly, and only
265	allows a single RPC call to execute concurrently.	476	allows a single RPC call to execute concurrently.
266		477
267	Setting C<async> to a true value switches to another implementation that	478	Setting C<async> to a true value switches to another implementation that
268	uses L<AnyEvent> in the child and allows multiple concurrent RPC calls.	479	uses L<AnyEvent> in the child and allows multiple concurrent RPC calls (it
		480	does not support recursion in the event loop however, blocking condvar
		481	calls will fail).
269		482
270	The actual API in the child is documented in the section that describes	483	The actual API in the child is documented in the section that describes
271	the calling semantics of the returned C<$rpc> function.	484	the calling semantics of the returned C<$rpc> function.
272		485
273	If you want to pre-load the actual back-end modules to enable memory	486	If you want to pre-load the actual back-end modules to enable memory
…		…
275	synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode.	488	synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode.
276		489
277	If you use a template process and want to fork both sync and async	490	If you use a template process and want to fork both sync and async
278	children, then it is permissible to load both modules.	491	children, then it is permissible to load both modules.
279		492
280	=item serialiser => $string (default: '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })')	493	=item serialiser => $string (default: $AnyEvent::Fork::RPC::STRING_SERIALISER)
281		494
282	All arguments, result data and event data have to be serialised to be	495	All arguments, result data and event data have to be serialised to be
283	transferred between the processes. For this, they have to be frozen and	496	transferred between the processes. For this, they have to be frozen and
284	thawed in both parent and child processes.	497	thawed in both parent and child processes.
285		498
286	By default, only octet strings can be passed between the processes, which	499	By default, only octet strings can be passed between the processes,
287	is reasonably fast and efficient.	500	which is reasonably fast and efficient and requires no extra modules
		501	(the C<AnyEvent::Fork::RPC> distribution does not provide these extra
		502	serialiser modules).
288		503
289	For more complicated use cases, you can provide your own freeze and thaw	504	For more complicated use cases, you can provide your own freeze and thaw
290	functions, by specifying a string with perl source code. It's supposed to	505	functions, by specifying a string with perl source code. It's supposed to
291	return two code references when evaluated: the first receives a list of	506	return two code references when evaluated: the first receives a list of
292	perl values and must return an octet string. The second receives the octet	507	perl values and must return an octet string. The second receives the octet
…		…
294		509
295	If you need an external module for serialisation, then you can either	510	If you need an external module for serialisation, then you can either
296	pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use>	511	pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use>
297	or C<require> statement into the serialiser string. Or both.	512	or C<require> statement into the serialiser string. Or both.
298		513
		514	Here are some examples - all of them are also available as global
		515	variables that make them easier to use.
		516
		517	=over 4
		518
		519	=item C<$AnyEvent::Fork::RPC::STRING_SERIALISER> - octet strings only
		520
		521	This serialiser (currently the default) concatenates length-prefixes octet
		522	strings, and is the default. That means you can only pass (and return)
		523	strings containing character codes 0-255.
		524
		525	The main advantages of this serialiser are the high speed and that it
		526	doesn't need another module. The main disadvantage is that you are very
		527	limited in what you can pass - only octet strings.
		528
		529	Implementation:
		530
		531	(
		532	sub { pack "(w/a)", @_ },
		533	sub { unpack "(w/a)", shift }
		534	)
		535
		536	=item C<$AnyEvent::Fork::RPC::CBOR_XS_SERIALISER> - uses L<CBOR::XS>
		537
		538	This serialiser creates CBOR::XS arrays - you have to make sure the
		539	L<CBOR::XS> module is installed for this serialiser to work. It can be
		540	beneficial for sharing when you preload the L<CBOR::XS> module in a template
		541	process.
		542
		543	L<CBOR::XS> is about as fast as the octet string serialiser, but supports
		544	complex data structures (similar to JSON) and is faster than any of the
		545	other serialisers. If you have the L<CBOR::XS> module available, it's the
		546	best choice.
		547
		548	The encoder enables C<allow_sharing> (so this serialisation method can
		549	encode cyclic and self-referencing data structures).
		550
		551	Implementation:
		552
		553	use CBOR::XS ();
		554	(
		555	sub { CBOR::XS::encode_cbor_sharing \@_ },
		556	sub { @{ CBOR::XS::decode_cbor shift } }
		557	)
		558
		559	=item C<$AnyEvent::Fork::RPC::JSON_SERIALISER> - uses L<JSON::XS> or L<JSON>
		560
		561	This serialiser creates JSON arrays - you have to make sure the L<JSON>
		562	module is installed for this serialiser to work. It can be beneficial for
		563	sharing when you preload the L<JSON> module in a template process.
		564
		565	L<JSON> (with L<JSON::XS> installed) is slower than the octet string
		566	serialiser, but usually much faster than L<Storable>, unless big chunks of
		567	binary data need to be transferred.
		568
		569	Implementation:
		570
		571	use JSON ();
		572	(
		573	sub { JSON::encode_json \@_ },
		574	sub { @{ JSON::decode_json shift } }
		575	)
		576
		577	=item C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER> - L<Storable>
		578
		579	This serialiser uses L<Storable>, which means it has high chance of
		580	serialising just about anything you throw at it, at the cost of having
		581	very high overhead per operation. It also comes with perl. It should be
		582	used when you need to serialise complex data structures.
		583
		584	Implementation:
		585
		586	use Storable ();
		587	(
		588	sub { Storable::freeze \@_ },
		589	sub { @{ Storable::thaw shift } }
		590	)
		591
		592	=item C<$AnyEvent::Fork::RPC::NSTORABLE_SERIALISER> - portable Storable
		593
		594	This serialiser also uses L<Storable>, but uses it's "network" format
		595	to serialise data, which makes it possible to talk to different
		596	perl binaries (for example, when talking to a process created with
		597	L<AnyEvent::Fork::Remote>).
		598
		599	Implementation:
		600
		601	use Storable ();
		602	(
		603	sub { Storable::nfreeze \@_ },
		604	sub { @{ Storable::thaw shift } }
		605	)
		606
		607	=back
		608
299	=back	609	=back
300		610
301	See the examples section earlier in this document for some actual	611	See the examples section earlier in this document for some actual
302	examples.	612	examples.
303		613
304	=cut	614	=cut
305		615
306	our $STRING_SERIALISER = '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })';	616	our $STRING_SERIALISER = '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })';
		617	our $CBOR_XS_SERIALISER = 'use CBOR::XS (); (sub { CBOR::XS::encode_cbor_sharing \@_ }, sub { @{ CBOR::XS::decode_cbor shift } })';
		618	our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })';
		619	our $STORABLE_SERIALISER = 'use Storable (); (sub { Storable::freeze \@_ }, sub { @{ Storable::thaw shift } })';
		620	our $NSTORABLE_SERIALISER = 'use Storable (); (sub { Storable::nfreeze \@_ }, sub { @{ Storable::thaw shift } })';
307		621
308	sub run {	622	sub run {
309	my ($self, $function, %arg) = @_;	623	my ($self, $function, %arg) = @_;
310		624
311	my $serialiser = delete $arg{serialiser} \|\| $STRING_SERIALISER;	625	my $serialiser = delete $arg{serialiser} \|\| $STRING_SERIALISER;
…		…
314	my $on_destroy = delete $arg{on_destroy};	628	my $on_destroy = delete $arg{on_destroy};
315		629
316	# default for on_error is to on_event, if specified	630	# default for on_error is to on_event, if specified
317	$on_error \|\|= $on_event	631	$on_error \|\|= $on_event
318	? sub { $on_event->(error => shift) }	632	? sub { $on_event->(error => shift) }
319	: sub { die "AnyEvent::Fork::RPC: uncaught error: $_[0].\n" };	633	: sub { AE::log die => "AnyEvent::Fork::RPC: uncaught error: $_[0]." };
320		634
321	# default for on_event is to raise an error	635	# default for on_event is to raise an error
322	$on_event \|\|= sub { $on_error->("event received, but no on_event handler") };	636	$on_event \|\|= sub { $on_error->("event received, but no on_event handler") };
323		637
324	my ($f, $t) = eval $serialiser; die $@ if $@;	638	my ($f, $t) = eval $serialiser; die $@ if $@;
…		…
345	};	659	};
346		660
347	my $module = "AnyEvent::Fork::RPC::" . ($arg{async} ? "Async" : "Sync");	661	my $module = "AnyEvent::Fork::RPC::" . ($arg{async} ? "Async" : "Sync");
348		662
349	$self->require ($module)	663	$self->require ($module)
350	->send_arg ($function, $arg{init}, $serialiser)	664	->send_arg ($function, $arg{init}, $serialiser, $arg{done} \|\| "$module\::do_exit")
351	->run ("$module\::run", sub {	665	->run ("$module\::run", sub {
352	$fh = shift;	666	$fh = shift;
353		667
354	my ($id, $len);	668	my ($id, $len);
355	$rw = AE::io $fh, 0, sub {	669	$rw = AE::io $fh, 0, sub {
356	$rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf;	670	$rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf;
357	$len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf;	671	$len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf;
358		672
359	if ($len) {	673	if ($len) {
360	while (8 <= length $rbuf) {	674	while (8 <= length $rbuf) {
361	($id, $len) = unpack "LL", $rbuf;	675	($id, $len) = unpack "NN", $rbuf;
362	8 + $len <= length $rbuf	676	8 + $len <= length $rbuf
363	or last;	677	or last;
364		678
365	my @r = $t->(substr $rbuf, 8, $len);	679	my @r = $t->(substr $rbuf, 8, $len);
366	substr $rbuf, 0, 8 + $len, "";	680	substr $rbuf, 0, 8 + $len, "";
…		…
380	}	694	}
381	} elsif (defined $len) {	695	} elsif (defined $len) {
382	undef $rw; undef $ww; # it ends here	696	undef $rw; undef $ww; # it ends here
383		697
384	if (@rcb \|\| %rcb) {	698	if (@rcb \|\| %rcb) {
385	use Data::Dump;ddx[\@rcb,\%rcb];#d#
386	$on_error->("unexpected eof");	699	$on_error->("unexpected eof");
387	} else {	700	} else {
388	$on_destroy->();	701	$on_destroy->()
		702	if $on_destroy;
389	}	703	}
390	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {	704	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {
391	undef $rw; undef $ww; # it ends here	705	undef $rw; undef $ww; # it ends here
392	$on_error->("read: $!");	706	$on_error->("read: $!");
393	}	707	}
…		…
396	$ww \|\|= AE::io $fh, 1, $wcb;	710	$ww \|\|= AE::io $fh, 1, $wcb;
397	});	711	});
398		712
399	my $guard = Guard::guard {	713	my $guard = Guard::guard {
400	$shutdown = 1;	714	$shutdown = 1;
401	$ww \|\|= $fh && AE::io $fh, 1, $wcb;	715
		716	shutdown $fh, 1 if $fh && !$ww;
402	};	717	};
403		718
404	my $id;	719	my $id;
405		720
406	$arg{async}	721	$arg{async}
…		…
408	$id = ($id == 0xffffffff ? 0 : $id) + 1;	723	$id = ($id == 0xffffffff ? 0 : $id) + 1;
409	$id = ($id == 0xffffffff ? 0 : $id) + 1 while exists $rcb{$id}; # rarely loops	724	$id = ($id == 0xffffffff ? 0 : $id) + 1 while exists $rcb{$id}; # rarely loops
410		725
411	$rcb{$id} = pop;	726	$rcb{$id} = pop;
412		727
413	$guard; # keep it alive	728	$guard if 0; # keep it alive
414		729
415	$wbuf .= pack "LL/a*", $id, &$f;	730	$wbuf .= pack "NN/a*", $id, &$f;
416	$ww \|\|= $fh && AE::io $fh, 1, $wcb;	731	$ww \|\|= $fh && AE::io $fh, 1, $wcb;
417	}	732	}
418	: sub {	733	: sub {
419	push @rcb, pop;	734	push @rcb, pop;
420		735
421	$guard; # keep it alive	736	$guard; # keep it alive
422		737
423	$wbuf .= pack "L/a*", &$f;	738	$wbuf .= pack "N/a*", &$f;
424	$ww \|\|= $fh && AE::io $fh, 1, $wcb;	739	$ww \|\|= $fh && AE::io $fh, 1, $wcb;
425	}	740	}
426	}	741	}
427		742
428	=item $rpc->(..., $cb->(...))	743	=item $rpc->(..., $cb->(...))
…		…
470	See the examples section earlier in this document for some actual	785	See the examples section earlier in this document for some actual
471	examples.	786	examples.
472		787
473	=back	788	=back
474		789
		790	=head2 PROCESS EXIT
		791
		792	If and when the child process exits depends on the backend and
		793	configuration. Apart from explicit exits (e.g. by calling C<exit>) or
		794	runtime conditions (uncaught exceptions, signals etc.), the backends exit
		795	under these conditions:
		796
		797	=over 4
		798
		799	=item Synchronous Backend
		800
		801	The synchronous backend is very simple: when the process waits for another
		802	request to arrive and the writing side (usually in the parent) is closed,
		803	it will exit normally, i.e. as if your main program reached the end of the
		804	file.
		805
		806	That means that if your parent process exits, the RPC process will usually
		807	exit as well, either because it is idle anyway, or because it executes a
		808	request. In the latter case, you will likely get an error when the RPc
		809	process tries to send the results to the parent (because agruably, you
		810	shouldn't exit your parent while there are still outstanding requests).
		811
		812	The process is usually quiescent when it happens, so it should rarely be a
		813	problem, and C<END> handlers can be used to clean up.
		814
		815	=item Asynchronous Backend
		816
		817	For the asynchronous backend, things are more complicated: Whenever it
		818	listens for another request by the parent, it might detect that the socket
		819	was closed (e.g. because the parent exited). It will sotp listening for
		820	new requests and instead try to write out any remaining data (if any) or
		821	simply check whether the socket can be written to. After this, the RPC
		822	process is effectively done - no new requests are incoming, no outstanding
		823	request data can be written back.
		824
		825	Since chances are high that there are event watchers that the RPC server
		826	knows nothing about (why else would one use the async backend if not for
		827	the ability to register watchers?), the event loop would often happily
		828	continue.
		829
		830	This is why the asynchronous backend explicitly calls C<CORE::exit> when
		831	it is done (under other circumstances, such as when there is an I/O error
		832	and there is outstanding data to write, it will log a fatal message via
		833	L<AnyEvent::Log>, also causing the program to exit).
		834
		835	You can override this by specifying a function name to call via the C<done>
		836	parameter instead.
		837
		838	=back
		839
		840	=head1 ADVANCED TOPICS
		841
		842	=head2 Choosing a backend
		843
		844	So how do you decide which backend to use? Well, that's your problem to
		845	solve, but here are some thoughts on the matter:
		846
		847	=over 4
		848
		849	=item Synchronous
		850
		851	The synchronous backend does not rely on any external modules (well,
		852	except L<common::sense>, which works around a bug in how perl's warning
		853	system works). This keeps the process very small, for example, on my
		854	system, an empty perl interpreter uses 1492kB RSS, which becomes 2020kB
		855	after C<use warnings; use strict> (for people who grew up with C64s around
		856	them this is probably shocking every single time they see it). The worker
		857	process in the first example in this document uses 1792kB.
		858
		859	Since the calls are done synchronously, slow jobs will keep newer jobs
		860	from executing.
		861
		862	The synchronous backend also has no overhead due to running an event loop
		863	- reading requests is therefore very efficient, while writing responses is
		864	less so, as every response results in a write syscall.
		865
		866	If the parent process is busy and a bit slow reading responses, the child
		867	waits instead of processing further requests. This also limits the amount
		868	of memory needed for buffering, as never more than one response has to be
		869	buffered.
		870
		871	The API in the child is simple - you just have to define a function that
		872	does something and returns something.
		873
		874	It's hard to use modules or code that relies on an event loop, as the
		875	child cannot execute anything while it waits for more input.
		876
		877	=item Asynchronous
		878
		879	The asynchronous backend relies on L<AnyEvent>, which tries to be small,
		880	but still comes at a price: On my system, the worker from example 1a uses
		881	3420kB RSS (for L<AnyEvent>, which loads L<EV>, which needs L<XSLoader>
		882	which in turn loads a lot of other modules such as L<warnings>, L<strict>,
		883	L<vars>, L<Exporter>...).
		884
		885	It batches requests and responses reasonably efficiently, doing only as
		886	few reads and writes as needed, but needs to poll for events via the event
		887	loop.
		888
		889	Responses are queued when the parent process is busy. This means the child
		890	can continue to execute any queued requests. It also means that a child
		891	might queue a lot of responses in memory when it generates them and the
		892	parent process is slow accepting them.
		893
		894	The API is not a straightforward RPC pattern - you have to call a
		895	"done" callback to pass return values and signal completion. Also, more
		896	importantly, the API starts jobs as fast as possible - when 1000 jobs
		897	are queued and the jobs are slow, they will all run concurrently. The
		898	child must implement some queueing/limiting mechanism if this causes
		899	problems. Alternatively, the parent could limit the amount of rpc calls
		900	that are outstanding.
		901
		902	Blocking use of condvars is not supported (in the main thread, outside of
		903	e.g. L<Coro> threads).
		904
		905	Using event-based modules such as L<IO::AIO>, L<Gtk2>, L<Tk> and so on is
		906	easy.
		907
		908	=back
		909
		910	=head2 Passing file descriptors
		911
		912	Unlike L<AnyEvent::Fork>, this module has no in-built file handle or file
		913	descriptor passing abilities.
		914
		915	The reason is that passing file descriptors is extraordinary tricky
		916	business, and conflicts with efficient batching of messages.
		917
		918	There still is a method you can use: Create a
		919	C<AnyEvent::Util::portable_socketpair> and C<send_fh> one half of it to
		920	the process before you pass control to C<AnyEvent::Fork::RPC::run>.
		921
		922	Whenever you want to pass a file descriptor, send an rpc request to the
		923	child process (so it expects the descriptor), then send it over the other
		924	half of the socketpair. The child should fetch the descriptor from the
		925	half it has passed earlier.
		926
		927	Here is some (untested) pseudocode to that effect:
		928
		929	use AnyEvent::Util;
		930	use AnyEvent::Fork;
		931	use AnyEvent::Fork::RPC;
		932	use IO::FDPass;
		933
		934	my ($s1, $s2) = AnyEvent::Util::portable_socketpair;
		935
		936	my $rpc = AnyEvent::Fork
		937	->new
		938	->send_fh ($s2)
		939	->require ("MyWorker")
		940	->AnyEvent::Fork::RPC::run ("MyWorker::run"
		941	init => "MyWorker::init",
		942	);
		943
		944	undef $s2; # no need to keep it around
		945
		946	# pass an fd
		947	$rpc->("i'll send some fd now, please expect it!", my $cv = AE::cv);
		948
		949	IO::FDPass fileno $s1, fileno $handle_to_pass;
		950
		951	$cv->recv;
		952
		953	The MyWorker module could look like this:
		954
		955	package MyWorker;
		956
		957	use IO::FDPass;
		958
		959	my $s2;
		960
		961	sub init {
		962	$s2 = $_[0];
		963	}
		964
		965	sub run {
		966	if ($_[0] eq "i'll send some fd now, please expect it!") {
		967	my $fd = IO::FDPass::recv fileno $s2;
		968	...
		969	}
		970	}
		971
		972	Of course, this might be blocking if you pass a lot of file descriptors,
		973	so you might want to look into L<AnyEvent::FDpasser> which can handle the
		974	gory details.
		975
		976	=head1 EXCEPTIONS
		977
		978	There are no provisions whatsoever for catching exceptions at this time -
		979	in the child, exceptions might kill the process, causing calls to be lost
		980	and the parent encountering a fatal error. In the parent, exceptions in
		981	the result callback will not be caught and cause undefined behaviour.
		982
475	=head1 SEE ALSO	983	=head1 SEE ALSO
476		984
477	L<AnyEvent::Fork> (to create the processes in the first place),	985	L<AnyEvent::Fork>, to create the processes in the first place.
		986
		987	L<AnyEvent::Fork::Remote>, likewise, but helpful for remote processes.
		988
478	L<AnyEvent::Fork::Pool> (to manage whole pools of processes).	989	L<AnyEvent::Fork::Pool>, to manage whole pools of processes.
479		990
480	=head1 AUTHOR AND CONTACT INFORMATION	991	=head1 AUTHOR AND CONTACT INFORMATION
481		992
482	Marc Lehmann <schmorp@schmorp.de>	993	Marc Lehmann <schmorp@schmorp.de>
483	http://software.schmorp.de/pkg/AnyEvent-Fork-RPC	994	http://software.schmorp.de/pkg/AnyEvent-Fork-RPC

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents): Revision 1.10 by root, Wed Apr 17 22:04:49 2013 UTC vs. Revision 1.39 by root, Thu May 12 16:54:43 2016 UTC

Diff Legend

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.10 by root, Wed Apr 17 22:04:49 2013 UTC vs.
Revision 1.39 by root, Thu May 12 16:54:43 2016 UTC