[ViewVC] Diff of: cvs/AnyEvent-Fork-RPC/RPC.pm

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.5 by root, Wed Apr 17 19:39:54 2013 UTC vs.
Revision 1.28 by root, Sun Apr 28 15:48:31 2013 UTC

…		…
12	->require ("MyModule")	12	->require ("MyModule")
13	->AnyEvent::Fork::RPC::run (	13	->AnyEvent::Fork::RPC::run (
14	"MyModule::server",	14	"MyModule::server",
15	);	15	);
16		16
		17	use AnyEvent;
		18
17	my $cv = AE::cv;	19	my $cv = AE::cv;
18		20
19	$rpc->(1, 2, 3, sub {	21	$rpc->(1, 2, 3, sub {
20	print "MyModule::server returned @_\n";	22	print "MyModule::server returned @_\n";
21	$cv->send;	23	$cv->send;
…		…
24	$cv->recv;	26	$cv->recv;
25		27
26	=head1 DESCRIPTION	28	=head1 DESCRIPTION
27		29
28	This module implements a simple RPC protocol and backend for processes	30	This module implements a simple RPC protocol and backend for processes
29	created via L<AnyEvent::Fork>, allowing you to call a function in the	31	created via L<AnyEvent::Fork> (or L<AnyEvent::Fork::Remote>), allowing you
30	child process and receive its return values (up to 4GB serialised).	32	to call a function in the child process and receive its return values (up
		33	to 4GB serialised).
31		34
32	It implements two different backends: a synchronous one that works like a	35	It implements two different backends: a synchronous one that works like a
33	normal function call, and an asynchronous one that can run multiple jobs	36	normal function call, and an asynchronous one that can run multiple jobs
34	concurrently in the child, using AnyEvent.	37	concurrently in the child, using AnyEvent.
35		38
36	It also implements an asynchronous event mechanism from the child to the	39	It also implements an asynchronous event mechanism from the child to the
37	parent, that could be used for progress indications or other information.	40	parent, that could be used for progress indications or other information.
38		41
39	=head1 EXAMPLES	42	=head1 EXAMPLES
40		43
41	=head2 Synchronous Backend	44	=head2 Example 1: Synchronous Backend
42		45
43	Here is a simple example that implements a backend that executes C<unlink>	46	Here is a simple example that implements a backend that executes C<unlink>
44	and C<rmdir> calls, and reports their status back. It also reports the	47	and C<rmdir> calls, and reports their status back. It also reports the
45	number of requests it has processed every three requests, which is clearly	48	number of requests it has processed every three requests, which is clearly
46	silly, but illustrates the use of events.	49	silly, but illustrates the use of events.
…		…
103	dies with a fatal error - obviously, you must never let this happen :).	106	dies with a fatal error - obviously, you must never let this happen :).
104		107
105	Eventually it returns the status value true if the command was successful,	108	Eventually it returns the status value true if the command was successful,
106	or the status value 0 and the stringified error message.	109	or the status value 0 and the stringified error message.
107		110
108	On my system, running the first cdoe fragment with the given	111	On my system, running the first code fragment with the given
109	F<MyWorker.pm> in the current directory yields:	112	F<MyWorker.pm> in the current directory yields:
110		113
111	/tmp/somepath/1: No such file or directory	114	/tmp/somepath/1: No such file or directory
112	/tmp/somepath/2: No such file or directory	115	/tmp/somepath/2: No such file or directory
113	3 requests handled	116	3 requests handled
…		…
134		137
135	And as a final remark, there is a fine module on CPAN that can	138	And as a final remark, there is a fine module on CPAN that can
136	asynchronously C<rmdir> and C<unlink> and a lot more, and more efficiently	139	asynchronously C<rmdir> and C<unlink> and a lot more, and more efficiently
137	than this example, namely L<IO::AIO>.	140	than this example, namely L<IO::AIO>.
138		141
		142	=head3 Example 1a: the same with the asynchronous backend
		143
		144	This example only shows what needs to be changed to use the async backend
		145	instead. Doing this is not very useful, the purpose of this example is
		146	to show the minimum amount of change that is required to go from the
		147	synchronous to the asynchronous backend.
		148
		149	To use the async backend in the previous example, you need to add the
		150	C<async> parameter to the C<AnyEvent::Fork::RPC::run> call:
		151
		152	->AnyEvent::Fork::RPC::run ("MyWorker::run",
		153	async => 1,
		154	...
		155
		156	And since the function call protocol is now changed, you need to adopt
		157	C<MyWorker::run> to the async API.
		158
		159	First, you need to accept the extra initial C<$done> callback:
		160
		161	sub run {
		162	my ($done, $cmd, $path) = @_;
		163
		164	And since a response is now generated when C<$done> is called, as opposed
		165	to when the function returns, we need to call the C<$done> function with
		166	the status:
		167
		168	$done->($status or (0, "$!"));
		169
		170	A few remarks are in order. First, it's quite pointless to use the async
		171	backend for this example - but it I<is> possible. Second, you can call
		172	C<$done> before or after returning from the function. Third, having both
		173	returned from the function and having called the C<$done> callback, the
		174	child process may exit at any time, so you should call C<$done> only when
		175	you really I<are> done.
		176
		177	=head2 Example 2: Asynchronous Backend
		178
		179	This example implements multiple count-downs in the child, using
		180	L<AnyEvent> timers. While this is a bit silly (one could use timers in te
		181	parent just as well), it illustrates the ability to use AnyEvent in the
		182	child and the fact that responses can arrive in a different order then the
		183	requests.
		184
		185	It also shows how to embed the actual child code into a C<__DATA__>
		186	section, so it doesn't need any external files at all.
		187
		188	And when your parent process is often busy, and you have stricter timing
		189	requirements, then running timers in a child process suddenly doesn't look
		190	so silly anymore.
		191
		192	Without further ado, here is the code:
		193
		194	use AnyEvent;
		195	use AnyEvent::Fork;
		196	use AnyEvent::Fork::RPC;
		197
		198	my $done = AE::cv;
		199
		200	my $rpc = AnyEvent::Fork
		201	->new
		202	->require ("AnyEvent::Fork::RPC::Async")
		203	->eval (do { local $/; <DATA> })
		204	->AnyEvent::Fork::RPC::run ("run",
		205	async => 1,
		206	on_error => sub { warn "FATAL: $_[0]"; exit 1 },
		207	on_event => sub { print $_[0] },
		208	on_destroy => $done,
		209	);
		210
		211	for my $count (3, 2, 1) {
		212	$rpc->($count, sub {
		213	warn "job $count finished\n";
		214	});
		215	}
		216
		217	undef $rpc;
		218
		219	$done->recv;
		220
		221	__DATA__
		222
		223	# this ends up in main, as we don't use a package declaration
		224
		225	use AnyEvent;
		226
		227	sub run {
		228	my ($done, $count) = @_;
		229
		230	my $n;
		231
		232	AnyEvent::Fork::RPC::event "starting to count up to $count\n";
		233
		234	my $w; $w = AE::timer 1, 1, sub {
		235	++$n;
		236
		237	AnyEvent::Fork::RPC::event "count $n of $count\n";
		238
		239	if ($n == $count) {
		240	undef $w;
		241	$done->();
		242	}
		243	};
		244	}
		245
		246	The parent part (the one before the C<__DATA__> section) isn't very
		247	different from the earlier examples. It sets async mode, preloads
		248	the backend module (so the C<AnyEvent::Fork::RPC::event> function is
		249	declared), uses a slightly different C<on_event> handler (which we use
		250	simply for logging purposes) and then, instead of loading a module with
		251	the actual worker code, it C<eval>'s the code from the data section in the
		252	child process.
		253
		254	It then starts three countdowns, from 3 to 1 seconds downwards, destroys
		255	the rpc object so the example finishes eventually, and then just waits for
		256	the stuff to trickle in.
		257
		258	The worker code uses the event function to log some progress messages, but
		259	mostly just creates a recurring one-second timer.
		260
		261	The timer callback increments a counter, logs a message, and eventually,
		262	when the count has been reached, calls the finish callback.
		263
		264	On my system, this results in the following output. Since all timers fire
		265	at roughly the same time, the actual order isn't guaranteed, but the order
		266	shown is very likely what you would get, too.
		267
		268	starting to count up to 3
		269	starting to count up to 2
		270	starting to count up to 1
		271	count 1 of 3
		272	count 1 of 2
		273	count 1 of 1
		274	job 1 finished
		275	count 2 of 2
		276	job 2 finished
		277	count 2 of 3
		278	count 3 of 3
		279	job 3 finished
		280
		281	While the overall ordering isn't guaranteed, the async backend still
		282	guarantees that events and responses are delivered to the parent process
		283	in the exact same ordering as they were generated in the child process.
		284
		285	And unless your system is I<very> busy, it should clearly show that the
		286	job started last will finish first, as it has the lowest count.
		287
		288	This concludes the async example. Since L<AnyEvent::Fork> does not
		289	actually fork, you are free to use about any module in the child, not just
		290	L<AnyEvent>, but also L<IO::AIO>, or L<Tk> for example.
		291
139	=head1 PARENT PROCESS USAGE	292	=head1 PARENT PROCESS USAGE
140		293
141	This module exports nothing, and only implements a single function:	294	This module exports nothing, and only implements a single function:
142		295
143	=over 4	296	=over 4
…		…
150		303
151	use Errno ();	304	use Errno ();
152	use Guard ();	305	use Guard ();
153		306
154	use AnyEvent;	307	use AnyEvent;
155	#use AnyEvent::Fork;
156		308
157	our $VERSION = 0.1;	309	our $VERSION = 1.1;
158		310
159	=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]	311	=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]
160		312
161	The traditional way to call it. But it is way cooler to call it in the	313	The traditional way to call it. But it is way cooler to call it in the
162	following way:	314	following way:
…		…
198		350
199	Called when the C<$rpc> object has been destroyed and all requests have	351	Called when the C<$rpc> object has been destroyed and all requests have
200	been successfully handled. This is useful when you queue some requests and	352	been successfully handled. This is useful when you queue some requests and
201	want the child to go away after it has handled them. The problem is that	353	want the child to go away after it has handled them. The problem is that
202	the parent must not exit either until all requests have been handled, and	354	the parent must not exit either until all requests have been handled, and
203	this cna be accomplished by waiting for this callback.	355	this can be accomplished by waiting for this callback.
204		356
205	=item init => $function (default none)	357	=item init => $function (default none)
206		358
207	When specified (by name), this function is called in the child as the very	359	When specified (by name), this function is called in the child as the very
208	first thing when taking over the process, with all the arguments normally	360	first thing when taking over the process, with all the arguments normally
…		…
221		373
222	The default server used in the child does all I/O blockingly, and only	374	The default server used in the child does all I/O blockingly, and only
223	allows a single RPC call to execute concurrently.	375	allows a single RPC call to execute concurrently.
224		376
225	Setting C<async> to a true value switches to another implementation that	377	Setting C<async> to a true value switches to another implementation that
226	uses L<AnyEvent> in the child and allows multiple concurrent RPC calls.	378	uses L<AnyEvent> in the child and allows multiple concurrent RPC calls (it
		379	does not support recursion in the event loop however, blocking condvar
		380	calls will fail).
227		381
228	The actual API in the child is documented in the section that describes	382	The actual API in the child is documented in the section that describes
229	the calling semantics of the returned C<$rpc> function.	383	the calling semantics of the returned C<$rpc> function.
230		384
231	If you want to pre-load the actual back-end modules to enable memory	385	If you want to pre-load the actual back-end modules to enable memory
232	sharing, then you should load C<AnyEvent::Fork::RPC::Sync> for	386	sharing, then you should load C<AnyEvent::Fork::RPC::Sync> for
233	synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode.	387	synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode.
234		388
235	If you use a template process and want to fork both sync and async	389	If you use a template process and want to fork both sync and async
236	children, then it is permissible to laod both modules.	390	children, then it is permissible to load both modules.
237		391
238	=item serialiser => $string (default: '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })')	392	=item serialiser => $string (default: $AnyEvent::Fork::RPC::STRING_SERIALISER)
239		393
240	All arguments, result data and event data have to be serialised to be	394	All arguments, result data and event data have to be serialised to be
241	transferred between the processes. For this, they have to be frozen and	395	transferred between the processes. For this, they have to be frozen and
242	thawed in both parent and child processes.	396	thawed in both parent and child processes.
243		397
244	By default, only octet strings can be passed between the processes, which	398	By default, only octet strings can be passed between the processes, which
245	is reasonably fast and efficient.	399	is reasonably fast and efficient and requires no extra modules.
246		400
247	For more complicated use cases, you can provide your own freeze and thaw	401	For more complicated use cases, you can provide your own freeze and thaw
248	functions, by specifying a string with perl source code. It's supposed to	402	functions, by specifying a string with perl source code. It's supposed to
249	return two code references when evaluated: the first receives a list of	403	return two code references when evaluated: the first receives a list of
250	perl values and must return an octet string. The second receives the octet	404	perl values and must return an octet string. The second receives the octet
…		…
252		406
253	If you need an external module for serialisation, then you can either	407	If you need an external module for serialisation, then you can either
254	pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use>	408	pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use>
255	or C<require> statement into the serialiser string. Or both.	409	or C<require> statement into the serialiser string. Or both.
256		410
		411	Here are some examples - some of them are also available as global
		412	variables that make them easier to use.
		413
		414	=over 4
		415
		416	=item octet strings - C<$AnyEvent::Fork::RPC::STRING_SERIALISER>
		417
		418	This serialiser concatenates length-prefixes octet strings, and is the
		419	default.
		420
		421	Implementation:
		422
		423	(
		424	sub { pack "(w/a)", @_ },
		425	sub { unpack "(w/a)", shift }
		426	)
		427
		428	=item json - C<$AnyEvent::Fork::RPC::JSON_SERIALISER>
		429
		430	This serialiser creates JSON arrays - you have to make sure the L<JSON>
		431	module is installed for this serialiser to work. It can be beneficial for
		432	sharing when you preload the L<JSON> module in a template process.
		433
		434	L<JSON> (with L<JSON::XS> installed) is slower than the octet string
		435	serialiser, but usually much faster than L<Storable>, unless big chunks of
		436	binary data need to be transferred.
		437
		438	Implementation:
		439
		440	use JSON ();
		441	(
		442	sub { JSON::encode_json \@_ },
		443	sub { @{ JSON::decode_json shift } }
		444	)
		445
		446	=item storable - C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER>
		447
		448	This serialiser uses L<Storable>, which means it has high chance of
		449	serialising just about anything you throw at it, at the cost of having
		450	very high overhead per operation. It also comes with perl.
		451
		452	Implementation:
		453
		454	use Storable ();
		455	(
		456	sub { Storable::freeze \@_ },
		457	sub { @{ Storable::thaw shift } }
		458	)
		459
		460	=item portable storable - C<$AnyEvent::Fork::RPC::NSTORABLE_SERIALISER>
		461
		462	This serialiser also uses L<Storable>, but uses it's "network" format
		463	to serialise data, which makes it possible to talk to incompatible
		464	perl versions (for example, when talking to a process created with
		465	L<AnyEvent::Fork::Remote>).
		466
		467	Implementation:
		468
		469	use Storable ();
		470	(
		471	sub { Storable::nfreeze \@_ },
		472	sub { @{ Storable::thaw shift } }
		473	)
		474
257	=back	475	=back
258		476
		477	=back
		478
		479	See the examples section earlier in this document for some actual
		480	examples.
		481
259	=cut	482	=cut
260		483
261	our $STRING_SERIALISER = '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })';	484	our $STRING_SERIALISER = '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })';
		485	our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })';
		486	our $STORABLE_SERIALISER = 'use Storable (); (sub { Storable::freeze \@_ }, sub { @{ Storable::thaw shift } })';
		487	our $NSTORABLE_SERIALISER = 'use Storable (); (sub { Storable::nfreeze \@_ }, sub { @{ Storable::thaw shift } })';
262		488
263	sub run {	489	sub run {
264	my ($self, $function, %arg) = @_;	490	my ($self, $function, %arg) = @_;
265		491
266	my $serialiser = delete $arg{serialiser} \|\| $STRING_SERIALISER;	492	my $serialiser = delete $arg{serialiser} \|\| $STRING_SERIALISER;
…		…
276	# default for on_event is to raise an error	502	# default for on_event is to raise an error
277	$on_event \|\|= sub { $on_error->("event received, but no on_event handler") };	503	$on_event \|\|= sub { $on_error->("event received, but no on_event handler") };
278		504
279	my ($f, $t) = eval $serialiser; die $@ if $@;	505	my ($f, $t) = eval $serialiser; die $@ if $@;
280		506
281	my (@rcb, $fh, $shutdown, $wbuf, $ww, $rw);	507	my (@rcb, %rcb, $fh, $shutdown, $wbuf, $ww);
282	my ($rlen, $rbuf) = 512 - 16;	508	my ($rlen, $rbuf, $rw) = 512 - 16;
283		509
284	my $wcb = sub {	510	my $wcb = sub {
285	my $len = syswrite $fh, $wbuf;	511	my $len = syswrite $fh, $wbuf;
286		512
287	if (!defined $len) {	513	unless (defined $len) {
288	if ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {	514	if ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {
289	undef $rw; undef $ww; # it ends here	515	undef $rw; undef $ww; # it ends here
290	$on_error->("$!");	516	$on_error->("$!");
291	}	517	}
292	}	518	}
…		…
303		529
304	$self->require ($module)	530	$self->require ($module)
305	->send_arg ($function, $arg{init}, $serialiser)	531	->send_arg ($function, $arg{init}, $serialiser)
306	->run ("$module\::run", sub {	532	->run ("$module\::run", sub {
307	$fh = shift;	533	$fh = shift;
		534
		535	my ($id, $len);
308	$rw = AE::io $fh, 0, sub {	536	$rw = AE::io $fh, 0, sub {
309	$rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf;	537	$rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf;
310	my $len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf;	538	$len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf;
311		539
312	if ($len) {	540	if ($len) {
313	while (5 <= length $rbuf) {	541	while (8 <= length $rbuf) {
314	$len = unpack "L", $rbuf;	542	($id, $len) = unpack "NN", $rbuf;
315	4 + $len <= length $rbuf	543	8 + $len <= length $rbuf
316	or last;	544	or last;
317		545
318	my @r = $t->(substr $rbuf, 4, $len);	546	my @r = $t->(substr $rbuf, 8, $len);
319	substr $rbuf, 0, $len + 4, "";	547	substr $rbuf, 0, 8 + $len, "";
		548
		549	if ($id) {
		550	if (@rcb) {
		551	(shift @rcb)->(@r);
		552	} elsif (my $cb = delete $rcb{$id}) {
		553	$cb->(@r);
		554	} else {
		555	undef $rw; undef $ww;
		556	$on_error->("unexpected data from child");
320		557	}
321	if (pop @r) {	558	} else {
322	$on_event->(@r);	559	$on_event->(@r);
323	} elsif (@rcb) {
324	(shift @rcb)->(@r);
325	} else {
326	undef $rw; undef $ww;
327	$on_error->("unexpected data from child");
328	}	560	}
329	}	561	}
330	} elsif (defined $len) {	562	} elsif (defined $len) {
331	undef $rw; undef $ww; # it ends here	563	undef $rw; undef $ww; # it ends here
332		564
333	if (@rcb) {	565	if (@rcb \|\| %rcb) {
334	$on_error->("unexpected eof");	566	$on_error->("unexpected eof");
335	} else {	567	} else {
336	$on_destroy->();	568	$on_destroy->()
		569	if $on_destroy;
337	}	570	}
338	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {	571	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {
339	undef $rw; undef $ww; # it ends here	572	undef $rw; undef $ww; # it ends here
340	$on_error->("read: $!");	573	$on_error->("read: $!");
341	}	574	}
…		…
344	$ww \|\|= AE::io $fh, 1, $wcb;	577	$ww \|\|= AE::io $fh, 1, $wcb;
345	});	578	});
346		579
347	my $guard = Guard::guard {	580	my $guard = Guard::guard {
348	$shutdown = 1;	581	$shutdown = 1;
349	$ww \|\|= $fh && AE::io $fh, 1, $wcb;	582
		583	shutdown $fh, 1 if $fh && !$ww;
350	};	584	};
351		585
		586	my $id;
		587
		588	$arg{async}
352	sub {	589	? sub {
353	push @rcb, pop;	590	$id = ($id == 0xffffffff ? 0 : $id) + 1;
		591	$id = ($id == 0xffffffff ? 0 : $id) + 1 while exists $rcb{$id}; # rarely loops
354		592
		593	$rcb{$id} = pop;
		594
355	$guard; # keep it alive	595	$guard if 0; # keep it alive
356		596
357	$wbuf .= pack "L/a*", &$f;	597	$wbuf .= pack "NN/a*", $id, &$f;
358	$ww \|\|= $fh && AE::io $fh, 1, $wcb;	598	$ww \|\|= $fh && AE::io $fh, 1, $wcb;
359	}	599	}
		600	: sub {
		601	push @rcb, pop;
		602
		603	$guard; # keep it alive
		604
		605	$wbuf .= pack "N/a*", &$f;
		606	$ww \|\|= $fh && AE::io $fh, 1, $wcb;
		607	}
360	}	608	}
361		609
362	=item $rpc->(..., $cb->(...))	610	=item $rpc->(..., $cb->(...))
363		611
364	The RPC object returned by C<AnyEvent::Fork::RPC::run> is actually a code	612	The RPC object returned by C<AnyEvent::Fork::RPC::run> is actually a code
…		…
379		627
380	The other thing that can be done with the RPC object is to destroy it. In	628	The other thing that can be done with the RPC object is to destroy it. In
381	this case, the child process will execute all remaining RPC calls, report	629	this case, the child process will execute all remaining RPC calls, report
382	their results, and then exit.	630	their results, and then exit.
383		631
		632	See the examples section earlier in this document for some actual
		633	examples.
		634
384	=back	635	=back
385		636
386	=head1 CHILD PROCESS USAGE	637	=head1 CHILD PROCESS USAGE
387		638
388	The following function is not available in this module. They are only	639	The following function is not available in this module. They are only
…		…
396		647
397	Send an event to the parent. Events are a bit like RPC calls made by the	648	Send an event to the parent. Events are a bit like RPC calls made by the
398	child process to the parent, except that there is no notion of return	649	child process to the parent, except that there is no notion of return
399	values.	650	values.
400		651
		652	See the examples section earlier in this document for some actual
		653	examples.
		654
401	=back	655	=back
402		656
		657	=head1 ADVANCED TOPICS
		658
		659	=head2 Choosing a backend
		660
		661	So how do you decide which backend to use? Well, that's your problem to
		662	solve, but here are some thoughts on the matter:
		663
		664	=over 4
		665
		666	=item Synchronous
		667
		668	The synchronous backend does not rely on any external modules (well,
		669	except L<common::sense>, which works around a bug in how perl's warning
		670	system works). This keeps the process very small, for example, on my
		671	system, an empty perl interpreter uses 1492kB RSS, which becomes 2020kB
		672	after C<use warnings; use strict> (for people who grew up with C64s around
		673	them this is probably shocking every single time they see it). The worker
		674	process in the first example in this document uses 1792kB.
		675
		676	Since the calls are done synchronously, slow jobs will keep newer jobs
		677	from executing.
		678
		679	The synchronous backend also has no overhead due to running an event loop
		680	- reading requests is therefore very efficient, while writing responses is
		681	less so, as every response results in a write syscall.
		682
		683	If the parent process is busy and a bit slow reading responses, the child
		684	waits instead of processing further requests. This also limits the amount
		685	of memory needed for buffering, as never more than one response has to be
		686	buffered.
		687
		688	The API in the child is simple - you just have to define a function that
		689	does something and returns something.
		690
		691	It's hard to use modules or code that relies on an event loop, as the
		692	child cannot execute anything while it waits for more input.
		693
		694	=item Asynchronous
		695
		696	The asynchronous backend relies on L<AnyEvent>, which tries to be small,
		697	but still comes at a price: On my system, the worker from example 1a uses
		698	3420kB RSS (for L<AnyEvent>, which loads L<EV>, which needs L<XSLoader>
		699	which in turn loads a lot of other modules such as L<warnings>, L<strict>,
		700	L<vars>, L<Exporter>...).
		701
		702	It batches requests and responses reasonably efficiently, doing only as
		703	few reads and writes as needed, but needs to poll for events via the event
		704	loop.
		705
		706	Responses are queued when the parent process is busy. This means the child
		707	can continue to execute any queued requests. It also means that a child
		708	might queue a lot of responses in memory when it generates them and the
		709	parent process is slow accepting them.
		710
		711	The API is not a straightforward RPC pattern - you have to call a
		712	"done" callback to pass return values and signal completion. Also, more
		713	importantly, the API starts jobs as fast as possible - when 1000 jobs
		714	are queued and the jobs are slow, they will all run concurrently. The
		715	child must implement some queueing/limiting mechanism if this causes
		716	problems. Alternatively, the parent could limit the amount of rpc calls
		717	that are outstanding.
		718
		719	Blocking use of condvars is not supported.
		720
		721	Using event-based modules such as L<IO::AIO>, L<Gtk2>, L<Tk> and so on is
		722	easy.
		723
		724	=back
		725
		726	=head2 Passing file descriptors
		727
		728	Unlike L<AnyEvent::Fork>, this module has no in-built file handle or file
		729	descriptor passing abilities.
		730
		731	The reason is that passing file descriptors is extraordinary tricky
		732	business, and conflicts with efficient batching of messages.
		733
		734	There still is a method you can use: Create a
		735	C<AnyEvent::Util::portable_socketpair> and C<send_fh> one half of it to
		736	the process before you pass control to C<AnyEvent::Fork::RPC::run>.
		737
		738	Whenever you want to pass a file descriptor, send an rpc request to the
		739	child process (so it expects the descriptor), then send it over the other
		740	half of the socketpair. The child should fetch the descriptor from the
		741	half it has passed earlier.
		742
		743	Here is some (untested) pseudocode to that effect:
		744
		745	use AnyEvent::Util;
		746	use AnyEvent::Fork;
		747	use AnyEvent::Fork::RPC;
		748	use IO::FDPass;
		749
		750	my ($s1, $s2) = AnyEvent::Util::portable_socketpair;
		751
		752	my $rpc = AnyEvent::Fork
		753	->new
		754	->send_fh ($s2)
		755	->require ("MyWorker")
		756	->AnyEvent::Fork::RPC::run ("MyWorker::run"
		757	init => "MyWorker::init",
		758	);
		759
		760	undef $s2; # no need to keep it around
		761
		762	# pass an fd
		763	$rpc->("i'll send some fd now, please expect it!", my $cv = AE::cv);
		764
		765	IO::FDPass fileno $s1, fileno $handle_to_pass;
		766
		767	$cv->recv;
		768
		769	The MyWorker module could look like this:
		770
		771	package MyWorker;
		772
		773	use IO::FDPass;
		774
		775	my $s2;
		776
		777	sub init {
		778	$s2 = $_[0];
		779	}
		780
		781	sub run {
		782	if ($_[0] eq "i'll send some fd now, please expect it!") {
		783	my $fd = IO::FDPass::recv fileno $s2;
		784	...
		785	}
		786	}
		787
		788	Of course, this might be blocking if you pass a lot of file descriptors,
		789	so you might want to look into L<AnyEvent::FDpasser> which can handle the
		790	gory details.
		791
		792	=head1 EXCEPTIONS
		793
		794	There are no provisions whatsoever for catching exceptions at this time -
		795	in the child, exeptions might kill the process, causing calls to be lost
		796	and the parent encountering a fatal error. In the parent, exceptions in
		797	the result callback will not be caught and cause undefined behaviour.
		798
403	=head1 SEE ALSO	799	=head1 SEE ALSO
404		800
405	L<AnyEvent::Fork> (to create the processes in the first place),	801	L<AnyEvent::Fork>, to create the processes in the first place.
		802
		803	L<AnyEvent::Fork::Remote>, likewise, but helpful for remote processes.
		804
406	L<AnyEvent::Fork::Pool> (to manage whole pools of processes).	805	L<AnyEvent::Fork::Pool>, to manage whole pools of processes.
407		806
408	=head1 AUTHOR AND CONTACT INFORMATION	807	=head1 AUTHOR AND CONTACT INFORMATION
409		808
410	Marc Lehmann <schmorp@schmorp.de>	809	Marc Lehmann <schmorp@schmorp.de>
411	http://software.schmorp.de/pkg/AnyEvent-Fork-RPC	810	http://software.schmorp.de/pkg/AnyEvent-Fork-RPC

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents): Revision 1.5 by root, Wed Apr 17 19:39:54 2013 UTC vs. Revision 1.28 by root, Sun Apr 28 15:48:31 2013 UTC

Diff Legend

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.5 by root, Wed Apr 17 19:39:54 2013 UTC vs.
Revision 1.28 by root, Sun Apr 28 15:48:31 2013 UTC