[ViewVC] Diff of: cvs/AnyEvent-Fork-RPC/RPC.pm

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.2 by root, Wed Apr 17 17:08:16 2013 UTC vs.
Revision 1.16 by root, Thu Apr 18 14:07:15 2013 UTC

…		…
2		2
3	AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork	3	AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork
4		4
5	=head1 SYNOPSIS	5	=head1 SYNOPSIS
6		6
7	use AnyEvent::Fork;
8	use AnyEvent::Fork::RPC;	7	use AnyEvent::Fork::RPC;
		8	# use AnyEvent::Fork is not needed
9		9
10	my $rpc = AnyEvent::Fork	10	my $rpc = AnyEvent::Fork
11	->new	11	->new
12	->require ("MyModule")	12	->require ("MyModule")
13	->AnyEvent::Fork::RPC::run (	13	->AnyEvent::Fork::RPC::run (
14	"MyModule::server",	14	"MyModule::server",
15	);	15	);
16		16
		17	use AnyEvent;
		18
17	my $cv = AE::cv;	19	my $cv = AE::cv;
18		20
19	$rpc->(1, 2, 3, sub {	21	$rpc->(1, 2, 3, sub {
20	print "MyModule::server returned @_\n";	22	print "MyModule::server returned @_\n";
21	$cv->send;	23	$cv->send;
…		…
34	concurrently in the child, using AnyEvent.	36	concurrently in the child, using AnyEvent.
35		37
36	It also implements an asynchronous event mechanism from the child to the	38	It also implements an asynchronous event mechanism from the child to the
37	parent, that could be used for progress indications or other information.	39	parent, that could be used for progress indications or other information.
38		40
		41	Loading this module also always loads L<AnyEvent::Fork>, so you can make a
		42	separate C<use AnyEvent::Fork> if you wish, but you don't have to.
		43
		44	=head1 EXAMPLES
		45
		46	=head2 Example 1: Synchronous Backend
		47
		48	Here is a simple example that implements a backend that executes C<unlink>
		49	and C<rmdir> calls, and reports their status back. It also reports the
		50	number of requests it has processed every three requests, which is clearly
		51	silly, but illustrates the use of events.
		52
		53	First the parent process:
		54
		55	use AnyEvent;
		56	use AnyEvent::Fork::RPC;
		57
		58	my $done = AE::cv;
		59
		60	my $rpc = AnyEvent::Fork
		61	->new
		62	->require ("MyWorker")
		63	->AnyEvent::Fork::RPC::run ("MyWorker::run",
		64	on_error => sub { warn "FATAL: $_[0]"; exit 1 },
		65	on_event => sub { warn "$_[0] requests handled\n" },
		66	on_destroy => $done,
		67	);
		68
		69	for my $id (1..6) {
		70	$rpc->(rmdir => "/tmp/somepath/$id", sub {
		71	$_[0]
		72	or warn "/tmp/somepath/$id: $_[1]\n";
		73	});
		74	}
		75
		76	undef $rpc;
		77
		78	$done->recv;
		79
		80	The parent creates the process, queues a few rmdir's. It then forgets
		81	about the C<$rpc> object, so that the child exits after it has handled the
		82	requests, and then it waits till the requests have been handled.
		83
		84	The child is implemented using a separate module, C<MyWorker>, shown here:
		85
		86	package MyWorker;
		87
		88	my $count;
		89
		90	sub run {
		91	my ($cmd, $path) = @_;
		92
		93	AnyEvent::Fork::RPC::event ($count)
		94	unless ++$count % 3;
		95
		96	my $status = $cmd eq "rmdir" ? rmdir $path
		97	: $cmd eq "unlink" ? unlink $path
		98	: die "fatal error, illegal command '$cmd'";
		99
		100	$status or (0, "$!")
		101	}
		102
		103	1
		104
		105	The C<run> function first sends a "progress" event every three calls, and
		106	then executes C<rmdir> or C<unlink>, depending on the first parameter (or
		107	dies with a fatal error - obviously, you must never let this happen :).
		108
		109	Eventually it returns the status value true if the command was successful,
		110	or the status value 0 and the stringified error message.
		111
		112	On my system, running the first code fragment with the given
		113	F<MyWorker.pm> in the current directory yields:
		114
		115	/tmp/somepath/1: No such file or directory
		116	/tmp/somepath/2: No such file or directory
		117	3 requests handled
		118	/tmp/somepath/3: No such file or directory
		119	/tmp/somepath/4: No such file or directory
		120	/tmp/somepath/5: No such file or directory
		121	6 requests handled
		122	/tmp/somepath/6: No such file or directory
		123
		124	Obviously, none of the directories I am trying to delete even exist. Also,
		125	the events and responses are processed in exactly the same order as
		126	they were created in the child, which is true for both synchronous and
		127	asynchronous backends.
		128
		129	Note that the parentheses in the call to C<AnyEvent::Fork::RPC::event> are
		130	not optional. That is because the function isn't defined when the code is
		131	compiled. You can make sure it is visible by pre-loading the correct
		132	backend module in the call to C<require>:
		133
		134	->require ("AnyEvent::Fork::RPC::Sync", "MyWorker")
		135
		136	Since the backend module declares the C<event> function, loading it first
		137	ensures that perl will correctly interpret calls to it.
		138
		139	And as a final remark, there is a fine module on CPAN that can
		140	asynchronously C<rmdir> and C<unlink> and a lot more, and more efficiently
		141	than this example, namely L<IO::AIO>.
		142
		143	=head3 Example 1a: the same with the asynchronous backend
		144
		145	This example only shows what needs to be changed to use the async backend
		146	instead. Doing this is not very useful, the purpose of this example is
		147	to show the minimum amount of change that is required to go from the
		148	synchronous to the asynchronous backend.
		149
		150	To use the async backend in the previous example, you need to add the
		151	C<async> parameter to the C<AnyEvent::Fork::RPC::run> call:
		152
		153	->AnyEvent::Fork::RPC::run ("MyWorker::run",
		154	async => 1,
		155	...
		156
		157	And since the function call protocol is now changed, you need to adopt
		158	C<MyWorker::run> to the async API.
		159
		160	First, you need to accept the extra initial C<$done> callback:
		161
		162	sub run {
		163	my ($done, $cmd, $path) = @_;
		164
		165	And since a response is now generated when C<$done> is called, as opposed
		166	to when the function returns, we need to call the C<$done> function with
		167	the status:
		168
		169	$done->($status or (0, "$!"));
		170
		171	A few remarks are in order. First, it's quite pointless to use the async
		172	backend for this example - but it I<is> possible. Second, you can call
		173	C<$done> before or after returning from the function. Third, having both
		174	returned from the function and having called the C<$done> callback, the
		175	child process may exit at any time, so you should call C<$done> only when
		176	you really I<are> done.
		177
		178	=head2 Example 2: Asynchronous Backend
		179
		180	This example implements multiple count-downs in the child, using
		181	L<AnyEvent> timers. While this is a bit silly (one could use timers in te
		182	parent just as well), it illustrates the ability to use AnyEvent in the
		183	child and the fact that responses can arrive in a different order then the
		184	requests.
		185
		186	It also shows how to embed the actual child code into a C<__DATA__>
		187	section, so it doesn't need any external files at all.
		188
		189	And when your parent process is often busy, and you have stricter timing
		190	requirements, then running timers in a child process suddenly doesn't look
		191	so silly anymore.
		192
		193	Without further ado, here is the code:
		194
		195	use AnyEvent;
		196	use AnyEvent::Fork::RPC;
		197
		198	my $done = AE::cv;
		199
		200	my $rpc = AnyEvent::Fork
		201	->new
		202	->require ("AnyEvent::Fork::RPC::Async")
		203	->eval (do { local $/; <DATA> })
		204	->AnyEvent::Fork::RPC::run ("run",
		205	async => 1,
		206	on_error => sub { warn "FATAL: $_[0]"; exit 1 },
		207	on_event => sub { print $_[0] },
		208	on_destroy => $done,
		209	);
		210
		211	for my $count (3, 2, 1) {
		212	$rpc->($count, sub {
		213	warn "job $count finished\n";
		214	});
		215	}
		216
		217	undef $rpc;
		218
		219	$done->recv;
		220
		221	__DATA__
		222
		223	# this ends up in main, as we don't use a package declaration
		224
		225	use AnyEvent;
		226
		227	sub run {
		228	my ($done, $count) = @_;
		229
		230	my $n;
		231
		232	AnyEvent::Fork::RPC::event "starting to count up to $count\n";
		233
		234	my $w; $w = AE::timer 1, 1, sub {
		235	++$n;
		236
		237	AnyEvent::Fork::RPC::event "count $n of $count\n";
		238
		239	if ($n == $count) {
		240	undef $w;
		241	$done->();
		242	}
		243	};
		244	}
		245
		246	The parent part (the one before the C<__DATA__> section) isn't very
		247	different from the earlier examples. It sets async mode, preloads
		248	the backend module (so the C<AnyEvent::Fork::RPC::event> function is
		249	declared), uses a slightly different C<on_event> handler (which we use
		250	simply for logging purposes) and then, instead of loading a module with
		251	the actual worker code, it C<eval>'s the code from the data section in the
		252	child process.
		253
		254	It then starts three countdowns, from 3 to 1 seconds downwards, destroys
		255	the rpc object so the example finishes eventually, and then just waits for
		256	the stuff to trickle in.
		257
		258	The worker code uses the event function to log some progress messages, but
		259	mostly just creates a recurring one-second timer.
		260
		261	The timer callback increments a counter, logs a message, and eventually,
		262	when the count has been reached, calls the finish callback.
		263
		264	On my system, this results in the following output. Since all timers fire
		265	at roughly the same time, the actual order isn't guaranteed, but the order
		266	shown is very likely what you would get, too.
		267
		268	starting to count up to 3
		269	starting to count up to 2
		270	starting to count up to 1
		271	count 1 of 3
		272	count 1 of 2
		273	count 1 of 1
		274	job 1 finished
		275	count 2 of 2
		276	job 2 finished
		277	count 2 of 3
		278	count 3 of 3
		279	job 3 finished
		280
		281	While the overall ordering isn't guaranteed, the async backend still
		282	guarantees that events and responses are delivered to the parent process
		283	in the exact same ordering as they were generated in the child process.
		284
		285	And unless your system is I<very> busy, it should clearly show that the
		286	job started last will finish first, as it has the lowest count.
		287
		288	This concludes the async example. Since L<AnyEvent::Fork> does not
		289	actually fork, you are free to use about any module in the child, not just
		290	L<AnyEvent>, but also L<IO::AIO>, or L<Tk> for example.
		291
39	=head1 PARENT PROCESS USAGE	292	=head1 PARENT PROCESS USAGE
40		293
41	This module exports nothing, and only implements a single function:	294	This module exports nothing, and only implements a single function:
42		295
43	=over 4	296	=over 4
…		…
50		303
51	use Errno ();	304	use Errno ();
52	use Guard ();	305	use Guard ();
53		306
54	use AnyEvent;	307	use AnyEvent;
55	#use AnyEvent::Fork;	308	use AnyEvent::Fork; # we don't actually depend on it, this is for convenience
56		309
57	our $VERSION = 0.1;	310	our $VERSION = 0.1;
58		311
59	=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]	312	=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]
60		313
…		…
92	Called for every call to the C<AnyEvent::Fork::RPC::event> function in the	345	Called for every call to the C<AnyEvent::Fork::RPC::event> function in the
93	child, with the arguments of that function passed to the callback.	346	child, with the arguments of that function passed to the callback.
94		347
95	Also called on errors when no C<on_error> handler is provided.	348	Also called on errors when no C<on_error> handler is provided.
96		349
		350	=item on_destroy => $cb->()
		351
		352	Called when the C<$rpc> object has been destroyed and all requests have
		353	been successfully handled. This is useful when you queue some requests and
		354	want the child to go away after it has handled them. The problem is that
		355	the parent must not exit either until all requests have been handled, and
		356	this can be accomplished by waiting for this callback.
		357
97	=item init => $function (default none)	358	=item init => $function (default none)
98		359
99	When specified (by name), this function is called in the child as the very	360	When specified (by name), this function is called in the child as the very
100	first thing when taking over the process, with all the arguments normally	361	first thing when taking over the process, with all the arguments normally
101	passed to the C<AnyEvent::Fork::run> function, except the communications	362	passed to the C<AnyEvent::Fork::run> function, except the communications
102	socket.	363	socket.
103		364
104	It can be used to do one-time things in the child such as storing passed	365	It can be used to do one-time things in the child such as storing passed
105	parameters or opening database connections.	366	parameters or opening database connections.
106		367
		368	It is called very early - before the serialisers are created or the
		369	C<$function> name is resolved into a function reference, so it could be
		370	used to load any modules that provide the serialiser or function. It can
		371	not, however, create events.
		372
107	=item async => $boolean (default: 0)	373	=item async => $boolean (default: 0)
108		374
109	The default server used in the child does all I/O blockingly, and only	375	The default server used in the child does all I/O blockingly, and only
110	allows a single RPC call to execute concurrently.	376	allows a single RPC call to execute concurrently.
111		377
112	Setting C<async> to a true value switches to another implementation that	378	Setting C<async> to a true value switches to another implementation that
113	uses L<AnyEvent> in the child and allows multiple concurrent RPC calls.	379	uses L<AnyEvent> in the child and allows multiple concurrent RPC calls (it
		380	does not support recursion in the event loop however, blocking condvar
		381	calls will fail).
114		382
115	The actual API in the child is documented in the section that describes	383	The actual API in the child is documented in the section that describes
116	the calling semantics of the returned C<$rpc> function.	384	the calling semantics of the returned C<$rpc> function.
117		385
118	If you want to pre-load the actual back-end modules to enable memory	386	If you want to pre-load the actual back-end modules to enable memory
119	sharing, then you should load C<AnyEvent::Fork::RPC::Sync> for	387	sharing, then you should load C<AnyEvent::Fork::RPC::Sync> for
120	synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode.	388	synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode.
121		389
122	=item serialiser => $string (default: '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })')	390	If you use a template process and want to fork both sync and async
		391	children, then it is permissible to load both modules.
		392
		393	=item serialiser => $string (default: $AnyEvent::Fork::RPC::STRING_SERIALISER)
123		394
124	All arguments, result data and event data have to be serialised to be	395	All arguments, result data and event data have to be serialised to be
125	transferred between the processes. For this, they have to be frozen and	396	transferred between the processes. For this, they have to be frozen and
126	thawed in both parent and child processes.	397	thawed in both parent and child processes.
127		398
128	By default, only octet strings can be passed between the processes, which	399	By default, only octet strings can be passed between the processes, which
129	is reasonably fast and efficient.	400	is reasonably fast and efficient and requires no extra modules.
130		401
131	For more complicated use cases, you can provide your own freeze and thaw	402	For more complicated use cases, you can provide your own freeze and thaw
132	functions, by specifying a string with perl source code. It's supposed to	403	functions, by specifying a string with perl source code. It's supposed to
133	return two code references when evaluated: the first receives a list of	404	return two code references when evaluated: the first receives a list of
134	perl values and must return an octet string. The second receives the octet	405	perl values and must return an octet string. The second receives the octet
…		…
136		407
137	If you need an external module for serialisation, then you can either	408	If you need an external module for serialisation, then you can either
138	pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use>	409	pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use>
139	or C<require> statement into the serialiser string. Or both.	410	or C<require> statement into the serialiser string. Or both.
140		411
		412	Here are some examples - some of them are also available as global
		413	variables that make them easier to use.
		414
		415	=over 4
		416
		417	=item octet strings - C<$AnyEvent::Fork::RPC::STRING_SERIALISER>
		418
		419	This serialiser concatenates length-prefixes octet strings, and is the
		420	default.
		421
		422	Implementation:
		423
		424	(
		425	sub { pack "(w/a)", @_ },
		426	sub { unpack "(w/a)", shift }
		427	)
		428
		429	=item json - C<$AnyEvent::Fork::RPC::JSON_SERIALISER>
		430
		431	This serialiser creates JSON arrays - you have to make sure the L<JSON>
		432	module is installed for this serialiser to work. It can be beneficial for
		433	sharing when you preload the L<JSON> module in a template process.
		434
		435	L<JSON> (with L<JSON::XS> installed) is slower than the octet string
		436	serialiser, but usually much faster than L<Storable>, unless big chunks of
		437	binary data need to be transferred.
		438
		439	Implementation:
		440
		441	use JSON ();
		442	(
		443	sub { JSON::encode_json \@_ },
		444	sub { @{ JSON::decode_json shift } }
		445	)
		446
		447	=item storable - C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER>
		448
		449	This serialiser uses L<Storable>, which means it has high chance of
		450	serialising just about anything you throw at it, at the cost of having
		451	very high overhead per operation. It also comes with perl.
		452
		453	Implementation:
		454
		455	use Storable ();
		456	(
		457	sub { Storable::freeze \@_ },
		458	sub { @{ Storable::thaw shift } }
		459	)
		460
141	=back	461	=back
142		462
		463	=back
		464
		465	See the examples section earlier in this document for some actual
		466	examples.
		467
143	=cut	468	=cut
144		469
145	our $STRING_SERIALISER = '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })';	470	our $STRING_SERIALISER = '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })';
146		471	our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })';
147	# ideally, we want (SvLEN - SvCUR) \|\| 1024 or somesuch...	472	our $STORABLE_SERIALISER = 'use Storable (); (sub { Storable::freeze \@_ }, sub { @{ Storable::thaw shift } })';
148	sub rlen($) { ($_[0] < 384 ? 512 + 16 : 2 << int +(log $_[0] + 512) / log 2) - $_[0] - 16 }
149		473
150	sub run {	474	sub run {
151	my ($self, $function, %arg) = @_;	475	my ($self, $function, %arg) = @_;
152		476
153	my $serialiser = delete $arg{serialiser} \|\| $STRING_SERIALISER;	477	my $serialiser = delete $arg{serialiser} \|\| $STRING_SERIALISER;
154	my $on_event = delete $arg{on_event};	478	my $on_event = delete $arg{on_event};
155	my $on_error = delete $arg{on_error};	479	my $on_error = delete $arg{on_error};
		480	my $on_destroy = delete $arg{on_destroy};
156		481
157	# default for on_error is to on_event, if specified	482	# default for on_error is to on_event, if specified
158	$on_error \|\|= $on_event	483	$on_error \|\|= $on_event
159	? sub { $on_event->(error => shift) }	484	? sub { $on_event->(error => shift) }
160	: sub { die "AnyEvent::Fork::RPC: uncaught error: $_[0].\n" };	485	: sub { die "AnyEvent::Fork::RPC: uncaught error: $_[0].\n" };
…		…
162	# default for on_event is to raise an error	487	# default for on_event is to raise an error
163	$on_event \|\|= sub { $on_error->("event received, but no on_event handler") };	488	$on_event \|\|= sub { $on_error->("event received, but no on_event handler") };
164		489
165	my ($f, $t) = eval $serialiser; die $@ if $@;	490	my ($f, $t) = eval $serialiser; die $@ if $@;
166		491
167	my (@rcb, $fh, $shutdown, $wbuf, $ww, $rbuf, $rw);	492	my (@rcb, %rcb, $fh, $shutdown, $wbuf, $ww);
		493	my ($rlen, $rbuf, $rw) = 512 - 16;
168		494
169	my $wcb = sub {	495	my $wcb = sub {
170	my $len = syswrite $fh, $wbuf;	496	my $len = syswrite $fh, $wbuf;
171		497
172	if (!defined $len) {	498	unless (defined $len) {
173	if ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {	499	if ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {
174	undef $rw; undef $ww; # it ends here	500	undef $rw; undef $ww; # it ends here
175	$on_error->("$!");	501	$on_error->("$!");
176	}	502	}
177	}	503	}
…		…
188		514
189	$self->require ($module)	515	$self->require ($module)
190	->send_arg ($function, $arg{init}, $serialiser)	516	->send_arg ($function, $arg{init}, $serialiser)
191	->run ("$module\::run", sub {	517	->run ("$module\::run", sub {
192	$fh = shift;	518	$fh = shift;
		519
		520	my ($id, $len);
193	$rw = AE::io $fh, 0, sub {	521	$rw = AE::io $fh, 0, sub {
		522	$rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf;
194	my $len = sysread $fh, $rbuf, rlen length $rbuf, length $rbuf;	523	$len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf;
195		524
196	if ($len) {	525	if ($len) {
197	while (5 <= length $rbuf) {	526	while (8 <= length $rbuf) {
198	$len = unpack "L", $rbuf;	527	($id, $len) = unpack "LL", $rbuf;
199	4 + $len <= length $rbuf	528	8 + $len <= length $rbuf
200	or last;	529	or last;
201		530
202	my @r = $t->(substr $rbuf, 4, $len);	531	my @r = $t->(substr $rbuf, 8, $len);
203	substr $rbuf, 0, $len + 4, "";	532	substr $rbuf, 0, 8 + $len, "";
		533
		534	if ($id) {
		535	if (@rcb) {
		536	(shift @rcb)->(@r);
		537	} elsif (my $cb = delete $rcb{$id}) {
		538	$cb->(@r);
		539	} else {
		540	undef $rw; undef $ww;
		541	$on_error->("unexpected data from child");
204		542	}
205	if (pop @r) {	543	} else {
206	$on_event->(@r);	544	$on_event->(@r);
207	} elsif (@rcb) {
208	(shift @rcb)->(@r);
209	} else {
210	undef $rw; undef $ww;
211	$on_error->("unexpected data from child");
212	}	545	}
213	}	546	}
214	} elsif (defined $len) {	547	} elsif (defined $len) {
215	undef $rw; undef $ww; # it ends here	548	undef $rw; undef $ww; # it ends here
		549
		550	if (@rcb \|\| %rcb) {
216	$on_error->("unexpected eof")	551	$on_error->("unexpected eof");
217	if @rcb;	552	} else {
		553	$on_destroy->();
		554	}
218	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {	555	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {
219	undef $rw; undef $ww; # it ends here	556	undef $rw; undef $ww; # it ends here
220	$on_error->("read: $!");	557	$on_error->("read: $!");
221	}	558	}
222	};	559	};
…		…
227	my $guard = Guard::guard {	564	my $guard = Guard::guard {
228	$shutdown = 1;	565	$shutdown = 1;
229	$ww \|\|= $fh && AE::io $fh, 1, $wcb;	566	$ww \|\|= $fh && AE::io $fh, 1, $wcb;
230	};	567	};
231		568
		569	my $id;
		570
		571	$arg{async}
232	sub {	572	? sub {
233	push @rcb, pop;	573	$id = ($id == 0xffffffff ? 0 : $id) + 1;
		574	$id = ($id == 0xffffffff ? 0 : $id) + 1 while exists $rcb{$id}; # rarely loops
234		575
		576	$rcb{$id} = pop;
		577
235	$guard; # keep it alive	578	$guard; # keep it alive
236		579
237	$wbuf .= pack "L/a*", &$f;	580	$wbuf .= pack "LL/a*", $id, &$f;
238	$ww \|\|= $fh && AE::io $fh, 1, $wcb;	581	$ww \|\|= $fh && AE::io $fh, 1, $wcb;
239	}	582	}
		583	: sub {
		584	push @rcb, pop;
		585
		586	$guard; # keep it alive
		587
		588	$wbuf .= pack "L/a*", &$f;
		589	$ww \|\|= $fh && AE::io $fh, 1, $wcb;
		590	}
240	}	591	}
241		592
		593	=item $rpc->(..., $cb->(...))
		594
		595	The RPC object returned by C<AnyEvent::Fork::RPC::run> is actually a code
		596	reference. There are two things you can do with it: call it, and let it go
		597	out of scope (let it get destroyed).
		598
		599	If C<async> was false when C<$rpc> was created (the default), then, if you
		600	call C<$rpc>, the C<$function> is invoked with all arguments passed to
		601	C<$rpc> except the last one (the callback). When the function returns, the
		602	callback will be invoked with all the return values.
		603
		604	If C<async> was true, then the C<$function> receives an additional
		605	initial argument, the result callback. In this case, returning from
		606	C<$function> does nothing - the function only counts as "done" when the
		607	result callback is called, and any arguments passed to it are considered
		608	the return values. This makes it possible to "return" from event handlers
		609	or e.g. Coro threads.
		610
		611	The other thing that can be done with the RPC object is to destroy it. In
		612	this case, the child process will execute all remaining RPC calls, report
		613	their results, and then exit.
		614
		615	See the examples section earlier in this document for some actual
		616	examples.
		617
242	=back	618	=back
243		619
244	=head1 CHILD PROCESS USAGE	620	=head1 CHILD PROCESS USAGE
245		621
246	These functions are not available in this module. They are only available	622	The following function is not available in this module. They are only
247	in the namespace of this module when the child is running, without	623	available in the namespace of this module when the child is running,
248	having to load any extra module. They are part of the child-side API of	624	without having to load any extra modules. They are part of the child-side
249	L<AnyEvent::Fork::RPC>.	625	API of L<AnyEvent::Fork::RPC>.
250		626
251	=over 4	627	=over 4
252
253	=item AnyEvent::Fork::RPC::quit
254
255	This function can be called to gracefully stop the child process when it
256	is idle.
257
258	After this function is called, the process stops handling incoming RPC
259	requests, but outstanding events and function return values will be sent
260	to the parent. When all data has been sent, the process calls C<exit>.
261
262	Since the parent might not expect the child to exit at random points in
263	time, it is often better to signal the parent by sending an C<event> and
264	letting the parent close down the child process.
265		628
266	=item AnyEvent::Fork::RPC::event ...	629	=item AnyEvent::Fork::RPC::event ...
267		630
268	Send an event to the parent. Events are a bit like RPC calls made by the	631	Send an event to the parent. Events are a bit like RPC calls made by the
269	child process to the parent, except that there is no notion of return	632	child process to the parent, except that there is no notion of return
270	values.	633	values.
271		634
		635	See the examples section earlier in this document for some actual
		636	examples.
		637
272	=back	638	=back
273		639
		640	=head1 ADVANCED TOPICS
		641
		642	=head2 Choosing a backend
		643
		644	So how do you decide which backend to use? Well, that's your problem to
		645	solve, but here are some thoughts on the matter:
		646
		647	=over 4
		648
		649	=item Synchronous
		650
		651	The synchronous backend does not rely on any external modules (well,
		652	except L<common::sense>, which works around a bug in how perl's warning
		653	system works). This keeps the process very small, for example, on my
		654	system, an empty perl interpreter uses 1492kB RSS, which becomes 2020kB
		655	after C<use warnings; use strict> (for people who grew up with C64s around
		656	them this is probably shocking every single time they see it). The worker
		657	process in the first example in this document uses 1792kB.
		658
		659	Since the calls are done synchronously, slow jobs will keep newer jobs
		660	from executing.
		661
		662	The synchronous backend also has no overhead due to running an event loop
		663	- reading requests is therefore very efficient, while writing responses is
		664	less so, as every response results in a write syscall.
		665
		666	If the parent process is busy and a bit slow reading responses, the child
		667	waits instead of processing further requests. This also limits the amount
		668	of memory needed for buffering, as never more than one response has to be
		669	buffered.
		670
		671	The API in the child is simple - you just have to define a function that
		672	does something and returns something.
		673
		674	It's hard to use modules or code that relies on an event loop, as the
		675	child cannot execute anything while it waits for more input.
		676
		677	=item Asynchronous
		678
		679	The asynchronous backend relies on L<AnyEvent>, which tries to be small,
		680	but still comes at a price: On my system, the worker from example 1a uses
		681	3420kB RSS (for L<AnyEvent>, which loads L<EV>, which needs L<XSLoader>
		682	which in turn loads a lot of other modules such as L<warnings>, L<strict>,
		683	L<vars>, L<Exporter>...).
		684
		685	It batches requests and responses reasonably efficiently, doing only as
		686	few reads and writes as needed, but needs to poll for events via the event
		687	loop.
		688
		689	Responses are queued when the parent process is busy. This means the child
		690	can continue to execute any queued requests. It also means that a child
		691	might queue a lot of responses in memory when it generates them and the
		692	parent process is slow accepting them.
		693
		694	The API is not a straightforward RPC pattern - you have to call a
		695	"done" callback to pass return values and signal completion. Also, more
		696	importantly, the API starts jobs as fast as possible - when 1000 jobs
		697	are queued and the jobs are slow, they will all run concurrently. The
		698	child must implement some queueing/limiting mechanism if this causes
		699	problems. Alternatively, the parent could limit the amount of rpc calls
		700	that are outstanding.
		701
		702	Blocking use of condvars is not supported.
		703
		704	Using event-based modules such as L<IO::AIO>, L<Gtk2>, L<Tk> and so on is
		705	easy.
		706
		707	=back
		708
		709	=head2 Passing file descriptors
		710
		711	Unlike L<AnyEvent::Fork>, this module has no in-built file handle or file
		712	descriptor passing abilities.
		713
		714	The reason is that passing file descriptors is extraordinary tricky
		715	business, and conflicts with efficient batching of messages.
		716
		717	There still is a method you can use: Create a
		718	C<AnyEvent::Util::portable_socketpair> and C<send_fh> one half of it to
		719	the process before you pass control to C<AnyEvent::Fork::RPC::run>.
		720
		721	Whenever you want to pass a file descriptor, send an rpc request to the
		722	child process (so it expects the descriptor), then send it over the other
		723	half of the socketpair. The child should fetch the descriptor from the
		724	half it has passed earlier.
		725
		726	Here is some (untested) pseudocode to that effect:
		727
		728	use AnyEvent::Util;
		729	use AnyEvent::Fork::RPC;
		730	use IO::FDPass;
		731
		732	my ($s1, $s2) = AnyEvent::Util::portable_socketpair;
		733
		734	my $rpc = AnyEvent::Fork
		735	->new
		736	->send_fh ($s2)
		737	->require ("MyWorker")
		738	->AnyEvent::Fork::RPC::run ("MyWorker::run"
		739	init => "MyWorker::init",
		740	);
		741
		742	undef $s2; # no need to keep it around
		743
		744	# pass an fd
		745	$rpc->("i'll send some fd now, please expect it!", my $cv = AE::cv);
		746
		747	IO::FDPass fileno $s1, fileno $handle_to_pass;
		748
		749	$cv->recv;
		750
		751	The MyWorker module could look like this:
		752
		753	package MyWorker;
		754
		755	use IO::FDPass;
		756
		757	my $s2;
		758
		759	sub init {
		760	$s2 = $_[0];
		761	}
		762
		763	sub run {
		764	if ($_[0] eq "i'll send some fd now, please expect it!") {
		765	my $fd = IO::FDPass::recv fileno $s2;
		766	...
		767	}
		768	}
		769
		770	Of course, this might be blocking if you pass a lot of file descriptors,
		771	so you might want to look into L<AnyEvent::FDpasser> which can handle the
		772	gory details.
		773
274	=head1 SEE ALSO	774	=head1 SEE ALSO
275		775
276	L<AnyEvent::Fork> (to create the processes in the first place),	776	L<AnyEvent::Fork>, to create the processes in the first place.
		777
277	L<AnyEvent::Fork::Pool> (to manage whole pools of processes).	778	L<AnyEvent::Fork::Pool>, to manage whole pools of processes.
278		779
279	=head1 AUTHOR AND CONTACT INFORMATION	780	=head1 AUTHOR AND CONTACT INFORMATION
280		781
281	Marc Lehmann <schmorp@schmorp.de>	782	Marc Lehmann <schmorp@schmorp.de>
282	http://software.schmorp.de/pkg/AnyEvent-Fork-RPC	783	http://software.schmorp.de/pkg/AnyEvent-Fork-RPC

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents): Revision 1.2 by root, Wed Apr 17 17:08:16 2013 UTC vs. Revision 1.16 by root, Thu Apr 18 14:07:15 2013 UTC

Diff Legend

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.2 by root, Wed Apr 17 17:08:16 2013 UTC vs.
Revision 1.16 by root, Thu Apr 18 14:07:15 2013 UTC