[ViewVC] Diff of: cvs/AnyEvent-Fork-RPC/RPC.pm

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.2 by root, Wed Apr 17 17:08:16 2013 UTC vs.
Revision 1.25 by root, Sun Apr 28 14:17:59 2013 UTC

…		…
1	=head1 NAME	1	=head1 NAME
2		2
3	AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork	3	AnyEvent::Fork::RPC - simple RPC extension for AnyEvent::Fork
4		4
		5	THE API IS NOT FINISHED, CONSIDER THIS A BETA RELEASE
		6
5	=head1 SYNOPSIS	7	=head1 SYNOPSIS
6		8
7	use AnyEvent::Fork;
8	use AnyEvent::Fork::RPC;	9	use AnyEvent::Fork::RPC;
		10	# use AnyEvent::Fork is not needed
9		11
10	my $rpc = AnyEvent::Fork	12	my $rpc = AnyEvent::Fork
11	->new	13	->new
12	->require ("MyModule")	14	->require ("MyModule")
13	->AnyEvent::Fork::RPC::run (	15	->AnyEvent::Fork::RPC::run (
14	"MyModule::server",	16	"MyModule::server",
15	);	17	);
16		18
		19	use AnyEvent;
		20
17	my $cv = AE::cv;	21	my $cv = AE::cv;
18		22
19	$rpc->(1, 2, 3, sub {	23	$rpc->(1, 2, 3, sub {
20	print "MyModule::server returned @_\n";	24	print "MyModule::server returned @_\n";
21	$cv->send;	25	$cv->send;
34	concurrently in the child, using AnyEvent.	38	concurrently in the child, using AnyEvent.
35		39
36	It also implements an asynchronous event mechanism from the child to the	40	It also implements an asynchronous event mechanism from the child to the
37	parent, that could be used for progress indications or other information.	41	parent, that could be used for progress indications or other information.
38		42
		43	Loading this module also always loads L<AnyEvent::Fork>, so you can make a
		44	separate C<use AnyEvent::Fork> if you wish, but you don't have to.
		45
		46	=head1 EXAMPLES
		47
		48	=head2 Example 1: Synchronous Backend
		49
		50	Here is a simple example that implements a backend that executes C<unlink>
		51	and C<rmdir> calls, and reports their status back. It also reports the
		52	number of requests it has processed every three requests, which is clearly
		53	silly, but illustrates the use of events.
		54
		55	First the parent process:
		56
		57	use AnyEvent;
		58	use AnyEvent::Fork::RPC;
		59
		60	my $done = AE::cv;
		61
		62	my $rpc = AnyEvent::Fork
		63	->new
		64	->require ("MyWorker")
		65	->AnyEvent::Fork::RPC::run ("MyWorker::run",
		66	on_error => sub { warn "FATAL: $_[0]"; exit 1 },
		67	on_event => sub { warn "$_[0] requests handled\n" },
		68	on_destroy => $done,
		69	);
		70
		71	for my $id (1..6) {
		72	$rpc->(rmdir => "/tmp/somepath/$id", sub {
		73	$_[0]
		74	or warn "/tmp/somepath/$id: $_[1]\n";
		75	});
		76	}
		77
		78	undef $rpc;
		79
		80	$done->recv;
		81
		82	The parent creates the process, queues a few rmdir's. It then forgets
		83	about the C<$rpc> object, so that the child exits after it has handled the
		84	requests, and then it waits till the requests have been handled.
		85
		86	The child is implemented using a separate module, C<MyWorker>, shown here:
		87
		88	package MyWorker;
		89
		90	my $count;
		91
		92	sub run {
		93	my ($cmd, $path) = @_;
		94
		95	AnyEvent::Fork::RPC::event ($count)
		96	unless ++$count % 3;
		97
		98	my $status = $cmd eq "rmdir" ? rmdir $path
		99	: $cmd eq "unlink" ? unlink $path
		100	: die "fatal error, illegal command '$cmd'";
		101
		102	$status or (0, "$!")
		103	}
		104
		105	1
		106
		107	The C<run> function first sends a "progress" event every three calls, and
		108	then executes C<rmdir> or C<unlink>, depending on the first parameter (or
		109	dies with a fatal error - obviously, you must never let this happen :).
		110
		111	Eventually it returns the status value true if the command was successful,
		112	or the status value 0 and the stringified error message.
		113
		114	On my system, running the first code fragment with the given
		115	F<MyWorker.pm> in the current directory yields:
		116
		117	/tmp/somepath/1: No such file or directory
		118	/tmp/somepath/2: No such file or directory
		119	3 requests handled
		120	/tmp/somepath/3: No such file or directory
		121	/tmp/somepath/4: No such file or directory
		122	/tmp/somepath/5: No such file or directory
		123	6 requests handled
		124	/tmp/somepath/6: No such file or directory
		125
		126	Obviously, none of the directories I am trying to delete even exist. Also,
		127	the events and responses are processed in exactly the same order as
		128	they were created in the child, which is true for both synchronous and
		129	asynchronous backends.
		130
		131	Note that the parentheses in the call to C<AnyEvent::Fork::RPC::event> are
		132	not optional. That is because the function isn't defined when the code is
		133	compiled. You can make sure it is visible by pre-loading the correct
		134	backend module in the call to C<require>:
		135
		136	->require ("AnyEvent::Fork::RPC::Sync", "MyWorker")
		137
		138	Since the backend module declares the C<event> function, loading it first
		139	ensures that perl will correctly interpret calls to it.
		140
		141	And as a final remark, there is a fine module on CPAN that can
		142	asynchronously C<rmdir> and C<unlink> and a lot more, and more efficiently
		143	than this example, namely L<IO::AIO>.
		144
		145	=head3 Example 1a: the same with the asynchronous backend
		146
		147	This example only shows what needs to be changed to use the async backend
		148	instead. Doing this is not very useful, the purpose of this example is
		149	to show the minimum amount of change that is required to go from the
		150	synchronous to the asynchronous backend.
		151
		152	To use the async backend in the previous example, you need to add the
		153	C<async> parameter to the C<AnyEvent::Fork::RPC::run> call:
		154
		155	->AnyEvent::Fork::RPC::run ("MyWorker::run",
		156	async => 1,
		157	...
		158
		159	And since the function call protocol is now changed, you need to adopt
		160	C<MyWorker::run> to the async API.
		161
		162	First, you need to accept the extra initial C<$done> callback:
		163
		164	sub run {
		165	my ($done, $cmd, $path) = @_;
		166
		167	And since a response is now generated when C<$done> is called, as opposed
		168	to when the function returns, we need to call the C<$done> function with
		169	the status:
		170
		171	$done->($status or (0, "$!"));
		172
		173	A few remarks are in order. First, it's quite pointless to use the async
		174	backend for this example - but it I<is> possible. Second, you can call
		175	C<$done> before or after returning from the function. Third, having both
		176	returned from the function and having called the C<$done> callback, the
		177	child process may exit at any time, so you should call C<$done> only when
		178	you really I<are> done.
		179
		180	=head2 Example 2: Asynchronous Backend
		181
		182	This example implements multiple count-downs in the child, using
		183	L<AnyEvent> timers. While this is a bit silly (one could use timers in te
		184	parent just as well), it illustrates the ability to use AnyEvent in the
		185	child and the fact that responses can arrive in a different order then the
		186	requests.
		187
		188	It also shows how to embed the actual child code into a C<__DATA__>
		189	section, so it doesn't need any external files at all.
		190
		191	And when your parent process is often busy, and you have stricter timing
		192	requirements, then running timers in a child process suddenly doesn't look
		193	so silly anymore.
		194
		195	Without further ado, here is the code:
		196
		197	use AnyEvent;
		198	use AnyEvent::Fork::RPC;
		199
		200	my $done = AE::cv;
		201
		202	my $rpc = AnyEvent::Fork
		203	->new
		204	->require ("AnyEvent::Fork::RPC::Async")
		205	->eval (do { local $/; <DATA> })
		206	->AnyEvent::Fork::RPC::run ("run",
		207	async => 1,
		208	on_error => sub { warn "FATAL: $_[0]"; exit 1 },
		209	on_event => sub { print $_[0] },
		210	on_destroy => $done,
		211	);
		212
		213	for my $count (3, 2, 1) {
		214	$rpc->($count, sub {
		215	warn "job $count finished\n";
		216	});
		217	}
		218
		219	undef $rpc;
		220
		221	$done->recv;
		222
		223	__DATA__
		224
		225	# this ends up in main, as we don't use a package declaration
		226
		227	use AnyEvent;
		228
		229	sub run {
		230	my ($done, $count) = @_;
		231
		232	my $n;
		233
		234	AnyEvent::Fork::RPC::event "starting to count up to $count\n";
		235
		236	my $w; $w = AE::timer 1, 1, sub {
		237	++$n;
		238
		239	AnyEvent::Fork::RPC::event "count $n of $count\n";
		240
		241	if ($n == $count) {
		242	undef $w;
		243	$done->();
		244	}
		245	};
		246	}
		247
		248	The parent part (the one before the C<__DATA__> section) isn't very
		249	different from the earlier examples. It sets async mode, preloads
		250	the backend module (so the C<AnyEvent::Fork::RPC::event> function is
		251	declared), uses a slightly different C<on_event> handler (which we use
		252	simply for logging purposes) and then, instead of loading a module with
		253	the actual worker code, it C<eval>'s the code from the data section in the
		254	child process.
		255
		256	It then starts three countdowns, from 3 to 1 seconds downwards, destroys
		257	the rpc object so the example finishes eventually, and then just waits for
		258	the stuff to trickle in.
		259
		260	The worker code uses the event function to log some progress messages, but
		261	mostly just creates a recurring one-second timer.
		262
		263	The timer callback increments a counter, logs a message, and eventually,
		264	when the count has been reached, calls the finish callback.
		265
		266	On my system, this results in the following output. Since all timers fire
		267	at roughly the same time, the actual order isn't guaranteed, but the order
		268	shown is very likely what you would get, too.
		269
		270	starting to count up to 3
		271	starting to count up to 2
		272	starting to count up to 1
		273	count 1 of 3
		274	count 1 of 2
		275	count 1 of 1
		276	job 1 finished
		277	count 2 of 2
		278	job 2 finished
		279	count 2 of 3
		280	count 3 of 3
		281	job 3 finished
		282
		283	While the overall ordering isn't guaranteed, the async backend still
		284	guarantees that events and responses are delivered to the parent process
		285	in the exact same ordering as they were generated in the child process.
		286
		287	And unless your system is I<very> busy, it should clearly show that the
		288	job started last will finish first, as it has the lowest count.
		289
		290	This concludes the async example. Since L<AnyEvent::Fork> does not
		291	actually fork, you are free to use about any module in the child, not just
		292	L<AnyEvent>, but also L<IO::AIO>, or L<Tk> for example.
		293
39	=head1 PARENT PROCESS USAGE	294	=head1 PARENT PROCESS USAGE
40		295
41	This module exports nothing, and only implements a single function:	296	This module exports nothing, and only implements a single function:
42		297
43	=over 4	298	=over 4
…		…
50		305
51	use Errno ();	306	use Errno ();
52	use Guard ();	307	use Guard ();
53		308
54	use AnyEvent;	309	use AnyEvent;
55	#use AnyEvent::Fork;	310	# explicit version on next line, as some cpan-testers test with the 0.1 version,
		311	# ignoring dependencies, and this line will at least give a clear indication of that.
		312	use AnyEvent::Fork 0.6; # we don't actually depend on it, this is for convenience
56		313
57	our $VERSION = 0.1;	314	our $VERSION = 1.1;
58		315
59	=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]	316	=item my $rpc = AnyEvent::Fork::RPC::run $fork, $function, [key => value...]
60		317
61	The traditional way to call it. But it is way cooler to call it in the	318	The traditional way to call it. But it is way cooler to call it in the
62	following way:	319	following way:
…		…
92	Called for every call to the C<AnyEvent::Fork::RPC::event> function in the	349	Called for every call to the C<AnyEvent::Fork::RPC::event> function in the
93	child, with the arguments of that function passed to the callback.	350	child, with the arguments of that function passed to the callback.
94		351
95	Also called on errors when no C<on_error> handler is provided.	352	Also called on errors when no C<on_error> handler is provided.
96		353
		354	=item on_destroy => $cb->()
		355
		356	Called when the C<$rpc> object has been destroyed and all requests have
		357	been successfully handled. This is useful when you queue some requests and
		358	want the child to go away after it has handled them. The problem is that
		359	the parent must not exit either until all requests have been handled, and
		360	this can be accomplished by waiting for this callback.
		361
97	=item init => $function (default none)	362	=item init => $function (default none)
98		363
99	When specified (by name), this function is called in the child as the very	364	When specified (by name), this function is called in the child as the very
100	first thing when taking over the process, with all the arguments normally	365	first thing when taking over the process, with all the arguments normally
101	passed to the C<AnyEvent::Fork::run> function, except the communications	366	passed to the C<AnyEvent::Fork::run> function, except the communications
102	socket.	367	socket.
103		368
104	It can be used to do one-time things in the child such as storing passed	369	It can be used to do one-time things in the child such as storing passed
105	parameters or opening database connections.	370	parameters or opening database connections.
106		371
		372	It is called very early - before the serialisers are created or the
		373	C<$function> name is resolved into a function reference, so it could be
		374	used to load any modules that provide the serialiser or function. It can
		375	not, however, create events.
		376
107	=item async => $boolean (default: 0)	377	=item async => $boolean (default: 0)
108		378
109	The default server used in the child does all I/O blockingly, and only	379	The default server used in the child does all I/O blockingly, and only
110	allows a single RPC call to execute concurrently.	380	allows a single RPC call to execute concurrently.
111		381
112	Setting C<async> to a true value switches to another implementation that	382	Setting C<async> to a true value switches to another implementation that
113	uses L<AnyEvent> in the child and allows multiple concurrent RPC calls.	383	uses L<AnyEvent> in the child and allows multiple concurrent RPC calls (it
		384	does not support recursion in the event loop however, blocking condvar
		385	calls will fail).
114		386
115	The actual API in the child is documented in the section that describes	387	The actual API in the child is documented in the section that describes
116	the calling semantics of the returned C<$rpc> function.	388	the calling semantics of the returned C<$rpc> function.
117		389
118	If you want to pre-load the actual back-end modules to enable memory	390	If you want to pre-load the actual back-end modules to enable memory
119	sharing, then you should load C<AnyEvent::Fork::RPC::Sync> for	391	sharing, then you should load C<AnyEvent::Fork::RPC::Sync> for
120	synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode.	392	synchronous, and C<AnyEvent::Fork::RPC::Async> for asynchronous mode.
121		393
122	=item serialiser => $string (default: '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })')	394	If you use a template process and want to fork both sync and async
		395	children, then it is permissible to load both modules.
		396
		397	=item serialiser => $string (default: $AnyEvent::Fork::RPC::STRING_SERIALISER)
123		398
124	All arguments, result data and event data have to be serialised to be	399	All arguments, result data and event data have to be serialised to be
125	transferred between the processes. For this, they have to be frozen and	400	transferred between the processes. For this, they have to be frozen and
126	thawed in both parent and child processes.	401	thawed in both parent and child processes.
127		402
128	By default, only octet strings can be passed between the processes, which	403	By default, only octet strings can be passed between the processes, which
129	is reasonably fast and efficient.	404	is reasonably fast and efficient and requires no extra modules.
130		405
131	For more complicated use cases, you can provide your own freeze and thaw	406	For more complicated use cases, you can provide your own freeze and thaw
132	functions, by specifying a string with perl source code. It's supposed to	407	functions, by specifying a string with perl source code. It's supposed to
133	return two code references when evaluated: the first receives a list of	408	return two code references when evaluated: the first receives a list of
134	perl values and must return an octet string. The second receives the octet	409	perl values and must return an octet string. The second receives the octet
…		…
136		411
137	If you need an external module for serialisation, then you can either	412	If you need an external module for serialisation, then you can either
138	pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use>	413	pre-load it into your L<AnyEvent::Fork> process, or you can add a C<use>
139	or C<require> statement into the serialiser string. Or both.	414	or C<require> statement into the serialiser string. Or both.
140		415
		416	Here are some examples - some of them are also available as global
		417	variables that make them easier to use.
		418
		419	=over 4
		420
		421	=item octet strings - C<$AnyEvent::Fork::RPC::STRING_SERIALISER>
		422
		423	This serialiser concatenates length-prefixes octet strings, and is the
		424	default.
		425
		426	Implementation:
		427
		428	(
		429	sub { pack "(w/a)", @_ },
		430	sub { unpack "(w/a)", shift }
		431	)
		432
		433	=item json - C<$AnyEvent::Fork::RPC::JSON_SERIALISER>
		434
		435	This serialiser creates JSON arrays - you have to make sure the L<JSON>
		436	module is installed for this serialiser to work. It can be beneficial for
		437	sharing when you preload the L<JSON> module in a template process.
		438
		439	L<JSON> (with L<JSON::XS> installed) is slower than the octet string
		440	serialiser, but usually much faster than L<Storable>, unless big chunks of
		441	binary data need to be transferred.
		442
		443	Implementation:
		444
		445	use JSON ();
		446	(
		447	sub { JSON::encode_json \@_ },
		448	sub { @{ JSON::decode_json shift } }
		449	)
		450
		451	=item storable - C<$AnyEvent::Fork::RPC::STORABLE_SERIALISER>
		452
		453	This serialiser uses L<Storable>, which means it has high chance of
		454	serialising just about anything you throw at it, at the cost of having
		455	very high overhead per operation. It also comes with perl.
		456
		457	Implementation:
		458
		459	use Storable ();
		460	(
		461	sub { Storable::freeze \@_ },
		462	sub { @{ Storable::thaw shift } }
		463	)
		464
141	=back	465	=back
142		466
		467	=back
		468
		469	See the examples section earlier in this document for some actual
		470	examples.
		471
143	=cut	472	=cut
144		473
145	our $STRING_SERIALISER = '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })';	474	our $STRING_SERIALISER = '(sub { pack "(w/a)", @_ }, sub { unpack "(w/a)", shift })';
146		475	our $JSON_SERIALISER = 'use JSON (); (sub { JSON::encode_json \@_ }, sub { @{ JSON::decode_json shift } })';
147	# ideally, we want (SvLEN - SvCUR) \|\| 1024 or somesuch...	476	our $STORABLE_SERIALISER = 'use Storable (); (sub { Storable::freeze \@_ }, sub { @{ Storable::thaw shift } })';
148	sub rlen($) { ($_[0] < 384 ? 512 + 16 : 2 << int +(log $_[0] + 512) / log 2) - $_[0] - 16 }
149		477
150	sub run {	478	sub run {
151	my ($self, $function, %arg) = @_;	479	my ($self, $function, %arg) = @_;
152		480
153	my $serialiser = delete $arg{serialiser} \|\| $STRING_SERIALISER;	481	my $serialiser = delete $arg{serialiser} \|\| $STRING_SERIALISER;
154	my $on_event = delete $arg{on_event};	482	my $on_event = delete $arg{on_event};
155	my $on_error = delete $arg{on_error};	483	my $on_error = delete $arg{on_error};
		484	my $on_destroy = delete $arg{on_destroy};
156		485
157	# default for on_error is to on_event, if specified	486	# default for on_error is to on_event, if specified
158	$on_error \|\|= $on_event	487	$on_error \|\|= $on_event
159	? sub { $on_event->(error => shift) }	488	? sub { $on_event->(error => shift) }
160	: sub { die "AnyEvent::Fork::RPC: uncaught error: $_[0].\n" };	489	: sub { die "AnyEvent::Fork::RPC: uncaught error: $_[0].\n" };
…		…
162	# default for on_event is to raise an error	491	# default for on_event is to raise an error
163	$on_event \|\|= sub { $on_error->("event received, but no on_event handler") };	492	$on_event \|\|= sub { $on_error->("event received, but no on_event handler") };
164		493
165	my ($f, $t) = eval $serialiser; die $@ if $@;	494	my ($f, $t) = eval $serialiser; die $@ if $@;
166		495
167	my (@rcb, $fh, $shutdown, $wbuf, $ww, $rbuf, $rw);	496	my (@rcb, %rcb, $fh, $shutdown, $wbuf, $ww);
		497	my ($rlen, $rbuf, $rw) = 512 - 16;
168		498
169	my $wcb = sub {	499	my $wcb = sub {
170	my $len = syswrite $fh, $wbuf;	500	my $len = syswrite $fh, $wbuf;
171		501
172	if (!defined $len) {	502	unless (defined $len) {
173	if ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {	503	if ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {
174	undef $rw; undef $ww; # it ends here	504	undef $rw; undef $ww; # it ends here
175	$on_error->("$!");	505	$on_error->("$!");
176	}	506	}
177	}	507	}
…		…
188		518
189	$self->require ($module)	519	$self->require ($module)
190	->send_arg ($function, $arg{init}, $serialiser)	520	->send_arg ($function, $arg{init}, $serialiser)
191	->run ("$module\::run", sub {	521	->run ("$module\::run", sub {
192	$fh = shift;	522	$fh = shift;
		523
		524	my ($id, $len);
193	$rw = AE::io $fh, 0, sub {	525	$rw = AE::io $fh, 0, sub {
		526	$rlen = $rlen * 2 + 16 if $rlen - 128 < length $rbuf;
194	my $len = sysread $fh, $rbuf, rlen length $rbuf, length $rbuf;	527	$len = sysread $fh, $rbuf, $rlen - length $rbuf, length $rbuf;
195		528
196	if ($len) {	529	if ($len) {
197	while (5 <= length $rbuf) {	530	while (8 <= length $rbuf) {
198	$len = unpack "L", $rbuf;	531	($id, $len) = unpack "NN", $rbuf;
199	4 + $len <= length $rbuf	532	8 + $len <= length $rbuf
200	or last;	533	or last;
201		534
202	my @r = $t->(substr $rbuf, 4, $len);	535	my @r = $t->(substr $rbuf, 8, $len);
203	substr $rbuf, 0, $len + 4, "";	536	substr $rbuf, 0, 8 + $len, "";
		537
		538	if ($id) {
		539	if (@rcb) {
		540	(shift @rcb)->(@r);
		541	} elsif (my $cb = delete $rcb{$id}) {
		542	$cb->(@r);
		543	} else {
		544	undef $rw; undef $ww;
		545	$on_error->("unexpected data from child");
204		546	}
205	if (pop @r) {	547	} else {
206	$on_event->(@r);	548	$on_event->(@r);
207	} elsif (@rcb) {
208	(shift @rcb)->(@r);
209	} else {
210	undef $rw; undef $ww;
211	$on_error->("unexpected data from child");
212	}	549	}
213	}	550	}
214	} elsif (defined $len) {	551	} elsif (defined $len) {
215	undef $rw; undef $ww; # it ends here	552	undef $rw; undef $ww; # it ends here
		553
		554	if (@rcb \|\| %rcb) {
216	$on_error->("unexpected eof")	555	$on_error->("unexpected eof");
217	if @rcb;	556	} else {
		557	$on_destroy->()
		558	if $on_destroy;
		559	}
218	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {	560	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {
219	undef $rw; undef $ww; # it ends here	561	undef $rw; undef $ww; # it ends here
220	$on_error->("read: $!");	562	$on_error->("read: $!");
221	}	563	}
222	};	564	};
…		…
224	$ww \|\|= AE::io $fh, 1, $wcb;	566	$ww \|\|= AE::io $fh, 1, $wcb;
225	});	567	});
226		568
227	my $guard = Guard::guard {	569	my $guard = Guard::guard {
228	$shutdown = 1;	570	$shutdown = 1;
229	$ww \|\|= $fh && AE::io $fh, 1, $wcb;	571
		572	shutdown $fh, 1 if $fh && !$ww;
230	};	573	};
231		574
		575	my $id;
		576
		577	$arg{async}
232	sub {	578	? sub {
233	push @rcb, pop;	579	$id = ($id == 0xffffffff ? 0 : $id) + 1;
		580	$id = ($id == 0xffffffff ? 0 : $id) + 1 while exists $rcb{$id}; # rarely loops
234		581
		582	$rcb{$id} = pop;
		583
235	$guard; # keep it alive	584	$guard if 0; # keep it alive
236		585
237	$wbuf .= pack "L/a*", &$f;	586	$wbuf .= pack "NN/a*", $id, &$f;
238	$ww \|\|= $fh && AE::io $fh, 1, $wcb;	587	$ww \|\|= $fh && AE::io $fh, 1, $wcb;
239	}	588	}
		589	: sub {
		590	push @rcb, pop;
		591
		592	$guard; # keep it alive
		593
		594	$wbuf .= pack "N/a*", &$f;
		595	$ww \|\|= $fh && AE::io $fh, 1, $wcb;
		596	}
240	}	597	}
241		598
		599	=item $rpc->(..., $cb->(...))
		600
		601	The RPC object returned by C<AnyEvent::Fork::RPC::run> is actually a code
		602	reference. There are two things you can do with it: call it, and let it go
		603	out of scope (let it get destroyed).
		604
		605	If C<async> was false when C<$rpc> was created (the default), then, if you
		606	call C<$rpc>, the C<$function> is invoked with all arguments passed to
		607	C<$rpc> except the last one (the callback). When the function returns, the
		608	callback will be invoked with all the return values.
		609
		610	If C<async> was true, then the C<$function> receives an additional
		611	initial argument, the result callback. In this case, returning from
		612	C<$function> does nothing - the function only counts as "done" when the
		613	result callback is called, and any arguments passed to it are considered
		614	the return values. This makes it possible to "return" from event handlers
		615	or e.g. Coro threads.
		616
		617	The other thing that can be done with the RPC object is to destroy it. In
		618	this case, the child process will execute all remaining RPC calls, report
		619	their results, and then exit.
		620
		621	See the examples section earlier in this document for some actual
		622	examples.
		623
242	=back	624	=back
243		625
244	=head1 CHILD PROCESS USAGE	626	=head1 CHILD PROCESS USAGE
245		627
246	These functions are not available in this module. They are only available	628	The following function is not available in this module. They are only
247	in the namespace of this module when the child is running, without	629	available in the namespace of this module when the child is running,
248	having to load any extra module. They are part of the child-side API of	630	without having to load any extra modules. They are part of the child-side
249	L<AnyEvent::Fork::RPC>.	631	API of L<AnyEvent::Fork::RPC>.
250		632
251	=over 4	633	=over 4
252
253	=item AnyEvent::Fork::RPC::quit
254
255	This function can be called to gracefully stop the child process when it
256	is idle.
257
258	After this function is called, the process stops handling incoming RPC
259	requests, but outstanding events and function return values will be sent
260	to the parent. When all data has been sent, the process calls C<exit>.
261
262	Since the parent might not expect the child to exit at random points in
263	time, it is often better to signal the parent by sending an C<event> and
264	letting the parent close down the child process.
265		634
266	=item AnyEvent::Fork::RPC::event ...	635	=item AnyEvent::Fork::RPC::event ...
267		636
268	Send an event to the parent. Events are a bit like RPC calls made by the	637	Send an event to the parent. Events are a bit like RPC calls made by the
269	child process to the parent, except that there is no notion of return	638	child process to the parent, except that there is no notion of return
270	values.	639	values.
271		640
		641	See the examples section earlier in this document for some actual
		642	examples.
		643
272	=back	644	=back
273		645
		646	=head1 ADVANCED TOPICS
		647
		648	=head2 Choosing a backend
		649
		650	So how do you decide which backend to use? Well, that's your problem to
		651	solve, but here are some thoughts on the matter:
		652
		653	=over 4
		654
		655	=item Synchronous
		656
		657	The synchronous backend does not rely on any external modules (well,
		658	except L<common::sense>, which works around a bug in how perl's warning
		659	system works). This keeps the process very small, for example, on my
		660	system, an empty perl interpreter uses 1492kB RSS, which becomes 2020kB
		661	after C<use warnings; use strict> (for people who grew up with C64s around
		662	them this is probably shocking every single time they see it). The worker
		663	process in the first example in this document uses 1792kB.
		664
		665	Since the calls are done synchronously, slow jobs will keep newer jobs
		666	from executing.
		667
		668	The synchronous backend also has no overhead due to running an event loop
		669	- reading requests is therefore very efficient, while writing responses is
		670	less so, as every response results in a write syscall.
		671
		672	If the parent process is busy and a bit slow reading responses, the child
		673	waits instead of processing further requests. This also limits the amount
		674	of memory needed for buffering, as never more than one response has to be
		675	buffered.
		676
		677	The API in the child is simple - you just have to define a function that
		678	does something and returns something.
		679
		680	It's hard to use modules or code that relies on an event loop, as the
		681	child cannot execute anything while it waits for more input.
		682
		683	=item Asynchronous
		684
		685	The asynchronous backend relies on L<AnyEvent>, which tries to be small,
		686	but still comes at a price: On my system, the worker from example 1a uses
		687	3420kB RSS (for L<AnyEvent>, which loads L<EV>, which needs L<XSLoader>
		688	which in turn loads a lot of other modules such as L<warnings>, L<strict>,
		689	L<vars>, L<Exporter>...).
		690
		691	It batches requests and responses reasonably efficiently, doing only as
		692	few reads and writes as needed, but needs to poll for events via the event
		693	loop.
		694
		695	Responses are queued when the parent process is busy. This means the child
		696	can continue to execute any queued requests. It also means that a child
		697	might queue a lot of responses in memory when it generates them and the
		698	parent process is slow accepting them.
		699
		700	The API is not a straightforward RPC pattern - you have to call a
		701	"done" callback to pass return values and signal completion. Also, more
		702	importantly, the API starts jobs as fast as possible - when 1000 jobs
		703	are queued and the jobs are slow, they will all run concurrently. The
		704	child must implement some queueing/limiting mechanism if this causes
		705	problems. Alternatively, the parent could limit the amount of rpc calls
		706	that are outstanding.
		707
		708	Blocking use of condvars is not supported.
		709
		710	Using event-based modules such as L<IO::AIO>, L<Gtk2>, L<Tk> and so on is
		711	easy.
		712
		713	=back
		714
		715	=head2 Passing file descriptors
		716
		717	Unlike L<AnyEvent::Fork>, this module has no in-built file handle or file
		718	descriptor passing abilities.
		719
		720	The reason is that passing file descriptors is extraordinary tricky
		721	business, and conflicts with efficient batching of messages.
		722
		723	There still is a method you can use: Create a
		724	C<AnyEvent::Util::portable_socketpair> and C<send_fh> one half of it to
		725	the process before you pass control to C<AnyEvent::Fork::RPC::run>.
		726
		727	Whenever you want to pass a file descriptor, send an rpc request to the
		728	child process (so it expects the descriptor), then send it over the other
		729	half of the socketpair. The child should fetch the descriptor from the
		730	half it has passed earlier.
		731
		732	Here is some (untested) pseudocode to that effect:
		733
		734	use AnyEvent::Util;
		735	use AnyEvent::Fork::RPC;
		736	use IO::FDPass;
		737
		738	my ($s1, $s2) = AnyEvent::Util::portable_socketpair;
		739
		740	my $rpc = AnyEvent::Fork
		741	->new
		742	->send_fh ($s2)
		743	->require ("MyWorker")
		744	->AnyEvent::Fork::RPC::run ("MyWorker::run"
		745	init => "MyWorker::init",
		746	);
		747
		748	undef $s2; # no need to keep it around
		749
		750	# pass an fd
		751	$rpc->("i'll send some fd now, please expect it!", my $cv = AE::cv);
		752
		753	IO::FDPass fileno $s1, fileno $handle_to_pass;
		754
		755	$cv->recv;
		756
		757	The MyWorker module could look like this:
		758
		759	package MyWorker;
		760
		761	use IO::FDPass;
		762
		763	my $s2;
		764
		765	sub init {
		766	$s2 = $_[0];
		767	}
		768
		769	sub run {
		770	if ($_[0] eq "i'll send some fd now, please expect it!") {
		771	my $fd = IO::FDPass::recv fileno $s2;
		772	...
		773	}
		774	}
		775
		776	Of course, this might be blocking if you pass a lot of file descriptors,
		777	so you might want to look into L<AnyEvent::FDpasser> which can handle the
		778	gory details.
		779
		780	=head1 EXCEPTIONS
		781
		782	There are no provisions whatsoever for catching exceptions at this time -
		783	in the child, exeptions might kill the process, causing calls to be lost
		784	and the parent encountering a fatal error. In the parent, exceptions in
		785	the result callback will not be caught and cause undefined behaviour.
		786
274	=head1 SEE ALSO	787	=head1 SEE ALSO
275		788
276	L<AnyEvent::Fork> (to create the processes in the first place),	789	L<AnyEvent::Fork>, to create the processes in the first place.
		790
277	L<AnyEvent::Fork::Pool> (to manage whole pools of processes).	791	L<AnyEvent::Fork::Pool>, to manage whole pools of processes.
278		792
279	=head1 AUTHOR AND CONTACT INFORMATION	793	=head1 AUTHOR AND CONTACT INFORMATION
280		794
281	Marc Lehmann <schmorp@schmorp.de>	795	Marc Lehmann <schmorp@schmorp.de>
282	http://software.schmorp.de/pkg/AnyEvent-Fork-RPC	796	http://software.schmorp.de/pkg/AnyEvent-Fork-RPC

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents): Revision 1.2 by root, Wed Apr 17 17:08:16 2013 UTC vs. Revision 1.25 by root, Sun Apr 28 14:17:59 2013 UTC

Diff Legend

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.2 by root, Wed Apr 17 17:08:16 2013 UTC vs.
Revision 1.25 by root, Sun Apr 28 14:17:59 2013 UTC