[ViewVC] Diff of: cvs/AnyEvent-Fork-RPC/RPC.pm

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.10 by root, Wed Apr 17 22:04:49 2013 UTC vs.
Revision 1.13 by root, Thu Apr 18 11:11:26 2013 UTC

…		…
49	silly, but illustrates the use of events.	49	silly, but illustrates the use of events.
50		50
51	First the parent process:	51	First the parent process:
52		52
53	use AnyEvent;	53	use AnyEvent;
54	use AnyEvent::Fork;
55	use AnyEvent::Fork::RPC;	54	use AnyEvent::Fork::RPC;
56		55
57	my $done = AE::cv;	56	my $done = AE::cv;
58		57
59	my $rpc = AnyEvent::Fork	58	my $rpc = AnyEvent::Fork
…		…
174	child process may exit at any time, so you should call C<$done> only when	173	child process may exit at any time, so you should call C<$done> only when
175	you really I<are> done.	174	you really I<are> done.
176		175
177	=head2 Example 2: Asynchronous Backend	176	=head2 Example 2: Asynchronous Backend
178		177
179	#TODO	178	This example implements multiple count-downs in the child, using
		179	L<AnyEvent> timers. While this is a bit silly (one could use timers in te
		180	parent just as well), it illustrates the ability to use AnyEvent in the
		181	child and the fact that responses can arrive in a different order then the
		182	requests.
		183
		184	It also shows how to embed the actual child code into a C<__DATA__>
		185	section, so it doesn't need any external files at all.
		186
		187	And when your parent process is often busy, and you have stricter timing
		188	requirements, then running timers in a child process suddenly doesn't look
		189	so silly anymore.
		190
		191	Without further ado, here is the code:
		192
		193	use AnyEvent;
		194	use AnyEvent::Fork::RPC;
		195
		196	my $done = AE::cv;
		197
		198	my $rpc = AnyEvent::Fork
		199	->new
		200	->require ("AnyEvent::Fork::RPC::Async")
		201	->eval (do { local $/; <DATA> })
		202	->AnyEvent::Fork::RPC::run ("run",
		203	async => 1,
		204	on_error => sub { warn "FATAL: $_[0]"; exit 1 },
		205	on_event => sub { print $_[0] },
		206	on_destroy => $done,
		207	);
		208
		209	for my $count (3, 2, 1) {
		210	$rpc->($count, sub {
		211	warn "job $count finished\n";
		212	});
		213	}
		214
		215	undef $rpc;
		216
		217	$done->recv;
		218
		219	__DATA__
		220
		221	# this ends up in main, as we don't use a package declaration
		222
		223	use AnyEvent;
		224
		225	sub run {
		226	my ($done, $count) = @_;
		227
		228	my $n;
		229
		230	AnyEvent::Fork::RPC::event "starting to count up to $count\n";
		231
		232	my $w; $w = AE::timer 1, 1, sub {
		233	++$n;
		234
		235	AnyEvent::Fork::RPC::event "count $n of $count\n";
		236
		237	if ($n == $count) {
		238	undef $w;
		239	$done->();
		240	}
		241	};
		242	}
		243
		244	The parent part (the one before the C<__DATA__> section) isn't very
		245	different from the earlier examples. It sets async mode, preloads
		246	the backend module (so the C<AnyEvent::Fork::RPC::event> function is
		247	declared), uses a slightly different C<on_event> handler (which we use
		248	simply for logging purposes) and then, instead of loading a module with
		249	the actual worker code, it C<eval>'s the code from the data section in the
		250	child process.
		251
		252	It then starts three countdowns, from 3 to 1 seconds downwards, destroys
		253	the rpc object so the example finishes eventually, and then just waits for
		254	the stuff to trickle in.
		255
		256	The worker code uses the event function to log some progress messages, but
		257	mostly just creates a recurring one-second timer.
		258
		259	The timer callback increments a counter, logs a message, and eventually,
		260	when the count has been reached, calls the finish callback.
		261
		262	On my system, this results in the following output. Since all timers fire
		263	at roughly the same time, the actual order isn't guaranteed, but the order
		264	shown is very likely what you would get, too.
		265
		266	starting to count up to 3
		267	starting to count up to 2
		268	starting to count up to 1
		269	count 1 of 3
		270	count 1 of 2
		271	count 1 of 1
		272	job 1 finished
		273	count 2 of 2
		274	job 2 finished
		275	count 2 of 3
		276	count 3 of 3
		277	job 3 finished
		278
		279	While the overall ordering isn't guaranteed, the async backend still
		280	guarantees that events and responses are delivered to the parent process
		281	in the exact same ordering as they were generated in the child process.
		282
		283	And unless your system is I<very> busy, it should clearly show that the
		284	job started last will finish first, as it has the lowest count.
		285
		286	This concludes the async example. Since L<AnyEvent::Fork> does not
		287	actually fork, you are free to use about any module in the child, not just
		288	L<AnyEvent>, but also L<IO::AIO>, or L<Tk> for example.
180		289
181	=head1 PARENT PROCESS USAGE	290	=head1 PARENT PROCESS USAGE
182		291
183	This module exports nothing, and only implements a single function:	292	This module exports nothing, and only implements a single function:
184		293
…		…
380	}	489	}
381	} elsif (defined $len) {	490	} elsif (defined $len) {
382	undef $rw; undef $ww; # it ends here	491	undef $rw; undef $ww; # it ends here
383		492
384	if (@rcb \|\| %rcb) {	493	if (@rcb \|\| %rcb) {
385	use Data::Dump;ddx[\@rcb,\%rcb];#d#
386	$on_error->("unexpected eof");	494	$on_error->("unexpected eof");
387	} else {	495	} else {
388	$on_destroy->();	496	$on_destroy->();
389	}	497	}
390	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {	498	} elsif ($! != Errno::EAGAIN && $! != Errno::EWOULDBLOCK) {
…		…
470	See the examples section earlier in this document for some actual	578	See the examples section earlier in this document for some actual
471	examples.	579	examples.
472		580
473	=back	581	=back
474		582
		583	=head1 ADVANCED TOPICS
		584
		585	=head2 Choosing a backend
		586
		587	So how do you decide which backend to use? Well, that's your problem to
		588	solve, but here are some thoughts on the matter:
		589
		590	=over 4
		591
		592	=item Synchronous
		593
		594	The synchronous backend does not rely on any external modules (well,
		595	except L<common::sense>, which works around a bug in how perl's warning
		596	system works). This keeps the process very small, for example, on my
		597	system, an empty perl interpreter uses 1492kB RSS, which becomes 2020kB
		598	after C<use warnings; use strict> (for people who grew up with C64s around
		599	them this is probably shocking every single time they see it). The worker
		600	process in the first example in this document uses 1792kB.
		601
		602	Since the calls are done synchronously, slow jobs will keep newer jobs
		603	from executing.
		604
		605	The synchronous backend also has no overhead due to running an event loop
		606	- reading requests is therefore very efficient, while writing responses is
		607	less so, as every response results in a write syscall.
		608
		609	If the parent process is busy and a bit slow reading responses, the child
		610	waits instead of processing further requests. This also limits the amount
		611	of memory needed for buffering, as never more than one response has to be
		612	buffered.
		613
		614	The API in the child is simple - you just have to define a function that
		615	does something and returns something.
		616
		617	It's hard to use modules or code that relies on an event loop, as the
		618	child cannot execute anything while it waits for more input.
		619
		620	=item Asynchronous
		621
		622	The asynchronous backend relies on L<AnyEvent>, which tries to be small,
		623	but still comes at a price: On my system, the worker from example 1a uses
		624	3420kB RSS (for L<AnyEvent>, which loads L<EV>, which needs L<XSLoader>
		625	which in turn loads a lot of other modules such as L<warnings>, L<strict>,
		626	L<vars>, L<Exporter>...).
		627
		628	It batches requests and responses reasonably efficiently, doing only as
		629	few reads and writes as needed, but needs to poll for events via the event
		630	loop.
		631
		632	Responses are queued when the parent process is busy. This means the child
		633	can continue to execute any queued requests. It also means that a child
		634	might queue a lot of responses in memory when it generates them and the
		635	parent process is slow accepting them.
		636
		637	The API is not a straightforward RPC pattern - you have to call a
		638	"done" callback to pass return values and signal completion. Also, more
		639	importantly, the API starts jobs as fast as possible - when 1000 jobs
		640	are queued and the jobs are slow, they will all run concurrently. The
		641	child must implement some queueing/limiting mechanism if this causes
		642	problems. Alternatively, the parent could limit the amount of rpc calls
		643	that are outstanding.
		644
		645	Using event-based modules such as L<IO::AIO>, L<Gtk2>, L<Tk> and so on is
		646	easy.
		647
		648	=back
		649
		650	=head2 Passing file descriptors
		651
		652	Unlike L<AnyEvent::Fork>, this module has no in-built file handle or file
		653	descriptor passing abilities.
		654
		655	The reason is that passing file descriptors is extraordinary tricky
		656	business, and conflicts with efficient batching of messages.
		657
		658	There still is a method you can use: Create a
		659	C<AnyEvent::Util::portable_socketpair> and C<send_fh> one half of it to
		660	the process before you pass control to C<AnyEvent::Fork::RPC::run>.
		661
		662	Whenever you want to pass a file descriptor, send an rpc request to the
		663	child process (so it expects the descriptor), then send it over the other
		664	half of the socketpair. The child should fetch the descriptor from the
		665	half it has passed earlier.
		666
		667	Here is some (untested) pseudocode to that effect:
		668
		669	use AnyEvent::Util;
		670	use AnyEvent::Fork::RPC;
		671	use IO::FDPass;
		672
		673	my ($s1, $s2) = AnyEvent::Util::portable_socketpair;
		674
		675	my $rpc = AnyEvent::Fork
		676	->new
		677	->send_fh ($s2)
		678	->require ("MyWorker")
		679	->AnyEvent::Fork::RPC::run ("MyWorker::run"
		680	init => "MyWorker::init",
		681	);
		682
		683	undef $s2; # no need to keep it around
		684
		685	# pass an fd
		686	$rpc->("i'll send some fd now, please expect it!", my $cv = AE::cv);
		687
		688	IO::FDPass fileno $s1, fileno $handle_to_pass;
		689
		690	$cv->recv;
		691
		692	The MyWorker module could look like this:
		693
		694	package MyWorker;
		695
		696	use IO::FDPass;
		697
		698	my $s2;
		699
		700	sub init {
		701	$s2 = $_[0];
		702	}
		703
		704	sub run {
		705	if ($_[0] eq "i'll send some fd now, please expect it!") {
		706	my $fd = IO::FDPass::recv fileno $s2;
		707	...
		708	}
		709	}
		710
		711	Of course, this might be blocking if you pass a lot of file descriptors,
		712	so you might want to look into L<AnyEvent::FDpasser> which can handle the
		713	gory details.
		714
475	=head1 SEE ALSO	715	=head1 SEE ALSO
476		716
477	L<AnyEvent::Fork> (to create the processes in the first place),	717	L<AnyEvent::Fork> (to create the processes in the first place),
478	L<AnyEvent::Fork::Pool> (to manage whole pools of processes).	718	L<AnyEvent::Fork::Pool> (to manage whole pools of processes).
479		719

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents): Revision 1.10 by root, Wed Apr 17 22:04:49 2013 UTC vs. Revision 1.13 by root, Thu Apr 18 11:11:26 2013 UTC

Diff Legend

Comparing AnyEvent-Fork-RPC/RPC.pm (file contents):
Revision 1.10 by root, Wed Apr 17 22:04:49 2013 UTC vs.
Revision 1.13 by root, Thu Apr 18 11:11:26 2013 UTC