[ViewVC] Diff of: cvs/AnyEvent-Fork/Fork.pm

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.39 by root, Sat Apr 6 22:39:37 2013 UTC vs.
Revision 1.48 by root, Fri Apr 19 10:51:41 2013 UTC

…		…
34	This module only creates processes and lets you pass file handles and	34	This module only creates processes and lets you pass file handles and
35	strings to it, and run perl code. It does not implement any kind of RPC -	35	strings to it, and run perl code. It does not implement any kind of RPC -
36	there is no back channel from the process back to you, and there is no RPC	36	there is no back channel from the process back to you, and there is no RPC
37	or message passing going on.	37	or message passing going on.
38		38
39	If you need some form of RPC, you can either implement it yourself	39	If you need some form of RPC, you could use the L<AnyEvent::Fork::RPC>
40	in whatever way you like, use some message-passing module such	40	companion module, which adds simple RPC/job queueing to a process created
41	as L<AnyEvent::MP>, some pipe such as L<AnyEvent::ZeroMQ>, use	41	by this module.
42	L<AnyEvent::Handle> on both sides to send e.g. JSON or Storable messages,	42
43	and so on.	43	Or you can implement it yourself in whatever way you like, use some
		44	message-passing module such as L<AnyEvent::MP>, some pipe such as
		45	L<AnyEvent::ZeroMQ>, use L<AnyEvent::Handle> on both sides to send
		46	e.g. JSON or Storable messages, and so on.
44		47
45	=head2 COMPARISON TO OTHER MODULES	48	=head2 COMPARISON TO OTHER MODULES
46		49
47	There is an abundance of modules on CPAN that do "something fork", such as	50	There is an abundance of modules on CPAN that do "something fork", such as
48	L<Parallel::ForkManager>, L<AnyEvent::ForkManager>, L<AnyEvent::Worker>	51	L<Parallel::ForkManager>, L<AnyEvent::ForkManager>, L<AnyEvent::Worker>
…		…
221	}	224	}
222	}	225	}
223		226
224	=head2 use AnyEvent::Fork as a faster fork+exec	227	=head2 use AnyEvent::Fork as a faster fork+exec
225		228
226	This runs C</bin/echo hi>, with stdandard output redirected to /tmp/log	229	This runs C</bin/echo hi>, with standard output redirected to F</tmp/log>
227	and standard error redirected to the communications socket. It is usually	230	and standard error redirected to the communications socket. It is usually
228	faster than fork+exec, but still lets you prepare the environment.	231	faster than fork+exec, but still lets you prepare the environment.
229		232
230	open my $output, ">/tmp/log" or die "$!";	233	open my $output, ">/tmp/log" or die "$!";
231		234
232	AnyEvent::Fork	235	AnyEvent::Fork
233	->new	236	->new
234	->eval ('	237	->eval ('
		238	# compile a helper function for later use
235	sub run {	239	sub run {
236	my ($fh, $output, @cmd) = @_;	240	my ($fh, $output, @cmd) = @_;
237		241
238	# perl will clear close-on-exec on STDOUT/STDERR	242	# perl will clear close-on-exec on STDOUT/STDERR
239	open STDOUT, ">&", $output or die;	243	open STDOUT, ">&", $output or die;
…		…
250		254
251	=head1 CONCEPTS	255	=head1 CONCEPTS
252		256
253	This module can create new processes either by executing a new perl	257	This module can create new processes either by executing a new perl
254	process, or by forking from an existing "template" process.	258	process, or by forking from an existing "template" process.
		259
		260	All these processes are called "child processes" (whether they are direct
		261	children or not), while the process that manages them is called the
		262	"parent process".
255		263
256	Each such process comes with its own file handle that can be used to	264	Each such process comes with its own file handle that can be used to
257	communicate with it (it's actually a socket - one end in the new process,	265	communicate with it (it's actually a socket - one end in the new process,
258	one end in the main process), and among the things you can do in it are	266	one end in the main process), and among the things you can do in it are
259	load modules, fork new processes, send file handles to it, and execute	267	load modules, fork new processes, send file handles to it, and execute
…		…
369	use AnyEvent;	377	use AnyEvent;
370	use AnyEvent::Util ();	378	use AnyEvent::Util ();
371		379
372	use IO::FDPass;	380	use IO::FDPass;
373		381
374	our $VERSION = 0.5;	382	our $VERSION = 0.7;
375
376	our $PERL; # the path to the perl interpreter, deduces with various forms of magic
377
378	=over 4
379
380	=back
381
382	=cut
383		383
384	# the early fork template process	384	# the early fork template process
385	our $EARLY;	385	our $EARLY;
386		386
387	# the empty template process	387	# the empty template process
388	our $TEMPLATE;	388	our $TEMPLATE;
		389
		390	sub QUEUE() { 0 }
		391	sub FH() { 1 }
		392	sub WW() { 2 }
		393	sub PID() { 3 }
		394	sub CB() { 4 }
		395
		396	sub _new {
		397	my ($self, $fh, $pid) = @_;
		398
		399	AnyEvent::Util::fh_nonblocking $fh, 1;
		400
		401	$self = bless [
		402	[], # write queue - strings or fd's
		403	$fh,
		404	undef, # AE watcher
		405	$pid,
		406	], $self;
		407
		408	$self
		409	}
389		410
390	sub _cmd {	411	sub _cmd {
391	my $self = shift;	412	my $self = shift;
392		413
393	# ideally, we would want to use "a (w/a)*" as format string, but perl	414	# ideally, we would want to use "a (w/a)*" as format string, but perl
394	# versions from at least 5.8.9 to 5.16.3 are all buggy and can't unpack	415	# versions from at least 5.8.9 to 5.16.3 are all buggy and can't unpack
395	# it.	416	# it.
396	push @{ $self->[2] }, pack "a L/a*", $_[0], $_[1];	417	push @{ $self->[QUEUE] }, pack "a L/a*", $_[0], $_[1];
397		418
398	$self->[3] \|\|= AE::io $self->[1], 1, sub {	419	$self->[WW] \|\|= AE::io $self->[FH], 1, sub {
399	do {	420	do {
400	# send the next "thing" in the queue - either a reference to an fh,	421	# send the next "thing" in the queue - either a reference to an fh,
401	# or a plain string.	422	# or a plain string.
402		423
403	if (ref $self->[2][0]) {	424	if (ref $self->[QUEUE][0]) {
404	# send fh	425	# send fh
405	unless (IO::FDPass::send fileno $self->[1], fileno ${ $self->[2][0] }) {	426	unless (IO::FDPass::send fileno $self->[FH], fileno ${ $self->[QUEUE][0] }) {
406	return if $! == Errno::EAGAIN \|\| $! == Errno::EWOULDBLOCK;	427	return if $! == Errno::EAGAIN \|\| $! == Errno::EWOULDBLOCK;
407	undef $self->[3];	428	undef $self->[WW];
408	die "AnyEvent::Fork: file descriptor send failure: $!";	429	die "AnyEvent::Fork: file descriptor send failure: $!";
409	}	430	}
410		431
411	shift @{ $self->[2] };	432	shift @{ $self->[QUEUE] };
412		433
413	} else {	434	} else {
414	# send string	435	# send string
415	my $len = syswrite $self->[1], $self->[2][0];	436	my $len = syswrite $self->[FH], $self->[QUEUE][0];
416		437
417	unless ($len) {	438	unless ($len) {
418	return if $! == Errno::EAGAIN \|\| $! == Errno::EWOULDBLOCK;	439	return if $! == Errno::EAGAIN \|\| $! == Errno::EWOULDBLOCK;
419	undef $self->[3];	440	undef $self->[3];
420	die "AnyEvent::Fork: command write failure: $!";	441	die "AnyEvent::Fork: command write failure: $!";
421	}	442	}
422		443
423	substr $self->[2][0], 0, $len, "";	444	substr $self->[QUEUE][0], 0, $len, "";
424	shift @{ $self->[2] } unless length $self->[2][0];	445	shift @{ $self->[QUEUE] } unless length $self->[QUEUE][0];
425	}	446	}
426	} while @{ $self->[2] };	447	} while @{ $self->[QUEUE] };
427		448
428	# everything written	449	# everything written
429	undef $self->[3];	450	undef $self->[WW];
430		451
431	# invoke run callback, if any	452	# invoke run callback, if any
432	$self->[4]->($self->[1]) if $self->[4];	453	if ($self->[CB]) {
		454	$self->[CB]->($self->[FH]);
		455	@$self = ();
		456	}
433	};	457	};
434		458
435	() # make sure we don't leak the watcher	459	() # make sure we don't leak the watcher
436	}
437
438	sub _new {
439	my ($self, $fh, $pid) = @_;
440
441	AnyEvent::Util::fh_nonblocking $fh, 1;
442
443	$self = bless [
444	$pid,
445	$fh,
446	[], # write queue - strings or fd's
447	undef, # AE watcher
448	], $self;
449
450	$self
451	}	460	}
452		461
453	# fork template from current process, used by AnyEvent::Fork::Early/Template	462	# fork template from current process, used by AnyEvent::Fork::Early/Template
454	sub _new_fork {	463	sub _new_fork {
455	my ($fh, $slave) = AnyEvent::Util::portable_socketpair;	464	my ($fh, $slave) = AnyEvent::Util::portable_socketpair;
…		…
460	if ($pid eq 0) {	469	if ($pid eq 0) {
461	require AnyEvent::Fork::Serve;	470	require AnyEvent::Fork::Serve;
462	$AnyEvent::Fork::Serve::OWNER = $parent;	471	$AnyEvent::Fork::Serve::OWNER = $parent;
463	close $fh;	472	close $fh;
464	$0 = "$_[1] of $parent";	473	$0 = "$_[1] of $parent";
465	$SIG{CHLD} = 'IGNORE';
466	AnyEvent::Fork::Serve::serve ($slave);	474	AnyEvent::Fork::Serve::serve ($slave);
467	exit 0;	475	exit 0;
468	} elsif (!$pid) {	476	} elsif (!$pid) {
469	die "AnyEvent::Fork::Early/Template: unable to fork template process: $!";	477	die "AnyEvent::Fork::Early/Template: unable to fork template process: $!";
470	}	478	}
…		…
589	AnyEvent::Fork itself.	597	AnyEvent::Fork itself.
590		598
591	=cut	599	=cut
592		600
593	sub pid {	601	sub pid {
594	$_[0][0]	602	$_[0][PID]
595	}	603	}
596		604
597	=item $proc = $proc->eval ($perlcode, @args)	605	=item $proc = $proc->eval ($perlcode, @args)
598		606
599	Evaluates the given C<$perlcode> as ... perl code, while setting C<@_> to	607	Evaluates the given C<$perlcode> as ... Perl code, while setting C<@_> to
600	the strings specified by C<@args>, in the "main" package.	608	the strings specified by C<@args>, in the "main" package.
601		609
602	This call is meant to do any custom initialisation that might be required	610	This call is meant to do any custom initialisation that might be required
603	(for example, the C<require> method uses it). It's not supposed to be used	611	(for example, the C<require> method uses it). It's not supposed to be used
604	to completely take over the process, use C<run> for that.	612	to completely take over the process, use C<run> for that.
…		…
666	sub send_fh {	674	sub send_fh {
667	my ($self, @fh) = @_;	675	my ($self, @fh) = @_;
668		676
669	for my $fh (@fh) {	677	for my $fh (@fh) {
670	$self->_cmd ("h");	678	$self->_cmd ("h");
671	push @{ $self->[2] }, \$fh;	679	push @{ $self->[QUEUE] }, \$fh;
672	}	680	}
673		681
674	$self	682	$self
675	}	683	}
676		684
…		…
762	=cut	770	=cut
763		771
764	sub run {	772	sub run {
765	my ($self, $func, $cb) = @_;	773	my ($self, $func, $cb) = @_;
766		774
767	$self->[4] = $cb;	775	$self->[CB] = $cb;
768	$self->_cmd (r => $func);	776	$self->_cmd (r => $func);
		777	}
		778
		779	=item $proc->to_fh ($cb->($fh))
		780
		781	Flushes all commands out to the process and then calls the callback with
		782	the communications socket.
		783
		784	The process object becomes unusable on return from this function - any
		785	further method calls result in undefined behaviour.
		786
		787	The point of this method is to give you a file handle thta you cna pass
		788	to another process. In that other process, you can call C<new_from_fh
		789	AnyEvent::Fork::RPC> to create a new C<AnyEvent::Fork> object from it,
		790	thereby effectively passing a fork object to another process.
		791
		792	=cut
		793
		794	sub to_fh {
		795	my ($self, $cb) = @_;
		796
		797	$self->[CB] = $cb;
		798
		799	unless ($self->[WW]) {
		800	$self->[CB]->($self->[FH]);
		801	@$self = ();
		802	}
		803	}
		804
		805	=item new_from_fh AnyEvent::Fork $fh
		806
		807	Takes a file handle originally rceeived by the C<to_fh> method and creates
		808	a new C<AnyEvent:Fork> object. The child process itself will not change in
		809	any way, i.e. it will keep all the modifications done to it before calling
		810	C<to_fh>.
		811
		812	The new object is very much like the original object, except that the
		813	C<pid> method will return C<undef> even if the process is a direct child.
		814
		815	=cut
		816
		817	sub new_from_fh {
		818	my ($class, $fh) = @_;
		819
		820	$class->_new ($fh)
769	}	821	}
770		822
771	=back	823	=back
772		824
773	=head1 PERFORMANCE	825	=head1 PERFORMANCE
…		…
783		835
784	2079 new processes per second, using manual socketpair + fork	836	2079 new processes per second, using manual socketpair + fork
785		837
786	Then I did the same thing, but instead of calling fork, I called	838	Then I did the same thing, but instead of calling fork, I called
787	AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the	839	AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the
788	socket form the child to close on exit. This does the same thing as manual	840	socket from the child to close on exit. This does the same thing as manual
789	socket pair + fork, except that what is forked is the template process	841	socket pair + fork, except that what is forked is the template process
790	(2440kB), and the socket needs to be passed to the server at the other end	842	(2440kB), and the socket needs to be passed to the server at the other end
791	of the socket first.	843	of the socket first.
792		844
793	2307 new processes per second, using AnyEvent::Fork->new	845	2307 new processes per second, using AnyEvent::Fork->new
…		…
800	So how can C<< AnyEvent->new >> be faster than a standard fork, even	852	So how can C<< AnyEvent->new >> be faster than a standard fork, even
801	though it uses the same operations, but adds a lot of overhead?	853	though it uses the same operations, but adds a lot of overhead?
802		854
803	The difference is simply the process size: forking the 5MB process takes	855	The difference is simply the process size: forking the 5MB process takes
804	so much longer than forking the 2.5MB template process that the extra	856	so much longer than forking the 2.5MB template process that the extra
805	overhead introduced is canceled out.	857	overhead is canceled out.
806		858
807	If the benchmark process grows, the normal fork becomes even slower:	859	If the benchmark process grows, the normal fork becomes even slower:
808		860
809	1340 new processes, manual fork of a 20MB process	861	1340 new processes, manual fork of a 20MB process
810	731 new processes, manual fork of a 200MB process	862	731 new processes, manual fork of a 200MB process
…		…
870	initialising them, for example, by calling C<init Gtk2> manually.	922	initialising them, for example, by calling C<init Gtk2> manually.
871		923
872	=item exiting calls object destructors	924	=item exiting calls object destructors
873		925
874	This only applies to users of L<AnyEvent::Fork:Early> and	926	This only applies to users of L<AnyEvent::Fork:Early> and
875	L<AnyEvent::Fork::Template>, or when initialiasing code creates objects	927	L<AnyEvent::Fork::Template>, or when initialising code creates objects
876	that reference external resources.	928	that reference external resources.
877		929
878	When a process created by AnyEvent::Fork exits, it might do so by calling	930	When a process created by AnyEvent::Fork exits, it might do so by calling
879	exit, or simply letting perl reach the end of the program. At which point	931	exit, or simply letting perl reach the end of the program. At which point
880	Perl runs all destructors.	932	Perl runs all destructors.
…		…
905	Cygwin perl is not supported at the moment due to some hilarious	957	Cygwin perl is not supported at the moment due to some hilarious
906	shortcomings of its API - see L<IO::FDPoll> for more details.	958	shortcomings of its API - see L<IO::FDPoll> for more details.
907		959
908	=head1 SEE ALSO	960	=head1 SEE ALSO
909		961
910	L<AnyEvent::Fork::Early> (to avoid executing a perl interpreter),	962	L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all
		963	(part of this distribution).
		964
911	L<AnyEvent::Fork::Template> (to create a process by forking the main	965	L<AnyEvent::Fork::Template>, to create a process by forking the main
912	program at a convenient time).	966	program at a convenient time (part of this distribution).
913		967
914	=head1 AUTHOR	968	L<AnyEvent::Fork::RPC>, for simple RPC to child processes (on CPAN).
		969
		970	=head1 AUTHOR AND CONTACT INFORMATION
915		971
916	Marc Lehmann <schmorp@schmorp.de>	972	Marc Lehmann <schmorp@schmorp.de>
917	http://home.schmorp.de/	973	http://software.schmorp.de/pkg/AnyEvent-Fork
918		974
919	=cut	975	=cut
920		976
921	1	977	1
922		978

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork/Fork.pm (file contents): Revision 1.39 by root, Sat Apr 6 22:39:37 2013 UTC vs. Revision 1.48 by root, Fri Apr 19 10:51:41 2013 UTC

Diff Legend

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.39 by root, Sat Apr 6 22:39:37 2013 UTC vs.
Revision 1.48 by root, Fri Apr 19 10:51:41 2013 UTC