[ViewVC] Diff of: cvs/AnyEvent-Fork/Fork.pm

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.58 by root, Sun Aug 25 21:10:10 2013 UTC vs.
Revision 1.71 by root, Wed Jul 25 22:35:00 2018 UTC

…		…
56	or L<AnyEvent::Subprocess>. There are modules that implement their own	56	or L<AnyEvent::Subprocess>. There are modules that implement their own
57	process management, such as L<AnyEvent::DBI>.	57	process management, such as L<AnyEvent::DBI>.
58		58
59	The problems that all these modules try to solve are real, however, none	59	The problems that all these modules try to solve are real, however, none
60	of them (from what I have seen) tackle the very real problems of unwanted	60	of them (from what I have seen) tackle the very real problems of unwanted
61	memory sharing, efficiency, not being able to use event processing or	61	memory sharing, efficiency or not being able to use event processing, GUI
62	similar modules in the processes they create.	62	toolkits or similar modules in the processes they create.
63		63
64	This module doesn't try to replace any of them - instead it tries to solve	64	This module doesn't try to replace any of them - instead it tries to solve
65	the problem of creating processes with a minimum of fuss and overhead (and	65	the problem of creating processes with a minimum of fuss and overhead (and
66	also luxury). Ideally, most of these would use AnyEvent::Fork internally,	66	also luxury). Ideally, most of these would use AnyEvent::Fork internally,
67	except they were written before AnyEvent:Fork was available, so obviously	67	except they were written before AnyEvent:Fork was available, so obviously
…		…
89		89
90	=item Forking usually creates a copy-on-write copy of the parent	90	=item Forking usually creates a copy-on-write copy of the parent
91	process.	91	process.
92		92
93	For example, modules or data files that are loaded will not use additional	93	For example, modules or data files that are loaded will not use additional
94	memory after a fork. When exec'ing a new process, modules and data files	94	memory after a fork. Exec'ing a new process, in contrast, means modules
95	might need to be loaded again, at extra CPU and memory cost. But when	95	and data files might need to be loaded again, at extra CPU and memory
96	forking, literally all data structures are copied - if the program frees	96	cost.
		97
		98	But when forking, you still create a copy of your data structures - if
97	them and replaces them by new data, the child processes will retain the	99	the program frees them and replaces them by new data, the child processes
98	old version even if it isn't used, which can suddenly and unexpectedly	100	will retain the old version even if it isn't used, which can suddenly and
99	increase memory usage when freeing memory.	101	unexpectedly increase memory usage when freeing memory.
100		102
		103	For example, L<Gtk2::CV> is an image viewer optimised for large
		104	directories (millions of pictures). It also forks subprocesses for
		105	thumbnail generation, which inherit the data structure that stores all
		106	file information. If the user changes the directory, it gets freed in
		107	the main process, leaving a copy in the thumbnailer processes. This can
		108	lead to many times the memory usage that would actually be required. The
		109	solution is to fork early (and being unable to dynamically generate more
		110	subprocesses or do this from a module)... or to use L<AnyEvent:Fork>.
		111
101	The trade-off is between more sharing with fork (which can be good or	112	There is a trade-off between more sharing with fork (which can be good or
102	bad), and no sharing with exec.	113	bad), and no sharing with exec.
103		114
104	This module allows the main program to do a controlled fork, and allows	115	This module allows the main program to do a controlled fork, and allows
105	modules to exec processes safely at any time. When creating a custom	116	modules to exec processes safely at any time. When creating a custom
106	process pool you can take advantage of data sharing via fork without	117	process pool you can take advantage of data sharing via fork without
…		…
111	shared and what isn't, at all times.	122	shared and what isn't, at all times.
112		123
113	=item Exec'ing a new perl process might be difficult.	124	=item Exec'ing a new perl process might be difficult.
114		125
115	For example, it is not easy to find the correct path to the perl	126	For example, it is not easy to find the correct path to the perl
116	interpreter - C<$^X> might not be a perl interpreter at all.	127	interpreter - C<$^X> might not be a perl interpreter at all. Worse, there
		128	might not even be a perl binary installed on the system.
117		129
118	This module tries hard to identify the correct path to the perl	130	This module tries hard to identify the correct path to the perl
119	interpreter. With a cooperative main program, exec'ing the interpreter	131	interpreter. With a cooperative main program, exec'ing the interpreter
120	might not even be necessary, but even without help from the main program,	132	might not even be necessary, but even without help from the main program,
121	it will still work when used from a module.	133	it will still work when used from a module.
…		…
127	and modules are no longer loadable because they refer to a different	139	and modules are no longer loadable because they refer to a different
128	perl version, or parts of a distribution are newer than the ones already	140	perl version, or parts of a distribution are newer than the ones already
129	loaded.	141	loaded.
130		142
131	This module supports creating pre-initialised perl processes to be used as	143	This module supports creating pre-initialised perl processes to be used as
132	a template for new processes.	144	a template for new processes at a later time, e.g. for use in a process
		145	pool.
133		146
134	=item Forking might be impossible when a program is running.	147	=item Forking might be impossible when a program is running.
135		148
136	For example, POSIX makes it almost impossible to fork from a	149	For example, POSIX makes it almost impossible to fork from a
137	multi-threaded program while doing anything useful in the child - in	150	multi-threaded program while doing anything useful in the child - in
138	fact, if your perl program uses POSIX threads (even indirectly via	151	fact, if your perl program uses POSIX threads (even indirectly via
139	e.g. L<IO::AIO> or L<threads>), you cannot call fork on the perl level	152	e.g. L<IO::AIO> or L<threads>), you cannot call fork on the perl level
140	anymore without risking corruption issues on a number of operating	153	anymore without risking memory corruption or worse on a number of
141	systems.	154	operating systems.
142		155
143	This module can safely fork helper processes at any time, by calling	156	This module can safely fork helper processes at any time, by calling
144	fork+exec in C, in a POSIX-compatible way (via L<Proc::FastSpawn>).	157	fork+exec in C, in a POSIX-compatible way (via L<Proc::FastSpawn>).
145		158
146	=item Parallel processing with fork might be inconvenient or difficult	159	=item Parallel processing with fork might be inconvenient or difficult
…		…
165		178
166	=back	179	=back
167		180
168	=head1 EXAMPLES	181	=head1 EXAMPLES
169		182
		183	This is where the wall of text ends and code speaks.
		184
170	=head2 Create a single new process, tell it to run your worker function.	185	=head2 Create a single new process, tell it to run your worker function.
171		186
172	AnyEvent::Fork	187	AnyEvent::Fork
173	->new	188	->new
174	->require ("MyModule")	189	->require ("MyModule")
…		…
185		200
186	sub worker {	201	sub worker {
187	my ($slave_filehandle) = @_;	202	my ($slave_filehandle) = @_;
188		203
189	# now $slave_filehandle is connected to the $master_filehandle	204	# now $slave_filehandle is connected to the $master_filehandle
190	# in the original prorcess. have fun!	205	# in the original process. have fun!
191	}	206	}
192		207
193	=head2 Create a pool of server processes all accepting on the same socket.	208	=head2 Create a pool of server processes all accepting on the same socket.
194		209
195	# create listener socket	210	# create listener socket
…		…
256		271
257	my $stderr = $cv->recv;	272	my $stderr = $cv->recv;
258		273
259	=head2 For stingy users: put the worker code into a C<DATA> section.	274	=head2 For stingy users: put the worker code into a C<DATA> section.
260		275
261	When you want to be stingy with files, you cna put your code into the	276	When you want to be stingy with files, you can put your code into the
262	C<DATA> section of your module (or program):	277	C<DATA> section of your module (or program):
263		278
264	use AnyEvent::Fork;	279	use AnyEvent::Fork;
265		280
266	AnyEvent::Fork	281	AnyEvent::Fork
…		…
276		291
277	=head2 For stingy standalone programs: do not rely on external files at	292	=head2 For stingy standalone programs: do not rely on external files at
278	all.	293	all.
279		294
280	For single-file scripts it can be inconvenient to rely on external	295	For single-file scripts it can be inconvenient to rely on external
281	files - even when using < C<DATA> section, you still need to C<exec>	296	files - even when using a C<DATA> section, you still need to C<exec> an
282	an external perl interpreter, which might not be available when using	297	external perl interpreter, which might not be available when using
283	L<App::Staticperl>, L<Urlader> or L<PAR::Packer> for example.	298	L<App::Staticperl>, L<Urlader> or L<PAR::Packer> for example.
284		299
285	Two modules help here - L<AnyEvent::Fork::Early> forks a template process	300	Two modules help here - L<AnyEvent::Fork::Early> forks a template process
286	for all further calls to C<new_exec>, and L<AnyEvent::Fork::Template>	301	for all further calls to C<new_exec>, and L<AnyEvent::Fork::Template>
287	forks the main program as a template process.	302	forks the main program as a template process.
…		…
304	my ($fh, @args) = @_;	319	my ($fh, @args) = @_;
305	...	320	...
306	}	321	}
307		322
308	# now preserve everything so far as AnyEvent::Fork object	323	# now preserve everything so far as AnyEvent::Fork object
309	# in §TEMPLATE.	324	# in $TEMPLATE.
310	use AnyEvent::Fork::Template;	325	use AnyEvent::Fork::Template;
311		326
312	# do not put code outside of BEGIN blocks until here	327	# do not put code outside of BEGIN blocks until here
313		328
314	# now use the $TEMPLATE process in any way you like	329	# now use the $TEMPLATE process in any way you like
…		…
450	use AnyEvent;	465	use AnyEvent;
451	use AnyEvent::Util ();	466	use AnyEvent::Util ();
452		467
453	use IO::FDPass;	468	use IO::FDPass;
454		469
455	our $VERSION = 1.1;	470	our $VERSION = 1.31;
456		471
457	# the early fork template process	472	# the early fork template process
458	our $EARLY;	473	our $EARLY;
459		474
460	# the empty template process	475	# the empty template process
…		…
541		556
542	if ($pid eq 0) {	557	if ($pid eq 0) {
543	require AnyEvent::Fork::Serve;	558	require AnyEvent::Fork::Serve;
544	$AnyEvent::Fork::Serve::OWNER = $parent;	559	$AnyEvent::Fork::Serve::OWNER = $parent;
545	close $fh;	560	close $fh;
546	$0 = "$_[1] of $parent";	561	$0 = "$parent AnyEvent::Fork/exec";
547	AnyEvent::Fork::Serve::serve ($slave);	562	AnyEvent::Fork::Serve::serve ($slave);
548	exit 0;	563	exit 0;
549	} elsif (!$pid) {	564	} elsif (!$pid) {
550	die "AnyEvent::Fork::Early/Template: unable to fork template process: $!";	565	die "AnyEvent::Fork::Early/Template: unable to fork template process: $!";
551	}	566	}
…		…
609	The path to the perl interpreter is divined using various methods - first	624	The path to the perl interpreter is divined using various methods - first
610	C<$^X> is investigated to see if the path ends with something that looks	625	C<$^X> is investigated to see if the path ends with something that looks
611	as if it were the perl interpreter. Failing this, the module falls back to	626	as if it were the perl interpreter. Failing this, the module falls back to
612	using C<$Config::Config{perlpath}>.	627	using C<$Config::Config{perlpath}>.
613		628
614	The path to perl can also be overriden by setting the global variable	629	The path to perl can also be overridden by setting the global variable
615	C<$AnyEvent::Fork::PERL> - it's value will be used for all subsequent	630	C<$AnyEvent::Fork::PERL> - it's value will be used for all subsequent
616	invocations.	631	invocations.
617		632
618	=cut	633	=cut
619		634
…		…
625	return $EARLY->fork	640	return $EARLY->fork
626	if $EARLY;	641	if $EARLY;
627		642
628	unless (defined $PERL) {	643	unless (defined $PERL) {
629	# first find path of perl	644	# first find path of perl
630	my $perl = $;	645	my $perl = $^X;
631		646
632	# first we try $^X, but the path must be absolute (always on win32), and end in sth.	647	# first we try $^X, but the path must be absolute (always on win32), and end in sth.
633	# that looks like perl. this obviously only works for posix and win32	648	# that looks like perl. this obviously only works for posix and win32
634	unless (	649	unless (
635	($^O eq "MSWin32" \|\| $perl =~ m%^/%)	650	($^O eq "MSWin32" \|\| $perl =~ m%^/%)
…		…
658	my %env = %ENV;	673	my %env = %ENV;
659	$env{PERL5LIB} = join +($^O eq "MSWin32" ? ";" : ":"), grep !ref, @INC;	674	$env{PERL5LIB} = join +($^O eq "MSWin32" ? ";" : ":"), grep !ref, @INC;
660		675
661	my $pid = Proc::FastSpawn::spawn (	676	my $pid = Proc::FastSpawn::spawn (
662	$PERL,	677	$PERL,
663	["perl", "-MAnyEvent::Fork::Serve", "-e", "AnyEvent::Fork::Serve::me", fileno $slave, $$],	678	[$PERL, "-MAnyEvent::Fork::Serve", "-e", "AnyEvent::Fork::Serve::me", fileno $slave, $$],
664	[map "$_=$env{$_}", keys %env],	679	[map "$_=$env{$_}", keys %env],
665	) or die "unable to spawn AnyEvent::Fork server: $!";	680	) or die "unable to spawn AnyEvent::Fork server: $!";
666		681
667	$self->_new ($fh, $pid)	682	$self->_new ($fh, $pid)
668	}	683	}
669		684
670	=item $pid = $proc->pid	685	=item $pid = $proc->pid
671		686
672	Returns the process id of the process I<iff it is a direct child of the	687	Returns the process id of the process I<iff it is a direct child of the
673	process running AnyEvent::Fork>, and C<undef> otherwise.	688	process running AnyEvent::Fork>, and C<undef> otherwise. As a general
		689	rule (that you cannot rely upon), processes created via C<new_exec>,
		690	L<AnyEvent::Fork::Early> or L<AnyEvent::Fork::Template> are direct
		691	children, while all other processes are not.
674		692
675	Normally, only processes created via C<< AnyEvent::Fork->new_exec >> and	693	Or in other words, you do not normally have to take care of zombies for
676	L<AnyEvent::Fork::Template> are direct children, and you are responsible	694	processes created via C<new>, but when in doubt, or zombies are a problem,
677	to clean up their zombies when they die.	695	you need to check whether a process is a diretc child by calling this
678		696	method, and possibly creating a child watcher or reap it manually.
679	All other processes are not direct children, and will be cleaned up by
680	AnyEvent::Fork itself.
681		697
682	=cut	698	=cut
683		699
684	sub pid {	700	sub pid {
685	$_[0][PID]	701	$_[0][PID]
…		…
703	it via C<run>. This also gives you access to any arguments passed via the	719	it via C<run>. This also gives you access to any arguments passed via the
704	C<send_xxx> methods, such as file handles. See the L<use AnyEvent::Fork as	720	C<send_xxx> methods, such as file handles. See the L<use AnyEvent::Fork as
705	a faster fork+exec> example to see it in action.	721	a faster fork+exec> example to see it in action.
706		722
707	Returns the process object for easy chaining of method calls.	723	Returns the process object for easy chaining of method calls.
		724
		725	It's common to want to call an iniitalisation function with some
		726	arguments. Make sure you actually pass C<@_> to that function (for example
		727	by using C<&name> syntax), and do not just specify a function name:
		728
		729	$proc->eval ('&MyModule::init', $string1, $string2);
708		730
709	=cut	731	=cut
710		732
711	sub eval {	733	sub eval {
712	my ($self, $code, @args) = @_;	734	my ($self, $code, @args) = @_;
…		…
885	$self->_cmd (r => $func);	907	$self->_cmd (r => $func);
886	}	908	}
887		909
888	=back	910	=back
889		911
		912
		913	=head2 CHILD PROCESS INTERFACE
		914
		915	This module has a limited API for use in child processes.
		916
		917	=over 4
		918
		919	=item @args = AnyEvent::Fork::Serve::run_args
		920
		921	This function, which only exists before the C<run> method is called,
		922	returns the arguments that would be passed to the run function, and clears
		923	them.
		924
		925	This is mainly useful to get any file handles passed via C<send_fh>, but
		926	works for any arguments passed via C<< send_I<xxx> >> methods.
		927
		928	=back
		929
		930
890	=head2 EXPERIMENTAL METHODS	931	=head2 EXPERIMENTAL METHODS
891		932
892	These methods might go away completely or change behaviour, at any time.	933	These methods might go away completely or change behaviour, at any time.
893		934
894	=over 4	935	=over 4
…		…
1079	Cygwin perl is not supported at the moment due to some hilarious	1120	Cygwin perl is not supported at the moment due to some hilarious
1080	shortcomings of its API - see L<IO::FDPoll> for more details. If you never	1121	shortcomings of its API - see L<IO::FDPoll> for more details. If you never
1081	use C<send_fh> and always use C<new_exec> to create processes, it should	1122	use C<send_fh> and always use C<new_exec> to create processes, it should
1082	work though.	1123	work though.
1083		1124
		1125	=head1 USING AnyEvent::Fork IN SUBPROCESSES
		1126
		1127	AnyEvent::Fork itself cannot generally be used in subprocesses. As long as
		1128	only one process ever forks new processes, sharing the template processes
		1129	is possible (you could use a pipe as a lock by writing a byte into it to
		1130	unlock, and reading the byte to lock for example)
		1131
		1132	To make concurrent calls possible after fork, you should get rid of the
		1133	template and early fork processes. AnyEvent::Fork will create a new
		1134	template process as needed.
		1135
		1136	undef $AnyEvent::Fork::EARLY;
		1137	undef $AnyEvent::Fork::TEMPLATE;
		1138
		1139	It doesn't matter whether you get rid of them in the parent or child after
		1140	a fork.
		1141
1084	=head1 SEE ALSO	1142	=head1 SEE ALSO
1085		1143
1086	L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all	1144	L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all
1087	(part of this distribution).	1145	(part of this distribution).
1088		1146

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork/Fork.pm (file contents): Revision 1.58 by root, Sun Aug 25 21:10:10 2013 UTC vs. Revision 1.71 by root, Wed Jul 25 22:35:00 2018 UTC

Diff Legend

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.58 by root, Sun Aug 25 21:10:10 2013 UTC vs.
Revision 1.71 by root, Wed Jul 25 22:35:00 2018 UTC