[ViewVC] Diff of: cvs/AnyEvent-Fork/README

Comparing AnyEvent-Fork/README (file contents):
Revision 1.7 by root, Sun Apr 21 12:26:00 2013 UTC vs.
Revision 1.12 by root, Wed Jan 26 16:44:16 2022 UTC

…		…
51	AnyEvent::Subprocess. There are modules that implement their own process	51	AnyEvent::Subprocess. There are modules that implement their own process
52	management, such as AnyEvent::DBI.	52	management, such as AnyEvent::DBI.
53		53
54	The problems that all these modules try to solve are real, however, none	54	The problems that all these modules try to solve are real, however, none
55	of them (from what I have seen) tackle the very real problems of	55	of them (from what I have seen) tackle the very real problems of
56	unwanted memory sharing, efficiency, not being able to use event	56	unwanted memory sharing, efficiency or not being able to use event
57	processing or similar modules in the processes they create.	57	processing, GUI toolkits or similar modules in the processes they
		58	create.
58		59
59	This module doesn't try to replace any of them - instead it tries to	60	This module doesn't try to replace any of them - instead it tries to
60	solve the problem of creating processes with a minimum of fuss and	61	solve the problem of creating processes with a minimum of fuss and
61	overhead (and also luxury). Ideally, most of these would use	62	overhead (and also luxury). Ideally, most of these would use
62	AnyEvent::Fork internally, except they were written before AnyEvent:Fork	63	AnyEvent::Fork internally, except they were written before AnyEvent:Fork
…		…
79	vfork where possible. This gives the speed of vfork, with the	80	vfork where possible. This gives the speed of vfork, with the
80	flexibility of fork.	81	flexibility of fork.
81		82
82	Forking usually creates a copy-on-write copy of the parent process.	83	Forking usually creates a copy-on-write copy of the parent process.
83	For example, modules or data files that are loaded will not use	84	For example, modules or data files that are loaded will not use
84	additional memory after a fork. When exec'ing a new process, modules	85	additional memory after a fork. Exec'ing a new process, in contrast,
85	and data files might need to be loaded again, at extra CPU and	86	means modules and data files might need to be loaded again, at extra
86	memory cost. But when forking, literally all data structures are	87	CPU and memory cost.
		88
		89	But when forking, you still create a copy of your data structures -
87	copied - if the program frees them and replaces them by new data,	90	if the program frees them and replaces them by new data, the child
88	the child processes will retain the old version even if it isn't	91	processes will retain the old version even if it isn't used, which
89	used, which can suddenly and unexpectedly increase memory usage when	92	can suddenly and unexpectedly increase memory usage when freeing
90	freeing memory.	93	memory.
91		94
		95	For example, Gtk2::CV is an image viewer optimised for large
		96	directories (millions of pictures). It also forks subprocesses for
		97	thumbnail generation, which inherit the data structure that stores
		98	all file information. If the user changes the directory, it gets
		99	freed in the main process, leaving a copy in the thumbnailer
		100	processes. This can lead to many times the memory usage that would
		101	actually be required. The solution is to fork early (and being
		102	unable to dynamically generate more subprocesses or do this from a
		103	module)... or to use <AnyEvent:Fork>.
		104
92	The trade-off is between more sharing with fork (which can be good	105	There is a trade-off between more sharing with fork (which can be
93	or bad), and no sharing with exec.	106	good or bad), and no sharing with exec.
94		107
95	This module allows the main program to do a controlled fork, and	108	This module allows the main program to do a controlled fork, and
96	allows modules to exec processes safely at any time. When creating a	109	allows modules to exec processes safely at any time. When creating a
97	custom process pool you can take advantage of data sharing via fork	110	custom process pool you can take advantage of data sharing via fork
98	without risking to share large dynamic data structures that will	111	without risking to share large dynamic data structures that will
…		…
101	In other words, this module puts you into control over what is being	114	In other words, this module puts you into control over what is being
102	shared and what isn't, at all times.	115	shared and what isn't, at all times.
103		116
104	Exec'ing a new perl process might be difficult.	117	Exec'ing a new perl process might be difficult.
105	For example, it is not easy to find the correct path to the perl	118	For example, it is not easy to find the correct path to the perl
106	interpreter - $^X might not be a perl interpreter at all.	119	interpreter - $^X might not be a perl interpreter at all. Worse,
		120	there might not even be a perl binary installed on the system.
107		121
108	This module tries hard to identify the correct path to the perl	122	This module tries hard to identify the correct path to the perl
109	interpreter. With a cooperative main program, exec'ing the	123	interpreter. With a cooperative main program, exec'ing the
110	interpreter might not even be necessary, but even without help from	124	interpreter might not even be necessary, but even without help from
111	the main program, it will still work when used from a module.	125	the main program, it will still work when used from a module.
…		…
116	and modules are no longer loadable because they refer to a different	130	and modules are no longer loadable because they refer to a different
117	perl version, or parts of a distribution are newer than the ones	131	perl version, or parts of a distribution are newer than the ones
118	already loaded.	132	already loaded.
119		133
120	This module supports creating pre-initialised perl processes to be	134	This module supports creating pre-initialised perl processes to be
121	used as a template for new processes.	135	used as a template for new processes at a later time, e.g. for use
		136	in a process pool.
122		137
123	Forking might be impossible when a program is running.	138	Forking might be impossible when a program is running.
124	For example, POSIX makes it almost impossible to fork from a	139	For example, POSIX makes it almost impossible to fork from a
125	multi-threaded program while doing anything useful in the child - in	140	multi-threaded program while doing anything useful in the child - in
126	fact, if your perl program uses POSIX threads (even indirectly via	141	fact, if your perl program uses POSIX threads (even indirectly via
127	e.g. IO::AIO or threads), you cannot call fork on the perl level	142	e.g. IO::AIO or threads), you cannot call fork on the perl level
128	anymore without risking corruption issues on a number of operating	143	anymore without risking memory corruption or worse on a number of
129	systems.	144	operating systems.
130		145
131	This module can safely fork helper processes at any time, by calling	146	This module can safely fork helper processes at any time, by calling
132	fork+exec in C, in a POSIX-compatible way (via Proc::FastSpawn).	147	fork+exec in C, in a POSIX-compatible way (via Proc::FastSpawn).
133		148
134	Parallel processing with fork might be inconvenient or difficult to	149	Parallel processing with fork might be inconvenient or difficult to
…		…
150	is still safe to do so) - all other processes are created via	165	is still safe to do so) - all other processes are created via
151	fork+exec, which makes it possible to use modules such as event	166	fork+exec, which makes it possible to use modules such as event
152	loops or window interfaces safely.	167	loops or window interfaces safely.
153		168
154	EXAMPLES	169	EXAMPLES
		170	This is where the wall of text ends and code speaks.
		171
155	Create a single new process, tell it to run your worker function.	172	Create a single new process, tell it to run your worker function.
156	AnyEvent::Fork	173	AnyEvent::Fork
157	->new	174	->new
158	->require ("MyModule")	175	->require ("MyModule")
159	->run ("MyModule::worker, sub {	176	->run ("MyModule::worker, sub {
…		…
169		186
170	sub worker {	187	sub worker {
171	my ($slave_filehandle) = @_;	188	my ($slave_filehandle) = @_;
172		189
173	# now $slave_filehandle is connected to the $master_filehandle	190	# now $slave_filehandle is connected to the $master_filehandle
174	# in the original prorcess. have fun!	191	# in the original process. have fun!
175	}	192	}
176		193
177	Create a pool of server processes all accepting on the same socket.	194	Create a pool of server processes all accepting on the same socket.
178	# create listener socket	195	# create listener socket
179	my $listener = ...;	196	my $listener = ...;
…		…
238	->run ("run", my $cv = AE::cv);	255	->run ("run", my $cv = AE::cv);
239		256
240	my $stderr = $cv->recv;	257	my $stderr = $cv->recv;
241		258
242	For stingy users: put the worker code into a "DATA" section.	259	For stingy users: put the worker code into a "DATA" section.
243	When you want to be stingy with files, you cna put your code into the	260	When you want to be stingy with files, you can put your code into the
244	"DATA" section of your module (or program):	261	"DATA" section of your module (or program):
245		262
246	use AnyEvent::Fork;	263	use AnyEvent::Fork;
247		264
248	AnyEvent::Fork	265	AnyEvent::Fork
…		…
257	}	274	}
258		275
259	For stingy standalone programs: do not rely on external files at	276	For stingy standalone programs: do not rely on external files at
260	all.	277	all.
261	For single-file scripts it can be inconvenient to rely on external files	278	For single-file scripts it can be inconvenient to rely on external files
262	- even when using < "DATA" section, you still need to "exec" an external	279	- even when using a "DATA" section, you still need to "exec" an external
263	perl interpreter, which might not be available when using	280	perl interpreter, which might not be available when using
264	App::Staticperl, Urlader or PAR::Packer for example.	281	App::Staticperl, Urlader or PAR::Packer for example.
265		282
266	Two modules help here - AnyEvent::Fork::Early forks a template process	283	Two modules help here - AnyEvent::Fork::Early forks a template process
267	for all further calls to "new_exec", and AnyEvent::Fork::Template forks	284	for all further calls to "new_exec", and AnyEvent::Fork::Template forks
…		…
285	my ($fh, @args) = @_;	302	my ($fh, @args) = @_;
286	...	303	...
287	}	304	}
288		305
289	# now preserve everything so far as AnyEvent::Fork object	306	# now preserve everything so far as AnyEvent::Fork object
290	# in §TEMPLATE.	307	# in $TEMPLATE.
291	use AnyEvent::Fork::Template;	308	use AnyEvent::Fork::Template;
292		309
293	# do not put code outside of BEGIN blocks until here	310	# do not put code outside of BEGIN blocks until here
294		311
295	# now use the $TEMPLATE process in any way you like	312	# now use the $TEMPLATE process in any way you like
…		…
304	$cv->end;	321	$cv->end;
305	});	322	});
306	}	323	}
307	$cv->recv;	324	$cv->recv;
308		325
309	lhead1 CONCEPTS	326	CONCEPTS
310
311	This module can create new processes either by executing a new perl	327	This module can create new processes either by executing a new perl
312	process, or by forking from an existing "template" process.	328	process, or by forking from an existing "template" process.
313		329
314	All these processes are called "child processes" (whether they are	330	All these processes are called "child processes" (whether they are
315	direct children or not), while the process that manages them is called	331	direct children or not), while the process that manages them is called
…		…
440	You should use "new" whenever possible, except when having a	456	You should use "new" whenever possible, except when having a
441	template process around is unacceptable.	457	template process around is unacceptable.
442		458
443	The path to the perl interpreter is divined using various methods -	459	The path to the perl interpreter is divined using various methods -
444	first $^X is investigated to see if the path ends with something	460	first $^X is investigated to see if the path ends with something
445	that sounds as if it were the perl interpreter. Failing this, the	461	that looks as if it were the perl interpreter. Failing this, the
446	module falls back to using $Config::Config{perlpath}.	462	module falls back to using $Config::Config{perlpath}.
		463
		464	The path to perl can also be overridden by setting the global
		465	variable $AnyEvent::Fork::PERL - it's value will be used for all
		466	subsequent invocations.
447		467
448	$pid = $proc->pid	468	$pid = $proc->pid
449	Returns the process id of the process *iff it is a direct child of	469	Returns the process id of the process *iff it is a direct child of
450	the process running AnyEvent::Fork*, and "undef" otherwise.	470	the process running AnyEvent::Fork*, and "undef" otherwise. As a
		471	general rule (that you cannot rely upon), processes created via
		472	"new_exec", AnyEvent::Fork::Early or AnyEvent::Fork::Template are
		473	direct children, while all other processes are not.
451		474
452	Normally, only processes created via "AnyEvent::Fork->new_exec" and	475	Or in other words, you do not normally have to take care of zombies
453	AnyEvent::Fork::Template are direct children, and you are	476	for processes created via "new", but when in doubt, or zombies are a
454	responsible to clean up their zombies when they die.	477	problem, you need to check whether a process is a diretc child by
455		478	calling this method, and possibly creating a child watcher or reap
456	All other processes are not direct children, and will be cleaned up	479	it manually.
457	by AnyEvent::Fork itself.
458		480
459	$proc = $proc->eval ($perlcode, @args)	481	$proc = $proc->eval ($perlcode, @args)
460	Evaluates the given $perlcode as ... Perl code, while setting @_ to	482	Evaluates the given $perlcode as ... Perl code, while setting @_ to
461	the strings specified by @args, in the "main" package.	483	the strings specified by @args, in the "main" package (so you can
		484	access the args using $_[0] and so on, but not using implicit "shit"
		485	as the latter works on @ARGV).
462		486
463	This call is meant to do any custom initialisation that might be	487	This call is meant to do any custom initialisation that might be
464	required (for example, the "require" method uses it). It's not	488	required (for example, the "require" method uses it). It's not
465	supposed to be used to completely take over the process, use "run"	489	supposed to be used to completely take over the process, use "run"
466	for that.	490	for that.
…		…
477	See the "use AnyEvent::Fork as a faster fork+exec" example to see it	501	See the "use AnyEvent::Fork as a faster fork+exec" example to see it
478	in action.	502	in action.
479		503
480	Returns the process object for easy chaining of method calls.	504	Returns the process object for easy chaining of method calls.
481		505
		506	It's common to want to call an iniitalisation function with some
		507	arguments. Make sure you actually pass @_ to that function (for
		508	example by using &name syntax), and do not just specify a function
		509	name:
		510
		511	$proc->eval ('&MyModule::init', $string1, $string2);
		512
482	$proc = $proc->require ($module, ...)	513	$proc = $proc->require ($module, ...)
483	Tries to load the given module(s) into the process	514	Tries to load the given module(s) into the process
484		515
485	Returns the process object for easy chaining of method calls.	516	Returns the process object for easy chaining of method calls.
486		517
…		…
544	Even if not used otherwise, the socket can be a good indicator for	575	Even if not used otherwise, the socket can be a good indicator for
545	the existence of the process - if the other process exits, you get a	576	the existence of the process - if the other process exits, you get a
546	readable event on it, because exiting the process closes the socket	577	readable event on it, because exiting the process closes the socket
547	(if it didn't create any children using fork).	578	(if it didn't create any children using fork).
548		579
		580	Compatibility to AnyEvent::Fork::Remote
		581	If you want to write code that works with both this module and
		582	AnyEvent::Fork::Remote, you need to write your code so that it
		583	assumes there are two file handles for communications, which
		584	might not be unix domain sockets. The "run" function should
		585	start like this:
		586
		587	sub run {
		588	my ($rfh, @args) = @_; # @args is your normal arguments
		589	my $wfh = fileno $rfh ? $rfh : *STDOUT;
		590
		591	# now use $rfh for reading and $wfh for writing
		592	}
		593
		594	This checks whether the passed file handle is, in fact, the
		595	process "STDIN" handle. If it is, then the function was invoked
		596	visa AnyEvent::Fork::Remote, so STDIN should be used for reading
		597	and "STDOUT" should be used for writing.
		598
		599	In all other cases, the function was called via this module, and
		600	there is only one file handle that should be sued for reading
		601	and writing.
		602
549	Example: create a template for a process pool, pass a few strings,	603	Example: create a template for a process pool, pass a few strings,
550	some file handles, then fork, pass one more string, and run some	604	some file handles, then fork, pass one more string, and run some
551	code.	605	code.
552		606
553	my $pool = AnyEvent::Fork	607	my $pool = AnyEvent::Fork
…		…
576	my ($fh, $str1, $str2, $fh1, $fh2, $str3) = @_;	630	my ($fh, $str1, $str2, $fh1, $fh2, $str3) = @_;
577		631
578	print scalar <$fh>; # prints "hi #1\n" and "hi #2\n" in any order	632	print scalar <$fh>; # prints "hi #1\n" and "hi #2\n" in any order
579	}	633	}
580		634
		635	CHILD PROCESS INTERFACE
		636	This module has a limited API for use in child processes.
		637
		638	@args = AnyEvent::Fork::Serve::run_args
		639	This function, which only exists before the "run" method is called,
		640	returns the arguments that would be passed to the run function, and
		641	clears them.
		642
		643	This is mainly useful to get any file handles passed via "send_fh",
		644	but works for any arguments passed via "send_xxx" methods.
		645
		646	EXPERIMENTAL METHODS
		647	These methods might go away completely or change behaviour, at any time.
		648
581	$proc->to_fh ($cb->($fh)) # EXPERIMENTAL, MIGHT BE REMOVED	649	$proc->to_fh ($cb->($fh)) # EXPERIMENTAL, MIGHT BE REMOVED
582	Flushes all commands out to the process and then calls the callback	650	Flushes all commands out to the process and then calls the callback
583	with the communications socket.	651	with the communications socket.
584		652
585	The process object becomes unusable on return from this function -	653	The process object becomes unusable on return from this function -
586	any further method calls result in undefined behaviour.	654	any further method calls result in undefined behaviour.
587		655
588	The point of this method is to give you a file handle thta you cna	656	The point of this method is to give you a file handle that you can
589	pass to another process. In that other process, you can call	657	pass to another process. In that other process, you can call
590	"new_from_fh AnyEvent::Fork" to create a new "AnyEvent::Fork" object	658	"new_from_fh AnyEvent::Fork $fh" to create a new "AnyEvent::Fork"
591	from it, thereby effectively passing a fork object to another	659	object from it, thereby effectively passing a fork object to another
592	process.	660	process.
593		661
594	new_from_fh AnyEvent::Fork $fh # EXPERIMENTAL, MIGHT BE REMOVED	662	new_from_fh AnyEvent::Fork $fh # EXPERIMENTAL, MIGHT BE REMOVED
595	Takes a file handle originally rceeived by the "to_fh" method and	663	Takes a file handle originally rceeived by the "to_fh" method and
596	creates a new "AnyEvent:Fork" object. The child process itself will	664	creates a new "AnyEvent:Fork" object. The child process itself will
…		…
734	Cygwin perl is not supported at the moment due to some hilarious	802	Cygwin perl is not supported at the moment due to some hilarious
735	shortcomings of its API - see IO::FDPoll for more details. If you never	803	shortcomings of its API - see IO::FDPoll for more details. If you never
736	use "send_fh" and always use "new_exec" to create processes, it should	804	use "send_fh" and always use "new_exec" to create processes, it should
737	work though.	805	work though.
738		806
		807	USING AnyEvent::Fork IN SUBPROCESSES
		808	AnyEvent::Fork itself cannot generally be used in subprocesses. As long
		809	as only one process ever forks new processes, sharing the template
		810	processes is possible (you could use a pipe as a lock by writing a byte
		811	into it to unlock, and reading the byte to lock for example)
		812
		813	To make concurrent calls possible after fork, you should get rid of the
		814	template and early fork processes. AnyEvent::Fork will create a new
		815	template process as needed.
		816
		817	undef $AnyEvent::Fork::EARLY;
		818	undef $AnyEvent::Fork::TEMPLATE;
		819
		820	It doesn't matter whether you get rid of them in the parent or child
		821	after a fork.
		822
739	SEE ALSO	823	SEE ALSO
740	AnyEvent::Fork::Early, to avoid executing a perl interpreter at all	824	AnyEvent::Fork::Early, to avoid executing a perl interpreter at all
741	(part of this distribution).	825	(part of this distribution).
742		826
743	AnyEvent::Fork::Template, to create a process by forking the main	827	AnyEvent::Fork::Template, to create a process by forking the main
744	program at a convenient time (part of this distribution).	828	program at a convenient time (part of this distribution).
745		829
		830	AnyEvent::Fork::Remote, for another way to create processes that is
		831	mostly compatible to this module and modules building on top of it, but
		832	works better with remote processes.
		833
746	AnyEvent::Fork::RPC, for simple RPC to child processes (on CPAN).	834	AnyEvent::Fork::RPC, for simple RPC to child processes (on CPAN).
747		835
748	AnyEvent::Fork::Pool, for simple worker process pool (on CPAN).	836	AnyEvent::Fork::Pool, for simple worker process pool (on CPAN).
749		837
750	AUTHOR AND CONTACT INFORMATION	838	AUTHOR AND CONTACT INFORMATION

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork/README (file contents): Revision 1.7 by root, Sun Apr 21 12:26:00 2013 UTC vs. Revision 1.12 by root, Wed Jan 26 16:44:16 2022 UTC

Diff Legend

Comparing AnyEvent-Fork/README (file contents):
Revision 1.7 by root, Sun Apr 21 12:26:00 2013 UTC vs.
Revision 1.12 by root, Wed Jan 26 16:44:16 2022 UTC