[ViewVC] Diff of: cvs/AnyEvent-Fork/Fork.pm

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.25 by root, Sat Apr 6 08:55:16 2013 UTC vs.
Revision 1.37 by root, Sat Apr 6 20:06:39 2013 UTC

…		…
125	becomes very hard to use the event loop from a child program, as the	125	becomes very hard to use the event loop from a child program, as the
126	watchers already exist but are only meaningful in the parent. Worse, a	126	watchers already exist but are only meaningful in the parent. Worse, a
127	module might want to use such a module, not knowing whether another module	127	module might want to use such a module, not knowing whether another module
128	or the main program also does, leading to problems.	128	or the main program also does, leading to problems.
129		129
		130	Apart from event loops, graphical toolkits also commonly fall into the
		131	"unsafe module" category, or just about anything that communicates with
		132	the external world, such as network libraries and file I/O modules, which
		133	usually don't like being copied and then allowed to continue in two
		134	processes.
		135
130	With this module only the main program is allowed to create new processes	136	With this module only the main program is allowed to create new processes
131	by forking (because only the main program can know when it is still safe	137	by forking (because only the main program can know when it is still safe
132	to do so) - all other processes are created via fork+exec, which makes it	138	to do so) - all other processes are created via fork+exec, which makes it
133	possible to use modules such as event loops or window interfaces safely.	139	possible to use modules such as event loops or window interfaces safely.
134		140
…		…
146		152
147	# now $master_filehandle is connected to the	153	# now $master_filehandle is connected to the
148	# $slave_filehandle in the new process.	154	# $slave_filehandle in the new process.
149	});	155	});
150		156
151	# MyModule::worker might look like this	157	C<MyModule> might look like this:
		158
		159	package MyModule;
		160
152	sub MyModule::worker {	161	sub worker {
153	my ($slave_filehandle) = @_;	162	my ($slave_filehandle) = @_;
154		163
155	# now $slave_filehandle is connected to the $master_filehandle	164	# now $slave_filehandle is connected to the $master_filehandle
156	# in the original prorcess. have fun!	165	# in the original prorcess. have fun!
157	}	166	}
…		…
176	}	185	}
177		186
178	# now do other things - maybe use the filehandle provided by run	187	# now do other things - maybe use the filehandle provided by run
179	# to wait for the processes to die. or whatever.	188	# to wait for the processes to die. or whatever.
180		189
181	# My::Server::run might look like this	190	C<My::Server> might look like this:
182	sub My::Server::run {	191
		192	package My::Server;
		193
		194	sub run {
183	my ($slave, $listener, $id) = @_;	195	my ($slave, $listener, $id) = @_;
184		196
185	close $slave; # we do not use the socket, so close it to save resources	197	close $slave; # we do not use the socket, so close it to save resources
186		198
187	# we could go ballistic and use e.g. AnyEvent here, or IO::AIO,	199	# we could go ballistic and use e.g. AnyEvent here, or IO::AIO,
…		…
191	}	203	}
192	}	204	}
193		205
194	=head2 use AnyEvent::Fork as a faster fork+exec	206	=head2 use AnyEvent::Fork as a faster fork+exec
195		207
196	This runs /bin/echo hi, with stdout redirected to /tmp/log and stderr to	208	This runs C</bin/echo hi>, with stdandard output redirected to /tmp/log
197	the communications socket. It is usually faster than fork+exec, but still	209	and standard error redirected to the communications socket. It is usually
198	let's you prepare the environment.	210	faster than fork+exec, but still lets you prepare the environment.
199		211
200	open my $output, ">/tmp/log" or die "$!";	212	open my $output, ">/tmp/log" or die "$!";
201		213
202	AnyEvent::Fork	214	AnyEvent::Fork
203	->new	215	->new
…		…
303	my ($fork_fh) = @_;	315	my ($fork_fh) = @_;
304	});	316	});
305		317
306	=back	318	=back
307		319
308	=head1 FUNCTIONS	320	=head1 THE C<AnyEvent::Fork> CLASS
		321
		322	This module exports nothing, and only implements a single class -
		323	C<AnyEvent::Fork>.
		324
		325	There are two class constructors that both create new processes - C<new>
		326	and C<new_exec>. The C<fork> method creates a new process by forking an
		327	existing one and could be considered a third constructor.
		328
		329	Most of the remaining methods deal with preparing the new process, by
		330	loading code, evaluating code and sending data to the new process. They
		331	usually return the process object, so you can chain method calls.
		332
		333	If a process object is destroyed before calling its C<run> method, then
		334	the process simply exits. After C<run> is called, all responsibility is
		335	passed to the specified function.
		336
		337	As long as there is any outstanding work to be done, process objects
		338	resist being destroyed, so there is no reason to store them unless you
		339	need them later - configure and forget works just fine.
309		340
310	=over 4	341	=over 4
311		342
312	=cut	343	=cut
313		344
…		…
323	use IO::FDPass;	354	use IO::FDPass;
324		355
325	our $VERSION = 0.5;	356	our $VERSION = 0.5;
326		357
327	our $PERL; # the path to the perl interpreter, deduces with various forms of magic	358	our $PERL; # the path to the perl interpreter, deduces with various forms of magic
328
329	=item my $pool = new AnyEvent::Fork key => value...
330
331	Create a new process pool. The following named parameters are supported:
332		359
333	=over 4	360	=over 4
334		361
335	=back	362	=back
336		363
…		…
432	Create a new "empty" perl interpreter process and returns its process	459	Create a new "empty" perl interpreter process and returns its process
433	object for further manipulation.	460	object for further manipulation.
434		461
435	The new process is forked from a template process that is kept around	462	The new process is forked from a template process that is kept around
436	for this purpose. When it doesn't exist yet, it is created by a call to	463	for this purpose. When it doesn't exist yet, it is created by a call to
437	C<new_exec> and kept around for future calls.	464	C<new_exec> first and then stays around for future calls.
438
439	When the process object is destroyed, it will release the file handle
440	that connects it with the new process. When the new process has not yet
441	called C<run>, then the process will exit. Otherwise, what happens depends
442	entirely on the code that is executed.
443		465
444	=cut	466	=cut
445		467
446	sub new {	468	sub new {
447	my $class = shift;	469	my $class = shift;
…		…
537	}	559	}
538		560
539	=item $pid = $proc->pid	561	=item $pid = $proc->pid
540		562
541	Returns the process id of the process I<iff it is a direct child of the	563	Returns the process id of the process I<iff it is a direct child of the
542	process> running AnyEvent::Fork, and C<undef> otherwise.	564	process running AnyEvent::Fork>, and C<undef> otherwise.
543		565
544	Normally, only processes created via C<< AnyEvent::Fork->new_exec >> and	566	Normally, only processes created via C<< AnyEvent::Fork->new_exec >> and
545	L<AnyEvent::Fork::Template> are direct children, and you are responsible	567	L<AnyEvent::Fork::Template> are direct children, and you are responsible
546	to clean up their zombies when they die.	568	to clean up their zombies when they die.
547		569
548	All other processes are not direct children, and will be cleaned up by	570	All other processes are not direct children, and will be cleaned up by
549	AnyEvent::Fork.	571	AnyEvent::Fork itself.
550		572
551	=cut	573	=cut
552		574
553	sub pid {	575	sub pid {
554	$_[0][0]	576	$_[0][0]
…		…
565		587
566	The code will usually be executed after this call returns, and there is no	588	The code will usually be executed after this call returns, and there is no
567	way to pass anything back to the calling process. Any evaluation errors	589	way to pass anything back to the calling process. Any evaluation errors
568	will be reported to stderr and cause the process to exit.	590	will be reported to stderr and cause the process to exit.
569		591
570	If you want to execute some code to take over the process (see the	592	If you want to execute some code (that isn't in a module) to take over the
571	"fork+exec" example in the SYNOPSIS), you should compile a function via	593	process, you should compile a function via C<eval> first, and then call
572	C<eval> first, and then call it via C<run>. This also gives you access to	594	it via C<run>. This also gives you access to any arguments passed via the
573	any arguments passed via the C<send_xxx> methods, such as file handles.	595	C<send_xxx> methods, such as file handles. See the L<use AnyEvent::Fork as
		596	a faster fork+exec> example to see it in action.
574		597
575	Returns the process object for easy chaining of method calls.	598	Returns the process object for easy chaining of method calls.
576		599
577	=cut	600	=cut
578		601
…		…
604	=item $proc = $proc->send_fh ($handle, ...)	627	=item $proc = $proc->send_fh ($handle, ...)
605		628
606	Send one or more file handles (I<not> file descriptors) to the process,	629	Send one or more file handles (I<not> file descriptors) to the process,
607	to prepare a call to C<run>.	630	to prepare a call to C<run>.
608		631
609	The process object keeps a reference to the handles until this is done,	632	The process object keeps a reference to the handles until they have
610	so you must not explicitly close the handles. This is most easily	633	been passed over to the process, so you must not explicitly close the
611	accomplished by simply not storing the file handles anywhere after passing	634	handles. This is most easily accomplished by simply not storing the file
612	them to this method.	635	handles anywhere after passing them to this method - when AnyEvent::Fork
		636	is finished using them, perl will automatically close them.
613		637
614	Returns the process object for easy chaining of method calls.	638	Returns the process object for easy chaining of method calls.
615		639
616	Example: pass a file handle to a process, and release it without	640	Example: pass a file handle to a process, and release it without
617	closing. It will be closed automatically when it is no longer used.	641	closing. It will be closed automatically when it is no longer used.
…		…
633	}	657	}
634		658
635	=item $proc = $proc->send_arg ($string, ...)	659	=item $proc = $proc->send_arg ($string, ...)
636		660
637	Send one or more argument strings to the process, to prepare a call to	661	Send one or more argument strings to the process, to prepare a call to
638	C<run>. The strings can be any octet string.	662	C<run>. The strings can be any octet strings.
639		663
640	The protocol is optimised to pass a moderate number of relatively short	664	The protocol is optimised to pass a moderate number of relatively short
641	strings - while you can pass up to 4GB of data in one go, this is more	665	strings - while you can pass up to 4GB of data in one go, this is more
642	meant to pass some ID information or other startup info, not big chunks of	666	meant to pass some ID information or other startup info, not big chunks of
643	data.	667	data.
…		…
659	Enter the function specified by the function name in C<$func> in the	683	Enter the function specified by the function name in C<$func> in the
660	process. The function is called with the communication socket as first	684	process. The function is called with the communication socket as first
661	argument, followed by all file handles and string arguments sent earlier	685	argument, followed by all file handles and string arguments sent earlier
662	via C<send_fh> and C<send_arg> methods, in the order they were called.	686	via C<send_fh> and C<send_arg> methods, in the order they were called.
663		687
		688	The process object becomes unusable on return from this function - any
		689	further method calls result in undefined behaviour.
		690
664	The function name should be fully qualified, but if it isn't, it will be	691	The function name should be fully qualified, but if it isn't, it will be
665	looked up in the main package.	692	looked up in the C<main> package.
666		693
667	If the called function returns, doesn't exist, or any error occurs, the	694	If the called function returns, doesn't exist, or any error occurs, the
668	process exits.	695	process exits.
669		696
670	Preparing the process is done in the background - when all commands have	697	Preparing the process is done in the background - when all commands have
671	been sent, the callback is invoked with the local communications socket	698	been sent, the callback is invoked with the local communications socket
672	as argument. At this point you can start using the socket in any way you	699	as argument. At this point you can start using the socket in any way you
673	like.	700	like.
674
675	The process object becomes unusable on return from this function - any
676	further method calls result in undefined behaviour.
677		701
678	If the communication socket isn't used, it should be closed on both sides,	702	If the communication socket isn't used, it should be closed on both sides,
679	to save on kernel memory.	703	to save on kernel memory.
680		704
681	The socket is non-blocking in the parent, and blocking in the newly	705	The socket is non-blocking in the parent, and blocking in the newly
…		…
756	479 vfork+execs per second, using AnyEvent::Fork->new_exec	780	479 vfork+execs per second, using AnyEvent::Fork->new_exec
757		781
758	So how can C<< AnyEvent->new >> be faster than a standard fork, even	782	So how can C<< AnyEvent->new >> be faster than a standard fork, even
759	though it uses the same operations, but adds a lot of overhead?	783	though it uses the same operations, but adds a lot of overhead?
760		784
761	The difference is simply the process size: forking the 6MB process takes	785	The difference is simply the process size: forking the 5MB process takes
762	so much longer than forking the 2.5MB template process that the overhead	786	so much longer than forking the 2.5MB template process that the extra
763	introduced is canceled out.	787	overhead introduced is canceled out.
764		788
765	If the benchmark process grows, the normal fork becomes even slower:	789	If the benchmark process grows, the normal fork becomes even slower:
766		790
767	1340 new processes, manual fork in a 20MB process	791	1340 new processes, manual fork of a 20MB process
768	731 new processes, manual fork in a 200MB process	792	731 new processes, manual fork of a 200MB process
769	235 new processes, manual fork in a 2000MB process	793	235 new processes, manual fork of a 2000MB process
770		794
771	What that means (to me) is that I can use this module without having a	795	What that means (to me) is that I can use this module without having a bad
772	very bad conscience because of the extra overhead required to start new	796	conscience because of the extra overhead required to start new processes.
773	processes.
774		797
775	=head1 TYPICAL PROBLEMS	798	=head1 TYPICAL PROBLEMS
776		799
777	This section lists typical problems that remain. I hope by recognising	800	This section lists typical problems that remain. I hope by recognising
778	them, most can be avoided.	801	them, most can be avoided.
779		802
780	=over 4	803	=over 4
781		804
782	=item "leaked" file descriptors for exec'ed processes	805	=item leaked file descriptors for exec'ed processes
783		806
784	POSIX systems inherit file descriptors by default when exec'ing a new	807	POSIX systems inherit file descriptors by default when exec'ing a new
785	process. While perl itself laudably sets the close-on-exec flags on new	808	process. While perl itself laudably sets the close-on-exec flags on new
786	file handles, most C libraries don't care, and even if all cared, it's	809	file handles, most C libraries don't care, and even if all cared, it's
787	often not possible to set the flag in a race-free manner.	810	often not possible to set the flag in a race-free manner.
…		…
807	libraries or the code that leaks those file descriptors.	830	libraries or the code that leaks those file descriptors.
808		831
809	Fortunately, most of these leaked descriptors do no harm, other than	832	Fortunately, most of these leaked descriptors do no harm, other than
810	sitting on some resources.	833	sitting on some resources.
811		834
812	=item "leaked" file descriptors for fork'ed processes	835	=item leaked file descriptors for fork'ed processes
813		836
814	Normally, L<AnyEvent::Fork> does start new processes by exec'ing them,	837	Normally, L<AnyEvent::Fork> does start new processes by exec'ing them,
815	which closes file descriptors not marked for being inherited.	838	which closes file descriptors not marked for being inherited.
816		839
817	However, L<AnyEvent::Fork::Early> and L<AnyEvent::Fork::Template> offer	840	However, L<AnyEvent::Fork::Early> and L<AnyEvent::Fork::Template> offer
…		…
826		849
827	The solution is to either not load these modules before use'ing	850	The solution is to either not load these modules before use'ing
828	L<AnyEvent::Fork::Early> or L<AnyEvent::Fork::Template>, or to delay	851	L<AnyEvent::Fork::Early> or L<AnyEvent::Fork::Template>, or to delay
829	initialising them, for example, by calling C<init Gtk2> manually.	852	initialising them, for example, by calling C<init Gtk2> manually.
830		853
831	=item exit runs destructors	854	=item exiting calls object destructors
832		855
833	This only applies to users of Lc<AnyEvent::Fork:Early> and	856	This only applies to users of Lc<AnyEvent::Fork:Early> and
834	L<AnyEvent::Fork::Template>.	857	L<AnyEvent::Fork::Template>.
835		858
836	When a process created by AnyEvent::Fork exits, it might do so by calling	859	When a process created by AnyEvent::Fork exits, it might do so by calling
…		…
858	to make it so, mostly due to the bloody broken perl that nobody seems to	881	to make it so, mostly due to the bloody broken perl that nobody seems to
859	care about. The fork emulation is a bad joke - I have yet to see something	882	care about. The fork emulation is a bad joke - I have yet to see something
860	useful that you can do with it without running into memory corruption	883	useful that you can do with it without running into memory corruption
861	issues or other braindamage. Hrrrr.	884	issues or other braindamage. Hrrrr.
862		885
863	Cygwin perl is not supported at the moment, as it should implement fd	886	Cygwin perl is not supported at the moment due to some hilarious
864	passing, but doesn't, and rolling my own is hard, as cygwin doesn't	887	shortcomings of its API - see L<IO::FDPoll> for more details.
865	support enough functionality to do it.
866		888
867	=head1 SEE ALSO	889	=head1 SEE ALSO
868		890
869	L<AnyEvent::Fork::Early> (to avoid executing a perl interpreter),	891	L<AnyEvent::Fork::Early> (to avoid executing a perl interpreter),
870	L<AnyEvent::Fork::Template> (to create a process by forking the main	892	L<AnyEvent::Fork::Template> (to create a process by forking the main

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork/Fork.pm (file contents): Revision 1.25 by root, Sat Apr 6 08:55:16 2013 UTC vs. Revision 1.37 by root, Sat Apr 6 20:06:39 2013 UTC

Diff Legend

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.25 by root, Sat Apr 6 08:55:16 2013 UTC vs.
Revision 1.37 by root, Sat Apr 6 20:06:39 2013 UTC