ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork/Fork.pm
(Generate patch)

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.48 by root, Fri Apr 19 10:51:41 2013 UTC vs.
Revision 1.57 by root, Sun Aug 25 17:38:43 2013 UTC

… …
38 38
39If you need some form of RPC, you could use the L<AnyEvent::Fork::RPC> 39If you need some form of RPC, you could use the L<AnyEvent::Fork::RPC>
40companion module, which adds simple RPC/job queueing to a process created 40companion module, which adds simple RPC/job queueing to a process created
41by this module. 41by this module.
42 42
43And if you need some automatic process pool management on top of
44L<AnyEvent::Fork::RPC>, you can look at the L<AnyEvent::Fork::Pool>
45companion module.
46
43Or you can implement it yourself in whatever way you like, use some 47Or you can implement it yourself in whatever way you like: use some
44message-passing module such as L<AnyEvent::MP>, some pipe such as 48message-passing module such as L<AnyEvent::MP>, some pipe such as
45L<AnyEvent::ZeroMQ>, use L<AnyEvent::Handle> on both sides to send 49L<AnyEvent::ZeroMQ>, use L<AnyEvent::Handle> on both sides to send
46e.g. JSON or Storable messages, and so on. 50e.g. JSON or Storable messages, and so on.
47 51
48=head2 COMPARISON TO OTHER MODULES 52=head2 COMPARISON TO OTHER MODULES
… …
250 ->send_arg ("/bin/echo", "hi") 254 ->send_arg ("/bin/echo", "hi")
251 ->run ("run", my $cv = AE::cv); 255 ->run ("run", my $cv = AE::cv);
252 256
253 my $stderr = $cv->recv; 257 my $stderr = $cv->recv;
254 258
259=head2 For stingy users: put the worker code into a C<DATA> section.
260
261When you want to be stingy with files, you cna put your code into the
262C<DATA> section of your module (or program):
263
264 use AnyEvent::Fork;
265
266 AnyEvent::Fork
267 ->new
268 ->eval (do { local $/; <DATA> })
269 ->run ("doit", sub { ... });
270
271 __DATA__
272
273 sub doit {
274 ... do something!
275 }
276
277=head2 For stingy standalone programs: do not rely on external files at
278all.
279
280For single-file scripts it can be inconvenient to rely on external
281files - even when using < C<DATA> section, you still need to C<exec>
282an external perl interpreter, which might not be available when using
283L<App::Staticperl>, L<Urlader> or L<PAR::Packer> for example.
284
285Two modules help here - L<AnyEvent::Fork::Early> forks a template process
286for all further calls to C<new_exec>, and L<AnyEvent::Fork::Template>
287forks the main program as a template process.
288
289Here is how your main program should look like:
290
291 #! perl
292
293 # optional, as the very first thing.
294 # in case modules want to create their own processes.
295 use AnyEvent::Fork::Early;
296
297 # next, load all modules you need in your template process
298 use Example::My::Module
299 use Example::Whatever;
300
301 # next, put your run function definition and anything else you
302 # need, but do not use code outside of BEGIN blocks.
303 sub worker_run {
304 my ($fh, @args) = @_;
305 ...
306 }
307
308 # now preserve everything so far as AnyEvent::Fork object
309 # in §TEMPLATE.
310 use AnyEvent::Fork::Template;
311
312 # do not put code outside of BEGIN blocks until here
313
314 # now use the $TEMPLATE process in any way you like
315
316 # for example: create 10 worker processes
317 my @worker;
318 my $cv = AE::cv;
319 for (1..10) {
320 $cv->begin;
321 $TEMPLATE->fork->send_arg ($_)->run ("worker_run", sub {
322 push @worker, shift;
323 $cv->end;
324 });
325 }
326 $cv->recv;
327
255=head1 CONCEPTS 328=head1 CONCEPTS
256 329
257This module can create new processes either by executing a new perl 330This module can create new processes either by executing a new perl
258process, or by forking from an existing "template" process. 331process, or by forking from an existing "template" process.
259 332
… …
377use AnyEvent; 450use AnyEvent;
378use AnyEvent::Util (); 451use AnyEvent::Util ();
379 452
380use IO::FDPass; 453use IO::FDPass;
381 454
382our $VERSION = 0.7; 455our $VERSION = 1.1;
383 456
384# the early fork template process 457# the early fork template process
385our $EARLY; 458our $EARLY;
386 459
387# the empty template process 460# the empty template process
… …
435 # send string 508 # send string
436 my $len = syswrite $self->[FH], $self->[QUEUE][0]; 509 my $len = syswrite $self->[FH], $self->[QUEUE][0];
437 510
438 unless ($len) { 511 unless ($len) {
439 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK; 512 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
440 undef $self->[3]; 513 undef $self->[WW];
441 die "AnyEvent::Fork: command write failure: $!"; 514 die "AnyEvent::Fork: command write failure: $!";
442 } 515 }
443 516
444 substr $self->[QUEUE][0], 0, $len, ""; 517 substr $self->[QUEUE][0], 0, $len, "";
445 shift @{ $self->[QUEUE] } unless length $self->[QUEUE][0]; 518 shift @{ $self->[QUEUE] } unless length $self->[QUEUE][0];
… …
532 605
533You should use C<new> whenever possible, except when having a template 606You should use C<new> whenever possible, except when having a template
534process around is unacceptable. 607process around is unacceptable.
535 608
536The path to the perl interpreter is divined using various methods - first 609The path to the perl interpreter is divined using various methods - first
537C<$^X> is investigated to see if the path ends with something that sounds 610C<$^X> is investigated to see if the path ends with something that looks
538as if it were the perl interpreter. Failing this, the module falls back to 611as if it were the perl interpreter. Failing this, the module falls back to
539using C<$Config::Config{perlpath}>. 612using C<$Config::Config{perlpath}>.
540 613
614The path to perl can also be overriden by setting the global variable
615C<$AnyEvent::Fork::PERL> - it's value will be used for all subsequent
616invocations.
617
541=cut 618=cut
619
620our $PERL;
542 621
543sub new_exec { 622sub new_exec {
544 my ($self) = @_; 623 my ($self) = @_;
545 624
546 return $EARLY->fork 625 return $EARLY->fork
547 if $EARLY; 626 if $EARLY;
548 627
628 unless (defined $PERL) {
549 # first find path of perl 629 # first find path of perl
550 my $perl = $; 630 my $perl = $;
551 631
552 # first we try $^X, but the path must be absolute (always on win32), and end in sth. 632 # first we try $^X, but the path must be absolute (always on win32), and end in sth.
553 # that looks like perl. this obviously only works for posix and win32 633 # that looks like perl. this obviously only works for posix and win32
554 unless ( 634 unless (
555 ($^O eq "MSWin32" || $perl =~ m%^/%) 635 ($^O eq "MSWin32" || $perl =~ m%^/%)
556 && $perl =~ m%[/\\]perl(?:[0-9]+(\.[0-9]+)+)?(\.exe)?$%i 636 && $perl =~ m%[/\\]perl(?:[0-9]+(\.[0-9]+)+)?(\.exe)?$%i
557 ) { 637 ) {
558 # if it doesn't look perlish enough, try Config 638 # if it doesn't look perlish enough, try Config
559 require Config; 639 require Config;
560 $perl = $Config::Config{perlpath}; 640 $perl = $Config::Config{perlpath};
561 $perl =~ s/(?:\Q$Config::Config{_exe}\E)?$/$Config::Config{_exe}/; 641 $perl =~ s/(?:\Q$Config::Config{_exe}\E)?$/$Config::Config{_exe}/;
642 }
643
644 $PERL = $perl;
562 } 645 }
563 646
564 require Proc::FastSpawn; 647 require Proc::FastSpawn;
565 648
566 my ($fh, $slave) = AnyEvent::Util::portable_socketpair; 649 my ($fh, $slave) = AnyEvent::Util::portable_socketpair;
… …
574 #local $ENV{PERL5LIB} = join ":", grep !ref, @INC; 657 #local $ENV{PERL5LIB} = join ":", grep !ref, @INC;
575 my %env = %ENV; 658 my %env = %ENV;
576 $env{PERL5LIB} = join +($^O eq "MSWin32" ? ";" : ":"), grep !ref, @INC; 659 $env{PERL5LIB} = join +($^O eq "MSWin32" ? ";" : ":"), grep !ref, @INC;
577 660
578 my $pid = Proc::FastSpawn::spawn ( 661 my $pid = Proc::FastSpawn::spawn (
579 $perl, 662 $PERL,
580 ["perl", "-MAnyEvent::Fork::Serve", "-e", "AnyEvent::Fork::Serve::me", fileno $slave, $$], 663 ["perl", "-MAnyEvent::Fork::Serve", "-e", "AnyEvent::Fork::Serve::me", fileno $slave, $$],
581 [map "$_=$env{$_}", keys %env], 664 [map "$_=$env{$_}", keys %env],
582 ) or die "unable to spawn AnyEvent::Fork server: $!"; 665 ) or die "unable to spawn AnyEvent::Fork server: $!";
583 666
584 $self->_new ($fh, $pid) 667 $self->_new ($fh, $pid)
… …
774 857
775 $self->[CB] = $cb; 858 $self->[CB] = $cb;
776 $self->_cmd (r => $func); 859 $self->_cmd (r => $func);
777} 860}
778 861
779=item $proc->to_fh ($cb->($fh)) 862=back
863
864=head2 EXPERIMENTAL METHODS
865
866These methods might go away completely or change behaviour, at any time.
867
868=over 4
869
870=item $proc->to_fh ($cb->($fh)) # EXPERIMENTAL, MIGHT BE REMOVED
780 871
781Flushes all commands out to the process and then calls the callback with 872Flushes all commands out to the process and then calls the callback with
782the communications socket. 873the communications socket.
783 874
784The process object becomes unusable on return from this function - any 875The process object becomes unusable on return from this function - any
785further method calls result in undefined behaviour. 876further method calls result in undefined behaviour.
786 877
787The point of this method is to give you a file handle thta you cna pass 878The point of this method is to give you a file handle thta you cna pass
788to another process. In that other process, you can call C<new_from_fh 879to another process. In that other process, you can call C<new_from_fh
789AnyEvent::Fork::RPC> to create a new C<AnyEvent::Fork> object from it, 880AnyEvent::Fork> to create a new C<AnyEvent::Fork> object from it, thereby
790thereby effectively passing a fork object to another process. 881effectively passing a fork object to another process.
791 882
792=cut 883=cut
793 884
794sub to_fh { 885sub to_fh {
795 my ($self, $cb) = @_; 886 my ($self, $cb) = @_;
… …
800 $self->[CB]->($self->[FH]); 891 $self->[CB]->($self->[FH]);
801 @$self = (); 892 @$self = ();
802 } 893 }
803} 894}
804 895
805=item new_from_fh AnyEvent::Fork $fh 896=item new_from_fh AnyEvent::Fork $fh # EXPERIMENTAL, MIGHT BE REMOVED
806 897
807Takes a file handle originally rceeived by the C<to_fh> method and creates 898Takes a file handle originally rceeived by the C<to_fh> method and creates
808a new C<AnyEvent:Fork> object. The child process itself will not change in 899a new C<AnyEvent:Fork> object. The child process itself will not change in
809any way, i.e. it will keep all the modifications done to it before calling 900any way, i.e. it will keep all the modifications done to it before calling
810C<to_fh>. 901C<to_fh>.
… …
952to make it so, mostly due to the bloody broken perl that nobody seems to 1043to make it so, mostly due to the bloody broken perl that nobody seems to
953care about. The fork emulation is a bad joke - I have yet to see something 1044care about. The fork emulation is a bad joke - I have yet to see something
954useful that you can do with it without running into memory corruption 1045useful that you can do with it without running into memory corruption
955issues or other braindamage. Hrrrr. 1046issues or other braindamage. Hrrrr.
956 1047
1048Since fork is endlessly broken on win32 perls (it doesn't even remotely
1049work within it's documented limits) and quite obviously it's not getting
1050improved any time soon, the best way to proceed on windows would be to
1051always use C<new_exec> and thus never rely on perl's fork "emulation".
1052
957Cygwin perl is not supported at the moment due to some hilarious 1053Cygwin perl is not supported at the moment due to some hilarious
958shortcomings of its API - see L<IO::FDPoll> for more details. 1054shortcomings of its API - see L<IO::FDPoll> for more details. If you never
1055use C<send_fh> and always use C<new_exec> to create processes, it should
1056work though.
959 1057
960=head1 SEE ALSO 1058=head1 SEE ALSO
961 1059
962L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all 1060L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all
963(part of this distribution). 1061(part of this distribution).
964 1062
965L<AnyEvent::Fork::Template>, to create a process by forking the main 1063L<AnyEvent::Fork::Template>, to create a process by forking the main
966program at a convenient time (part of this distribution). 1064program at a convenient time (part of this distribution).
967 1065
1066L<AnyEvent::Fork::Remote>, for another way to create processes that is
1067mostly compatible to this module and modules building on top of it, but
1068works better with remote processes.
1069
968L<AnyEvent::Fork::RPC>, for simple RPC to child processes (on CPAN). 1070L<AnyEvent::Fork::RPC>, for simple RPC to child processes (on CPAN).
1071
1072L<AnyEvent::Fork::Pool>, for simple worker process pool (on CPAN).
969 1073
970=head1 AUTHOR AND CONTACT INFORMATION 1074=head1 AUTHOR AND CONTACT INFORMATION
971 1075
972 Marc Lehmann <schmorp@schmorp.de> 1076 Marc Lehmann <schmorp@schmorp.de>
973 http://software.schmorp.de/pkg/AnyEvent-Fork 1077 http://software.schmorp.de/pkg/AnyEvent-Fork

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines