ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork/Fork.pm
(Generate patch)

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.47 by root, Thu Apr 18 20:17:34 2013 UTC vs.
Revision 1.51 by root, Sun Apr 21 12:25:50 2013 UTC

… …
38 38
39If you need some form of RPC, you could use the L<AnyEvent::Fork::RPC> 39If you need some form of RPC, you could use the L<AnyEvent::Fork::RPC>
40companion module, which adds simple RPC/job queueing to a process created 40companion module, which adds simple RPC/job queueing to a process created
41by this module. 41by this module.
42 42
43And if you need some automatic process pool management on top of
44L<AnyEvent::Fork::RPC>, you can look at the L<AnyEvent::Fork::Pool>
45companion module.
46
43Or you can implement it yourself in whatever way you like, use some 47Or you can implement it yourself in whatever way you like: use some
44message-passing module such as L<AnyEvent::MP>, some pipe such as 48message-passing module such as L<AnyEvent::MP>, some pipe such as
45L<AnyEvent::ZeroMQ>, use L<AnyEvent::Handle> on both sides to send 49L<AnyEvent::ZeroMQ>, use L<AnyEvent::Handle> on both sides to send
46e.g. JSON or Storable messages, and so on. 50e.g. JSON or Storable messages, and so on.
47 51
48=head2 COMPARISON TO OTHER MODULES 52=head2 COMPARISON TO OTHER MODULES
… …
250 ->send_arg ("/bin/echo", "hi") 254 ->send_arg ("/bin/echo", "hi")
251 ->run ("run", my $cv = AE::cv); 255 ->run ("run", my $cv = AE::cv);
252 256
253 my $stderr = $cv->recv; 257 my $stderr = $cv->recv;
254 258
259=head2 For stingy users: put the worker code into a C<DATA> section.
260
261When you want to be stingy with files, you cna put your code into the
262C<DATA> section of your module (or program):
263
264 use AnyEvent::Fork;
265
266 AnyEvent::Fork
267 ->new
268 ->eval (do { local $/; <DATA> })
269 ->run ("doit", sub { ... });
270
271 __DATA__
272
273 sub doit {
274 ... do something!
275 }
276
277=head2 For stingy standalone programs: do not rely on external files at
278all.
279
280For single-file scripts it can be inconvenient to rely on external
281files - even when using < C<DATA> section, you still need to C<exec>
282an external perl interpreter, which might not be available when using
283L<App::Staticperl>, L<Urlader> or L<PAR::Packer> for example.
284
285Two modules help here - L<AnyEvent::Fork::Early> forks a template process
286for all further calls to C<new_exec>, and L<AnyEvent::Fork::Template>
287forks the main program as a template process.
288
289Here is how your main program should look like:
290
291 #! perl
292
293 # optional, as the very first thing.
294 # in case modules want to create their own processes.
295 use AnyEvent::Fork::Early;
296
297 # next, load all modules you need in your template process
298 use Example::My::Module
299 use Example::Whatever;
300
301 # next, put your run function definition and anything else you
302 # need, but do not use code outside of BEGIN blocks.
303 sub worker_run {
304 my ($fh, @args) = @_;
305 ...
306 }
307
308 # now preserve everything so far as AnyEvent::Fork object
309 # in §TEMPLATE.
310 use AnyEvent::Fork::Template;
311
312 # do not put code outside of BEGIN blocks until here
313
314 # now use the $TEMPLATE process in any way you like
315
316 # for example: create 10 worker processes
317 my @worker;
318 my $cv = AE::cv;
319 for (1..10) {
320 $cv->begin;
321 $TEMPLATE->fork->send_arg ($_)->run ("worker_run", sub {
322 push @worker, shift;
323 $cv->end;
324 });
325 }
326 $cv->recv;
327
255=head1 CONCEPTS 328lhead1 CONCEPTS
256 329
257This module can create new processes either by executing a new perl 330This module can create new processes either by executing a new perl
258process, or by forking from an existing "template" process. 331process, or by forking from an existing "template" process.
259 332
260All these processes are called "child processes" (whether they are direct 333All these processes are called "child processes" (whether they are direct
… …
377use AnyEvent; 450use AnyEvent;
378use AnyEvent::Util (); 451use AnyEvent::Util ();
379 452
380use IO::FDPass; 453use IO::FDPass;
381 454
382our $VERSION = 0.7; 455our $VERSION = '1.0';
383 456
384# the early fork template process 457# the early fork template process
385our $EARLY; 458our $EARLY;
386 459
387# the empty template process 460# the empty template process
… …
435 # send string 508 # send string
436 my $len = syswrite $self->[FH], $self->[QUEUE][0]; 509 my $len = syswrite $self->[FH], $self->[QUEUE][0];
437 510
438 unless ($len) { 511 unless ($len) {
439 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK; 512 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
440 undef $self->[3]; 513 undef $self->[WW];
441 die "AnyEvent::Fork: command write failure: $!"; 514 die "AnyEvent::Fork: command write failure: $!";
442 } 515 }
443 516
444 substr $self->[QUEUE][0], 0, $len, ""; 517 substr $self->[QUEUE][0], 0, $len, "";
445 shift @{ $self->[QUEUE] } unless length $self->[QUEUE][0]; 518 shift @{ $self->[QUEUE] } unless length $self->[QUEUE][0];
… …
448 521
449 # everything written 522 # everything written
450 undef $self->[WW]; 523 undef $self->[WW];
451 524
452 # invoke run callback, if any 525 # invoke run callback, if any
526 if ($self->[CB]) {
453 $self->[CB]->($self->[FH]) if $self->[CB]; 527 $self->[CB]->($self->[FH]);
528 @$self = ();
529 }
454 }; 530 };
455 531
456 () # make sure we don't leak the watcher 532 () # make sure we don't leak the watcher
457} 533}
458 534
… …
771 847
772 $self->[CB] = $cb; 848 $self->[CB] = $cb;
773 $self->_cmd (r => $func); 849 $self->_cmd (r => $func);
774} 850}
775 851
852=item $proc->to_fh ($cb->($fh)) # EXPERIMENTAL, MIGHT BE REMOVED
853
854Flushes all commands out to the process and then calls the callback with
855the communications socket.
856
857The process object becomes unusable on return from this function - any
858further method calls result in undefined behaviour.
859
860The point of this method is to give you a file handle thta you cna pass
861to another process. In that other process, you can call C<new_from_fh
862AnyEvent::Fork> to create a new C<AnyEvent::Fork> object from it, thereby
863effectively passing a fork object to another process.
864
865=cut
866
867sub to_fh {
868 my ($self, $cb) = @_;
869
870 $self->[CB] = $cb;
871
872 unless ($self->[WW]) {
873 $self->[CB]->($self->[FH]);
874 @$self = ();
875 }
876}
877
878=item new_from_fh AnyEvent::Fork $fh # EXPERIMENTAL, MIGHT BE REMOVED
879
880Takes a file handle originally rceeived by the C<to_fh> method and creates
881a new C<AnyEvent:Fork> object. The child process itself will not change in
882any way, i.e. it will keep all the modifications done to it before calling
883C<to_fh>.
884
885The new object is very much like the original object, except that the
886C<pid> method will return C<undef> even if the process is a direct child.
887
888=cut
889
890sub new_from_fh {
891 my ($class, $fh) = @_;
892
893 $class->_new ($fh)
894}
895
776=back 896=back
777 897
778=head1 PERFORMANCE 898=head1 PERFORMANCE
779 899
780Now for some unscientific benchmark numbers (all done on an amd64 900Now for some unscientific benchmark numbers (all done on an amd64
… …
788 908
789 2079 new processes per second, using manual socketpair + fork 909 2079 new processes per second, using manual socketpair + fork
790 910
791Then I did the same thing, but instead of calling fork, I called 911Then I did the same thing, but instead of calling fork, I called
792AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the 912AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the
793socket form the child to close on exit. This does the same thing as manual 913socket from the child to close on exit. This does the same thing as manual
794socket pair + fork, except that what is forked is the template process 914socket pair + fork, except that what is forked is the template process
795(2440kB), and the socket needs to be passed to the server at the other end 915(2440kB), and the socket needs to be passed to the server at the other end
796of the socket first. 916of the socket first.
797 917
798 2307 new processes per second, using AnyEvent::Fork->new 918 2307 new processes per second, using AnyEvent::Fork->new
… …
905to make it so, mostly due to the bloody broken perl that nobody seems to 1025to make it so, mostly due to the bloody broken perl that nobody seems to
906care about. The fork emulation is a bad joke - I have yet to see something 1026care about. The fork emulation is a bad joke - I have yet to see something
907useful that you can do with it without running into memory corruption 1027useful that you can do with it without running into memory corruption
908issues or other braindamage. Hrrrr. 1028issues or other braindamage. Hrrrr.
909 1029
1030Since fork is endlessly broken on win32 perls (it doesn't even remotely
1031work within it's documented limits) and quite obviously it's not getting
1032improved any time soon, the best way to proceed on windows would be to
1033always use C<new_exec> and thus never rely on perl's fork "emulation".
1034
910Cygwin perl is not supported at the moment due to some hilarious 1035Cygwin perl is not supported at the moment due to some hilarious
911shortcomings of its API - see L<IO::FDPoll> for more details. 1036shortcomings of its API - see L<IO::FDPoll> for more details. If you never
1037use C<send_fh> and always use C<new_exec> to create processes, it should
1038work though.
912 1039
913=head1 SEE ALSO 1040=head1 SEE ALSO
914 1041
915L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all 1042L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all
916(part of this distribution). 1043(part of this distribution).
… …
918L<AnyEvent::Fork::Template>, to create a process by forking the main 1045L<AnyEvent::Fork::Template>, to create a process by forking the main
919program at a convenient time (part of this distribution). 1046program at a convenient time (part of this distribution).
920 1047
921L<AnyEvent::Fork::RPC>, for simple RPC to child processes (on CPAN). 1048L<AnyEvent::Fork::RPC>, for simple RPC to child processes (on CPAN).
922 1049
1050L<AnyEvent::Fork::Pool>, for simple worker process pool (on CPAN).
1051
923=head1 AUTHOR AND CONTACT INFORMATION 1052=head1 AUTHOR AND CONTACT INFORMATION
924 1053
925 Marc Lehmann <schmorp@schmorp.de> 1054 Marc Lehmann <schmorp@schmorp.de>
926 http://software.schmorp.de/pkg/AnyEvent-Fork 1055 http://software.schmorp.de/pkg/AnyEvent-Fork
927 1056

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines