ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork/Fork.pm
(Generate patch)

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.31 by root, Sat Apr 6 09:29:26 2013 UTC vs.
Revision 1.52 by root, Thu Apr 25 11:40:42 2013 UTC

27 27
28Special care has been taken to make this module useful from other modules, 28Special care has been taken to make this module useful from other modules,
29while still supporting specialised environments such as L<App::Staticperl> 29while still supporting specialised environments such as L<App::Staticperl>
30or L<PAR::Packer>. 30or L<PAR::Packer>.
31 31
32=head1 WHAT THIS MODULE IS NOT 32=head2 WHAT THIS MODULE IS NOT
33 33
34This module only creates processes and lets you pass file handles and 34This module only creates processes and lets you pass file handles and
35strings to it, and run perl code. It does not implement any kind of RPC - 35strings to it, and run perl code. It does not implement any kind of RPC -
36there is no back channel from the process back to you, and there is no RPC 36there is no back channel from the process back to you, and there is no RPC
37or message passing going on. 37or message passing going on.
38 38
39If you need some form of RPC, you can either implement it yourself 39If you need some form of RPC, you could use the L<AnyEvent::Fork::RPC>
40in whatever way you like, use some message-passing module such 40companion module, which adds simple RPC/job queueing to a process created
41as L<AnyEvent::MP>, some pipe such as L<AnyEvent::ZeroMQ>, use 41by this module.
42L<AnyEvent::Handle> on both sides to send e.g. JSON or Storable messages,
43and so on.
44 42
43And if you need some automatic process pool management on top of
44L<AnyEvent::Fork::RPC>, you can look at the L<AnyEvent::Fork::Pool>
45companion module.
46
47Or you can implement it yourself in whatever way you like: use some
48message-passing module such as L<AnyEvent::MP>, some pipe such as
49L<AnyEvent::ZeroMQ>, use L<AnyEvent::Handle> on both sides to send
50e.g. JSON or Storable messages, and so on.
51
52=head2 COMPARISON TO OTHER MODULES
53
54There is an abundance of modules on CPAN that do "something fork", such as
55L<Parallel::ForkManager>, L<AnyEvent::ForkManager>, L<AnyEvent::Worker>
56or L<AnyEvent::Subprocess>. There are modules that implement their own
57process management, such as L<AnyEvent::DBI>.
58
59The problems that all these modules try to solve are real, however, none
60of them (from what I have seen) tackle the very real problems of unwanted
61memory sharing, efficiency, not being able to use event processing or
62similar modules in the processes they create.
63
64This module doesn't try to replace any of them - instead it tries to solve
65the problem of creating processes with a minimum of fuss and overhead (and
66also luxury). Ideally, most of these would use AnyEvent::Fork internally,
67except they were written before AnyEvent:Fork was available, so obviously
68had to roll their own.
69
45=head1 PROBLEM STATEMENT 70=head2 PROBLEM STATEMENT
46 71
47There are two traditional ways to implement parallel processing on UNIX 72There are two traditional ways to implement parallel processing on UNIX
48like operating systems - fork and process, and fork+exec and process. They 73like operating systems - fork and process, and fork+exec and process. They
49have different advantages and disadvantages that I describe below, 74have different advantages and disadvantages that I describe below,
50together with how this module tries to mitigate the disadvantages. 75together with how this module tries to mitigate the disadvantages.
152 177
153 # now $master_filehandle is connected to the 178 # now $master_filehandle is connected to the
154 # $slave_filehandle in the new process. 179 # $slave_filehandle in the new process.
155 }); 180 });
156 181
157MyModule might look like this: 182C<MyModule> might look like this:
158 183
159 package MyModule; 184 package MyModule;
160 185
161 sub worker { 186 sub worker {
162 my ($slave_filehandle) = @_; 187 my ($slave_filehandle) = @_;
185 } 210 }
186 211
187 # now do other things - maybe use the filehandle provided by run 212 # now do other things - maybe use the filehandle provided by run
188 # to wait for the processes to die. or whatever. 213 # to wait for the processes to die. or whatever.
189 214
190My::Server might look like this: 215C<My::Server> might look like this:
191 216
192 package My::Server; 217 package My::Server;
193 218
194 sub run { 219 sub run {
195 my ($slave, $listener, $id) = @_; 220 my ($slave, $listener, $id) = @_;
203 } 228 }
204 } 229 }
205 230
206=head2 use AnyEvent::Fork as a faster fork+exec 231=head2 use AnyEvent::Fork as a faster fork+exec
207 232
208This runs /bin/echo hi, with stdout redirected to /tmp/log and stderr to 233This runs C</bin/echo hi>, with standard output redirected to F</tmp/log>
209the communications socket. It is usually faster than fork+exec, but still 234and standard error redirected to the communications socket. It is usually
210let's you prepare the environment. 235faster than fork+exec, but still lets you prepare the environment.
211 236
212 open my $output, ">/tmp/log" or die "$!"; 237 open my $output, ">/tmp/log" or die "$!";
213 238
214 AnyEvent::Fork 239 AnyEvent::Fork
215 ->new 240 ->new
216 ->eval (' 241 ->eval ('
242 # compile a helper function for later use
217 sub run { 243 sub run {
218 my ($fh, $output, @cmd) = @_; 244 my ($fh, $output, @cmd) = @_;
219 245
220 # perl will clear close-on-exec on STDOUT/STDERR 246 # perl will clear close-on-exec on STDOUT/STDERR
221 open STDOUT, ">&", $output or die; 247 open STDOUT, ">&", $output or die;
228 ->send_arg ("/bin/echo", "hi") 254 ->send_arg ("/bin/echo", "hi")
229 ->run ("run", my $cv = AE::cv); 255 ->run ("run", my $cv = AE::cv);
230 256
231 my $stderr = $cv->recv; 257 my $stderr = $cv->recv;
232 258
259=head2 For stingy users: put the worker code into a C<DATA> section.
260
261When you want to be stingy with files, you cna put your code into the
262C<DATA> section of your module (or program):
263
264 use AnyEvent::Fork;
265
266 AnyEvent::Fork
267 ->new
268 ->eval (do { local $/; <DATA> })
269 ->run ("doit", sub { ... });
270
271 __DATA__
272
273 sub doit {
274 ... do something!
275 }
276
277=head2 For stingy standalone programs: do not rely on external files at
278all.
279
280For single-file scripts it can be inconvenient to rely on external
281files - even when using < C<DATA> section, you still need to C<exec>
282an external perl interpreter, which might not be available when using
283L<App::Staticperl>, L<Urlader> or L<PAR::Packer> for example.
284
285Two modules help here - L<AnyEvent::Fork::Early> forks a template process
286for all further calls to C<new_exec>, and L<AnyEvent::Fork::Template>
287forks the main program as a template process.
288
289Here is how your main program should look like:
290
291 #! perl
292
293 # optional, as the very first thing.
294 # in case modules want to create their own processes.
295 use AnyEvent::Fork::Early;
296
297 # next, load all modules you need in your template process
298 use Example::My::Module
299 use Example::Whatever;
300
301 # next, put your run function definition and anything else you
302 # need, but do not use code outside of BEGIN blocks.
303 sub worker_run {
304 my ($fh, @args) = @_;
305 ...
306 }
307
308 # now preserve everything so far as AnyEvent::Fork object
309 # in §TEMPLATE.
310 use AnyEvent::Fork::Template;
311
312 # do not put code outside of BEGIN blocks until here
313
314 # now use the $TEMPLATE process in any way you like
315
316 # for example: create 10 worker processes
317 my @worker;
318 my $cv = AE::cv;
319 for (1..10) {
320 $cv->begin;
321 $TEMPLATE->fork->send_arg ($_)->run ("worker_run", sub {
322 push @worker, shift;
323 $cv->end;
324 });
325 }
326 $cv->recv;
327
233=head1 CONCEPTS 328=head1 CONCEPTS
234 329
235This module can create new processes either by executing a new perl 330This module can create new processes either by executing a new perl
236process, or by forking from an existing "template" process. 331process, or by forking from an existing "template" process.
332
333All these processes are called "child processes" (whether they are direct
334children or not), while the process that manages them is called the
335"parent process".
237 336
238Each such process comes with its own file handle that can be used to 337Each such process comes with its own file handle that can be used to
239communicate with it (it's actually a socket - one end in the new process, 338communicate with it (it's actually a socket - one end in the new process,
240one end in the main process), and among the things you can do in it are 339one end in the main process), and among the things you can do in it are
241load modules, fork new processes, send file handles to it, and execute 340load modules, fork new processes, send file handles to it, and execute
351use AnyEvent; 450use AnyEvent;
352use AnyEvent::Util (); 451use AnyEvent::Util ();
353 452
354use IO::FDPass; 453use IO::FDPass;
355 454
356our $VERSION = 0.5; 455our $VERSION = '1.0';
357
358our $PERL; # the path to the perl interpreter, deduces with various forms of magic
359
360=over 4
361
362=back
363
364=cut
365 456
366# the early fork template process 457# the early fork template process
367our $EARLY; 458our $EARLY;
368 459
369# the empty template process 460# the empty template process
370our $TEMPLATE; 461our $TEMPLATE;
462
463sub QUEUE() { 0 }
464sub FH() { 1 }
465sub WW() { 2 }
466sub PID() { 3 }
467sub CB() { 4 }
468
469sub _new {
470 my ($self, $fh, $pid) = @_;
471
472 AnyEvent::Util::fh_nonblocking $fh, 1;
473
474 $self = bless [
475 [], # write queue - strings or fd's
476 $fh,
477 undef, # AE watcher
478 $pid,
479 ], $self;
480
481 $self
482}
371 483
372sub _cmd { 484sub _cmd {
373 my $self = shift; 485 my $self = shift;
374 486
375 # ideally, we would want to use "a (w/a)*" as format string, but perl 487 # ideally, we would want to use "a (w/a)*" as format string, but perl
376 # versions from at least 5.8.9 to 5.16.3 are all buggy and can't unpack 488 # versions from at least 5.8.9 to 5.16.3 are all buggy and can't unpack
377 # it. 489 # it.
378 push @{ $self->[2] }, pack "a L/a*", $_[0], $_[1]; 490 push @{ $self->[QUEUE] }, pack "a L/a*", $_[0], $_[1];
379 491
380 $self->[3] ||= AE::io $self->[1], 1, sub { 492 $self->[WW] ||= AE::io $self->[FH], 1, sub {
381 do { 493 do {
382 # send the next "thing" in the queue - either a reference to an fh, 494 # send the next "thing" in the queue - either a reference to an fh,
383 # or a plain string. 495 # or a plain string.
384 496
385 if (ref $self->[2][0]) { 497 if (ref $self->[QUEUE][0]) {
386 # send fh 498 # send fh
387 unless (IO::FDPass::send fileno $self->[1], fileno ${ $self->[2][0] }) { 499 unless (IO::FDPass::send fileno $self->[FH], fileno ${ $self->[QUEUE][0] }) {
388 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK; 500 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
389 undef $self->[3]; 501 undef $self->[WW];
390 die "AnyEvent::Fork: file descriptor send failure: $!"; 502 die "AnyEvent::Fork: file descriptor send failure: $!";
391 } 503 }
392 504
393 shift @{ $self->[2] }; 505 shift @{ $self->[QUEUE] };
394 506
395 } else { 507 } else {
396 # send string 508 # send string
397 my $len = syswrite $self->[1], $self->[2][0]; 509 my $len = syswrite $self->[FH], $self->[QUEUE][0];
398 510
399 unless ($len) { 511 unless ($len) {
400 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK; 512 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
401 undef $self->[3]; 513 undef $self->[WW];
402 die "AnyEvent::Fork: command write failure: $!"; 514 die "AnyEvent::Fork: command write failure: $!";
403 } 515 }
404 516
405 substr $self->[2][0], 0, $len, ""; 517 substr $self->[QUEUE][0], 0, $len, "";
406 shift @{ $self->[2] } unless length $self->[2][0]; 518 shift @{ $self->[QUEUE] } unless length $self->[QUEUE][0];
407 } 519 }
408 } while @{ $self->[2] }; 520 } while @{ $self->[QUEUE] };
409 521
410 # everything written 522 # everything written
411 undef $self->[3]; 523 undef $self->[WW];
412 524
413 # invoke run callback, if any 525 # invoke run callback, if any
414 $self->[4]->($self->[1]) if $self->[4]; 526 if ($self->[CB]) {
527 $self->[CB]->($self->[FH]);
528 @$self = ();
529 }
415 }; 530 };
416 531
417 () # make sure we don't leak the watcher 532 () # make sure we don't leak the watcher
418}
419
420sub _new {
421 my ($self, $fh, $pid) = @_;
422
423 AnyEvent::Util::fh_nonblocking $fh, 1;
424
425 $self = bless [
426 $pid,
427 $fh,
428 [], # write queue - strings or fd's
429 undef, # AE watcher
430 ], $self;
431
432 $self
433} 533}
434 534
435# fork template from current process, used by AnyEvent::Fork::Early/Template 535# fork template from current process, used by AnyEvent::Fork::Early/Template
436sub _new_fork { 536sub _new_fork {
437 my ($fh, $slave) = AnyEvent::Util::portable_socketpair; 537 my ($fh, $slave) = AnyEvent::Util::portable_socketpair;
442 if ($pid eq 0) { 542 if ($pid eq 0) {
443 require AnyEvent::Fork::Serve; 543 require AnyEvent::Fork::Serve;
444 $AnyEvent::Fork::Serve::OWNER = $parent; 544 $AnyEvent::Fork::Serve::OWNER = $parent;
445 close $fh; 545 close $fh;
446 $0 = "$_[1] of $parent"; 546 $0 = "$_[1] of $parent";
447 $SIG{CHLD} = 'IGNORE';
448 AnyEvent::Fork::Serve::serve ($slave); 547 AnyEvent::Fork::Serve::serve ($slave);
449 exit 0; 548 exit 0;
450 } elsif (!$pid) { 549 } elsif (!$pid) {
451 die "AnyEvent::Fork::Early/Template: unable to fork template process: $!"; 550 die "AnyEvent::Fork::Early/Template: unable to fork template process: $!";
452 } 551 }
559} 658}
560 659
561=item $pid = $proc->pid 660=item $pid = $proc->pid
562 661
563Returns the process id of the process I<iff it is a direct child of the 662Returns the process id of the process I<iff it is a direct child of the
564process> running AnyEvent::Fork, and C<undef> otherwise. 663process running AnyEvent::Fork>, and C<undef> otherwise.
565 664
566Normally, only processes created via C<< AnyEvent::Fork->new_exec >> and 665Normally, only processes created via C<< AnyEvent::Fork->new_exec >> and
567L<AnyEvent::Fork::Template> are direct children, and you are responsible 666L<AnyEvent::Fork::Template> are direct children, and you are responsible
568to clean up their zombies when they die. 667to clean up their zombies when they die.
569 668
571AnyEvent::Fork itself. 670AnyEvent::Fork itself.
572 671
573=cut 672=cut
574 673
575sub pid { 674sub pid {
576 $_[0][0] 675 $_[0][PID]
577} 676}
578 677
579=item $proc = $proc->eval ($perlcode, @args) 678=item $proc = $proc->eval ($perlcode, @args)
580 679
581Evaluates the given C<$perlcode> as ... perl code, while setting C<@_> to 680Evaluates the given C<$perlcode> as ... Perl code, while setting C<@_> to
582the strings specified by C<@args>, in the "main" package. 681the strings specified by C<@args>, in the "main" package.
583 682
584This call is meant to do any custom initialisation that might be required 683This call is meant to do any custom initialisation that might be required
585(for example, the C<require> method uses it). It's not supposed to be used 684(for example, the C<require> method uses it). It's not supposed to be used
586to completely take over the process, use C<run> for that. 685to completely take over the process, use C<run> for that.
587 686
588The code will usually be executed after this call returns, and there is no 687The code will usually be executed after this call returns, and there is no
589way to pass anything back to the calling process. Any evaluation errors 688way to pass anything back to the calling process. Any evaluation errors
590will be reported to stderr and cause the process to exit. 689will be reported to stderr and cause the process to exit.
591 690
592If you want to execute some code to take over the process (see the 691If you want to execute some code (that isn't in a module) to take over the
593"fork+exec" example in the SYNOPSIS), you should compile a function via 692process, you should compile a function via C<eval> first, and then call
594C<eval> first, and then call it via C<run>. This also gives you access to 693it via C<run>. This also gives you access to any arguments passed via the
595any arguments passed via the C<send_xxx> methods, such as file handles. 694C<send_xxx> methods, such as file handles. See the L<use AnyEvent::Fork as
695a faster fork+exec> example to see it in action.
596 696
597Returns the process object for easy chaining of method calls. 697Returns the process object for easy chaining of method calls.
598 698
599=cut 699=cut
600 700
626=item $proc = $proc->send_fh ($handle, ...) 726=item $proc = $proc->send_fh ($handle, ...)
627 727
628Send one or more file handles (I<not> file descriptors) to the process, 728Send one or more file handles (I<not> file descriptors) to the process,
629to prepare a call to C<run>. 729to prepare a call to C<run>.
630 730
631The process object keeps a reference to the handles until this is done, 731The process object keeps a reference to the handles until they have
632so you must not explicitly close the handles. This is most easily 732been passed over to the process, so you must not explicitly close the
633accomplished by simply not storing the file handles anywhere after passing 733handles. This is most easily accomplished by simply not storing the file
634them to this method. 734handles anywhere after passing them to this method - when AnyEvent::Fork
735is finished using them, perl will automatically close them.
635 736
636Returns the process object for easy chaining of method calls. 737Returns the process object for easy chaining of method calls.
637 738
638Example: pass a file handle to a process, and release it without 739Example: pass a file handle to a process, and release it without
639closing. It will be closed automatically when it is no longer used. 740closing. It will be closed automatically when it is no longer used.
646sub send_fh { 747sub send_fh {
647 my ($self, @fh) = @_; 748 my ($self, @fh) = @_;
648 749
649 for my $fh (@fh) { 750 for my $fh (@fh) {
650 $self->_cmd ("h"); 751 $self->_cmd ("h");
651 push @{ $self->[2] }, \$fh; 752 push @{ $self->[QUEUE] }, \$fh;
652 } 753 }
653 754
654 $self 755 $self
655} 756}
656 757
657=item $proc = $proc->send_arg ($string, ...) 758=item $proc = $proc->send_arg ($string, ...)
658 759
659Send one or more argument strings to the process, to prepare a call to 760Send one or more argument strings to the process, to prepare a call to
660C<run>. The strings can be any octet string. 761C<run>. The strings can be any octet strings.
661 762
662The protocol is optimised to pass a moderate number of relatively short 763The protocol is optimised to pass a moderate number of relatively short
663strings - while you can pass up to 4GB of data in one go, this is more 764strings - while you can pass up to 4GB of data in one go, this is more
664meant to pass some ID information or other startup info, not big chunks of 765meant to pass some ID information or other startup info, not big chunks of
665data. 766data.
681Enter the function specified by the function name in C<$func> in the 782Enter the function specified by the function name in C<$func> in the
682process. The function is called with the communication socket as first 783process. The function is called with the communication socket as first
683argument, followed by all file handles and string arguments sent earlier 784argument, followed by all file handles and string arguments sent earlier
684via C<send_fh> and C<send_arg> methods, in the order they were called. 785via C<send_fh> and C<send_arg> methods, in the order they were called.
685 786
787The process object becomes unusable on return from this function - any
788further method calls result in undefined behaviour.
789
686The function name should be fully qualified, but if it isn't, it will be 790The function name should be fully qualified, but if it isn't, it will be
687looked up in the main package. 791looked up in the C<main> package.
688 792
689If the called function returns, doesn't exist, or any error occurs, the 793If the called function returns, doesn't exist, or any error occurs, the
690process exits. 794process exits.
691 795
692Preparing the process is done in the background - when all commands have 796Preparing the process is done in the background - when all commands have
693been sent, the callback is invoked with the local communications socket 797been sent, the callback is invoked with the local communications socket
694as argument. At this point you can start using the socket in any way you 798as argument. At this point you can start using the socket in any way you
695like. 799like.
696
697The process object becomes unusable on return from this function - any
698further method calls result in undefined behaviour.
699 800
700If the communication socket isn't used, it should be closed on both sides, 801If the communication socket isn't used, it should be closed on both sides,
701to save on kernel memory. 802to save on kernel memory.
702 803
703The socket is non-blocking in the parent, and blocking in the newly 804The socket is non-blocking in the parent, and blocking in the newly
742=cut 843=cut
743 844
744sub run { 845sub run {
745 my ($self, $func, $cb) = @_; 846 my ($self, $func, $cb) = @_;
746 847
747 $self->[4] = $cb; 848 $self->[CB] = $cb;
748 $self->_cmd (r => $func); 849 $self->_cmd (r => $func);
850}
851
852=item $proc->to_fh ($cb->($fh)) # EXPERIMENTAL, MIGHT BE REMOVED
853
854Flushes all commands out to the process and then calls the callback with
855the communications socket.
856
857The process object becomes unusable on return from this function - any
858further method calls result in undefined behaviour.
859
860The point of this method is to give you a file handle thta you cna pass
861to another process. In that other process, you can call C<new_from_fh
862AnyEvent::Fork> to create a new C<AnyEvent::Fork> object from it, thereby
863effectively passing a fork object to another process.
864
865=cut
866
867sub to_fh {
868 my ($self, $cb) = @_;
869
870 $self->[CB] = $cb;
871
872 unless ($self->[WW]) {
873 $self->[CB]->($self->[FH]);
874 @$self = ();
875 }
876}
877
878=item new_from_fh AnyEvent::Fork $fh # EXPERIMENTAL, MIGHT BE REMOVED
879
880Takes a file handle originally rceeived by the C<to_fh> method and creates
881a new C<AnyEvent:Fork> object. The child process itself will not change in
882any way, i.e. it will keep all the modifications done to it before calling
883C<to_fh>.
884
885The new object is very much like the original object, except that the
886C<pid> method will return C<undef> even if the process is a direct child.
887
888=cut
889
890sub new_from_fh {
891 my ($class, $fh) = @_;
892
893 $class->_new ($fh)
749} 894}
750 895
751=back 896=back
752 897
753=head1 PERFORMANCE 898=head1 PERFORMANCE
763 908
764 2079 new processes per second, using manual socketpair + fork 909 2079 new processes per second, using manual socketpair + fork
765 910
766Then I did the same thing, but instead of calling fork, I called 911Then I did the same thing, but instead of calling fork, I called
767AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the 912AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the
768socket form the child to close on exit. This does the same thing as manual 913socket from the child to close on exit. This does the same thing as manual
769socket pair + fork, except that what is forked is the template process 914socket pair + fork, except that what is forked is the template process
770(2440kB), and the socket needs to be passed to the server at the other end 915(2440kB), and the socket needs to be passed to the server at the other end
771of the socket first. 916of the socket first.
772 917
773 2307 new processes per second, using AnyEvent::Fork->new 918 2307 new processes per second, using AnyEvent::Fork->new
778 479 vfork+execs per second, using AnyEvent::Fork->new_exec 923 479 vfork+execs per second, using AnyEvent::Fork->new_exec
779 924
780So how can C<< AnyEvent->new >> be faster than a standard fork, even 925So how can C<< AnyEvent->new >> be faster than a standard fork, even
781though it uses the same operations, but adds a lot of overhead? 926though it uses the same operations, but adds a lot of overhead?
782 927
783The difference is simply the process size: forking the 6MB process takes 928The difference is simply the process size: forking the 5MB process takes
784so much longer than forking the 2.5MB template process that the overhead 929so much longer than forking the 2.5MB template process that the extra
785introduced is canceled out. 930overhead is canceled out.
786 931
787If the benchmark process grows, the normal fork becomes even slower: 932If the benchmark process grows, the normal fork becomes even slower:
788 933
789 1340 new processes, manual fork in a 20MB process 934 1340 new processes, manual fork of a 20MB process
790 731 new processes, manual fork in a 200MB process 935 731 new processes, manual fork of a 200MB process
791 235 new processes, manual fork in a 2000MB process 936 235 new processes, manual fork of a 2000MB process
792 937
793What that means (to me) is that I can use this module without having a 938What that means (to me) is that I can use this module without having a bad
794very bad conscience because of the extra overhead required to start new 939conscience because of the extra overhead required to start new processes.
795processes.
796 940
797=head1 TYPICAL PROBLEMS 941=head1 TYPICAL PROBLEMS
798 942
799This section lists typical problems that remain. I hope by recognising 943This section lists typical problems that remain. I hope by recognising
800them, most can be avoided. 944them, most can be avoided.
801 945
802=over 4 946=over 4
803 947
804=item "leaked" file descriptors for exec'ed processes 948=item leaked file descriptors for exec'ed processes
805 949
806POSIX systems inherit file descriptors by default when exec'ing a new 950POSIX systems inherit file descriptors by default when exec'ing a new
807process. While perl itself laudably sets the close-on-exec flags on new 951process. While perl itself laudably sets the close-on-exec flags on new
808file handles, most C libraries don't care, and even if all cared, it's 952file handles, most C libraries don't care, and even if all cared, it's
809often not possible to set the flag in a race-free manner. 953often not possible to set the flag in a race-free manner.
829libraries or the code that leaks those file descriptors. 973libraries or the code that leaks those file descriptors.
830 974
831Fortunately, most of these leaked descriptors do no harm, other than 975Fortunately, most of these leaked descriptors do no harm, other than
832sitting on some resources. 976sitting on some resources.
833 977
834=item "leaked" file descriptors for fork'ed processes 978=item leaked file descriptors for fork'ed processes
835 979
836Normally, L<AnyEvent::Fork> does start new processes by exec'ing them, 980Normally, L<AnyEvent::Fork> does start new processes by exec'ing them,
837which closes file descriptors not marked for being inherited. 981which closes file descriptors not marked for being inherited.
838 982
839However, L<AnyEvent::Fork::Early> and L<AnyEvent::Fork::Template> offer 983However, L<AnyEvent::Fork::Early> and L<AnyEvent::Fork::Template> offer
848 992
849The solution is to either not load these modules before use'ing 993The solution is to either not load these modules before use'ing
850L<AnyEvent::Fork::Early> or L<AnyEvent::Fork::Template>, or to delay 994L<AnyEvent::Fork::Early> or L<AnyEvent::Fork::Template>, or to delay
851initialising them, for example, by calling C<init Gtk2> manually. 995initialising them, for example, by calling C<init Gtk2> manually.
852 996
853=item exit runs destructors 997=item exiting calls object destructors
854 998
855This only applies to users of Lc<AnyEvent::Fork:Early> and 999This only applies to users of L<AnyEvent::Fork:Early> and
856L<AnyEvent::Fork::Template>. 1000L<AnyEvent::Fork::Template>, or when initialising code creates objects
1001that reference external resources.
857 1002
858When a process created by AnyEvent::Fork exits, it might do so by calling 1003When a process created by AnyEvent::Fork exits, it might do so by calling
859exit, or simply letting perl reach the end of the program. At which point 1004exit, or simply letting perl reach the end of the program. At which point
860Perl runs all destructors. 1005Perl runs all destructors.
861 1006
880to make it so, mostly due to the bloody broken perl that nobody seems to 1025to make it so, mostly due to the bloody broken perl that nobody seems to
881care about. The fork emulation is a bad joke - I have yet to see something 1026care about. The fork emulation is a bad joke - I have yet to see something
882useful that you can do with it without running into memory corruption 1027useful that you can do with it without running into memory corruption
883issues or other braindamage. Hrrrr. 1028issues or other braindamage. Hrrrr.
884 1029
885Cygwin perl is not supported at the moment, as it should implement fd 1030Since fork is endlessly broken on win32 perls (it doesn't even remotely
886passing, but doesn't, and rolling my own is hard, as cygwin doesn't 1031work within it's documented limits) and quite obviously it's not getting
887support enough functionality to do it. 1032improved any time soon, the best way to proceed on windows would be to
1033always use C<new_exec> and thus never rely on perl's fork "emulation".
1034
1035Cygwin perl is not supported at the moment due to some hilarious
1036shortcomings of its API - see L<IO::FDPoll> for more details. If you never
1037use C<send_fh> and always use C<new_exec> to create processes, it should
1038work though.
888 1039
889=head1 SEE ALSO 1040=head1 SEE ALSO
890 1041
891L<AnyEvent::Fork::Early> (to avoid executing a perl interpreter), 1042L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all
1043(part of this distribution).
1044
892L<AnyEvent::Fork::Template> (to create a process by forking the main 1045L<AnyEvent::Fork::Template>, to create a process by forking the main
893program at a convenient time). 1046program at a convenient time (part of this distribution).
894 1047
895=head1 AUTHOR 1048L<AnyEvent::Fork::RPC>, for simple RPC to child processes (on CPAN).
1049
1050L<AnyEvent::Fork::Pool>, for simple worker process pool (on CPAN).
1051
1052=head1 AUTHOR AND CONTACT INFORMATION
896 1053
897 Marc Lehmann <schmorp@schmorp.de> 1054 Marc Lehmann <schmorp@schmorp.de>
898 http://home.schmorp.de/ 1055 http://software.schmorp.de/pkg/AnyEvent-Fork
899 1056
900=cut 1057=cut
901 1058
9021 10591
903 1060

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines