ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/cvsroot/AnyEvent-Fork/Fork.pm
(Generate patch)

Comparing cvsroot/AnyEvent-Fork/Fork.pm (file contents):
Revision 1.15 by root, Fri Apr 5 08:56:36 2013 UTC vs.
Revision 1.16 by root, Fri Apr 5 23:35:07 2013 UTC

… …
76than using fork+exec in big processes. 76than using fork+exec in big processes.
77 77
78Special care has been taken to make this module useful from other modules, 78Special care has been taken to make this module useful from other modules,
79while still supporting specialised environments such as L<App::Staticperl> 79while still supporting specialised environments such as L<App::Staticperl>
80or L<PAR::Packer>. 80or L<PAR::Packer>.
81
82=head1 WHAT THIS MODULE IS NOT
83
84This module only creates processes and lets you pass file handles and
85strings to it, and run perl code. It does not implement any kind of RPC -
86there is no back channel from the process back to you, and there is no RPC
87or message passing going on.
88
89If you need some form of RPC, you can either implement it yourself
90in whatever way you like, use some message-passing module such
91as L<AnyEvent::MP>, some pipe such as L<AnyEvent::ZeroMQ>, use
92L<AnyEvent::Handle> on both sides to send e.g. JSON or Storable messages,
93and so on.
81 94
82=head1 PROBLEM STATEMENT 95=head1 PROBLEM STATEMENT
83 96
84There are two ways to implement parallel processing on UNIX like operating 97There are two ways to implement parallel processing on UNIX like operating
85systems - fork and process, and fork+exec and process. They have different 98systems - fork and process, and fork+exec and process. They have different
… …
272 285
273 #TODO: maybe append the packet to any existing string command already in the queue 286 #TODO: maybe append the packet to any existing string command already in the queue
274 287
275 # ideally, we would want to use "a (w/a)*" as format string, but perl versions 288 # ideally, we would want to use "a (w/a)*" as format string, but perl versions
276 # from at least 5.8.9 to 5.16.3 are all buggy and can't unpack it. 289 # from at least 5.8.9 to 5.16.3 are all buggy and can't unpack it.
277 push @{ $self->[2] }, pack "N/a*", pack "(w/a*)*", @_; 290 push @{ $self->[2] }, pack "L/a*", pack "(w/a*)*", @_;
278 291
279 $self->[3] ||= AE::io $self->[1], 1, sub { 292 $self->[3] ||= AE::io $self->[1], 1, sub {
280 # send the next "thing" in the queue - either a reference to an fh, 293 # send the next "thing" in the queue - either a reference to an fh,
281 # or a plain string. 294 # or a plain string.
282 295
… …
329 if ($pid eq 0) { 342 if ($pid eq 0) {
330 require AnyEvent::Fork::Serve; 343 require AnyEvent::Fork::Serve;
331 $AnyEvent::Fork::Serve::OWNER = $parent; 344 $AnyEvent::Fork::Serve::OWNER = $parent;
332 close $fh; 345 close $fh;
333 $0 = "$_[1] of $parent"; 346 $0 = "$_[1] of $parent";
347 $SIG{CHLD} = 'IGNORE';
334 AnyEvent::Fork::Serve::serve ($slave); 348 AnyEvent::Fork::Serve::serve ($slave);
335 exit 0; 349 exit 0;
336 } elsif (!$pid) { 350 } elsif (!$pid) {
337 die "AnyEvent::Fork::Early/Template: unable to fork template process: $!"; 351 die "AnyEvent::Fork::Early/Template: unable to fork template process: $!";
338 } 352 }
… …
603 $self->_cmd (r => $func); 617 $self->_cmd (r => $func);
604} 618}
605 619
606=back 620=back
607 621
622=head1 PERFORMANCE
623
624Now for some unscientific benchmark numbers (all done on an amd64
625GNU/Linux box). These are intended to give you an idea of the relative
626performance you can expect.
627
628Ok, so, I ran a simple benchmark that creates a socketpair, forks, calls
629exit in the child and waits for the socket to close in the parent. I did
630load AnyEvent, EV and AnyEvent::Fork, for a total process size of 6312kB.
631
632 2079 new processes per second, using socketpair + fork manually
633
634Then I did the same thing, but instead of calling fork, I called
635AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the
636socket form the child to close on exit. This does the same thing as manual
637socketpair + fork, except that what is forked is the template process
638(2440kB), and the socket needs to be passed to the server at the other end
639of the socket first.
640
641 2307 new processes per second, using AnyEvent::Fork->new
642
643And finally, using C<new_exec> instead C<new>, using vforks+execs to exec
644a new perl interpreter and compile the small server each time, I get:
645
646 479 vfork+execs per second, using AnyEvent::Fork->new_exec
647
648So how can C<< AnyEvent->new >> be faster than a standard fork, een though
649it uses the same operations, but adds a lot of overhead?
650
651The difference is simply the process size: forking the 6MB process takes
652so much longer than forking the 2.5MB template process that the overhead
653introduced is canceled out.
654
655If the benchmark process grows, the normal fork becomes even slower:
656
657 1340 new processes, manual fork in a 20MB process
658 731 new processes, manual fork in a 200MB process
659 235 new processes, manual fork in a 2000MB process
660
661What that means (to me) is that I can use this module without havign a
662very bad conscience because of the extra overhead requried to strat new
663processes.
664
608=head1 TYPICAL PROBLEMS 665=head1 TYPICAL PROBLEMS
609 666
610This section lists typical problems that remain. I hope by recognising 667This section lists typical problems that remain. I hope by recognising
611them, most can be avoided. 668them, most can be avoided.
612 669

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines