ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork/README
(Generate patch)

Comparing AnyEvent-Fork/README (file contents):
Revision 1.6 by root, Thu Apr 18 20:17:35 2013 UTC vs.
Revision 1.9 by root, Wed Sep 25 11:05:30 2013 UTC

… …
34 34
35 If you need some form of RPC, you could use the AnyEvent::Fork::RPC 35 If you need some form of RPC, you could use the AnyEvent::Fork::RPC
36 companion module, which adds simple RPC/job queueing to a process 36 companion module, which adds simple RPC/job queueing to a process
37 created by this module. 37 created by this module.
38 38
39 And if you need some automatic process pool management on top of
40 AnyEvent::Fork::RPC, you can look at the AnyEvent::Fork::Pool companion
41 module.
42
39 Or you can implement it yourself in whatever way you like, use some 43 Or you can implement it yourself in whatever way you like: use some
40 message-passing module such as AnyEvent::MP, some pipe such as 44 message-passing module such as AnyEvent::MP, some pipe such as
41 AnyEvent::ZeroMQ, use AnyEvent::Handle on both sides to send e.g. JSON 45 AnyEvent::ZeroMQ, use AnyEvent::Handle on both sides to send e.g. JSON
42 or Storable messages, and so on. 46 or Storable messages, and so on.
43 47
44 COMPARISON TO OTHER MODULES 48 COMPARISON TO OTHER MODULES
… …
233 ->send_arg ("/bin/echo", "hi") 237 ->send_arg ("/bin/echo", "hi")
234 ->run ("run", my $cv = AE::cv); 238 ->run ("run", my $cv = AE::cv);
235 239
236 my $stderr = $cv->recv; 240 my $stderr = $cv->recv;
237 241
242 For stingy users: put the worker code into a "DATA" section.
243 When you want to be stingy with files, you cna put your code into the
244 "DATA" section of your module (or program):
245
246 use AnyEvent::Fork;
247
248 AnyEvent::Fork
249 ->new
250 ->eval (do { local $/; <DATA> })
251 ->run ("doit", sub { ... });
252
253 __DATA__
254
255 sub doit {
256 ... do something!
257 }
258
259 For stingy standalone programs: do not rely on external files at
260all.
261 For single-file scripts it can be inconvenient to rely on external files
262 - even when using < "DATA" section, you still need to "exec" an external
263 perl interpreter, which might not be available when using
264 App::Staticperl, Urlader or PAR::Packer for example.
265
266 Two modules help here - AnyEvent::Fork::Early forks a template process
267 for all further calls to "new_exec", and AnyEvent::Fork::Template forks
268 the main program as a template process.
269
270 Here is how your main program should look like:
271
272 #! perl
273
274 # optional, as the very first thing.
275 # in case modules want to create their own processes.
276 use AnyEvent::Fork::Early;
277
278 # next, load all modules you need in your template process
279 use Example::My::Module
280 use Example::Whatever;
281
282 # next, put your run function definition and anything else you
283 # need, but do not use code outside of BEGIN blocks.
284 sub worker_run {
285 my ($fh, @args) = @_;
286 ...
287 }
288
289 # now preserve everything so far as AnyEvent::Fork object
290 # in §TEMPLATE.
291 use AnyEvent::Fork::Template;
292
293 # do not put code outside of BEGIN blocks until here
294
295 # now use the $TEMPLATE process in any way you like
296
297 # for example: create 10 worker processes
298 my @worker;
299 my $cv = AE::cv;
300 for (1..10) {
301 $cv->begin;
302 $TEMPLATE->fork->send_arg ($_)->run ("worker_run", sub {
303 push @worker, shift;
304 $cv->end;
305 });
306 }
307 $cv->recv;
308
238CONCEPTS 309CONCEPTS
239 This module can create new processes either by executing a new perl 310 This module can create new processes either by executing a new perl
240 process, or by forking from an existing "template" process. 311 process, or by forking from an existing "template" process.
241 312
242 All these processes are called "child processes" (whether they are 313 All these processes are called "child processes" (whether they are
… …
368 You should use "new" whenever possible, except when having a 439 You should use "new" whenever possible, except when having a
369 template process around is unacceptable. 440 template process around is unacceptable.
370 441
371 The path to the perl interpreter is divined using various methods - 442 The path to the perl interpreter is divined using various methods -
372 first $^X is investigated to see if the path ends with something 443 first $^X is investigated to see if the path ends with something
373 that sounds as if it were the perl interpreter. Failing this, the 444 that looks as if it were the perl interpreter. Failing this, the
374 module falls back to using $Config::Config{perlpath}. 445 module falls back to using $Config::Config{perlpath}.
446
447 The path to perl can also be overriden by setting the global
448 variable $AnyEvent::Fork::PERL - it's value will be used for all
449 subsequent invocations.
375 450
376 $pid = $proc->pid 451 $pid = $proc->pid
377 Returns the process id of the process *iff it is a direct child of 452 Returns the process id of the process *iff it is a direct child of
378 the process running AnyEvent::Fork*, and "undef" otherwise. 453 the process running AnyEvent::Fork*, and "undef" otherwise. As a
454 general rule (that you cannot rely upon), processes created via
455 "new_exec", AnyEvent::Fork::Early or AnyEvent::Fork::Template are
456 direct children, while all other processes are not.
379 457
380 Normally, only processes created via "AnyEvent::Fork->new_exec" and 458 Or in other words, you do not normally have to take care of zombies
381 AnyEvent::Fork::Template are direct children, and you are 459 for processes created via "new", but when in doubt, or zombies are a
382 responsible to clean up their zombies when they die. 460 problem, you need to check whether a process is a diretc child by
383 461 calling this method, and possibly creating a child watcher or reap
384 All other processes are not direct children, and will be cleaned up 462 it manually.
385 by AnyEvent::Fork itself.
386 463
387 $proc = $proc->eval ($perlcode, @args) 464 $proc = $proc->eval ($perlcode, @args)
388 Evaluates the given $perlcode as ... Perl code, while setting @_ to 465 Evaluates the given $perlcode as ... Perl code, while setting @_ to
389 the strings specified by @args, in the "main" package. 466 the strings specified by @args, in the "main" package.
390 467
… …
472 Even if not used otherwise, the socket can be a good indicator for 549 Even if not used otherwise, the socket can be a good indicator for
473 the existence of the process - if the other process exits, you get a 550 the existence of the process - if the other process exits, you get a
474 readable event on it, because exiting the process closes the socket 551 readable event on it, because exiting the process closes the socket
475 (if it didn't create any children using fork). 552 (if it didn't create any children using fork).
476 553
554 Compatibility to AnyEvent::Fork::Remote
555 If you want to write code that works with both this module and
556 AnyEvent::Fork::Remote, you need to write your code so that it
557 assumes there are two file handles for communications, which
558 might not be unix domain sockets. The "run" function should
559 start like this:
560
561 sub run {
562 my ($rfh, @args) = @_; # @args is your normal arguments
563 my $wfh = fileno $rfh ? $rfh : *STDOUT;
564
565 # now use $rfh for reading and $wfh for writing
566 }
567
568 This checks whether the passed file handle is, in fact, the
569 process "STDIN" handle. If it is, then the function was invoked
570 visa AnyEvent::Fork::Remote, so STDIN should be used for reading
571 and "STDOUT" should be used for writing.
572
573 In all other cases, the function was called via this module, and
574 there is only one file handle that should be sued for reading
575 and writing.
576
477 Example: create a template for a process pool, pass a few strings, 577 Example: create a template for a process pool, pass a few strings,
478 some file handles, then fork, pass one more string, and run some 578 some file handles, then fork, pass one more string, and run some
479 code. 579 code.
480 580
481 my $pool = AnyEvent::Fork 581 my $pool = AnyEvent::Fork
… …
504 my ($fh, $str1, $str2, $fh1, $fh2, $str3) = @_; 604 my ($fh, $str1, $str2, $fh1, $fh2, $str3) = @_;
505 605
506 print scalar <$fh>; # prints "hi #1\n" and "hi #2\n" in any order 606 print scalar <$fh>; # prints "hi #1\n" and "hi #2\n" in any order
507 } 607 }
508 608
609 EXPERIMENTAL METHODS
610 These methods might go away completely or change behaviour, at any time.
611
612 $proc->to_fh ($cb->($fh)) # EXPERIMENTAL, MIGHT BE REMOVED
613 Flushes all commands out to the process and then calls the callback
614 with the communications socket.
615
616 The process object becomes unusable on return from this function -
617 any further method calls result in undefined behaviour.
618
619 The point of this method is to give you a file handle that you can
620 pass to another process. In that other process, you can call
621 "new_from_fh AnyEvent::Fork $fh" to create a new "AnyEvent::Fork"
622 object from it, thereby effectively passing a fork object to another
623 process.
624
625 new_from_fh AnyEvent::Fork $fh # EXPERIMENTAL, MIGHT BE REMOVED
626 Takes a file handle originally rceeived by the "to_fh" method and
627 creates a new "AnyEvent:Fork" object. The child process itself will
628 not change in any way, i.e. it will keep all the modifications done
629 to it before calling "to_fh".
630
631 The new object is very much like the original object, except that
632 the "pid" method will return "undef" even if the process is a direct
633 child.
634
509PERFORMANCE 635PERFORMANCE
510 Now for some unscientific benchmark numbers (all done on an amd64 636 Now for some unscientific benchmark numbers (all done on an amd64
511 GNU/Linux box). These are intended to give you an idea of the relative 637 GNU/Linux box). These are intended to give you an idea of the relative
512 performance you can expect, they are not meant to be absolute 638 performance you can expect, they are not meant to be absolute
513 performance numbers. 639 performance numbers.
… …
519 645
520 2079 new processes per second, using manual socketpair + fork 646 2079 new processes per second, using manual socketpair + fork
521 647
522 Then I did the same thing, but instead of calling fork, I called 648 Then I did the same thing, but instead of calling fork, I called
523 AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the 649 AnyEvent::Fork->new->run ("CORE::exit") and then again waited for the
524 socket form the child to close on exit. This does the same thing as 650 socket from the child to close on exit. This does the same thing as
525 manual socket pair + fork, except that what is forked is the template 651 manual socket pair + fork, except that what is forked is the template
526 process (2440kB), and the socket needs to be passed to the server at the 652 process (2440kB), and the socket needs to be passed to the server at the
527 other end of the socket first. 653 other end of the socket first.
528 654
529 2307 new processes per second, using AnyEvent::Fork->new 655 2307 new processes per second, using AnyEvent::Fork->new
… …
629 and sweat to make it so, mostly due to the bloody broken perl that 755 and sweat to make it so, mostly due to the bloody broken perl that
630 nobody seems to care about. The fork emulation is a bad joke - I have 756 nobody seems to care about. The fork emulation is a bad joke - I have
631 yet to see something useful that you can do with it without running into 757 yet to see something useful that you can do with it without running into
632 memory corruption issues or other braindamage. Hrrrr. 758 memory corruption issues or other braindamage. Hrrrr.
633 759
760 Since fork is endlessly broken on win32 perls (it doesn't even remotely
761 work within it's documented limits) and quite obviously it's not getting
762 improved any time soon, the best way to proceed on windows would be to
763 always use "new_exec" and thus never rely on perl's fork "emulation".
764
634 Cygwin perl is not supported at the moment due to some hilarious 765 Cygwin perl is not supported at the moment due to some hilarious
635 shortcomings of its API - see IO::FDPoll for more details. 766 shortcomings of its API - see IO::FDPoll for more details. If you never
767 use "send_fh" and always use "new_exec" to create processes, it should
768 work though.
636 769
637SEE ALSO 770SEE ALSO
638 AnyEvent::Fork::Early, to avoid executing a perl interpreter at all 771 AnyEvent::Fork::Early, to avoid executing a perl interpreter at all
639 (part of this distribution). 772 (part of this distribution).
640 773
641 AnyEvent::Fork::Template, to create a process by forking the main 774 AnyEvent::Fork::Template, to create a process by forking the main
642 program at a convenient time (part of this distribution). 775 program at a convenient time (part of this distribution).
643 776
777 AnyEvent::Fork::Remote, for another way to create processes that is
778 mostly compatible to this module and modules building on top of it, but
779 works better with remote processes.
780
644 AnyEvent::Fork::RPC, for simple RPC to child processes (on CPAN). 781 AnyEvent::Fork::RPC, for simple RPC to child processes (on CPAN).
782
783 AnyEvent::Fork::Pool, for simple worker process pool (on CPAN).
645 784
646AUTHOR AND CONTACT INFORMATION 785AUTHOR AND CONTACT INFORMATION
647 Marc Lehmann <schmorp@schmorp.de> 786 Marc Lehmann <schmorp@schmorp.de>
648 http://software.schmorp.de/pkg/AnyEvent-Fork 787 http://software.schmorp.de/pkg/AnyEvent-Fork
649 788

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines