ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork/Fork.pm
(Generate patch)

Comparing AnyEvent-Fork/Fork.pm (file contents):
Revision 1.29 by root, Sat Apr 6 09:15:49 2013 UTC vs.
Revision 1.47 by root, Thu Apr 18 20:17:34 2013 UTC

27 27
28Special care has been taken to make this module useful from other modules, 28Special care has been taken to make this module useful from other modules,
29while still supporting specialised environments such as L<App::Staticperl> 29while still supporting specialised environments such as L<App::Staticperl>
30or L<PAR::Packer>. 30or L<PAR::Packer>.
31 31
32=head1 WHAT THIS MODULE IS NOT 32=head2 WHAT THIS MODULE IS NOT
33 33
34This module only creates processes and lets you pass file handles and 34This module only creates processes and lets you pass file handles and
35strings to it, and run perl code. It does not implement any kind of RPC - 35strings to it, and run perl code. It does not implement any kind of RPC -
36there is no back channel from the process back to you, and there is no RPC 36there is no back channel from the process back to you, and there is no RPC
37or message passing going on. 37or message passing going on.
38 38
39If you need some form of RPC, you can either implement it yourself 39If you need some form of RPC, you could use the L<AnyEvent::Fork::RPC>
40in whatever way you like, use some message-passing module such 40companion module, which adds simple RPC/job queueing to a process created
41as L<AnyEvent::MP>, some pipe such as L<AnyEvent::ZeroMQ>, use 41by this module.
42L<AnyEvent::Handle> on both sides to send e.g. JSON or Storable messages,
43and so on.
44 42
43Or you can implement it yourself in whatever way you like, use some
44message-passing module such as L<AnyEvent::MP>, some pipe such as
45L<AnyEvent::ZeroMQ>, use L<AnyEvent::Handle> on both sides to send
46e.g. JSON or Storable messages, and so on.
47
48=head2 COMPARISON TO OTHER MODULES
49
50There is an abundance of modules on CPAN that do "something fork", such as
51L<Parallel::ForkManager>, L<AnyEvent::ForkManager>, L<AnyEvent::Worker>
52or L<AnyEvent::Subprocess>. There are modules that implement their own
53process management, such as L<AnyEvent::DBI>.
54
55The problems that all these modules try to solve are real, however, none
56of them (from what I have seen) tackle the very real problems of unwanted
57memory sharing, efficiency, not being able to use event processing or
58similar modules in the processes they create.
59
60This module doesn't try to replace any of them - instead it tries to solve
61the problem of creating processes with a minimum of fuss and overhead (and
62also luxury). Ideally, most of these would use AnyEvent::Fork internally,
63except they were written before AnyEvent:Fork was available, so obviously
64had to roll their own.
65
45=head1 PROBLEM STATEMENT 66=head2 PROBLEM STATEMENT
46 67
47There are two traditional ways to implement parallel processing on UNIX 68There are two traditional ways to implement parallel processing on UNIX
48like operating systems - fork and process, and fork+exec and process. They 69like operating systems - fork and process, and fork+exec and process. They
49have different advantages and disadvantages that I describe below, 70have different advantages and disadvantages that I describe below,
50together with how this module tries to mitigate the disadvantages. 71together with how this module tries to mitigate the disadvantages.
152 173
153 # now $master_filehandle is connected to the 174 # now $master_filehandle is connected to the
154 # $slave_filehandle in the new process. 175 # $slave_filehandle in the new process.
155 }); 176 });
156 177
157 # MyModule::worker might look like this 178C<MyModule> might look like this:
179
180 package MyModule;
181
158 sub MyModule::worker { 182 sub worker {
159 my ($slave_filehandle) = @_; 183 my ($slave_filehandle) = @_;
160 184
161 # now $slave_filehandle is connected to the $master_filehandle 185 # now $slave_filehandle is connected to the $master_filehandle
162 # in the original prorcess. have fun! 186 # in the original prorcess. have fun!
163 } 187 }
182 } 206 }
183 207
184 # now do other things - maybe use the filehandle provided by run 208 # now do other things - maybe use the filehandle provided by run
185 # to wait for the processes to die. or whatever. 209 # to wait for the processes to die. or whatever.
186 210
187 # My::Server::run might look like this 211C<My::Server> might look like this:
188 sub My::Server::run { 212
213 package My::Server;
214
215 sub run {
189 my ($slave, $listener, $id) = @_; 216 my ($slave, $listener, $id) = @_;
190 217
191 close $slave; # we do not use the socket, so close it to save resources 218 close $slave; # we do not use the socket, so close it to save resources
192 219
193 # we could go ballistic and use e.g. AnyEvent here, or IO::AIO, 220 # we could go ballistic and use e.g. AnyEvent here, or IO::AIO,
197 } 224 }
198 } 225 }
199 226
200=head2 use AnyEvent::Fork as a faster fork+exec 227=head2 use AnyEvent::Fork as a faster fork+exec
201 228
202This runs /bin/echo hi, with stdout redirected to /tmp/log and stderr to 229This runs C</bin/echo hi>, with standard output redirected to F</tmp/log>
203the communications socket. It is usually faster than fork+exec, but still 230and standard error redirected to the communications socket. It is usually
204let's you prepare the environment. 231faster than fork+exec, but still lets you prepare the environment.
205 232
206 open my $output, ">/tmp/log" or die "$!"; 233 open my $output, ">/tmp/log" or die "$!";
207 234
208 AnyEvent::Fork 235 AnyEvent::Fork
209 ->new 236 ->new
210 ->eval (' 237 ->eval ('
238 # compile a helper function for later use
211 sub run { 239 sub run {
212 my ($fh, $output, @cmd) = @_; 240 my ($fh, $output, @cmd) = @_;
213 241
214 # perl will clear close-on-exec on STDOUT/STDERR 242 # perl will clear close-on-exec on STDOUT/STDERR
215 open STDOUT, ">&", $output or die; 243 open STDOUT, ">&", $output or die;
226 254
227=head1 CONCEPTS 255=head1 CONCEPTS
228 256
229This module can create new processes either by executing a new perl 257This module can create new processes either by executing a new perl
230process, or by forking from an existing "template" process. 258process, or by forking from an existing "template" process.
259
260All these processes are called "child processes" (whether they are direct
261children or not), while the process that manages them is called the
262"parent process".
231 263
232Each such process comes with its own file handle that can be used to 264Each such process comes with its own file handle that can be used to
233communicate with it (it's actually a socket - one end in the new process, 265communicate with it (it's actually a socket - one end in the new process,
234one end in the main process), and among the things you can do in it are 266one end in the main process), and among the things you can do in it are
235load modules, fork new processes, send file handles to it, and execute 267load modules, fork new processes, send file handles to it, and execute
345use AnyEvent; 377use AnyEvent;
346use AnyEvent::Util (); 378use AnyEvent::Util ();
347 379
348use IO::FDPass; 380use IO::FDPass;
349 381
350our $VERSION = 0.5; 382our $VERSION = 0.7;
351
352our $PERL; # the path to the perl interpreter, deduces with various forms of magic
353
354=over 4
355
356=back
357
358=cut
359 383
360# the early fork template process 384# the early fork template process
361our $EARLY; 385our $EARLY;
362 386
363# the empty template process 387# the empty template process
364our $TEMPLATE; 388our $TEMPLATE;
389
390sub QUEUE() { 0 }
391sub FH() { 1 }
392sub WW() { 2 }
393sub PID() { 3 }
394sub CB() { 4 }
395
396sub _new {
397 my ($self, $fh, $pid) = @_;
398
399 AnyEvent::Util::fh_nonblocking $fh, 1;
400
401 $self = bless [
402 [], # write queue - strings or fd's
403 $fh,
404 undef, # AE watcher
405 $pid,
406 ], $self;
407
408 $self
409}
365 410
366sub _cmd { 411sub _cmd {
367 my $self = shift; 412 my $self = shift;
368 413
369 # ideally, we would want to use "a (w/a)*" as format string, but perl 414 # ideally, we would want to use "a (w/a)*" as format string, but perl
370 # versions from at least 5.8.9 to 5.16.3 are all buggy and can't unpack 415 # versions from at least 5.8.9 to 5.16.3 are all buggy and can't unpack
371 # it. 416 # it.
372 push @{ $self->[2] }, pack "a L/a*", $_[0], $_[1]; 417 push @{ $self->[QUEUE] }, pack "a L/a*", $_[0], $_[1];
373 418
374 $self->[3] ||= AE::io $self->[1], 1, sub { 419 $self->[WW] ||= AE::io $self->[FH], 1, sub {
375 do { 420 do {
376 # send the next "thing" in the queue - either a reference to an fh, 421 # send the next "thing" in the queue - either a reference to an fh,
377 # or a plain string. 422 # or a plain string.
378 423
379 if (ref $self->[2][0]) { 424 if (ref $self->[QUEUE][0]) {
380 # send fh 425 # send fh
381 unless (IO::FDPass::send fileno $self->[1], fileno ${ $self->[2][0] }) { 426 unless (IO::FDPass::send fileno $self->[FH], fileno ${ $self->[QUEUE][0] }) {
382 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK; 427 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
383 undef $self->[3]; 428 undef $self->[WW];
384 die "AnyEvent::Fork: file descriptor send failure: $!"; 429 die "AnyEvent::Fork: file descriptor send failure: $!";
385 } 430 }
386 431
387 shift @{ $self->[2] }; 432 shift @{ $self->[QUEUE] };
388 433
389 } else { 434 } else {
390 # send string 435 # send string
391 my $len = syswrite $self->[1], $self->[2][0]; 436 my $len = syswrite $self->[FH], $self->[QUEUE][0];
392 437
393 unless ($len) { 438 unless ($len) {
394 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK; 439 return if $! == Errno::EAGAIN || $! == Errno::EWOULDBLOCK;
395 undef $self->[3]; 440 undef $self->[3];
396 die "AnyEvent::Fork: command write failure: $!"; 441 die "AnyEvent::Fork: command write failure: $!";
397 } 442 }
398 443
399 substr $self->[2][0], 0, $len, ""; 444 substr $self->[QUEUE][0], 0, $len, "";
400 shift @{ $self->[2] } unless length $self->[2][0]; 445 shift @{ $self->[QUEUE] } unless length $self->[QUEUE][0];
401 } 446 }
402 } while @{ $self->[2] }; 447 } while @{ $self->[QUEUE] };
403 448
404 # everything written 449 # everything written
405 undef $self->[3]; 450 undef $self->[WW];
406 451
407 # invoke run callback, if any 452 # invoke run callback, if any
408 $self->[4]->($self->[1]) if $self->[4]; 453 $self->[CB]->($self->[FH]) if $self->[CB];
409 }; 454 };
410 455
411 () # make sure we don't leak the watcher 456 () # make sure we don't leak the watcher
412}
413
414sub _new {
415 my ($self, $fh, $pid) = @_;
416
417 AnyEvent::Util::fh_nonblocking $fh, 1;
418
419 $self = bless [
420 $pid,
421 $fh,
422 [], # write queue - strings or fd's
423 undef, # AE watcher
424 ], $self;
425
426 $self
427} 457}
428 458
429# fork template from current process, used by AnyEvent::Fork::Early/Template 459# fork template from current process, used by AnyEvent::Fork::Early/Template
430sub _new_fork { 460sub _new_fork {
431 my ($fh, $slave) = AnyEvent::Util::portable_socketpair; 461 my ($fh, $slave) = AnyEvent::Util::portable_socketpair;
436 if ($pid eq 0) { 466 if ($pid eq 0) {
437 require AnyEvent::Fork::Serve; 467 require AnyEvent::Fork::Serve;
438 $AnyEvent::Fork::Serve::OWNER = $parent; 468 $AnyEvent::Fork::Serve::OWNER = $parent;
439 close $fh; 469 close $fh;
440 $0 = "$_[1] of $parent"; 470 $0 = "$_[1] of $parent";
441 $SIG{CHLD} = 'IGNORE';
442 AnyEvent::Fork::Serve::serve ($slave); 471 AnyEvent::Fork::Serve::serve ($slave);
443 exit 0; 472 exit 0;
444 } elsif (!$pid) { 473 } elsif (!$pid) {
445 die "AnyEvent::Fork::Early/Template: unable to fork template process: $!"; 474 die "AnyEvent::Fork::Early/Template: unable to fork template process: $!";
446 } 475 }
553} 582}
554 583
555=item $pid = $proc->pid 584=item $pid = $proc->pid
556 585
557Returns the process id of the process I<iff it is a direct child of the 586Returns the process id of the process I<iff it is a direct child of the
558process> running AnyEvent::Fork, and C<undef> otherwise. 587process running AnyEvent::Fork>, and C<undef> otherwise.
559 588
560Normally, only processes created via C<< AnyEvent::Fork->new_exec >> and 589Normally, only processes created via C<< AnyEvent::Fork->new_exec >> and
561L<AnyEvent::Fork::Template> are direct children, and you are responsible 590L<AnyEvent::Fork::Template> are direct children, and you are responsible
562to clean up their zombies when they die. 591to clean up their zombies when they die.
563 592
564All other processes are not direct children, and will be cleaned up by 593All other processes are not direct children, and will be cleaned up by
565AnyEvent::Fork. 594AnyEvent::Fork itself.
566 595
567=cut 596=cut
568 597
569sub pid { 598sub pid {
570 $_[0][0] 599 $_[0][PID]
571} 600}
572 601
573=item $proc = $proc->eval ($perlcode, @args) 602=item $proc = $proc->eval ($perlcode, @args)
574 603
575Evaluates the given C<$perlcode> as ... perl code, while setting C<@_> to 604Evaluates the given C<$perlcode> as ... Perl code, while setting C<@_> to
576the strings specified by C<@args>, in the "main" package. 605the strings specified by C<@args>, in the "main" package.
577 606
578This call is meant to do any custom initialisation that might be required 607This call is meant to do any custom initialisation that might be required
579(for example, the C<require> method uses it). It's not supposed to be used 608(for example, the C<require> method uses it). It's not supposed to be used
580to completely take over the process, use C<run> for that. 609to completely take over the process, use C<run> for that.
581 610
582The code will usually be executed after this call returns, and there is no 611The code will usually be executed after this call returns, and there is no
583way to pass anything back to the calling process. Any evaluation errors 612way to pass anything back to the calling process. Any evaluation errors
584will be reported to stderr and cause the process to exit. 613will be reported to stderr and cause the process to exit.
585 614
586If you want to execute some code to take over the process (see the 615If you want to execute some code (that isn't in a module) to take over the
587"fork+exec" example in the SYNOPSIS), you should compile a function via 616process, you should compile a function via C<eval> first, and then call
588C<eval> first, and then call it via C<run>. This also gives you access to 617it via C<run>. This also gives you access to any arguments passed via the
589any arguments passed via the C<send_xxx> methods, such as file handles. 618C<send_xxx> methods, such as file handles. See the L<use AnyEvent::Fork as
619a faster fork+exec> example to see it in action.
590 620
591Returns the process object for easy chaining of method calls. 621Returns the process object for easy chaining of method calls.
592 622
593=cut 623=cut
594 624
620=item $proc = $proc->send_fh ($handle, ...) 650=item $proc = $proc->send_fh ($handle, ...)
621 651
622Send one or more file handles (I<not> file descriptors) to the process, 652Send one or more file handles (I<not> file descriptors) to the process,
623to prepare a call to C<run>. 653to prepare a call to C<run>.
624 654
625The process object keeps a reference to the handles until this is done, 655The process object keeps a reference to the handles until they have
626so you must not explicitly close the handles. This is most easily 656been passed over to the process, so you must not explicitly close the
627accomplished by simply not storing the file handles anywhere after passing 657handles. This is most easily accomplished by simply not storing the file
628them to this method. 658handles anywhere after passing them to this method - when AnyEvent::Fork
659is finished using them, perl will automatically close them.
629 660
630Returns the process object for easy chaining of method calls. 661Returns the process object for easy chaining of method calls.
631 662
632Example: pass a file handle to a process, and release it without 663Example: pass a file handle to a process, and release it without
633closing. It will be closed automatically when it is no longer used. 664closing. It will be closed automatically when it is no longer used.
640sub send_fh { 671sub send_fh {
641 my ($self, @fh) = @_; 672 my ($self, @fh) = @_;
642 673
643 for my $fh (@fh) { 674 for my $fh (@fh) {
644 $self->_cmd ("h"); 675 $self->_cmd ("h");
645 push @{ $self->[2] }, \$fh; 676 push @{ $self->[QUEUE] }, \$fh;
646 } 677 }
647 678
648 $self 679 $self
649} 680}
650 681
651=item $proc = $proc->send_arg ($string, ...) 682=item $proc = $proc->send_arg ($string, ...)
652 683
653Send one or more argument strings to the process, to prepare a call to 684Send one or more argument strings to the process, to prepare a call to
654C<run>. The strings can be any octet string. 685C<run>. The strings can be any octet strings.
655 686
656The protocol is optimised to pass a moderate number of relatively short 687The protocol is optimised to pass a moderate number of relatively short
657strings - while you can pass up to 4GB of data in one go, this is more 688strings - while you can pass up to 4GB of data in one go, this is more
658meant to pass some ID information or other startup info, not big chunks of 689meant to pass some ID information or other startup info, not big chunks of
659data. 690data.
675Enter the function specified by the function name in C<$func> in the 706Enter the function specified by the function name in C<$func> in the
676process. The function is called with the communication socket as first 707process. The function is called with the communication socket as first
677argument, followed by all file handles and string arguments sent earlier 708argument, followed by all file handles and string arguments sent earlier
678via C<send_fh> and C<send_arg> methods, in the order they were called. 709via C<send_fh> and C<send_arg> methods, in the order they were called.
679 710
711The process object becomes unusable on return from this function - any
712further method calls result in undefined behaviour.
713
680The function name should be fully qualified, but if it isn't, it will be 714The function name should be fully qualified, but if it isn't, it will be
681looked up in the main package. 715looked up in the C<main> package.
682 716
683If the called function returns, doesn't exist, or any error occurs, the 717If the called function returns, doesn't exist, or any error occurs, the
684process exits. 718process exits.
685 719
686Preparing the process is done in the background - when all commands have 720Preparing the process is done in the background - when all commands have
687been sent, the callback is invoked with the local communications socket 721been sent, the callback is invoked with the local communications socket
688as argument. At this point you can start using the socket in any way you 722as argument. At this point you can start using the socket in any way you
689like. 723like.
690
691The process object becomes unusable on return from this function - any
692further method calls result in undefined behaviour.
693 724
694If the communication socket isn't used, it should be closed on both sides, 725If the communication socket isn't used, it should be closed on both sides,
695to save on kernel memory. 726to save on kernel memory.
696 727
697The socket is non-blocking in the parent, and blocking in the newly 728The socket is non-blocking in the parent, and blocking in the newly
736=cut 767=cut
737 768
738sub run { 769sub run {
739 my ($self, $func, $cb) = @_; 770 my ($self, $func, $cb) = @_;
740 771
741 $self->[4] = $cb; 772 $self->[CB] = $cb;
742 $self->_cmd (r => $func); 773 $self->_cmd (r => $func);
743} 774}
744 775
745=back 776=back
746 777
772 479 vfork+execs per second, using AnyEvent::Fork->new_exec 803 479 vfork+execs per second, using AnyEvent::Fork->new_exec
773 804
774So how can C<< AnyEvent->new >> be faster than a standard fork, even 805So how can C<< AnyEvent->new >> be faster than a standard fork, even
775though it uses the same operations, but adds a lot of overhead? 806though it uses the same operations, but adds a lot of overhead?
776 807
777The difference is simply the process size: forking the 6MB process takes 808The difference is simply the process size: forking the 5MB process takes
778so much longer than forking the 2.5MB template process that the overhead 809so much longer than forking the 2.5MB template process that the extra
779introduced is canceled out. 810overhead is canceled out.
780 811
781If the benchmark process grows, the normal fork becomes even slower: 812If the benchmark process grows, the normal fork becomes even slower:
782 813
783 1340 new processes, manual fork in a 20MB process 814 1340 new processes, manual fork of a 20MB process
784 731 new processes, manual fork in a 200MB process 815 731 new processes, manual fork of a 200MB process
785 235 new processes, manual fork in a 2000MB process 816 235 new processes, manual fork of a 2000MB process
786 817
787What that means (to me) is that I can use this module without having a 818What that means (to me) is that I can use this module without having a bad
788very bad conscience because of the extra overhead required to start new 819conscience because of the extra overhead required to start new processes.
789processes.
790 820
791=head1 TYPICAL PROBLEMS 821=head1 TYPICAL PROBLEMS
792 822
793This section lists typical problems that remain. I hope by recognising 823This section lists typical problems that remain. I hope by recognising
794them, most can be avoided. 824them, most can be avoided.
795 825
796=over 4 826=over 4
797 827
798=item "leaked" file descriptors for exec'ed processes 828=item leaked file descriptors for exec'ed processes
799 829
800POSIX systems inherit file descriptors by default when exec'ing a new 830POSIX systems inherit file descriptors by default when exec'ing a new
801process. While perl itself laudably sets the close-on-exec flags on new 831process. While perl itself laudably sets the close-on-exec flags on new
802file handles, most C libraries don't care, and even if all cared, it's 832file handles, most C libraries don't care, and even if all cared, it's
803often not possible to set the flag in a race-free manner. 833often not possible to set the flag in a race-free manner.
823libraries or the code that leaks those file descriptors. 853libraries or the code that leaks those file descriptors.
824 854
825Fortunately, most of these leaked descriptors do no harm, other than 855Fortunately, most of these leaked descriptors do no harm, other than
826sitting on some resources. 856sitting on some resources.
827 857
828=item "leaked" file descriptors for fork'ed processes 858=item leaked file descriptors for fork'ed processes
829 859
830Normally, L<AnyEvent::Fork> does start new processes by exec'ing them, 860Normally, L<AnyEvent::Fork> does start new processes by exec'ing them,
831which closes file descriptors not marked for being inherited. 861which closes file descriptors not marked for being inherited.
832 862
833However, L<AnyEvent::Fork::Early> and L<AnyEvent::Fork::Template> offer 863However, L<AnyEvent::Fork::Early> and L<AnyEvent::Fork::Template> offer
842 872
843The solution is to either not load these modules before use'ing 873The solution is to either not load these modules before use'ing
844L<AnyEvent::Fork::Early> or L<AnyEvent::Fork::Template>, or to delay 874L<AnyEvent::Fork::Early> or L<AnyEvent::Fork::Template>, or to delay
845initialising them, for example, by calling C<init Gtk2> manually. 875initialising them, for example, by calling C<init Gtk2> manually.
846 876
847=item exit runs destructors 877=item exiting calls object destructors
848 878
849This only applies to users of Lc<AnyEvent::Fork:Early> and 879This only applies to users of L<AnyEvent::Fork:Early> and
850L<AnyEvent::Fork::Template>. 880L<AnyEvent::Fork::Template>, or when initialising code creates objects
881that reference external resources.
851 882
852When a process created by AnyEvent::Fork exits, it might do so by calling 883When a process created by AnyEvent::Fork exits, it might do so by calling
853exit, or simply letting perl reach the end of the program. At which point 884exit, or simply letting perl reach the end of the program. At which point
854Perl runs all destructors. 885Perl runs all destructors.
855 886
874to make it so, mostly due to the bloody broken perl that nobody seems to 905to make it so, mostly due to the bloody broken perl that nobody seems to
875care about. The fork emulation is a bad joke - I have yet to see something 906care about. The fork emulation is a bad joke - I have yet to see something
876useful that you can do with it without running into memory corruption 907useful that you can do with it without running into memory corruption
877issues or other braindamage. Hrrrr. 908issues or other braindamage. Hrrrr.
878 909
879Cygwin perl is not supported at the moment, as it should implement fd 910Cygwin perl is not supported at the moment due to some hilarious
880passing, but doesn't, and rolling my own is hard, as cygwin doesn't 911shortcomings of its API - see L<IO::FDPoll> for more details.
881support enough functionality to do it.
882 912
883=head1 SEE ALSO 913=head1 SEE ALSO
884 914
885L<AnyEvent::Fork::Early> (to avoid executing a perl interpreter), 915L<AnyEvent::Fork::Early>, to avoid executing a perl interpreter at all
916(part of this distribution).
917
886L<AnyEvent::Fork::Template> (to create a process by forking the main 918L<AnyEvent::Fork::Template>, to create a process by forking the main
887program at a convenient time). 919program at a convenient time (part of this distribution).
888 920
889=head1 AUTHOR 921L<AnyEvent::Fork::RPC>, for simple RPC to child processes (on CPAN).
922
923=head1 AUTHOR AND CONTACT INFORMATION
890 924
891 Marc Lehmann <schmorp@schmorp.de> 925 Marc Lehmann <schmorp@schmorp.de>
892 http://home.schmorp.de/ 926 http://software.schmorp.de/pkg/AnyEvent-Fork
893 927
894=cut 928=cut
895 929
8961 9301
897 931

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines