[ViewVC] Diff of: cvs/AnyEvent-Fork-Pool/Pool.pm

Comparing AnyEvent-Fork-Pool/Pool.pm (file contents):
Revision 1.8 by root, Sun Apr 21 12:28:34 2013 UTC vs.
Revision 1.14 by root, Sun Oct 26 16:22:38 2014 UTC

 =head1 NAME
 AnyEvent::Fork::Pool - simple process pool manager on top of AnyEvent::Fork
-THE API IS NOT FINISHED, CONSIDER THIS AN ALPHA RELEASE
 =head1 SYNOPSIS
    use AnyEvent;
+   use AnyEvent::Fork;
    use AnyEvent::Fork::Pool;
-   # use AnyEvent::Fork is not needed
    # all possible parameters shown, with default values
    my $pool = AnyEvent::Fork
       ->new
       ->require ("MyWorker")
    $finish->recv;
 =head1 DESCRIPTION
-This module uses processes created via L<AnyEvent::Fork> and the RPC
+This module uses processes created via L<AnyEvent::Fork> (or
-protocol implement in L<AnyEvent::Fork::RPC> to create a load-balanced
+L<AnyEvent::Fork::Remote>) and the RPC protocol implement in
-pool of processes that handles jobs.
+L<AnyEvent::Fork::RPC> to create a load-balanced pool of processes that
+handles jobs.
 Understanding of L<AnyEvent::Fork> is helpful but not critical to be able
 to use this module, but a thorough understanding of L<AnyEvent::Fork::RPC>
 is, as it defines the actual API that needs to be implemented in the
 worker processes.
-=head1 EXAMPLES
 =head1 PARENT USAGE
 To create a pool, you first have to create a L<AnyEvent::Fork> object -
 this object becomes your template process. Whenever a new worker process
 use Guard ();
 use Array::Heap ();
 use AnyEvent;
-use AnyEvent::Fork; # we don't actually depend on it, this is for convenience
 use AnyEvent::Fork::RPC;
 # these are used for the first and last argument of events
 # in the hope of not colliding. yes, I don't like it either,
 # but didn't come up with an obviously better alternative.
 my $magic0 = ':t6Z@HK1N%Dx@_7?=~-7NQgWDdAs6a,jFN=wLO0*jD*1%P';
 my $magic1 = '<~53rexz.U`!]X[A235^"fyEoiTF\T~oH1l/N6+Djep9b~bI9`\1x%B~vWO1q*';
-our $VERSION = 0.1;
+our $VERSION = 1.1;
 =item my $pool = AnyEvent::Fork::Pool::run $fork, $function, [key => value...]
 The traditional way to call the pool creation function. But it is way
 cooler to call it in the following way:
 to this function are effectively read-only - modifying them after the call
 and before the callback is invoked causes undefined behaviour.
 =cut
+=item $cpus = AnyEvent::Fork::Pool::ncpu [$default_cpus]
+=item ($cpus, $eus) = AnyEvent::Fork::Pool::ncpu [$default_cpus]
+Tries to detect the number of CPUs (C<$cpus> often called CPU cores
+nowadays) and execution units (C<$eus>) which include e.g. extra
+hyperthreaded units). When C<$cpus> cannot be determined reliably,
+C<$default_cpus> is returned for both values, or C<1> if it is missing.
+For normal CPU bound uses, it is wise to have as many worker processes
+as CPUs in the system (C<$cpus>), if nothing else uses the CPU. Using
+hyperthreading is usually detrimental to performance, but in those rare
+cases where that really helps it might be beneficial to use more workers
+(C<$eus>).
+Currently, F</proc/cpuinfo> is parsed on GNU/Linux systems for both
+C<$cpus> and C<$eus>, and on {Free,Net,Open}BSD, F<sysctl -n hw.ncpu> is
+used for C<$cpus>.
+Example: create a worker pool with as many workers as CPU cores, or C<2>,
+if the actual number could not be determined.
+   $fork->AnyEvent::Fork::Pool::run ("myworker::function",
+      max => (scalar AnyEvent::Fork::Pool::ncpu 2),
+   );
+=cut
+BEGIN {
+   if ($^O eq "linux") {
+      *ncpu = sub(;$) {
+         my ($cpus, $eus);
+         if (open my $fh, "<", "/proc/cpuinfo") {
+            my %id;
+            while (<$fh>) {
+               if (/^core id\s*:\s*(\d+)/) {
+                  ++$eus;
+                  undef $id{$1};
+               }
+            }
+            $cpus = scalar keys %id;
+         } else {
+            $cpus = $eus = @_ ? shift : 1;
+         }
+         wantarray ? ($cpus, $eus) : $cpus
+      };
+   } elsif ($^O eq "freebsd" || $^O eq "netbsd" || $^O eq "openbsd") {
+      *ncpu = sub(;$) {
+         my $cpus = qx<sysctl -n hw.ncpu> * 1
+                 || (@_ ? shift : 1);
+         wantarray ? ($cpus, $cpus) : $cpus
+      };
+   } else {
+      *ncpu = sub(;$) {
+         my $cpus = @_ ? shift : 1;
+         wantarray ? ($cpus, $cpus) : $cpus
+      };
+   }
+}
 =back
 =head1 CHILD USAGE
 In addition to the L<AnyEvent::Fork::RPC> API, this module implements one
 =item AnyEvent::Fork::Pool::retire ()
 This function sends an event to the parent process to request retirement:
 the worker is removed from the pool and no new jobs will be sent to it,
-but it has to handle the jobs that are already queued.
+but it still has to handle the jobs that are already queued.
 The parentheses are part of the syntax: the function usually isn't defined
 when you compile your code (because that happens I<before> handing the
 template process over to C<AnyEvent::Fork::Pool::run>, so you need the
 empty parentheses to tell Perl that the function is indeed a function.
 Retiring a worker can be useful to gracefully shut it down when the worker
-deems this useful. For example, after executing a job, one could check
+deems this useful. For example, after executing a job, it could check the
-the process size or the number of jobs handled so far, and if either is
+process size or the number of jobs handled so far, and if either is too
-too high, the worker could ask to get retired, to avoid memory leaks to
+high, the worker could request to be retired, to avoid memory leaks to
 accumulate.
+Example: retire a worker after it has handled roughly 100 requests. It
+doesn't matter whether you retire at the beginning or end of your request,
+as the worker will continue to handle some outstanding requests. Likewise,
+it's ok to call retire multiple times.
+   my $count = 0;
+   sub my::worker {
+      ++$count == 100
+         and AnyEvent::Fork::Pool::retire ();
+      ... normal code goes here
+   }
 =back
 =head1 POOL PARAMETERS RECIPES
-This section describes some recipes for pool paramaters. These are mostly
+This section describes some recipes for pool parameters. These are mostly
 meant for the synchronous RPC backend, as the asynchronous RPC backend
 changes the rules considerably, making workers themselves responsible for
 their scheduling.
 =over 4
 =item high throughput, I/O bound jobs - set load >= 2, max = 1, or very high
 When your jobs are I/O bound, using more workers usually boils down to
 higher throughput, depending very much on your actual workload - sometimes
 having only one worker is best, for example, when you read or write big
-files at maixmum speed, as a second worker will increase seek times.
+files at maximum speed, as a second worker will increase seek times.
 =back
 =head1 EXCEPTIONS
-The same "policy" as with L<AnyEvent::Fork::RPC> applies - exceptins will
+The same "policy" as with L<AnyEvent::Fork::RPC> applies - exceptions
-not be caught, and exceptions in both worker and in callbacks causes
+will not be caught, and exceptions in both worker and in callbacks causes
 undesirable or undefined behaviour.
 =head1 SEE ALSO
 L<AnyEvent::Fork>, to create the processes in the first place.
+L<AnyEvent::Fork::Remote>, likewise, but helpful for remote processes.
 L<AnyEvent::Fork::RPC>, which implements the RPC protocol and API.
 =head1 AUTHOR AND CONTACT INFORMATION

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-Fork-Pool/Pool.pm (file contents): Revision 1.8 by root, Sun Apr 21 12:28:34 2013 UTC vs. Revision 1.14 by root, Sun Oct 26 16:22:38 2014 UTC

Diff Legend

Comparing AnyEvent-Fork-Pool/Pool.pm (file contents):
Revision 1.8 by root, Sun Apr 21 12:28:34 2013 UTC vs.
Revision 1.14 by root, Sun Oct 26 16:22:38 2014 UTC