ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-Fork-Pool/Pool.pm
(Generate patch)

Comparing AnyEvent-Fork-Pool/Pool.pm (file contents):
Revision 1.7 by root, Sun Apr 21 12:03:02 2013 UTC vs.
Revision 1.11 by root, Sun Apr 28 14:19:22 2013 UTC

1=head1 NAME 1=head1 NAME
2 2
3AnyEvent::Fork::Pool - simple process pool manager on top of AnyEvent::Fork 3AnyEvent::Fork::Pool - simple process pool manager on top of AnyEvent::Fork
4
5THE API IS NOT FINISHED, CONSIDER THIS AN ALPHA RELEASE
4 6
5=head1 SYNOPSIS 7=head1 SYNOPSIS
6 8
7 use AnyEvent; 9 use AnyEvent;
8 use AnyEvent::Fork::Pool; 10 use AnyEvent::Fork::Pool;
… …
87 89
88use Guard (); 90use Guard ();
89use Array::Heap (); 91use Array::Heap ();
90 92
91use AnyEvent; 93use AnyEvent;
94# explicit version on next line, as some cpan-testers test with the 0.1 version,
95# ignoring dependencies, and this line will at least give a clear indication of that.
92use AnyEvent::Fork; # we don't actually depend on it, this is for convenience 96use AnyEvent::Fork 0.6; # we don't actually depend on it, this is for convenience
93use AnyEvent::Fork::RPC; 97use AnyEvent::Fork::RPC;
94 98
95# these are used for the first and last argument of events 99# these are used for the first and last argument of events
96# in the hope of not colliding. yes, I don't like it either, 100# in the hope of not colliding. yes, I don't like it either,
97# but didn't come up with an obviously better alternative. 101# but didn't come up with an obviously better alternative.
98my $magic0 = ':t6Z@HK1N%Dx@_7?=~-7NQgWDdAs6a,jFN=wLO0*jD*1%P'; 102my $magic0 = ':t6Z@HK1N%Dx@_7?=~-7NQgWDdAs6a,jFN=wLO0*jD*1%P';
99my $magic1 = '<~53rexz.U`!]X[A235^"fyEoiTF\T~oH1l/N6+Djep9b~bI9`\1x%B~vWO1q*'; 103my $magic1 = '<~53rexz.U`!]X[A235^"fyEoiTF\T~oH1l/N6+Djep9b~bI9`\1x%B~vWO1q*';
100 104
101our $VERSION = 0.1; 105our $VERSION = 1.1;
102 106
103=item my $pool = AnyEvent::Fork::Pool::run $fork, $function, [key => value...] 107=item my $pool = AnyEvent::Fork::Pool::run $fork, $function, [key => value...]
104 108
105The traditional way to call the pool creation function. But it is way 109The traditional way to call the pool creation function. But it is way
106cooler to call it in the following way: 110cooler to call it in the following way:
… …
444to this function are effectively read-only - modifying them after the call 448to this function are effectively read-only - modifying them after the call
445and before the callback is invoked causes undefined behaviour. 449and before the callback is invoked causes undefined behaviour.
446 450
447=cut 451=cut
448 452
453=item $cpus = AnyEvent::Fork::Pool::ncpu [$default_cpus]
454
455=item ($cpus, $eus) = AnyEvent::Fork::Pool::ncpu [$default_cpus]
456
457Tries to detect the number of CPUs (C<$cpus> often called cpu cores
458nowadays) and execution units (C<$eus>) which include e.g. extra
459hyperthreaded units). When C<$cpus> cannot be determined reliably,
460C<$default_cpus> is returned for both values, or C<1> if it is missing.
461
462For normal CPU bound uses, it is wise to have as many worker processes
463as CPUs in the system (C<$cpus>), if nothing else uses the CPU. Using
464hyperthreading is usually detrimental to performance, but in those rare
465cases where that really helps it might be beneficial to use more workers
466(C<$eus>).
467
468Currently, F</proc/cpuinfo> is parsed on GNU/Linux systems for both
469C<$cpus> and C<$eu>, and on {Free,Net,Open}BSD, F<sysctl -n hw.ncpu> is
470used for C<$cpus>.
471
472Example: create a worker pool with as many workers as cpu cores, or C<2>,
473if the actual number could not be determined.
474
475 $fork->AnyEvent::Fork::Pool::run ("myworker::function",
476 max => (scalar AnyEvent::Fork::Pool::ncpu 2),
477 );
478
479=cut
480
481BEGIN {
482 if ($^O eq "linux") {
483 *ncpu = sub(;$) {
484 my ($cpus, $eus);
485
486 if (open my $fh, "<", "/proc/cpuinfo") {
487 my %id;
488
489 while (<$fh>) {
490 if (/^core id\s*:\s*(\d+)/) {
491 ++$eus;
492 undef $id{$1};
493 }
494 }
495
496 $cpus = scalar keys %id;
497 } else {
498 $cpus = $eus = @_ ? shift : 1;
499 }
500 wantarray ? ($cpus, $eus) : $cpus
501 };
502 } elsif ($^O eq "freebsd" || $^O eq "netbsd" || $^O eq "openbsd") {
503 *ncpu = sub(;$) {
504 my $cpus = qx<sysctl -n hw.ncpu> * 1
505 || (@_ ? shift : 1);
506 wantarray ? ($cpus, $cpus) : $cpus
507 };
508 } else {
509 *ncpu = sub(;$) {
510 my $cpus = @_ ? shift : 1;
511 wantarray ? ($cpus, $cpus) : $cpus
512 };
513 }
514}
515
449=back 516=back
450 517
451=head1 CHILD USAGE 518=head1 CHILD USAGE
452 519
453In addition to the L<AnyEvent::Fork::RPC> API, this module implements one 520In addition to the L<AnyEvent::Fork::RPC> API, this module implements one
… …
470deems this useful. For example, after executing a job, one could check 537deems this useful. For example, after executing a job, one could check
471the process size or the number of jobs handled so far, and if either is 538the process size or the number of jobs handled so far, and if either is
472too high, the worker could ask to get retired, to avoid memory leaks to 539too high, the worker could ask to get retired, to avoid memory leaks to
473accumulate. 540accumulate.
474 541
542Example: retire a worker after it has handled roughly 100 requests.
543
544 my $count = 0;
545
546 sub my::worker {
547
548 ++$count == 100
549 and AnyEvent::Fork::Pool::retire ();
550
551 ... normal code goes here
552 }
553
475=back 554=back
476 555
477=head1 POOL PARAMETERS RECIPES 556=head1 POOL PARAMETERS RECIPES
478 557
479This section describes some recipes for pool paramaters. These are mostly 558This section describes some recipes for pool paramaters. These are mostly

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines