ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/cvsroot/AnyEvent-SNMP/SNMP.pm
(Generate patch)

Comparing cvsroot/AnyEvent-SNMP/SNMP.pm (file contents):
Revision 1.7 by root, Sun Aug 2 14:24:23 2009 UTC vs.
Revision 1.11 by root, Sun Oct 31 18:26:27 2010 UTC

… …
23 my @result = $cv->wait; 23 my @result = $cv->wait;
24 24
25=head1 DESCRIPTION 25=head1 DESCRIPTION
26 26
27This module implements an alternative "event dispatcher" for Net::SNMP, 27This module implements an alternative "event dispatcher" for Net::SNMP,
28using AnyEvent as a backend. 28using AnyEvent as a backend. This integrates Net::SNMP into AnyEvent. That
29 29means you can make non-blocking Net::SNMP calls and as long as other
30This integrates Net::SNMP into AnyEvent: You can make non-blocking 30parts of your program also use AnyEvent (or some event loop supported by
31Net::SNMP calls and as long as other parts of your program also use 31AnyEvent), they will run in parallel.
32AnyEvent (or some event loop supported by AnyEvent), they will run in
33parallel.
34 32
35Also, the Net::SNMP scheduler is very inefficient with respect to both CPU 33Also, the Net::SNMP scheduler is very inefficient with respect to both CPU
36and memory usage. Most AnyEvent backends (including the pure-perl backend) 34and memory usage. Most AnyEvent backends (including the pure-perl backend)
37fare much better than the Net::SNMP dispatcher. 35fare much better than the Net::SNMP dispatcher.
38 36
37Another major added fetaure of this module over Net::SNMP is automatic
38rate-adjustments: Net::SNMP is so slow that firing a few thousand
39requests can cause many timeouts simply because Net::SNMP cannot process
40the replies in time. This module automatically adapts the send rate to
41avoid false timeouts caused by slow reply processing.
42
39A potential disadvantage is that replacing the dispatcher is not at all 43A potential disadvantage of this module is that replacing the dispatcher
40a documented thing to do, so future changes in Net::SNP might break this 44is not at all a documented thing to do, so future changes in Net::SNP
41module (or the many similar ones). 45might break this module (or the many similar ones).
42 46
43This module does not export anything and does not require you to do 47This module does not export anything and does not require you to do
44anything special apart from loading it I<before doing any non-blocking 48anything special apart from loading it I<before doing any non-blocking
45requests with Net::SNMP>. It is recommended but not required to load this 49requests with Net::SNMP>. It is recommended but not required to load this
46module before C<Net::SNMP>. 50module before C<Net::SNMP>.
… …
71 75
72Unfortunately, this number depends not only on processing speed and load 76Unfortunately, this number depends not only on processing speed and load
73of the machine running Net::SNMP, but also on the network latency and the 77of the machine running Net::SNMP, but also on the network latency and the
74speed of your SNMP agents. 78speed of your SNMP agents.
75 79
76AnyEvent::SNMP tries to dynamically adjust this number dynamically upwards 80AnyEvent::SNMP tries to dynamically adjust this number upwards and
77and downwards. 81downwards.
78 82
79Increasing C<$MAX_OUTSTANDING> will not automatically use the 83Increasing C<$MAX_OUTSTANDING> will not automatically use the
80C<extra request slots. To increase $MAX_OUTSTANDING> and make 84extra request slots. To increase C<$MAX_OUTSTANDING> and make
81C<C<AnyEvent::SNMP> make use of the extra paralellity, call 85C<AnyEvent::SNMP> make use of the extra paralellity, call
82C<AnyEvent::SNMP::set_max_outstanding> with the new value, e.g.: 86C<AnyEvent::SNMP::set_max_outstanding> with the new value, e.g.:
83 87
84 AnyEvent::SNMP::set_max_outstanding 500; 88 AnyEvent::SNMP::set_max_outstanding 500;
85 89
86Although due to the dynamic adjustment, this might have little lasting 90Although due to the dynamic adjustment, this might have little lasting
… …
131 135
132=cut 136=cut
133 137
134package AnyEvent::SNMP; 138package AnyEvent::SNMP;
135 139
136no warnings; 140use common::sense;
137use strict qw(subs vars);
138 141
139# it is possible to do this without loading 142# it is possible to do this without loading
140# Net::SNMP::Dispatcher, but much more awkward. 143# Net::SNMP::Dispatcher, but much more awkward.
141use Net::SNMP::Dispatcher; 144use Net::SNMP::Dispatcher;
142 145
… …
145} 148}
146 149
147use Net::SNMP (); 150use Net::SNMP ();
148use AnyEvent (); 151use AnyEvent ();
149 152
150our $VERSION = '0.2'; 153our $VERSION = '1.0';
151 154
152$Net::SNMP::DISPATCHER = instance Net::SNMP::Dispatcher; 155$Net::SNMP::DISPATCHER = instance Net::SNMP::Dispatcher;
153 156
154our $MESSAGE_PROCESSING = $Net::SNMP::Dispatcher::MESSAGE_PROCESSING; 157our $MESSAGE_PROCESSING = $Net::SNMP::Dispatcher::MESSAGE_PROCESSING;
155 158
156# avoid the method call
157my $timer = sub { shift->timer (@_) };
158AnyEvent::post_detect { $timer = AnyEvent->can ("timer") };
159
160our $BUSY; 159our $BUSY;
160our $DONE; # finished all jobs
161our @TRANSPORT; # fileno => [count, watcher] 161our @TRANSPORT; # fileno => [count, watcher]
162our @QUEUE; 162our @QUEUE;
163our $MAX_OUTSTANDING = 50; 163our $MAX_OUTSTANDING = 50;
164our $MIN_RECVQUEUE = 8; 164our $MIN_RECVQUEUE = 8;
165our $MAX_RECVQUEUE = 64; 165our $MAX_RECVQUEUE = 64;
… …
188 $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id) 188 $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id)
189 if $pdu->expect_response; 189 if $pdu->expect_response;
190 190
191 # A crude attempt to recover from temporary failures. 191 # A crude attempt to recover from temporary failures.
192 if ($retries-- > 0 && ($!{EAGAIN} || $!{EWOULDBLOCK} || $!{ENOSPC})) { 192 if ($retries-- > 0 && ($!{EAGAIN} || $!{EWOULDBLOCK} || $!{ENOSPC})) {
193 my $retry_w; $retry_w = AnyEvent->$timer (after => $pdu->timeout, cb => sub { 193 my $retry_w; $retry_w = AE::timer $pdu->timeout, 0, sub {
194 undef $retry_w; 194 undef $retry_w;
195 _send_pdu ($pdu, $retries); 195 _send_pdu ($pdu, $retries);
196 }); 196 };
197 } else { 197 } else {
198 --$BUSY; 198 --$BUSY;
199 kick_job; 199 kick_job;
200 } 200 }
201 201
… …
209 my $transport = $msg->transport; 209 my $transport = $msg->transport;
210 my $fileno = $transport->fileno; 210 my $fileno = $transport->fileno;
211 211
212 # register the transport 212 # register the transport
213 unless ($TRANSPORT[$fileno][0]++) { 213 unless ($TRANSPORT[$fileno][0]++) {
214 $TRANSPORT[$fileno][1] = AnyEvent->io (fh => $transport->socket, poll => 'r', cb => sub { 214 $TRANSPORT[$fileno][1] = AE::io $transport->socket, 0, sub {
215 for my $count (1..$MAX_RECVQUEUE) { # handle up to this many requests in one go 215 for my $count (1..$MAX_RECVQUEUE) { # handle up to this many requests in one go
216 # Create a new Message object to receive the response 216 # Create a new Message object to receive the response
217 my ($msg, $error) = Net::SNMP::Message->new (-transport => $transport); 217 my ($msg, $error) = Net::SNMP::Message->new (-transport => $transport);
218 218
219 if (!defined $msg) { 219 if (!defined $msg) {
… …
274 274
275 # when we end up here, we successfully handled $MAX_RECVQUEUE 275 # when we end up here, we successfully handled $MAX_RECVQUEUE
276 # replies in one iteration, so assume we are overloaded 276 # replies in one iteration, so assume we are overloaded
277 # and reduce the amount of parallelity. 277 # and reduce the amount of parallelity.
278 $MAX_OUTSTANDING = (int $MAX_OUTSTANDING * 0.95) || 1; 278 $MAX_OUTSTANDING = (int $MAX_OUTSTANDING * 0.95) || 1;
279 }); 279 };
280 } 280 }
281 281
282 $msg->timeout_id (\(my $rtimeout_w = 282 $msg->timeout_id (\(my $rtimeout_w =
283 AnyEvent->$timer (after => $pdu->timeout, cb => sub { 283 AE::timer $pdu->timeout, 0, sub {
284 my $rtimeout_w = $msg->timeout_id; 284 my $rtimeout_w = $msg->timeout_id;
285 if ($$rtimeout_w) { 285 if ($$rtimeout_w) {
286 undef $$rtimeout_w; 286 undef $$rtimeout_w;
287 delete $TRANSPORT[$fileno] 287 delete $TRANSPORT[$fileno]
288 unless --$TRANSPORT[$fileno][0]; 288 unless --$TRANSPORT[$fileno][0];
… …
296 296
297 --$BUSY; 297 --$BUSY;
298 kick_job; 298 kick_job;
299 } 299 }
300 }) 300 })
301 )); 301 );
302 } else { 302 } else {
303 --$BUSY; 303 --$BUSY;
304 kick_job; 304 kick_job;
305 } 305 }
306} 306}
… …
309 while ($BUSY < $MAX_OUTSTANDING) { 309 while ($BUSY < $MAX_OUTSTANDING) {
310 my $pdu = shift @QUEUE 310 my $pdu = shift @QUEUE
311 or last; 311 or last;
312 312
313 ++$BUSY; 313 ++$BUSY;
314
315 _send_pdu $pdu, $pdu->retries; 314 _send_pdu $pdu, $pdu->retries;
316 } 315 }
316
317 $DONE and $DONE->() unless $BUSY;
317} 318}
318 319
319sub send_pdu($$$) { 320sub send_pdu($$$) {
320 my (undef, $pdu, $delay) = @_; 321 my (undef, $pdu, $delay) = @_;
321 322
322 # $delay is not very sensibly implemented by AnyEvent::SNMP, 323 # $delay is not very sensibly implemented by AnyEvent::SNMP,
323 # but apparently it is not a very sensible feature. 324 # but apparently it is not a very sensible feature.
324 if ($delay > 0) { 325 if ($delay > 0) {
325 ++$BUSY; 326 ++$BUSY;
326 my $delay_w; $delay_w = AnyEvent->$timer (after => $delay, cb => sub { 327 my $delay_w; $delay_w = AE::timer $delay, 0, sub {
327 undef $delay_w; 328 undef $delay_w;
329 push @QUEUE, $pdu;
328 --$BUSY; 330 --$BUSY;
329 push @QUEUE, $pdu;
330 kick_job; 331 kick_job;
331 }); 332 };
332 return 1; 333 return 1;
333 } 334 }
334 335
335 push @QUEUE, $pdu; 336 push @QUEUE, $pdu;
336 kick_job; 337 kick_job;
337 338
338 1 339 1
339} 340}
340 341
341sub activate($) { 342sub activate($) {
342 AnyEvent->one_event while $BUSY; 343 while ($BUSY) {
344 $DONE = AE::cv;
345 $DONE->recv;
346 undef $DONE;
347 }
343} 348}
344 349
345sub one_event($) { 350sub one_event($) {
351 # should not ever be used
346 AnyEvent->one_event; 352 AnyEvent->one_event; #d# todo
347} 353}
348 354
349sub set_max_outstanding($) { 355sub set_max_outstanding($) {
350 $MAX_OUTSTANDING = $_[0]; 356 $MAX_OUTSTANDING = $_[0];
351 kick_job; 357 kick_job;

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines