ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/cvsroot/AnyEvent-SNMP/SNMP.pm
(Generate patch)

Comparing cvsroot/AnyEvent-SNMP/SNMP.pm (file contents):
Revision 1.7 by root, Sun Aug 2 14:24:23 2009 UTC vs.
Revision 1.14 by root, Wed Oct 9 18:22:59 2019 UTC

… …
23 my @result = $cv->wait; 23 my @result = $cv->wait;
24 24
25=head1 DESCRIPTION 25=head1 DESCRIPTION
26 26
27This module implements an alternative "event dispatcher" for Net::SNMP, 27This module implements an alternative "event dispatcher" for Net::SNMP,
28using AnyEvent as a backend. 28using AnyEvent as a backend. This integrates Net::SNMP into AnyEvent. That
29 29means you can make non-blocking Net::SNMP calls and as long as other
30This integrates Net::SNMP into AnyEvent: You can make non-blocking 30parts of your program also use AnyEvent (or some event loop supported by
31Net::SNMP calls and as long as other parts of your program also use 31AnyEvent), they will run in parallel.
32AnyEvent (or some event loop supported by AnyEvent), they will run in
33parallel.
34 32
35Also, the Net::SNMP scheduler is very inefficient with respect to both CPU 33Also, the Net::SNMP scheduler is very inefficient with respect to both CPU
36and memory usage. Most AnyEvent backends (including the pure-perl backend) 34and memory usage. Most AnyEvent backends (including the pure-perl backend)
37fare much better than the Net::SNMP dispatcher. 35fare much better than the Net::SNMP dispatcher.
38 36
37Another major added feature of this module over Net::SNMP is automatic
38rate-adjustments: Net::SNMP is so slow that firing a few thousand
39requests can cause many timeouts simply because Net::SNMP cannot process
40the replies in time. This module automatically adapts the send rate to
41avoid false timeouts caused by slow reply processing.
42
39A potential disadvantage is that replacing the dispatcher is not at all 43A potential disadvantage of this module is that replacing the dispatcher
40a documented thing to do, so future changes in Net::SNP might break this 44is not at all a documented thing to do, so future changes in Net::SNMP
41module (or the many similar ones). 45might break this module (or the many similar ones).
42 46
43This module does not export anything and does not require you to do 47This module does not export anything and does not require you to do
44anything special apart from loading it I<before doing any non-blocking 48anything special apart from loading it I<before doing any non-blocking
45requests with Net::SNMP>. It is recommended but not required to load this 49requests with Net::SNMP>. It is recommended but not required to load this
46module before C<Net::SNMP>. 50module before C<Net::SNMP>.
… …
65case, this can lead to packet loss, when the receive queue overflows and 69case, this can lead to packet loss, when the receive queue overflows and
66the kernel can no longer accept new packets. 70the kernel can no longer accept new packets.
67 71
68To avoid this, you can (and should) limit the number of outstanding 72To avoid this, you can (and should) limit the number of outstanding
69requests to a number low enough so that parsing time doesn't introduce 73requests to a number low enough so that parsing time doesn't introduce
70noticable delays. 74noticeable delays.
71 75
72Unfortunately, this number depends not only on processing speed and load 76Unfortunately, this number depends not only on processing speed and load
73of the machine running Net::SNMP, but also on the network latency and the 77of the machine running Net::SNMP, but also on the network latency and the
74speed of your SNMP agents. 78speed of your SNMP agents.
75 79
76AnyEvent::SNMP tries to dynamically adjust this number dynamically upwards 80AnyEvent::SNMP tries to dynamically adjust this number upwards and
77and downwards. 81downwards.
78 82
79Increasing C<$MAX_OUTSTANDING> will not automatically use the 83Increasing C<$MAX_OUTSTANDING> will not automatically use the
80C<extra request slots. To increase $MAX_OUTSTANDING> and make 84extra request slots. To increase C<$MAX_OUTSTANDING> and make
81C<C<AnyEvent::SNMP> make use of the extra paralellity, call 85C<AnyEvent::SNMP> make use of the extra parallelity, call
82C<AnyEvent::SNMP::set_max_outstanding> with the new value, e.g.: 86C<AnyEvent::SNMP::set_max_outstanding> with the new value, e.g.:
83 87
84 AnyEvent::SNMP::set_max_outstanding 500; 88 AnyEvent::SNMP::set_max_outstanding 500;
85 89
86Although due to the dynamic adjustment, this might have little lasting 90Although due to the dynamic adjustment, this might have little lasting
… …
99When AnyEvent::SNMP handles $MAX_RECVQUEUE or more packets per iteration 103When AnyEvent::SNMP handles $MAX_RECVQUEUE or more packets per iteration
100it will reduce $MAX_OUTSTANDING. If it handles less than $MIN_RECVQUEUE, 104it will reduce $MAX_OUTSTANDING. If it handles less than $MIN_RECVQUEUE,
101it increases $MAX_OUTSTANDING. 105it increases $MAX_OUTSTANDING.
102 106
103This has the result of adjusting the number of outstanding requests so that 107This has the result of adjusting the number of outstanding requests so that
104the recv queue is between the minimum and maximu, usually. 108the recv queue is between the minimum and maximum, usually.
105 109
106This algorithm works reasonably well as long as the responses, response 110This algorithm works reasonably well as long as the responses, response
107latencies and processing times are the same size per packet on average. 111latencies and processing times are the same per packet on average.
108 112
109=back 113=back
110 114
111=head1 COMPATIBILITY 115=head1 COMPATIBILITY
112 116
… …
131 135
132=cut 136=cut
133 137
134package AnyEvent::SNMP; 138package AnyEvent::SNMP;
135 139
136no warnings; 140use common::sense;
137use strict qw(subs vars);
138 141
139# it is possible to do this without loading 142# it is possible to do this without loading
140# Net::SNMP::Dispatcher, but much more awkward. 143# Net::SNMP::Dispatcher, but much more awkward.
141use Net::SNMP::Dispatcher; 144use Net::SNMP::Dispatcher;
142 145
146# we could inherit fro Net:SNMP::Dispatcher, but since this is undocumented,
147# I'd rather see it die (and reported) than silenty and subtly fail.
148*msg_handle_alloc = \&Net::SNMP::Dispatcher::msg_handle_alloc;
149
143sub Net::SNMP::Dispatcher::instance { 150sub Net::SNMP::Dispatcher::instance {
144 AnyEvent::SNMP:: 151 AnyEvent::SNMP::
145} 152}
146 153
147use Net::SNMP (); 154use Net::SNMP ();
148use AnyEvent (); 155use AnyEvent ();
149 156
150our $VERSION = '0.2'; 157our $VERSION = '6.02';
151 158
152$Net::SNMP::DISPATCHER = instance Net::SNMP::Dispatcher; 159$Net::SNMP::DISPATCHER = instance Net::SNMP::Dispatcher;
153 160
154our $MESSAGE_PROCESSING = $Net::SNMP::Dispatcher::MESSAGE_PROCESSING; 161our $MESSAGE_PROCESSING = $Net::SNMP::Dispatcher::MESSAGE_PROCESSING;
155 162
156# avoid the method call
157my $timer = sub { shift->timer (@_) };
158AnyEvent::post_detect { $timer = AnyEvent->can ("timer") };
159
160our $BUSY; 163our $BUSY;
164our $DONE; # finished all jobs
161our @TRANSPORT; # fileno => [count, watcher] 165our @TRANSPORT; # fileno => [count, watcher]
162our @QUEUE; 166our @QUEUE;
163our $MAX_OUTSTANDING = 50; 167our $MAX_OUTSTANDING = 50;
164our $MIN_RECVQUEUE = 8; 168our $MIN_RECVQUEUE = 8;
165our $MAX_RECVQUEUE = 64; 169our $MAX_RECVQUEUE = 64;
… …
188 $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id) 192 $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id)
189 if $pdu->expect_response; 193 if $pdu->expect_response;
190 194
191 # A crude attempt to recover from temporary failures. 195 # A crude attempt to recover from temporary failures.
192 if ($retries-- > 0 && ($!{EAGAIN} || $!{EWOULDBLOCK} || $!{ENOSPC})) { 196 if ($retries-- > 0 && ($!{EAGAIN} || $!{EWOULDBLOCK} || $!{ENOSPC})) {
193 my $retry_w; $retry_w = AnyEvent->$timer (after => $pdu->timeout, cb => sub { 197 my $retry_w; $retry_w = AE::timer $pdu->timeout, 0, sub {
194 undef $retry_w; 198 undef $retry_w;
195 _send_pdu ($pdu, $retries); 199 _send_pdu ($pdu, $retries);
196 }); 200 };
197 } else { 201 } else {
198 --$BUSY; 202 --$BUSY;
199 kick_job; 203 kick_job;
200 } 204 }
201 205
… …
209 my $transport = $msg->transport; 213 my $transport = $msg->transport;
210 my $fileno = $transport->fileno; 214 my $fileno = $transport->fileno;
211 215
212 # register the transport 216 # register the transport
213 unless ($TRANSPORT[$fileno][0]++) { 217 unless ($TRANSPORT[$fileno][0]++) {
214 $TRANSPORT[$fileno][1] = AnyEvent->io (fh => $transport->socket, poll => 'r', cb => sub { 218 $TRANSPORT[$fileno][1] = AE::io $transport->socket, 0, sub {
215 for my $count (1..$MAX_RECVQUEUE) { # handle up to this many requests in one go 219 for my $count (1..$MAX_RECVQUEUE) { # handle up to this many requests in one go
216 # Create a new Message object to receive the response 220 # Create a new Message object to receive the response
217 my ($msg, $error) = Net::SNMP::Message->new (-transport => $transport); 221 my ($msg, $error) = Net::SNMP::Message->new (-transport => $transport);
218 222
219 if (!defined $msg) { 223 if (!defined $msg) {
… …
253 257
254 # Set the error if applicable. 258 # Set the error if applicable.
255 $msg->error ($MESSAGE_PROCESSING->error) if $MESSAGE_PROCESSING->error; 259 $msg->error ($MESSAGE_PROCESSING->error) if $MESSAGE_PROCESSING->error;
256 260
257 # Notify the command generator to process the response. 261 # Notify the command generator to process the response.
258 $msg->process_response_pdu; 262 # Net::SNMP calls process_response_pdu, which simply calls callback_execute,
263 # but some errors cause $msg to be of type Net::SNMP::Message, not Net::SMMP::PDU,
264 # so we call the underlying callback_execute method which exists on both and
265 # seems to do the right thing.
266 $msg->callback_execute;
259 267
260 # Cancel the timeout. 268 # Cancel the timeout.
261 my $rtimeout_w = $msg->timeout_id; 269 my $rtimeout_w = $msg->timeout_id;
262 if ($$rtimeout_w) { 270 if ($$rtimeout_w) {
263 undef $$rtimeout_w; 271 undef $$rtimeout_w;
… …
274 282
275 # when we end up here, we successfully handled $MAX_RECVQUEUE 283 # when we end up here, we successfully handled $MAX_RECVQUEUE
276 # replies in one iteration, so assume we are overloaded 284 # replies in one iteration, so assume we are overloaded
277 # and reduce the amount of parallelity. 285 # and reduce the amount of parallelity.
278 $MAX_OUTSTANDING = (int $MAX_OUTSTANDING * 0.95) || 1; 286 $MAX_OUTSTANDING = (int $MAX_OUTSTANDING * 0.95) || 1;
279 }); 287 };
280 } 288 }
281 289
282 $msg->timeout_id (\(my $rtimeout_w = 290 $msg->timeout_id (\(my $rtimeout_w =
283 AnyEvent->$timer (after => $pdu->timeout, cb => sub { 291 AE::timer $pdu->timeout, 0, sub {
284 my $rtimeout_w = $msg->timeout_id; 292 my $rtimeout_w = $msg->timeout_id;
285 if ($$rtimeout_w) { 293 if ($$rtimeout_w) {
286 undef $$rtimeout_w; 294 undef $$rtimeout_w;
287 delete $TRANSPORT[$fileno] 295 delete $TRANSPORT[$fileno]
288 unless --$TRANSPORT[$fileno][0]; 296 unless --$TRANSPORT[$fileno][0];
… …
296 304
297 --$BUSY; 305 --$BUSY;
298 kick_job; 306 kick_job;
299 } 307 }
300 }) 308 })
301 )); 309 );
302 } else { 310 } else {
303 --$BUSY; 311 --$BUSY;
304 kick_job; 312 kick_job;
305 } 313 }
306} 314}
… …
309 while ($BUSY < $MAX_OUTSTANDING) { 317 while ($BUSY < $MAX_OUTSTANDING) {
310 my $pdu = shift @QUEUE 318 my $pdu = shift @QUEUE
311 or last; 319 or last;
312 320
313 ++$BUSY; 321 ++$BUSY;
314
315 _send_pdu $pdu, $pdu->retries; 322 _send_pdu $pdu, $pdu->retries;
316 } 323 }
324
325 $DONE and $DONE->() unless $BUSY;
317} 326}
318 327
319sub send_pdu($$$) { 328sub send_pdu($$$) {
320 my (undef, $pdu, $delay) = @_; 329 my (undef, $pdu, $delay) = @_;
321 330
322 # $delay is not very sensibly implemented by AnyEvent::SNMP, 331 # $delay is not very sensibly implemented by AnyEvent::SNMP,
323 # but apparently it is not a very sensible feature. 332 # but apparently it is not a very sensible feature.
324 if ($delay > 0) { 333 if ($delay > 0) {
325 ++$BUSY; 334 ++$BUSY;
326 my $delay_w; $delay_w = AnyEvent->$timer (after => $delay, cb => sub { 335 my $delay_w; $delay_w = AE::timer $delay, 0, sub {
327 undef $delay_w; 336 undef $delay_w;
337 push @QUEUE, $pdu;
328 --$BUSY; 338 --$BUSY;
329 push @QUEUE, $pdu;
330 kick_job; 339 kick_job;
331 }); 340 };
332 return 1; 341 return 1;
333 } 342 }
334 343
335 push @QUEUE, $pdu; 344 push @QUEUE, $pdu;
336 kick_job; 345 kick_job;
337 346
338 1 347 1
339} 348}
340 349
341sub activate($) { 350sub loop($) {
342 AnyEvent->one_event while $BUSY; 351 while ($BUSY) {
352 $DONE = AE::cv;
353 $DONE->recv;
354 undef $DONE;
355 }
343} 356}
357
358*activate = \&loop; # 5.x compatibility?
359*listen = \&loop; # 5.x compatibility?
344 360
345sub one_event($) { 361sub one_event($) {
362 # should not ever be used
346 AnyEvent->one_event; 363 AnyEvent->one_event; #d# todo
347} 364}
348 365
349sub set_max_outstanding($) { 366sub set_max_outstanding($) {
350 $MAX_OUTSTANDING = $_[0]; 367 $MAX_OUTSTANDING = $_[0];
351 kick_job; 368 kick_job;
352} 369}
353 370
371# not provided yet:
372# schedule # apparently only used by Net::SNMP::Dispatcher itself
373# register # apparently only used by Net::SNMP::Dispatcher itself
374# deregister # apparently only used by Net::SNMP::Dispatcher itself
375# cancel # apparently only used by Net::SNMP::Dispatcher itself
376# return_response_pdu # apparently not used at all?
377# error # only used by Net::SNMP::Dispatcher itself?
378# debug # only used by Net::SNMP::Dispatcher itself?
379
354=head1 SEE ALSO 380=head1 SEE ALSO
355 381
356L<AnyEvent>, L<Net::SNMP>, L<Net::SNMP::XS>, L<Net::SNMP::EV>. 382L<AnyEvent>, L<Net::SNMP>, L<Net::SNMP::XS>, L<Net::SNMP::EV>.
357 383
358=head1 AUTHOR 384=head1 AUTHOR

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines