ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-SNMP/SNMP.pm
Revision: 1.14
Committed: Wed Oct 9 18:22:59 2019 UTC (4 years, 7 months ago) by root
Branch: MAIN
CVS Tags: rel-6_02, HEAD
Changes since 1.13: +6 -2 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.1 =head1 NAME
2    
3 root 1.7 AnyEvent::SNMP - adaptor to integrate Net::SNMP into AnyEvent.
4 root 1.1
5     =head1 SYNOPSIS
6    
7     use AnyEvent::SNMP;
8     use Net::SNMP;
9    
10     # just use Net::SNMP and AnyEvent as you like:
11    
12     # use a condvar to transfer results, this is
13     # just an example, you can use a naked callback as well.
14     my $cv = AnyEvent->condvar;
15    
16     # ... start non-blocking snmp request(s)...
17     Net::SNMP->session (-hostname => "127.0.0.1",
18     -community => "public",
19     -nonblocking => 1)
20     ->get_request (-callback => sub { $cv->send (@_) });
21    
22     # ... do something else until the result is required
23     my @result = $cv->wait;
24    
25     =head1 DESCRIPTION
26    
27     This module implements an alternative "event dispatcher" for Net::SNMP,
28 root 1.10 using AnyEvent as a backend. This integrates Net::SNMP into AnyEvent. That
29     means you can make non-blocking Net::SNMP calls and as long as other
30     parts of your program also use AnyEvent (or some event loop supported by
31     AnyEvent), they will run in parallel.
32 root 1.1
33     Also, the Net::SNMP scheduler is very inefficient with respect to both CPU
34     and memory usage. Most AnyEvent backends (including the pure-perl backend)
35     fare much better than the Net::SNMP dispatcher.
36    
37 root 1.13 Another major added feature of this module over Net::SNMP is automatic
38 root 1.10 rate-adjustments: Net::SNMP is so slow that firing a few thousand
39     requests can cause many timeouts simply because Net::SNMP cannot process
40     the replies in time. This module automatically adapts the send rate to
41     avoid false timeouts caused by slow reply processing.
42    
43     A potential disadvantage of this module is that replacing the dispatcher
44 root 1.13 is not at all a documented thing to do, so future changes in Net::SNMP
45 root 1.10 might break this module (or the many similar ones).
46 root 1.1
47     This module does not export anything and does not require you to do
48     anything special apart from loading it I<before doing any non-blocking
49     requests with Net::SNMP>. It is recommended but not required to load this
50     module before C<Net::SNMP>.
51    
52 root 1.3 =head1 GLOBAL VARIABLES
53    
54     =over 4
55    
56     =item $AnyEvent::SNMP::MAX_OUTSTANDING (default: C<50>, dynamic)
57    
58 root 1.6 =item AnyEvent::SNMP::set_max_outstanding $new_value
59    
60 root 1.3 Use this package variable to restrict the number of outstanding SNMP
61     requests at any point in time.
62    
63     Net::SNMP is very fast at creating and sending SNMP requests, but much
64     slower at parsing (big, bulk) responses. This makes it easy to request a
65     lot of data that can take many seconds to parse.
66    
67     In the best case, this can lead to unnecessary delays (and even time-outs,
68     as the data has been received but not yet processed) and in the worst
69     case, this can lead to packet loss, when the receive queue overflows and
70     the kernel can no longer accept new packets.
71    
72 root 1.6 To avoid this, you can (and should) limit the number of outstanding
73     requests to a number low enough so that parsing time doesn't introduce
74 root 1.13 noticeable delays.
75 root 1.3
76     Unfortunately, this number depends not only on processing speed and load
77     of the machine running Net::SNMP, but also on the network latency and the
78     speed of your SNMP agents.
79    
80 root 1.11 AnyEvent::SNMP tries to dynamically adjust this number upwards and
81     downwards.
82 root 1.3
83 root 1.6 Increasing C<$MAX_OUTSTANDING> will not automatically use the
84 root 1.8 extra request slots. To increase C<$MAX_OUTSTANDING> and make
85 root 1.13 C<AnyEvent::SNMP> make use of the extra parallelity, call
86 root 1.6 C<AnyEvent::SNMP::set_max_outstanding> with the new value, e.g.:
87    
88     AnyEvent::SNMP::set_max_outstanding 500;
89    
90     Although due to the dynamic adjustment, this might have little lasting
91     effect.
92    
93 root 1.3 Note that you can use L<Net::SNMP::XS> to speed up parsing of responses
94     considerably.
95    
96 root 1.5 =item $AnyEvent::SNMP::MIN_RECVQUEUE (default: C<8>)
97 root 1.3
98     =item $AnyEvent::SNMP::MAX_RECVQUEUE (default: C<64>)
99    
100     These values specify the minimum and maximum receive queue length (in
101     units of one response packet).
102    
103     When AnyEvent::SNMP handles $MAX_RECVQUEUE or more packets per iteration
104     it will reduce $MAX_OUTSTANDING. If it handles less than $MIN_RECVQUEUE,
105     it increases $MAX_OUTSTANDING.
106    
107     This has the result of adjusting the number of outstanding requests so that
108 root 1.13 the recv queue is between the minimum and maximum, usually.
109 root 1.3
110     This algorithm works reasonably well as long as the responses, response
111 root 1.13 latencies and processing times are the same per packet on average.
112 root 1.3
113     =back
114    
115     =head1 COMPATIBILITY
116    
117     This module may be used as a drop in replacement for the
118     Net::SNMP::Dispatcher in existing programs. You can still call
119     C<snmp_dispatcher> to start the event-loop, but then you loose the benefit
120     of mixing Net::SNMP events with other events.
121    
122     use AnyEvent::SNMP;
123     use Net::SNMP;
124    
125     # just use Net::SNMP as before
126    
127     # ... start non-blocking snmp request(s)...
128     Net::SNMP->session (
129     -hostname => "127.0.0.1",
130     -community => "public",
131     -nonblocking => 1,
132     )->get_request (-callback => sub { ... });
133    
134     snmp_dispatcher;
135    
136 root 1.1 =cut
137    
138     package AnyEvent::SNMP;
139    
140 root 1.11 use common::sense;
141 root 1.1
142     # it is possible to do this without loading
143     # Net::SNMP::Dispatcher, but much more awkward.
144     use Net::SNMP::Dispatcher;
145    
146 root 1.12 # we could inherit fro Net:SNMP::Dispatcher, but since this is undocumented,
147     # I'd rather see it die (and reported) than silenty and subtly fail.
148     *msg_handle_alloc = \&Net::SNMP::Dispatcher::msg_handle_alloc;
149    
150 root 1.1 sub Net::SNMP::Dispatcher::instance {
151     AnyEvent::SNMP::
152     }
153    
154     use Net::SNMP ();
155     use AnyEvent ();
156    
157 root 1.14 our $VERSION = '6.02';
158 root 1.1
159     $Net::SNMP::DISPATCHER = instance Net::SNMP::Dispatcher;
160    
161     our $MESSAGE_PROCESSING = $Net::SNMP::Dispatcher::MESSAGE_PROCESSING;
162    
163     our $BUSY;
164 root 1.8 our $DONE; # finished all jobs
165 root 1.5 our @TRANSPORT; # fileno => [count, watcher]
166 root 1.3 our @QUEUE;
167     our $MAX_OUTSTANDING = 50;
168 root 1.5 our $MIN_RECVQUEUE = 8;
169 root 1.3 our $MAX_RECVQUEUE = 64;
170    
171 root 1.9 sub kick_job;
172 root 1.1
173     sub _send_pdu {
174     my ($pdu, $retries) = @_;
175    
176     # mostly copied from Net::SNMP::Dispatch
177    
178     # Pass the PDU to Message Processing so that it can
179     # create the new outgoing message.
180     my $msg = $MESSAGE_PROCESSING->prepare_outgoing_msg ($pdu);
181    
182     if (!defined $msg) {
183 root 1.9 --$BUSY;
184 root 1.3 kick_job;
185 root 1.1 # Inform the command generator about the Message Processing error.
186     $pdu->status_information ($MESSAGE_PROCESSING->error);
187     return;
188     }
189    
190     # Actually send the message.
191     if (!defined $msg->send) {
192     $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id)
193     if $pdu->expect_response;
194    
195     # A crude attempt to recover from temporary failures.
196     if ($retries-- > 0 && ($!{EAGAIN} || $!{EWOULDBLOCK} || $!{ENOSPC})) {
197 root 1.8 my $retry_w; $retry_w = AE::timer $pdu->timeout, 0, sub {
198 root 1.1 undef $retry_w;
199     _send_pdu ($pdu, $retries);
200 root 1.8 };
201 root 1.1 } else {
202 root 1.9 --$BUSY;
203 root 1.3 kick_job;
204 root 1.1 }
205    
206     # Inform the command generator about the send() error.
207     $pdu->status_information ($msg->error);
208     return;
209     }
210    
211     # Schedule the timeout handler if the message expects a response.
212     if ($pdu->expect_response) {
213     my $transport = $msg->transport;
214 root 1.5 my $fileno = $transport->fileno;
215 root 1.1
216     # register the transport
217 root 1.5 unless ($TRANSPORT[$fileno][0]++) {
218 root 1.8 $TRANSPORT[$fileno][1] = AE::io $transport->socket, 0, sub {
219 root 1.3 for my $count (1..$MAX_RECVQUEUE) { # handle up to this many requests in one go
220     # Create a new Message object to receive the response
221     my ($msg, $error) = Net::SNMP::Message->new (-transport => $transport);
222    
223     if (!defined $msg) {
224     die sprintf 'Failed to create Message object [%s]', $error;
225     }
226 root 1.1
227 root 1.3 # Read the message from the Transport Layer
228     if (!defined $msg->recv) {
229     if ($transport->connectionless) {
230 root 1.4 # if we handled very few replies and we have queued work, try
231     # to increase the parallelity as we probably can handle more.
232 root 1.3 if ($count < $MIN_RECVQUEUE && @QUEUE) {
233     ++$MAX_OUTSTANDING;
234     kick_job;
235     }
236     } else {
237     # for some reason, connected-oriented transports seem to need this
238 root 1.5 delete $TRANSPORT[$fileno]
239     unless --$TRANSPORT[$fileno][0];
240 root 1.3 }
241 root 1.1
242 root 1.3 $msg->error;
243     return;
244 root 1.1 }
245    
246 root 1.3 # For connection-oriented Transport Domains, it is possible to
247     # "recv" an empty buffer if reassembly is required.
248     if (!$msg->length) {
249     return;
250     }
251 root 1.1
252 root 1.3 # Hand the message over to Message Processing.
253     if (!defined $MESSAGE_PROCESSING->prepare_data_elements ($msg)) {
254     $MESSAGE_PROCESSING->error;
255     return;
256     }
257 root 1.1
258 root 1.3 # Set the error if applicable.
259     $msg->error ($MESSAGE_PROCESSING->error) if $MESSAGE_PROCESSING->error;
260 root 1.1
261 root 1.3 # Notify the command generator to process the response.
262 root 1.14 # Net::SNMP calls process_response_pdu, which simply calls callback_execute,
263     # but some errors cause $msg to be of type Net::SNMP::Message, not Net::SMMP::PDU,
264     # so we call the underlying callback_execute method which exists on both and
265     # seems to do the right thing.
266     $msg->callback_execute;
267 root 1.1
268 root 1.3 # Cancel the timeout.
269     my $rtimeout_w = $msg->timeout_id;
270     if ($$rtimeout_w) {
271     undef $$rtimeout_w;
272    
273 root 1.9 --$BUSY;
274 root 1.3 kick_job;
275    
276 root 1.5 unless (--$TRANSPORT[$fileno][0]) {
277     delete $TRANSPORT[$fileno];
278 root 1.3 return;
279     }
280     }
281 root 1.1 }
282    
283 root 1.4 # when we end up here, we successfully handled $MAX_RECVQUEUE
284     # replies in one iteration, so assume we are overloaded
285     # and reduce the amount of parallelity.
286 root 1.5 $MAX_OUTSTANDING = (int $MAX_OUTSTANDING * 0.95) || 1;
287 root 1.8 };
288 root 1.1 }
289    
290     $msg->timeout_id (\(my $rtimeout_w =
291 root 1.8 AE::timer $pdu->timeout, 0, sub {
292 root 1.1 my $rtimeout_w = $msg->timeout_id;
293     if ($$rtimeout_w) {
294     undef $$rtimeout_w;
295 root 1.5 delete $TRANSPORT[$fileno]
296     unless --$TRANSPORT[$fileno][0];
297 root 1.1 }
298    
299     if ($retries--) {
300     _send_pdu ($pdu, $retries);
301     } else {
302     $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id);
303     $pdu->status_information ("No response from remote host '%s'", $pdu->hostname);
304 root 1.3
305 root 1.9 --$BUSY;
306 root 1.3 kick_job;
307 root 1.1 }
308     })
309 root 1.8 );
310 root 1.1 } else {
311 root 1.9 --$BUSY;
312 root 1.3 kick_job;
313 root 1.1 }
314     }
315    
316 root 1.3 sub kick_job {
317     while ($BUSY < $MAX_OUTSTANDING) {
318     my $pdu = shift @QUEUE
319     or last;
320    
321     ++$BUSY;
322     _send_pdu $pdu, $pdu->retries;
323     }
324 root 1.8
325     $DONE and $DONE->() unless $BUSY;
326 root 1.3 }
327 root 1.6
328 root 1.1 sub send_pdu($$$) {
329     my (undef, $pdu, $delay) = @_;
330    
331 root 1.3 # $delay is not very sensibly implemented by AnyEvent::SNMP,
332     # but apparently it is not a very sensible feature.
333 root 1.1 if ($delay > 0) {
334 root 1.3 ++$BUSY;
335 root 1.8 my $delay_w; $delay_w = AE::timer $delay, 0, sub {
336 root 1.1 undef $delay_w;
337 root 1.3 push @QUEUE, $pdu;
338 root 1.9 --$BUSY;
339 root 1.3 kick_job;
340 root 1.8 };
341 root 1.1 return 1;
342     }
343    
344 root 1.3 push @QUEUE, $pdu;
345     kick_job;
346    
347 root 1.1 1
348     }
349    
350 root 1.12 sub loop($) {
351 root 1.8 while ($BUSY) {
352     $DONE = AE::cv;
353     $DONE->recv;
354     undef $DONE;
355     }
356 root 1.1 }
357    
358 root 1.12 *activate = \&loop; # 5.x compatibility?
359     *listen = \&loop; # 5.x compatibility?
360    
361 root 1.1 sub one_event($) {
362 root 1.9 # should not ever be used
363 root 1.8 AnyEvent->one_event; #d# todo
364 root 1.1 }
365    
366 root 1.6 sub set_max_outstanding($) {
367     $MAX_OUTSTANDING = $_[0];
368     kick_job;
369     }
370    
371 root 1.12 # not provided yet:
372     # schedule # apparently only used by Net::SNMP::Dispatcher itself
373     # register # apparently only used by Net::SNMP::Dispatcher itself
374     # deregister # apparently only used by Net::SNMP::Dispatcher itself
375     # cancel # apparently only used by Net::SNMP::Dispatcher itself
376     # return_response_pdu # apparently not used at all?
377     # error # only used by Net::SNMP::Dispatcher itself?
378     # debug # only used by Net::SNMP::Dispatcher itself?
379    
380 root 1.1 =head1 SEE ALSO
381    
382 root 1.3 L<AnyEvent>, L<Net::SNMP>, L<Net::SNMP::XS>, L<Net::SNMP::EV>.
383 root 1.1
384     =head1 AUTHOR
385    
386     Marc Lehmann <schmorp@schmorp.de>
387     http://home.schmorp.de/
388    
389     =cut
390    
391     1
392