ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-SNMP/SNMP.pm
Revision: 1.12
Committed: Sun Apr 15 11:02:07 2012 UTC (12 years, 1 month ago) by root
Branch: MAIN
CVS Tags: rel-6_0
Changes since 1.11: +18 -2 lines
Log Message:
6.0

File Contents

# User Rev Content
1 root 1.1 =head1 NAME
2    
3 root 1.7 AnyEvent::SNMP - adaptor to integrate Net::SNMP into AnyEvent.
4 root 1.1
5     =head1 SYNOPSIS
6    
7     use AnyEvent::SNMP;
8     use Net::SNMP;
9    
10     # just use Net::SNMP and AnyEvent as you like:
11    
12     # use a condvar to transfer results, this is
13     # just an example, you can use a naked callback as well.
14     my $cv = AnyEvent->condvar;
15    
16     # ... start non-blocking snmp request(s)...
17     Net::SNMP->session (-hostname => "127.0.0.1",
18     -community => "public",
19     -nonblocking => 1)
20     ->get_request (-callback => sub { $cv->send (@_) });
21    
22     # ... do something else until the result is required
23     my @result = $cv->wait;
24    
25     =head1 DESCRIPTION
26    
27     This module implements an alternative "event dispatcher" for Net::SNMP,
28 root 1.10 using AnyEvent as a backend. This integrates Net::SNMP into AnyEvent. That
29     means you can make non-blocking Net::SNMP calls and as long as other
30     parts of your program also use AnyEvent (or some event loop supported by
31     AnyEvent), they will run in parallel.
32 root 1.1
33     Also, the Net::SNMP scheduler is very inefficient with respect to both CPU
34     and memory usage. Most AnyEvent backends (including the pure-perl backend)
35     fare much better than the Net::SNMP dispatcher.
36    
37 root 1.10 Another major added fetaure of this module over Net::SNMP is automatic
38     rate-adjustments: Net::SNMP is so slow that firing a few thousand
39     requests can cause many timeouts simply because Net::SNMP cannot process
40     the replies in time. This module automatically adapts the send rate to
41     avoid false timeouts caused by slow reply processing.
42    
43     A potential disadvantage of this module is that replacing the dispatcher
44     is not at all a documented thing to do, so future changes in Net::SNP
45     might break this module (or the many similar ones).
46 root 1.1
47     This module does not export anything and does not require you to do
48     anything special apart from loading it I<before doing any non-blocking
49     requests with Net::SNMP>. It is recommended but not required to load this
50     module before C<Net::SNMP>.
51    
52 root 1.3 =head1 GLOBAL VARIABLES
53    
54     =over 4
55    
56     =item $AnyEvent::SNMP::MAX_OUTSTANDING (default: C<50>, dynamic)
57    
58 root 1.6 =item AnyEvent::SNMP::set_max_outstanding $new_value
59    
60 root 1.3 Use this package variable to restrict the number of outstanding SNMP
61     requests at any point in time.
62    
63     Net::SNMP is very fast at creating and sending SNMP requests, but much
64     slower at parsing (big, bulk) responses. This makes it easy to request a
65     lot of data that can take many seconds to parse.
66    
67     In the best case, this can lead to unnecessary delays (and even time-outs,
68     as the data has been received but not yet processed) and in the worst
69     case, this can lead to packet loss, when the receive queue overflows and
70     the kernel can no longer accept new packets.
71    
72 root 1.6 To avoid this, you can (and should) limit the number of outstanding
73     requests to a number low enough so that parsing time doesn't introduce
74     noticable delays.
75 root 1.3
76     Unfortunately, this number depends not only on processing speed and load
77     of the machine running Net::SNMP, but also on the network latency and the
78     speed of your SNMP agents.
79    
80 root 1.11 AnyEvent::SNMP tries to dynamically adjust this number upwards and
81     downwards.
82 root 1.3
83 root 1.6 Increasing C<$MAX_OUTSTANDING> will not automatically use the
84 root 1.8 extra request slots. To increase C<$MAX_OUTSTANDING> and make
85     C<AnyEvent::SNMP> make use of the extra paralellity, call
86 root 1.6 C<AnyEvent::SNMP::set_max_outstanding> with the new value, e.g.:
87    
88     AnyEvent::SNMP::set_max_outstanding 500;
89    
90     Although due to the dynamic adjustment, this might have little lasting
91     effect.
92    
93 root 1.3 Note that you can use L<Net::SNMP::XS> to speed up parsing of responses
94     considerably.
95    
96 root 1.5 =item $AnyEvent::SNMP::MIN_RECVQUEUE (default: C<8>)
97 root 1.3
98     =item $AnyEvent::SNMP::MAX_RECVQUEUE (default: C<64>)
99    
100     These values specify the minimum and maximum receive queue length (in
101     units of one response packet).
102    
103     When AnyEvent::SNMP handles $MAX_RECVQUEUE or more packets per iteration
104     it will reduce $MAX_OUTSTANDING. If it handles less than $MIN_RECVQUEUE,
105     it increases $MAX_OUTSTANDING.
106    
107     This has the result of adjusting the number of outstanding requests so that
108     the recv queue is between the minimum and maximu, usually.
109    
110     This algorithm works reasonably well as long as the responses, response
111     latencies and processing times are the same size per packet on average.
112    
113     =back
114    
115     =head1 COMPATIBILITY
116    
117     This module may be used as a drop in replacement for the
118     Net::SNMP::Dispatcher in existing programs. You can still call
119     C<snmp_dispatcher> to start the event-loop, but then you loose the benefit
120     of mixing Net::SNMP events with other events.
121    
122     use AnyEvent::SNMP;
123     use Net::SNMP;
124    
125     # just use Net::SNMP as before
126    
127     # ... start non-blocking snmp request(s)...
128     Net::SNMP->session (
129     -hostname => "127.0.0.1",
130     -community => "public",
131     -nonblocking => 1,
132     )->get_request (-callback => sub { ... });
133    
134     snmp_dispatcher;
135    
136 root 1.1 =cut
137    
138     package AnyEvent::SNMP;
139    
140 root 1.11 use common::sense;
141 root 1.1
142     # it is possible to do this without loading
143     # Net::SNMP::Dispatcher, but much more awkward.
144     use Net::SNMP::Dispatcher;
145    
146 root 1.12 # we could inherit fro Net:SNMP::Dispatcher, but since this is undocumented,
147     # I'd rather see it die (and reported) than silenty and subtly fail.
148     *msg_handle_alloc = \&Net::SNMP::Dispatcher::msg_handle_alloc;
149    
150 root 1.1 sub Net::SNMP::Dispatcher::instance {
151     AnyEvent::SNMP::
152     }
153    
154     use Net::SNMP ();
155     use AnyEvent ();
156    
157 root 1.12 our $VERSION = '6.0';
158 root 1.1
159     $Net::SNMP::DISPATCHER = instance Net::SNMP::Dispatcher;
160    
161     our $MESSAGE_PROCESSING = $Net::SNMP::Dispatcher::MESSAGE_PROCESSING;
162    
163     our $BUSY;
164 root 1.8 our $DONE; # finished all jobs
165 root 1.5 our @TRANSPORT; # fileno => [count, watcher]
166 root 1.3 our @QUEUE;
167     our $MAX_OUTSTANDING = 50;
168 root 1.5 our $MIN_RECVQUEUE = 8;
169 root 1.3 our $MAX_RECVQUEUE = 64;
170    
171 root 1.9 sub kick_job;
172 root 1.1
173     sub _send_pdu {
174     my ($pdu, $retries) = @_;
175    
176     # mostly copied from Net::SNMP::Dispatch
177    
178     # Pass the PDU to Message Processing so that it can
179     # create the new outgoing message.
180     my $msg = $MESSAGE_PROCESSING->prepare_outgoing_msg ($pdu);
181    
182     if (!defined $msg) {
183 root 1.9 --$BUSY;
184 root 1.3 kick_job;
185 root 1.1 # Inform the command generator about the Message Processing error.
186     $pdu->status_information ($MESSAGE_PROCESSING->error);
187     return;
188     }
189    
190     # Actually send the message.
191     if (!defined $msg->send) {
192     $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id)
193     if $pdu->expect_response;
194    
195     # A crude attempt to recover from temporary failures.
196     if ($retries-- > 0 && ($!{EAGAIN} || $!{EWOULDBLOCK} || $!{ENOSPC})) {
197 root 1.8 my $retry_w; $retry_w = AE::timer $pdu->timeout, 0, sub {
198 root 1.1 undef $retry_w;
199     _send_pdu ($pdu, $retries);
200 root 1.8 };
201 root 1.1 } else {
202 root 1.9 --$BUSY;
203 root 1.3 kick_job;
204 root 1.1 }
205    
206     # Inform the command generator about the send() error.
207     $pdu->status_information ($msg->error);
208     return;
209     }
210    
211     # Schedule the timeout handler if the message expects a response.
212     if ($pdu->expect_response) {
213     my $transport = $msg->transport;
214 root 1.5 my $fileno = $transport->fileno;
215 root 1.1
216     # register the transport
217 root 1.5 unless ($TRANSPORT[$fileno][0]++) {
218 root 1.8 $TRANSPORT[$fileno][1] = AE::io $transport->socket, 0, sub {
219 root 1.3 for my $count (1..$MAX_RECVQUEUE) { # handle up to this many requests in one go
220     # Create a new Message object to receive the response
221     my ($msg, $error) = Net::SNMP::Message->new (-transport => $transport);
222    
223     if (!defined $msg) {
224     die sprintf 'Failed to create Message object [%s]', $error;
225     }
226 root 1.1
227 root 1.3 # Read the message from the Transport Layer
228     if (!defined $msg->recv) {
229     if ($transport->connectionless) {
230 root 1.4 # if we handled very few replies and we have queued work, try
231     # to increase the parallelity as we probably can handle more.
232 root 1.3 if ($count < $MIN_RECVQUEUE && @QUEUE) {
233     ++$MAX_OUTSTANDING;
234     kick_job;
235     }
236     } else {
237     # for some reason, connected-oriented transports seem to need this
238 root 1.5 delete $TRANSPORT[$fileno]
239     unless --$TRANSPORT[$fileno][0];
240 root 1.3 }
241 root 1.1
242 root 1.3 $msg->error;
243     return;
244 root 1.1 }
245    
246 root 1.3 # For connection-oriented Transport Domains, it is possible to
247     # "recv" an empty buffer if reassembly is required.
248     if (!$msg->length) {
249     return;
250     }
251 root 1.1
252 root 1.3 # Hand the message over to Message Processing.
253     if (!defined $MESSAGE_PROCESSING->prepare_data_elements ($msg)) {
254     $MESSAGE_PROCESSING->error;
255     return;
256     }
257 root 1.1
258 root 1.3 # Set the error if applicable.
259     $msg->error ($MESSAGE_PROCESSING->error) if $MESSAGE_PROCESSING->error;
260 root 1.1
261 root 1.3 # Notify the command generator to process the response.
262     $msg->process_response_pdu;
263 root 1.1
264 root 1.3 # Cancel the timeout.
265     my $rtimeout_w = $msg->timeout_id;
266     if ($$rtimeout_w) {
267     undef $$rtimeout_w;
268    
269 root 1.9 --$BUSY;
270 root 1.3 kick_job;
271    
272 root 1.5 unless (--$TRANSPORT[$fileno][0]) {
273     delete $TRANSPORT[$fileno];
274 root 1.3 return;
275     }
276     }
277 root 1.1 }
278    
279 root 1.4 # when we end up here, we successfully handled $MAX_RECVQUEUE
280     # replies in one iteration, so assume we are overloaded
281     # and reduce the amount of parallelity.
282 root 1.5 $MAX_OUTSTANDING = (int $MAX_OUTSTANDING * 0.95) || 1;
283 root 1.8 };
284 root 1.1 }
285    
286     $msg->timeout_id (\(my $rtimeout_w =
287 root 1.8 AE::timer $pdu->timeout, 0, sub {
288 root 1.1 my $rtimeout_w = $msg->timeout_id;
289     if ($$rtimeout_w) {
290     undef $$rtimeout_w;
291 root 1.5 delete $TRANSPORT[$fileno]
292     unless --$TRANSPORT[$fileno][0];
293 root 1.1 }
294    
295     if ($retries--) {
296     _send_pdu ($pdu, $retries);
297     } else {
298     $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id);
299     $pdu->status_information ("No response from remote host '%s'", $pdu->hostname);
300 root 1.3
301 root 1.9 --$BUSY;
302 root 1.3 kick_job;
303 root 1.1 }
304     })
305 root 1.8 );
306 root 1.1 } else {
307 root 1.9 --$BUSY;
308 root 1.3 kick_job;
309 root 1.1 }
310     }
311    
312 root 1.3 sub kick_job {
313     while ($BUSY < $MAX_OUTSTANDING) {
314     my $pdu = shift @QUEUE
315     or last;
316    
317     ++$BUSY;
318     _send_pdu $pdu, $pdu->retries;
319     }
320 root 1.8
321     $DONE and $DONE->() unless $BUSY;
322 root 1.3 }
323 root 1.6
324 root 1.1 sub send_pdu($$$) {
325     my (undef, $pdu, $delay) = @_;
326    
327 root 1.3 # $delay is not very sensibly implemented by AnyEvent::SNMP,
328     # but apparently it is not a very sensible feature.
329 root 1.1 if ($delay > 0) {
330 root 1.3 ++$BUSY;
331 root 1.8 my $delay_w; $delay_w = AE::timer $delay, 0, sub {
332 root 1.1 undef $delay_w;
333 root 1.3 push @QUEUE, $pdu;
334 root 1.9 --$BUSY;
335 root 1.3 kick_job;
336 root 1.8 };
337 root 1.1 return 1;
338     }
339    
340 root 1.3 push @QUEUE, $pdu;
341     kick_job;
342    
343 root 1.1 1
344     }
345    
346 root 1.12 sub loop($) {
347 root 1.8 while ($BUSY) {
348     $DONE = AE::cv;
349     $DONE->recv;
350     undef $DONE;
351     }
352 root 1.1 }
353    
354 root 1.12 *activate = \&loop; # 5.x compatibility?
355     *listen = \&loop; # 5.x compatibility?
356    
357 root 1.1 sub one_event($) {
358 root 1.9 # should not ever be used
359 root 1.8 AnyEvent->one_event; #d# todo
360 root 1.1 }
361    
362 root 1.6 sub set_max_outstanding($) {
363     $MAX_OUTSTANDING = $_[0];
364     kick_job;
365     }
366    
367 root 1.12 # not provided yet:
368     # schedule # apparently only used by Net::SNMP::Dispatcher itself
369     # register # apparently only used by Net::SNMP::Dispatcher itself
370     # deregister # apparently only used by Net::SNMP::Dispatcher itself
371     # cancel # apparently only used by Net::SNMP::Dispatcher itself
372     # return_response_pdu # apparently not used at all?
373     # error # only used by Net::SNMP::Dispatcher itself?
374     # debug # only used by Net::SNMP::Dispatcher itself?
375    
376 root 1.1 =head1 SEE ALSO
377    
378 root 1.3 L<AnyEvent>, L<Net::SNMP>, L<Net::SNMP::XS>, L<Net::SNMP::EV>.
379 root 1.1
380     =head1 AUTHOR
381    
382     Marc Lehmann <schmorp@schmorp.de>
383     http://home.schmorp.de/
384    
385     =cut
386    
387     1
388