ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-SNMP/SNMP.pm
Revision: 1.10
Committed: Sun Oct 31 18:15:22 2010 UTC (13 years, 7 months ago) by root
Branch: MAIN
Changes since 1.9: +13 -9 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.1 =head1 NAME
2    
3 root 1.7 AnyEvent::SNMP - adaptor to integrate Net::SNMP into AnyEvent.
4 root 1.1
5     =head1 SYNOPSIS
6    
7     use AnyEvent::SNMP;
8     use Net::SNMP;
9    
10     # just use Net::SNMP and AnyEvent as you like:
11    
12     # use a condvar to transfer results, this is
13     # just an example, you can use a naked callback as well.
14     my $cv = AnyEvent->condvar;
15    
16     # ... start non-blocking snmp request(s)...
17     Net::SNMP->session (-hostname => "127.0.0.1",
18     -community => "public",
19     -nonblocking => 1)
20     ->get_request (-callback => sub { $cv->send (@_) });
21    
22     # ... do something else until the result is required
23     my @result = $cv->wait;
24    
25     =head1 DESCRIPTION
26    
27     This module implements an alternative "event dispatcher" for Net::SNMP,
28 root 1.10 using AnyEvent as a backend. This integrates Net::SNMP into AnyEvent. That
29     means you can make non-blocking Net::SNMP calls and as long as other
30     parts of your program also use AnyEvent (or some event loop supported by
31     AnyEvent), they will run in parallel.
32 root 1.1
33     Also, the Net::SNMP scheduler is very inefficient with respect to both CPU
34     and memory usage. Most AnyEvent backends (including the pure-perl backend)
35     fare much better than the Net::SNMP dispatcher.
36    
37 root 1.10 Another major added fetaure of this module over Net::SNMP is automatic
38     rate-adjustments: Net::SNMP is so slow that firing a few thousand
39     requests can cause many timeouts simply because Net::SNMP cannot process
40     the replies in time. This module automatically adapts the send rate to
41     avoid false timeouts caused by slow reply processing.
42    
43     A potential disadvantage of this module is that replacing the dispatcher
44     is not at all a documented thing to do, so future changes in Net::SNP
45     might break this module (or the many similar ones).
46 root 1.1
47     This module does not export anything and does not require you to do
48     anything special apart from loading it I<before doing any non-blocking
49     requests with Net::SNMP>. It is recommended but not required to load this
50     module before C<Net::SNMP>.
51    
52 root 1.3 =head1 GLOBAL VARIABLES
53    
54     =over 4
55    
56     =item $AnyEvent::SNMP::MAX_OUTSTANDING (default: C<50>, dynamic)
57    
58 root 1.6 =item AnyEvent::SNMP::set_max_outstanding $new_value
59    
60 root 1.3 Use this package variable to restrict the number of outstanding SNMP
61     requests at any point in time.
62    
63     Net::SNMP is very fast at creating and sending SNMP requests, but much
64     slower at parsing (big, bulk) responses. This makes it easy to request a
65     lot of data that can take many seconds to parse.
66    
67     In the best case, this can lead to unnecessary delays (and even time-outs,
68     as the data has been received but not yet processed) and in the worst
69     case, this can lead to packet loss, when the receive queue overflows and
70     the kernel can no longer accept new packets.
71    
72 root 1.6 To avoid this, you can (and should) limit the number of outstanding
73     requests to a number low enough so that parsing time doesn't introduce
74     noticable delays.
75 root 1.3
76     Unfortunately, this number depends not only on processing speed and load
77     of the machine running Net::SNMP, but also on the network latency and the
78     speed of your SNMP agents.
79    
80     AnyEvent::SNMP tries to dynamically adjust this number dynamically upwards
81     and downwards.
82    
83 root 1.6 Increasing C<$MAX_OUTSTANDING> will not automatically use the
84 root 1.8 extra request slots. To increase C<$MAX_OUTSTANDING> and make
85     C<AnyEvent::SNMP> make use of the extra paralellity, call
86 root 1.6 C<AnyEvent::SNMP::set_max_outstanding> with the new value, e.g.:
87    
88     AnyEvent::SNMP::set_max_outstanding 500;
89    
90     Although due to the dynamic adjustment, this might have little lasting
91     effect.
92    
93 root 1.3 Note that you can use L<Net::SNMP::XS> to speed up parsing of responses
94     considerably.
95    
96 root 1.5 =item $AnyEvent::SNMP::MIN_RECVQUEUE (default: C<8>)
97 root 1.3
98     =item $AnyEvent::SNMP::MAX_RECVQUEUE (default: C<64>)
99    
100     These values specify the minimum and maximum receive queue length (in
101     units of one response packet).
102    
103     When AnyEvent::SNMP handles $MAX_RECVQUEUE or more packets per iteration
104     it will reduce $MAX_OUTSTANDING. If it handles less than $MIN_RECVQUEUE,
105     it increases $MAX_OUTSTANDING.
106    
107     This has the result of adjusting the number of outstanding requests so that
108     the recv queue is between the minimum and maximu, usually.
109    
110     This algorithm works reasonably well as long as the responses, response
111     latencies and processing times are the same size per packet on average.
112    
113     =back
114    
115     =head1 COMPATIBILITY
116    
117     This module may be used as a drop in replacement for the
118     Net::SNMP::Dispatcher in existing programs. You can still call
119     C<snmp_dispatcher> to start the event-loop, but then you loose the benefit
120     of mixing Net::SNMP events with other events.
121    
122     use AnyEvent::SNMP;
123     use Net::SNMP;
124    
125     # just use Net::SNMP as before
126    
127     # ... start non-blocking snmp request(s)...
128     Net::SNMP->session (
129     -hostname => "127.0.0.1",
130     -community => "public",
131     -nonblocking => 1,
132     )->get_request (-callback => sub { ... });
133    
134     snmp_dispatcher;
135    
136 root 1.1 =cut
137    
138     package AnyEvent::SNMP;
139    
140     no warnings;
141     use strict qw(subs vars);
142    
143     # it is possible to do this without loading
144     # Net::SNMP::Dispatcher, but much more awkward.
145     use Net::SNMP::Dispatcher;
146    
147     sub Net::SNMP::Dispatcher::instance {
148     AnyEvent::SNMP::
149     }
150    
151     use Net::SNMP ();
152     use AnyEvent ();
153    
154 root 1.8 our $VERSION = '1.0';
155 root 1.1
156     $Net::SNMP::DISPATCHER = instance Net::SNMP::Dispatcher;
157    
158     our $MESSAGE_PROCESSING = $Net::SNMP::Dispatcher::MESSAGE_PROCESSING;
159    
160     our $BUSY;
161 root 1.8 our $DONE; # finished all jobs
162 root 1.5 our @TRANSPORT; # fileno => [count, watcher]
163 root 1.3 our @QUEUE;
164     our $MAX_OUTSTANDING = 50;
165 root 1.5 our $MIN_RECVQUEUE = 8;
166 root 1.3 our $MAX_RECVQUEUE = 64;
167    
168 root 1.9 sub kick_job;
169 root 1.1
170     sub _send_pdu {
171     my ($pdu, $retries) = @_;
172    
173     # mostly copied from Net::SNMP::Dispatch
174    
175     # Pass the PDU to Message Processing so that it can
176     # create the new outgoing message.
177     my $msg = $MESSAGE_PROCESSING->prepare_outgoing_msg ($pdu);
178    
179     if (!defined $msg) {
180 root 1.9 --$BUSY;
181 root 1.3 kick_job;
182 root 1.1 # Inform the command generator about the Message Processing error.
183     $pdu->status_information ($MESSAGE_PROCESSING->error);
184     return;
185     }
186    
187     # Actually send the message.
188     if (!defined $msg->send) {
189     $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id)
190     if $pdu->expect_response;
191    
192     # A crude attempt to recover from temporary failures.
193     if ($retries-- > 0 && ($!{EAGAIN} || $!{EWOULDBLOCK} || $!{ENOSPC})) {
194 root 1.8 my $retry_w; $retry_w = AE::timer $pdu->timeout, 0, sub {
195 root 1.1 undef $retry_w;
196     _send_pdu ($pdu, $retries);
197 root 1.8 };
198 root 1.1 } else {
199 root 1.9 --$BUSY;
200 root 1.3 kick_job;
201 root 1.1 }
202    
203     # Inform the command generator about the send() error.
204     $pdu->status_information ($msg->error);
205     return;
206     }
207    
208     # Schedule the timeout handler if the message expects a response.
209     if ($pdu->expect_response) {
210     my $transport = $msg->transport;
211 root 1.5 my $fileno = $transport->fileno;
212 root 1.1
213     # register the transport
214 root 1.5 unless ($TRANSPORT[$fileno][0]++) {
215 root 1.8 $TRANSPORT[$fileno][1] = AE::io $transport->socket, 0, sub {
216 root 1.3 for my $count (1..$MAX_RECVQUEUE) { # handle up to this many requests in one go
217     # Create a new Message object to receive the response
218     my ($msg, $error) = Net::SNMP::Message->new (-transport => $transport);
219    
220     if (!defined $msg) {
221     die sprintf 'Failed to create Message object [%s]', $error;
222     }
223 root 1.1
224 root 1.3 # Read the message from the Transport Layer
225     if (!defined $msg->recv) {
226     if ($transport->connectionless) {
227 root 1.4 # if we handled very few replies and we have queued work, try
228     # to increase the parallelity as we probably can handle more.
229 root 1.3 if ($count < $MIN_RECVQUEUE && @QUEUE) {
230     ++$MAX_OUTSTANDING;
231     kick_job;
232     }
233     } else {
234     # for some reason, connected-oriented transports seem to need this
235 root 1.5 delete $TRANSPORT[$fileno]
236     unless --$TRANSPORT[$fileno][0];
237 root 1.3 }
238 root 1.1
239 root 1.3 $msg->error;
240     return;
241 root 1.1 }
242    
243 root 1.3 # For connection-oriented Transport Domains, it is possible to
244     # "recv" an empty buffer if reassembly is required.
245     if (!$msg->length) {
246     return;
247     }
248 root 1.1
249 root 1.3 # Hand the message over to Message Processing.
250     if (!defined $MESSAGE_PROCESSING->prepare_data_elements ($msg)) {
251     $MESSAGE_PROCESSING->error;
252     return;
253     }
254 root 1.1
255 root 1.3 # Set the error if applicable.
256     $msg->error ($MESSAGE_PROCESSING->error) if $MESSAGE_PROCESSING->error;
257 root 1.1
258 root 1.3 # Notify the command generator to process the response.
259     $msg->process_response_pdu;
260 root 1.1
261 root 1.3 # Cancel the timeout.
262     my $rtimeout_w = $msg->timeout_id;
263     if ($$rtimeout_w) {
264     undef $$rtimeout_w;
265    
266 root 1.9 --$BUSY;
267 root 1.3 kick_job;
268    
269 root 1.5 unless (--$TRANSPORT[$fileno][0]) {
270     delete $TRANSPORT[$fileno];
271 root 1.3 return;
272     }
273     }
274 root 1.1 }
275    
276 root 1.4 # when we end up here, we successfully handled $MAX_RECVQUEUE
277     # replies in one iteration, so assume we are overloaded
278     # and reduce the amount of parallelity.
279 root 1.5 $MAX_OUTSTANDING = (int $MAX_OUTSTANDING * 0.95) || 1;
280 root 1.8 };
281 root 1.1 }
282    
283     $msg->timeout_id (\(my $rtimeout_w =
284 root 1.8 AE::timer $pdu->timeout, 0, sub {
285 root 1.1 my $rtimeout_w = $msg->timeout_id;
286     if ($$rtimeout_w) {
287     undef $$rtimeout_w;
288 root 1.5 delete $TRANSPORT[$fileno]
289     unless --$TRANSPORT[$fileno][0];
290 root 1.1 }
291    
292     if ($retries--) {
293     _send_pdu ($pdu, $retries);
294     } else {
295     $MESSAGE_PROCESSING->msg_handle_delete ($pdu->msg_id);
296     $pdu->status_information ("No response from remote host '%s'", $pdu->hostname);
297 root 1.3
298 root 1.9 --$BUSY;
299 root 1.3 kick_job;
300 root 1.1 }
301     })
302 root 1.8 );
303 root 1.1 } else {
304 root 1.9 --$BUSY;
305 root 1.3 kick_job;
306 root 1.1 }
307     }
308    
309 root 1.3 sub kick_job {
310     while ($BUSY < $MAX_OUTSTANDING) {
311     my $pdu = shift @QUEUE
312     or last;
313    
314     ++$BUSY;
315     _send_pdu $pdu, $pdu->retries;
316     }
317 root 1.8
318     $DONE and $DONE->() unless $BUSY;
319 root 1.3 }
320 root 1.6
321 root 1.1 sub send_pdu($$$) {
322     my (undef, $pdu, $delay) = @_;
323    
324 root 1.3 # $delay is not very sensibly implemented by AnyEvent::SNMP,
325     # but apparently it is not a very sensible feature.
326 root 1.1 if ($delay > 0) {
327 root 1.3 ++$BUSY;
328 root 1.8 my $delay_w; $delay_w = AE::timer $delay, 0, sub {
329 root 1.1 undef $delay_w;
330 root 1.3 push @QUEUE, $pdu;
331 root 1.9 --$BUSY;
332 root 1.3 kick_job;
333 root 1.8 };
334 root 1.1 return 1;
335     }
336    
337 root 1.3 push @QUEUE, $pdu;
338     kick_job;
339    
340 root 1.1 1
341     }
342    
343     sub activate($) {
344 root 1.8 while ($BUSY) {
345     $DONE = AE::cv;
346     $DONE->recv;
347     undef $DONE;
348     }
349 root 1.1 }
350    
351     sub one_event($) {
352 root 1.9 # should not ever be used
353 root 1.8 AnyEvent->one_event; #d# todo
354 root 1.1 }
355    
356 root 1.6 sub set_max_outstanding($) {
357     $MAX_OUTSTANDING = $_[0];
358     kick_job;
359     }
360    
361 root 1.1 =head1 SEE ALSO
362    
363 root 1.3 L<AnyEvent>, L<Net::SNMP>, L<Net::SNMP::XS>, L<Net::SNMP::EV>.
364 root 1.1
365     =head1 AUTHOR
366    
367     Marc Lehmann <schmorp@schmorp.de>
368     http://home.schmorp.de/
369    
370     =cut
371    
372     1
373