AnyEvent-FastPing/FastPing.pm

=head1 NAME

AnyEvent::FastPing - quickly ping a large number of hosts

=head1 SYNOPSIS

 use AnyEvent::FastPing;

=head1 DESCRIPTION

This module was written for a single purpose only: sending ICMP ECHO
REQUEST packets as quickly as possible to a large number of hosts
(thousands to millions).

It employs a sending thread and is fully event-driven (using AnyEvent), so
you have to run an event model supported by AnyEvent to use this module.

=head1 FUNCTIONS

=over 4

=cut

package AnyEvent::FastPing;

use common::sense;

use AnyEvent;

BEGIN {
   our $VERSION = '2.0';
   our @ISA = qw(Exporter);

   require Exporter;
   #Exporter::export_ok_tags (keys %EXPORT_TAGS);

   require XSLoader;
   XSLoader::load (__PACKAGE__, $VERSION);
}

our ($THR_RES_FD, $ICMP4_FD, $ICMP6_FD);

our $THR_RES_FH; open $THR_RES_FH, "<&=$THR_RES_FD" or die "FATAL: cannot fdopen";

our $ICMP4_FH; our $ICMP4_W = $ICMP4_FD >= 0 && (open $ICMP4_FH, "<&=$ICMP4_FD") && AE::io $ICMP4_FH, 0, \&_recv_icmp4;
our $ICMP6_FH; our $ICMP6_W = $ICMP6_FD >= 0 && (open $ICMP6_FH, "<&=$ICMP6_FD") && AE::io $ICMP6_FH, 0, \&_recv_icmp6;

=item AnyEvent::FastPing::ipv4_supported

Returns true iff IPv4 is supported in this module and on this system.

=item AnyEvent::FastPing::ipv6_supported

Returns true iff IPv6 is supported in this module and on this system.

=item AnyEvent::FastPing::icmp4_pktsize

Returns the number of octets per IPv4 ping packet (the whole IP packet
including headers, excluding lower-level headers or trailers such as
Ethernet).

Can be used to calculate e.g. octets/s from rate ...

   my $octets_per_second = $packets_per_second * AnyEvent::FastPing::icmp4_pktsize;

... or convert kilobit/second to packet rate ...

   my $packets_per_second = $kilobit_per_second
                            * (1000 / 8 / AnyEvent::FastPing::icmp4_pktsize);

etc.

=item AnyEvent::FastPing::icmp6_pktsize

Like AnyEvent::FastPing::icmp4_pktsize, but for IPv6.

=back

=head1 THE AnyEvent::FastPing CLASS

The AnyEvent::FastPing class represents a single "pinger". A "pinger"
comes with its own thread to send packets in the background, a rate-limit
machinery and separate idle/receive callbacks.

The recommended workflow (there are others) is this: 1. create a new
AnyEvent::FastPing object 2. configure the address lists and ranges to
ping, also configure an idle callback and optionally a receive callback
3. C<start> the pinger.

When the pinger has finished pinging all the configured addresses it will
call the idle callback.

The pinging process works like this: every range has a minimum interval
between sends, which is used to limit the rate at which hosts in that
range are being pinged. Distinct ranges are independent of each other,
which is why there is a per-pinger "global" minimum interval as well.

The pinger sends pings as fats as possible, while both obeying the pinger
rate limit as well as range limits.

When a range is exhausted, it is removed. When all ranges are exhausted,
the pinger waits another C<max_rtt> seconds and then exits, causing the
idle callback to trigger.

Performance: On my 2 GHz Opteron system with a pretty average nvidia
gigabit network card I can ping around 60k to 200k adresses per second,
depending on routing decisions.

Example: ping 10.0.0.1-10.0.0.15 with at most 100 packets/s, and
11.0.0.1-11.0.255.255 with at most 1000 packets/s. Also ping the IPv6
loopback address 5 times as fast as possible. Do not, however, exceed 1000
packets/s overall. Dump each received reply:

   use AnyEvent::Socket;
   use AnyEvent::FastPing;

   my $done = AnyEvent->condvar;

   my $pinger = new AnyEvent::FastPing;

   $pinger->interval (1/1000);
   $pinger->max_rtt (0.1); # reasonably fast/reliable network

   $pinger->add_range (v10.0.0.1, v10.0.0.15, 1/100);
   $pinger->add_range (v11.0.0.1, v11.0.255.255, 1/1000);
   $pinger->add_hosts ([ (v0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1) x 5 ]);

   $pinger->on_recv (sub {
      for (@{ $_[0] }) {
         printf "%s %g\n", (AnyEvent::Socket::format_address $_->[0]), $_->[1];
      }
   });

   $pinger->on_idle (sub {
      print "done\n";
      undef $pinger;
   });

   $pinger->start;
   $done->wait;

=head2 METHODS

=over 4

=item $pinger = new AnyEvent::FastPing

Creates a new pinger - right now there can be at most C<65536> pingers in
a process, although that limit might change to somethind drastically lower
- you should be stringy with your pinger objects.

=cut

sub new {
   my ($klass) = @_;

   _new $klass, (rand 65536), (rand 65536), (rand 65536)
}

our @IDLE_CB;

sub DESTROY {
   undef $IDLE_CB[ &id ];
   &_free;
}

=item $pinger->on_recv ($callback->([[$host, $rtt], ...]))

Registeres a callback to be called for ping replies. If no callback has
been registered than ping replies will be ignored, otherwise this module
calculates the round trip time, in seconds, for each reply and calls this
callback.

The callback receives a single argument, which is an array reference
with an entry for each reply packet (the replies will be batched for
efficiency). Each member in the array reference is again an array
reference with exactly two members: the binary host addresss (4 octets for
IPv4, 16 for IPv6) and the approximate round trip time, in seconds.

The replies will be passed to the callback as soon as they arrive, and
this callback can be called many times with batches of replies.

The receive callback will be called whenever a suitable reply arrives,
whether generated by this pinger or not, whether this pinger is started
or not. The packets will have a unique 64 bit ID to distinguish them from
other pinger objects and other generators, but this doesn't help against
malicious replies.

Note that very high packet rates can overwhelm your process, causing
replies to be dropped (configure your kernel with long receive queues for
raw sockets if this is a problem).

Example: register a callback which simply dumps the received data.

   use AnyEvent::Socket;

   $pinger->on_recv (sub {
      for (@{ $_[0] }) {
         printf "%s %g\n", (AnyEvent::Socket::format_address $_->[0]), $_->[1];
      }
   };

Example: a single ping reply with payload of 1 from C<::1> gets passed
like this:

   [
      [ "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1", 0.000280141830444336 ]
   ]

Example: ping replies for C<127.0.0.1> and C<127.0.0.2>:

   [
      [ "\177\0\0\1", 0.00015711784362793 ],
      [ "\177\0\0\2", 0.00090184211731 ]
   ]

=item $pinger->on_idle ($callback->())

Registers a callback to be called when the pinger becomes I<idle>, that
is, it has been started, has exhausted all ping ranges and waited for
the C<max_rtt> time. An idle pinger is also stopped, so the callback can
instantly add new ranges, if it so desires.

=cut

sub on_idle {
   $IDLE_CB[ &id ] = $_[1];
}

our $THR_RES_W = AE::io $THR_RES_FH, 0, sub {
   sysread $THR_RES_FH, my $buf, 8;

   for my $id (unpack "S*", $buf) {
      _stop_id $id;
      ($IDLE_CB[$id] || sub { })->();
   }
};

=item $pinger->interval ($seconds)

Configures the minimum interval between packet sends for this pinger - the
pinger will not send packets faster than this rate (or atcually 1 / rate),
even if individual ranges have a lower interval.

A value of C<0> selects the fastests possible speed (currently no faster
than 1_000_000 packets/s).

=item $pinger->max_rtt ($seconds)

If your idle callback were called instantly after all ranges were
exhausted and you destroyed the object inside (which is common), then
there would be no chance to receive some replies, as there would be no
time fo the packet to travel over the network.

This can be fixed by starting a timer in the idle callback, or more simply
by selecting a suitable C<max_rtt> value, which should be the maximum time
you allow a ping packet to travel to its destinazion and back.

The pinger thread automatically waits for this amount of time before becoming idle.

The default is currently C<0.5> seconds, which is usually plenty.

=item $pinger->add_range ($lo, $hi[, $interval])

Ping the IPv4 (or IPv6, but see below) address range, starting at binary
address C<$lo> and ending at C<$hi> (both C<$lo> and C<$hi> will be
pinged), generating no more than one ping per C<$interval> seconds (or as
fast as possible if omitted).

You can convert IP addresses from text to binary form by
using C<AnyEvent::Util::parse_address>, C<Socket::inet_aton>,
C<Socket6::inet_pton> or any other method that you like :)

The algorithm to select the next address is O(log n) on the number of
ranges, so even a large number of ranges (many thousands) is managable.

No storage is allocated per address.

Note that, while IPv6 addresses are currently supported, the usefulness of
this option is extremely limited and might be gone in future versions - if
you want to ping a number of IPv6 hosts, better specify them individually
using the C<add_hosts> method.

=item $pinger->add_hosts ([$host...], $interval, $interleave)

Similar to C<add_range>, but uses a list of single addresses instead. The
list is specified as an array reference as first argument. Each entry in
the array should be a binary host address, either IPv4 or IPv6. Currently,
all entries in the list must be either IPv4 B<OR> IPv6, so you have to
create two host ranges if you have mixed addresses.

Minimum C<$interval> is the same as for C<add_range> and can be left out.

C<$interlave> specifies an increment between addresses: often address
lists are generated in a way that results in clustering - first all
addresses from one subnet, then from the next, and so on. To avoid this,
you can specify an interleave factor. If it is C<1> (the default), then
every address is pinged in the order specified. If it is C<2>, then only
every second address will be pinged in the first round, followed by a
second round with the others. Higher factors will create C<$interleave>
runs of addresses spaced C<$interleave> indices in the list.

The special value C<0> selects a (hopefully) suitable interleave factor
automatically - currently C<256> for lists with less than 65536 addresses,
and the square root of the list length otherwise.

=item $pinger->start

Start the pinger, unless it is running already. While a pinger is running
you must not modify the pinger. If you want to change a parameter, you
have to C<stop> the pinger first.

The pinger will automatically stop when destroyed.

=item $pinger->stop

Stop the pinger, if it is running. A pinger can be stopped at any time,
after which it's current state is preserved - starting it again will
continue where it left off.

=cut

1;

=back

=head1 AUTHOR

   Marc Lehmann <schmorp@schmorp.de>
   http://home.schmorp.de/

=head1 LICENSE

   This software is distributed under the GENERAL PUBLIC LICENSE, version 2
   or any later version or, at your option, the Artistic License.

=cut

Revision:	1.13
Committed:	Wed Feb 2 19:26:45 2011 UTC (13 years, 3 months ago) by root
Branch:	MAIN
Changes since 1.12:	+219 -154 lines
Log Message:	docs
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			AnyEvent::FastPing - quickly ping a large number of hosts
4
5			=head1 SYNOPSIS
6
7			use AnyEvent::FastPing;
8
9			=head1 DESCRIPTION
10
11	root	1.2	This module was written for a single purpose only: sending ICMP ECHO
12	root	1.1	REQUEST packets as quickly as possible to a large number of hosts
13			(thousands to millions).
14
15			It employs a sending thread and is fully event-driven (using AnyEvent), so
16			you have to run an event model supported by AnyEvent to use this module.
17
18			=head1 FUNCTIONS
19
20			=over 4
21
22			=cut
23
24			package AnyEvent::FastPing;
25
26	root	1.9	use common::sense;
27	root	1.1
28			use AnyEvent;
29
30			BEGIN {
31	root	1.11	our $VERSION = '2.0';
32	root	1.1	our @ISA = qw(Exporter);
33
34			require Exporter;
35			#Exporter::export_ok_tags (keys %EXPORT_TAGS);
36
37			require XSLoader;
38			XSLoader::load (__PACKAGE__, $VERSION);
39			}
40
41	root	1.11	our ($THR_RES_FD, $ICMP4_FD, $ICMP6_FD);
42	root	1.1
43			our $THR_RES_FH; open $THR_RES_FH, "<&=$THR_RES_FD" or die "FATAL: cannot fdopen";
44
45	root	1.11	our $ICMP4_FH; our $ICMP4_W = $ICMP4_FD >= 0 && (open $ICMP4_FH, "<&=$ICMP4_FD") && AE::io $ICMP4_FH, 0, \&_recv_icmp4;
46			our $ICMP6_FH; our $ICMP6_W = $ICMP6_FD >= 0 && (open $ICMP6_FH, "<&=$ICMP6_FD") && AE::io $ICMP6_FH, 0, \&_recv_icmp6;
47	root	1.1
48			=item AnyEvent::FastPing::ipv4_supported
49
50	root	1.13	Returns true iff IPv4 is supported in this module and on this system.
51	root	1.1
52			=item AnyEvent::FastPing::ipv6_supported
53
54	root	1.13	Returns true iff IPv6 is supported in this module and on this system.
55	root	1.1
56			=item AnyEvent::FastPing::icmp4_pktsize
57
58	root	1.13	Returns the number of octets per IPv4 ping packet (the whole IP packet
59			including headers, excluding lower-level headers or trailers such as
60			Ethernet).
61
62			Can be used to calculate e.g. octets/s from rate ...
63
64			my $octets_per_second = $packets_per_second * AnyEvent::FastPing::icmp4_pktsize;
65
66			... or convert kilobit/second to packet rate ...
67
68			my $packets_per_second = $kilobit_per_second
69			* (1000 / 8 / AnyEvent::FastPing::icmp4_pktsize);
70
71			etc.
72	root	1.1
73			=item AnyEvent::FastPing::icmp6_pktsize
74
75	root	1.13	Like AnyEvent::FastPing::icmp4_pktsize, but for IPv6.
76
77			=back
78
79			=head1 THE AnyEvent::FastPing CLASS
80
81			The AnyEvent::FastPing class represents a single "pinger". A "pinger"
82			comes with its own thread to send packets in the background, a rate-limit
83			machinery and separate idle/receive callbacks.
84
85			The recommended workflow (there are others) is this: 1. create a new
86			AnyEvent::FastPing object 2. configure the address lists and ranges to
87			ping, also configure an idle callback and optionally a receive callback
88			3. C<start> the pinger.
89
90			When the pinger has finished pinging all the configured addresses it will
91			call the idle callback.
92
93			The pinging process works like this: every range has a minimum interval
94			between sends, which is used to limit the rate at which hosts in that
95			range are being pinged. Distinct ranges are independent of each other,
96			which is why there is a per-pinger "global" minimum interval as well.
97
98			The pinger sends pings as fats as possible, while both obeying the pinger
99			rate limit as well as range limits.
100
101			When a range is exhausted, it is removed. When all ranges are exhausted,
102			the pinger waits another C<max_rtt> seconds and then exits, causing the
103			idle callback to trigger.
104
105			Performance: On my 2 GHz Opteron system with a pretty average nvidia
106			gigabit network card I can ping around 60k to 200k adresses per second,
107			depending on routing decisions.
108
109			Example: ping 10.0.0.1-10.0.0.15 with at most 100 packets/s, and
110			11.0.0.1-11.0.255.255 with at most 1000 packets/s. Also ping the IPv6
111			loopback address 5 times as fast as possible. Do not, however, exceed 1000
112			packets/s overall. Dump each received reply:
113
114			use AnyEvent::Socket;
115			use AnyEvent::FastPing;
116
117			my $done = AnyEvent->condvar;
118
119			my $pinger = new AnyEvent::FastPing;
120
121			$pinger->interval (1/1000);
122			$pinger->max_rtt (0.1); # reasonably fast/reliable network
123
124			$pinger->add_range (v10.0.0.1, v10.0.0.15, 1/100);
125			$pinger->add_range (v11.0.0.1, v11.0.255.255, 1/1000);
126			$pinger->add_hosts ([ (v0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1) x 5 ]);
127
128			$pinger->on_recv (sub {
129			for (@{ $_[0] }) {
130			printf "%s %g\n", (AnyEvent::Socket::format_address $_->[0]), $_->[1];
131			}
132			});
133
134			$pinger->on_idle (sub {
135			print "done\n";
136			undef $pinger;
137			});
138
139			$pinger->start;
140			$done->wait;
141
142			=head2 METHODS
143
144			=over 4
145
146			=item $pinger = new AnyEvent::FastPing
147
148			Creates a new pinger - right now there can be at most C<65536> pingers in
149			a process, although that limit might change to somethind drastically lower
150			- you should be stringy with your pinger objects.
151	root	1.1
152	root	1.11	=cut
153
154			sub new {
155			my ($klass) = @_;
156
157			_new $klass, (rand 65536), (rand 65536), (rand 65536)
158			}
159
160			our @IDLE_CB;
161
162			sub DESTROY {
163			undef $IDLE_CB[ &id ];
164			&_free;
165			}
166
167	root	1.13	=item $pinger->on_recv ($callback->([[$host, $rtt], ...]))
168
169			Registeres a callback to be called for ping replies. If no callback has
170			been registered than ping replies will be ignored, otherwise this module
171			calculates the round trip time, in seconds, for each reply and calls this
172			callback.
173
174			The callback receives a single argument, which is an array reference
175			with an entry for each reply packet (the replies will be batched for
176			efficiency). Each member in the array reference is again an array
177			reference with exactly two members: the binary host addresss (4 octets for
178			IPv4, 16 for IPv6) and the approximate round trip time, in seconds.
179
180			The replies will be passed to the callback as soon as they arrive, and
181			this callback can be called many times with batches of replies.
182
183			The receive callback will be called whenever a suitable reply arrives,
184			whether generated by this pinger or not, whether this pinger is started
185			or not. The packets will have a unique 64 bit ID to distinguish them from
186			other pinger objects and other generators, but this doesn't help against
187			malicious replies.
188
189			Note that very high packet rates can overwhelm your process, causing
190			replies to be dropped (configure your kernel with long receive queues for
191			raw sockets if this is a problem).
192
193			Example: register a callback which simply dumps the received data.
194
195			use AnyEvent::Socket;
196
197			$pinger->on_recv (sub {
198			for (@{ $_[0] }) {
199			printf "%s %g\n", (AnyEvent::Socket::format_address $_->[0]), $_->[1];
200			}
201			};
202
203			Example: a single ping reply with payload of 1 from C<::1> gets passed
204			like this:
205
206			[
207			[ "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1", 0.000280141830444336 ]
208			]
209
210			Example: ping replies for C<127.0.0.1> and C<127.0.0.2>:
211
212			[
213			[ "\177\0\0\1", 0.00015711784362793 ],
214			[ "\177\0\0\2", 0.00090184211731 ]
215			]
216
217			=item $pinger->on_idle ($callback->())
218
219			Registers a callback to be called when the pinger becomes I<idle>, that
220			is, it has been started, has exhausted all ping ranges and waited for
221			the C<max_rtt> time. An idle pinger is also stopped, so the callback can
222			instantly add new ranges, if it so desires.
223
224			=cut
225
226	root	1.11	sub on_idle {
227			$IDLE_CB[ &id ] = $_[1];
228			}
229
230			our $THR_RES_W = AE::io $THR_RES_FH, 0, sub {
231			sysread $THR_RES_FH, my $buf, 8;
232
233			for my $id (unpack "S*", $buf) {
234			_stop_id $id;
235			($IDLE_CB[$id] \|\| sub { })->();
236			}
237			};
238
239	root	1.13	=item $pinger->interval ($seconds)
240	root	1.12
241	root	1.13	Configures the minimum interval between packet sends for this pinger - the
242			pinger will not send packets faster than this rate (or atcually 1 / rate),
243			even if individual ranges have a lower interval.
244	root	1.11
245	root	1.13	A value of C<0> selects the fastests possible speed (currently no faster
246			than 1_000_000 packets/s).
247	root	1.1
248	root	1.13	=item $pinger->max_rtt ($seconds)
249	root	1.1
250	root	1.13	If your idle callback were called instantly after all ranges were
251			exhausted and you destroyed the object inside (which is common), then
252			there would be no chance to receive some replies, as there would be no
253			time fo the packet to travel over the network.
254	root	1.1
255	root	1.13	This can be fixed by starting a timer in the idle callback, or more simply
256			by selecting a suitable C<max_rtt> value, which should be the maximum time
257			you allow a ping packet to travel to its destinazion and back.
258	root	1.1
259	root	1.13	The pinger thread automatically waits for this amount of time before becoming idle.
260	root	1.1
261	root	1.13	The default is currently C<0.5> seconds, which is usually plenty.
262	root	1.1
263	root	1.13	=item $pinger->add_range ($lo, $hi[, $interval])
264	root	1.1
265	root	1.13	Ping the IPv4 (or IPv6, but see below) address range, starting at binary
266			address C<$lo> and ending at C<$hi> (both C<$lo> and C<$hi> will be
267			pinged), generating no more than one ping per C<$interval> seconds (or as
268			fast as possible if omitted).
269	root	1.1
270	root	1.13	You can convert IP addresses from text to binary form by
271			using C<AnyEvent::Util::parse_address>, C<Socket::inet_aton>,
272			C<Socket6::inet_pton> or any other method that you like :)
273	root	1.1
274	root	1.13	The algorithm to select the next address is O(log n) on the number of
275			ranges, so even a large number of ranges (many thousands) is managable.
276	root	1.1
277	root	1.13	No storage is allocated per address.
278	root	1.1
279	root	1.13	Note that, while IPv6 addresses are currently supported, the usefulness of
280			this option is extremely limited and might be gone in future versions - if
281			you want to ping a number of IPv6 hosts, better specify them individually
282			using the C<add_hosts> method.
283
284			=item $pinger->add_hosts ([$host...], $interval, $interleave)
285
286			Similar to C<add_range>, but uses a list of single addresses instead. The
287			list is specified as an array reference as first argument. Each entry in
288			the array should be a binary host address, either IPv4 or IPv6. Currently,
289			all entries in the list must be either IPv4 B<OR> IPv6, so you have to
290			create two host ranges if you have mixed addresses.
291
292			Minimum C<$interval> is the same as for C<add_range> and can be left out.
293
294			C<$interlave> specifies an increment between addresses: often address
295			lists are generated in a way that results in clustering - first all
296			addresses from one subnet, then from the next, and so on. To avoid this,
297			you can specify an interleave factor. If it is C<1> (the default), then
298			every address is pinged in the order specified. If it is C<2>, then only
299			every second address will be pinged in the first round, followed by a
300			second round with the others. Higher factors will create C<$interleave>
301			runs of addresses spaced C<$interleave> indices in the list.
302
303			The special value C<0> selects a (hopefully) suitable interleave factor
304			automatically - currently C<256> for lists with less than 65536 addresses,
305			and the square root of the list length otherwise.
306
307			=item $pinger->start
308
309			Start the pinger, unless it is running already. While a pinger is running
310			you must not modify the pinger. If you want to change a parameter, you
311			have to C<stop> the pinger first.
312
313			The pinger will automatically stop when destroyed.
314
315			=item $pinger->stop
316
317			Stop the pinger, if it is running. A pinger can be stopped at any time,
318			after which it's current state is preserved - starting it again will
319			continue where it left off.
320	root	1.1
321			=cut
322
323			1;
324
325			=back
326
327			=head1 AUTHOR
328
329	root	1.4	Marc Lehmann <schmorp@schmorp.de>
330			http://home.schmorp.de/
331	root	1.1
332	root	1.4	=head1 LICENSE
333	root	1.1
334	root	1.4	This software is distributed under the GENERAL PUBLIC LICENSE, version 2
335			or any later version or, at your option, the Artistic License.
336	root	1.1
337			=cut
338