ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP.pm
(Generate patch)

Comparing AnyEvent-MP/MP.pm (file contents):
Revision 1.12 by root, Sun Aug 2 18:11:48 2009 UTC vs.
Revision 1.29 by root, Tue Aug 4 23:16:57 2009 UTC

4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use AnyEvent::MP; 7 use AnyEvent::MP;
8 8
9 NODE # returns this node identifier
10 $NODE # contains this node identifier 9 $NODE # contains this node's noderef
10 NODE # returns this node's noderef
11 NODE $port # returns the noderef of the port
11 12
12 snd $port, type => data...; 13 snd $port, type => data...;
14
15 $SELF # receiving/own port id in rcv callbacks
13 16
14 rcv $port, smartmatch => $cb->($port, @msg); 17 rcv $port, smartmatch => $cb->($port, @msg);
15 18
16 # examples: 19 # examples:
17 rcv $port2, ping => sub { snd $_[0], "pong"; 0 }; 20 rcv $port2, ping => sub { snd $_[0], "pong"; 0 };
27This module (-family) implements a simple message passing framework. 30This module (-family) implements a simple message passing framework.
28 31
29Despite its simplicity, you can securely message other processes running 32Despite its simplicity, you can securely message other processes running
30on the same or other hosts. 33on the same or other hosts.
31 34
35For an introduction to this module family, see the L<AnyEvent::MP::Intro>
36manual page.
37
32At the moment, this module family is severly brokena nd underdocumented, 38At the moment, this module family is severly broken and underdocumented,
33so do not use. This was uploaded mainly to resreve the CPAN namespace - 39so do not use. This was uploaded mainly to reserve the CPAN namespace -
34stay tuned! 40stay tuned! The basic API should be finished, however.
35 41
36=head1 CONCEPTS 42=head1 CONCEPTS
37 43
38=over 4 44=over 4
39 45
40=item port 46=item port
41 47
42A port is something you can send messages to with the C<snd> function, and 48A port is something you can send messages to (with the C<snd> function).
43you can register C<rcv> handlers with. All C<rcv> handlers will receive 49
44messages they match, messages will not be queued. 50Some ports allow you to register C<rcv> handlers that can match specific
51messages. All C<rcv> handlers will receive messages they match, messages
52will not be queued.
45 53
46=item port id - C<noderef#portname> 54=item port id - C<noderef#portname>
47 55
48A port id is always the noderef, a hash-mark (C<#>) as separator, followed 56A port id is normaly the concatenation of a noderef, a hash-mark (C<#>) as
49by a port name (a printable string of unspecified format). 57separator, and a port name (a printable string of unspecified format). An
58exception is the the node port, whose ID is identical to it's node
59reference.
50 60
51=item node 61=item node
52 62
53A node is a single process containing at least one port - the node 63A node is a single process containing at least one port - the node
54port. You can send messages to node ports to let them create new ports, 64port. You can send messages to node ports to find existing ports or to
55among other things. 65create new ports, among other things.
56 66
57Initially, nodes are either private (single-process only) or hidden 67Nodes are either private (single-process only), slaves (connected to a
58(connected to a master node only). Only when they epxlicitly "become 68master node only) or public nodes (connectable from unrelated nodes).
59public" can you send them messages from unrelated other nodes.
60 69
61=item noderef - C<host:port,host:port...>, C<id@noderef>, C<id> 70=item noderef - C<host:port,host:port...>, C<id@noderef>, C<id>
62 71
63A noderef is a string that either uniquely identifies a given node (for 72A node reference is a string that either simply identifies the node (for
64private and hidden nodes), or contains a recipe on how to reach a given 73private and slave nodes), or contains a recipe on how to reach a given
65node (for public nodes). 74node (for public nodes).
66 75
76This recipe is simply a comma-separated list of C<address:port> pairs (for
77TCP/IP, other protocols might look different).
78
79Node references come in two flavours: resolved (containing only numerical
80addresses) or unresolved (where hostnames are used instead of addresses).
81
82Before using an unresolved node reference in a message you first have to
83resolve it.
84
67=back 85=back
68 86
69=head1 VARIABLES/FUNCTIONS 87=head1 VARIABLES/FUNCTIONS
70 88
71=over 4 89=over 4
82 100
83use AE (); 101use AE ();
84 102
85use base "Exporter"; 103use base "Exporter";
86 104
87our $VERSION = '0.02'; 105our $VERSION = '0.1';
88our @EXPORT = qw( 106our @EXPORT = qw(
89 NODE $NODE $PORT snd rcv _any_ 107 NODE $NODE *SELF node_of _any_
90 create_port create_port_on 108 resolve_node
91 create_miniport
92 become_slave become_public 109 become_slave become_public
110 snd rcv mon kil reg psub
111 port
93); 112);
94 113
114our $SELF;
115
116sub _self_die() {
117 my $msg = $@;
118 $msg =~ s/\n+$// unless ref $msg;
119 kil $SELF, die => $msg;
120}
121
95=item NODE / $NODE 122=item $thisnode = NODE / $NODE
96 123
97The C<NODE ()> function and the C<$NODE> variable contain the noderef of 124The C<NODE> function returns, and the C<$NODE> variable contains
98the local node. The value is initialised by a call to C<become_public> or 125the noderef of the local node. The value is initialised by a call
99C<become_slave>, after which all local port identifiers become invalid. 126to C<become_public> or C<become_slave>, after which all local port
127identifiers become invalid.
128
129=item $noderef = node_of $portid
130
131Extracts and returns the noderef from a portid or a noderef.
132
133=item $cv = resolve_node $noderef
134
135Takes an unresolved node reference that may contain hostnames and
136abbreviated IDs, resolves all of them and returns a resolved node
137reference.
138
139In addition to C<address:port> pairs allowed in resolved noderefs, the
140following forms are supported:
141
142=over 4
143
144=item the empty string
145
146An empty-string component gets resolved as if the default port (4040) was
147specified.
148
149=item naked port numbers (e.g. C<1234>)
150
151These are resolved by prepending the local nodename and a colon, to be
152further resolved.
153
154=item hostnames (e.g. C<localhost:1234>, C<localhost>)
155
156These are resolved by using AnyEvent::DNS to resolve them, optionally
157looking up SRV records for the C<aemp=4040> port, if no port was
158specified.
159
160=back
161
162=item $SELF
163
164Contains the current port id while executing C<rcv> callbacks or C<psub>
165blocks.
166
167=item SELF, %SELF, @SELF...
168
169Due to some quirks in how perl exports variables, it is impossible to
170just export C<$SELF>, all the symbols called C<SELF> are exported by this
171module, but only C<$SELF> is currently used.
100 172
101=item snd $portid, type => @data 173=item snd $portid, type => @data
102 174
103=item snd $portid, @msg 175=item snd $portid, @msg
104 176
118JSON is used, then only strings, numbers and arrays and hashes consisting 190JSON is used, then only strings, numbers and arrays and hashes consisting
119of those are allowed (no objects). When Storable is used, then anything 191of those are allowed (no objects). When Storable is used, then anything
120that Storable can serialise and deserialise is allowed, and for the local 192that Storable can serialise and deserialise is allowed, and for the local
121node, anything can be passed. 193node, anything can be passed.
122 194
123=item $local_port = create_port 195=item kil $portid[, @reason]
124 196
125Create a new local port object. See the next section for allowed methods. 197Kill the specified port with the given C<@reason>.
126 198
127=cut 199If no C<@reason> is specified, then the port is killed "normally" (linked
200ports will not be kileld, or even notified).
128 201
129sub create_port { 202Otherwise, linked ports get killed with the same reason (second form of
130 my $id = "$AnyEvent::MP::Base::UNIQ." . ++$AnyEvent::MP::Base::ID; 203C<mon>, see below).
131 204
132 my $self = bless { 205Runtime errors while evaluating C<rcv> callbacks or inside C<psub> blocks
133 id => "$NODE#$id", 206will be reported as reason C<< die => $@ >>.
134 names => [$id],
135 }, "AnyEvent::MP::Port";
136 207
137 $AnyEvent::MP::Base::PORT{$id} = sub { 208Transport/communication errors are reported as C<< transport_error =>
138 unshift @_, $self; 209$message >>.
139 210
140 for (@{ $self->{rc0}{$_[1]} }) { 211=item $guard = mon $portid, $cb->(@reason)
141 $_ && &{$_->[0]} 212
142 && undef $_; 213=item $guard = mon $portid, $otherport
214
215=item $guard = mon $portid, $otherport, @msg
216
217Monitor the given port and do something when the port is killed.
218
219In the first form, the callback is simply called with any number
220of C<@reason> elements (no @reason means that the port was deleted
221"normally"). Note also that I<< the callback B<must> never die >>, so use
222C<eval> if unsure.
223
224In the second form, the other port will be C<kil>'ed with C<@reason>, iff
225a @reason was specified, i.e. on "normal" kils nothing happens, while
226under all other conditions, the other port is killed with the same reason.
227
228In the last form, a message of the form C<@msg, @reason> will be C<snd>.
229
230Example: call a given callback when C<$port> is killed.
231
232 mon $port, sub { warn "port died because of <@_>\n" };
233
234Example: kill ourselves when C<$port> is killed abnormally.
235
236 mon $port, $self;
237
238Example: send us a restart message another C<$port> is killed.
239
240 mon $port, $self => "restart";
241
242=cut
243
244sub mon {
245 my ($noderef, $port, $cb) = ((split /#/, shift, 2), shift);
246
247 my $node = $NODE{$noderef} || add_node $noderef;
248
249 #TODO: ports must not be references
250 if (!ref $cb or "AnyEvent::MP::Port" eq ref $cb) {
251 if (@_) {
252 # send a kill info message
253 my (@msg) = ($cb, @_);
254 $cb = sub { snd @msg, @_ };
255 } else {
256 # simply kill other port
257 my $port = $cb;
258 $cb = sub { kil $port, @_ if @_ };
143 } 259 }
260 }
144 261
145 for (@{ $self->{rcv}{$_[1]} }) { 262 $node->monitor ($port, $cb);
146 $_ && [@_[1 .. @{$_->[1]}]] ~~ $_->[1] 263
147 && &{$_->[0]} 264 defined wantarray
148 && undef $_; 265 and AnyEvent::Util::guard { $node->unmonitor ($port, $cb) }
266}
267
268=item $guard = mon_guard $port, $ref, $ref...
269
270Monitors the given C<$port> and keeps the passed references. When the port
271is killed, the references will be freed.
272
273Optionally returns a guard that will stop the monitoring.
274
275This function is useful when you create e.g. timers or other watchers and
276want to free them when the port gets killed:
277
278 $port->rcv (start => sub {
279 my $timer; $timer = mon_guard $port, AE::timer 1, 1, sub {
280 undef $timer if 0.9 < rand;
149 } 281 });
282 });
150 283
151 for (@{ $self->{any} }) { 284=cut
152 $_ && [@_[0 .. $#{$_->[1]}]] ~~ $_->[1] 285
153 && &{$_->[0]} 286sub mon_guard {
154 && undef $_; 287 my ($port, @refs) = @_;
155 } 288
289 mon $port, sub { 0 && @refs }
290}
291
292=item lnk $port1, $port2
293
294Link two ports. This is simply a shorthand for:
295
296 mon $port1, $port2;
297 mon $port2, $port1;
298
299It means that if either one is killed abnormally, the other one gets
300killed as well.
301
302=item $local_port = port
303
304Create a new local port object that supports message matching.
305
306=item $portid = port { my @msg = @_; $finished }
307
308Creates a "mini port", that is, a very lightweight port without any
309pattern matching behind it, and returns its ID.
310
311The block will be called for every message received on the port. When the
312callback returns a true value its job is considered "done" and the port
313will be destroyed. Otherwise it will stay alive.
314
315The message will be passed as-is, no extra argument (i.e. no port id) will
316be passed to the callback.
317
318If you need the local port id in the callback, this works nicely:
319
320 my $port; $port = miniport {
321 snd $otherport, reply => $port;
156 }; 322 };
157 323
158 $self 324=cut
325
326sub port(;&) {
327 my $id = "$UNIQ." . $ID++;
328 my $port = "$NODE#$id";
329
330 if (@_) {
331 my $cb = shift;
332 $PORT{$id} = sub {
333 local $SELF = $port;
334 eval {
335 &$cb
336 and kil $id;
337 };
338 _self_die if $@;
339 };
340 } else {
341 my $self = bless {
342 id => "$NODE#$id",
343 }, "AnyEvent::MP::Port";
344
345 $PORT_DATA{$id} = $self;
346 $PORT{$id} = sub {
347 local $SELF = $port;
348
349 eval {
350 for (@{ $self->{rc0}{$_[0]} }) {
351 $_ && &{$_->[0]}
352 && undef $_;
353 }
354
355 for (@{ $self->{rcv}{$_[0]} }) {
356 $_ && [@_[1 .. @{$_->[1]}]] ~~ $_->[1]
357 && &{$_->[0]}
358 && undef $_;
359 }
360
361 for (@{ $self->{any} }) {
362 $_ && [@_[0 .. $#{$_->[1]}]] ~~ $_->[1]
363 && &{$_->[0]}
364 && undef $_;
365 }
366 };
367 _self_die if $@;
368 };
369 }
370
371 $port
159} 372}
160 373
161=item $portid = create_miniport { } 374=item reg $portid, $name
162 375
163Creates a "mini port", that is, a port without much #TODO 376Registers the given port under the name C<$name>. If the name already
377exists it is replaced.
164 378
165=cut 379A port can only be registered under one well known name.
166 380
167sub create_miniport(&) { 381A port automatically becomes unregistered when it is killed.
168 my $cb = shift;
169 my $id = "$AnyEvent::MP::Base::UNIQ." . ++$AnyEvent::MP::Base::ID;
170 382
171 $AnyEvent::MP::Base::PORT{$id} = sub { 383=cut
172 $_[0] = "$NODE#$id";
173 &$cb
174 and delete $AnyEvent::MP::Base::PORT{$id};
175 };
176 384
177 "$NODE#$id" 385sub reg(@) {
386 my ($portid, $name) = @_;
387
388 $REG{$name} = $portid;
178} 389}
179 390
180package AnyEvent::MP::Port; 391=item rcv $portid, tagstring => $callback->(@msg), ...
181 392
182=back 393=item rcv $portid, $smartmatch => $callback->(@msg), ...
183 394
184=head1 METHODS FOR PORT OBJECTS
185
186=over 4
187
188=item "$port"
189
190A port object stringifies to its port ID, so can be used directly for
191C<snd> operations.
192
193=cut
194
195use overload
196 '""' => sub { $_[0]{id} },
197 fallback => 1;
198
199=item $port->rcv (type => $callback->($port, @msg))
200
201=item $port->rcv ($smartmatch => $callback->($port, @msg))
202
203=item $port->rcv ([$smartmatch...] => $callback->($port, @msg)) 395=item rcv $portid, [$smartmatch...] => $callback->(@msg), ...
204 396
205Register a callback on the given port. 397Register callbacks to be called on matching messages on the given port.
206 398
207The callback has to return a true value when its work is done, after 399The callback has to return a true value when its work is done, after
208which is will be removed, or a false value in which case it will stay 400which is will be removed, or a false value in which case it will stay
209registered. 401registered.
210 402
403The global C<$SELF> (exported by this module) contains C<$portid> while
404executing the callback.
405
406Runtime errors wdurign callback execution will result in the port being
407C<kil>ed.
408
211If the match is an array reference, then it will be matched against the 409If the match is an array reference, then it will be matched against the
212first elements of the message, otherwise only the first element is being 410first elements of the message, otherwise only the first element is being
213matched. 411matched.
214 412
215Any element in the match that is specified as C<_any_> (a function 413Any element in the match that is specified as C<_any_> (a function
220also the most efficient match (by far). 418also the most efficient match (by far).
221 419
222=cut 420=cut
223 421
224sub rcv($@) { 422sub rcv($@) {
225 my ($self, $match, $cb) = @_; 423 my ($noderef, $port) = split /#/, shift, 2;
226 424
425 ($NODE{$noderef} || add_node $noderef) == $NODE{""}
426 or Carp::croak "$noderef#$port: rcv can only be called on local ports, caught";
427
428 my $self = $PORT_DATA{$port}
429 or Carp::croak "$noderef#$port: rcv can only be called on message matching ports, caught";
430
431 "AnyEvent::MP::Port" eq ref $self
432 or Carp::croak "$noderef#$port: rcv can only be called on message matching ports, caught";
433
434 while (@_) {
435 my ($match, $cb) = splice @_, 0, 2;
436
227 if (!ref $match) { 437 if (!ref $match) {
228 push @{ $self->{rc0}{$match} }, [$cb]; 438 push @{ $self->{rc0}{$match} }, [$cb];
229 } elsif (("ARRAY" eq ref $match && !ref $match->[0])) { 439 } elsif (("ARRAY" eq ref $match && !ref $match->[0])) {
230 my ($type, @match) = @$match; 440 my ($type, @match) = @$match;
231 @match 441 @match
232 ? push @{ $self->{rcv}{$match->[0]} }, [$cb, \@match] 442 ? push @{ $self->{rcv}{$match->[0]} }, [$cb, \@match]
233 : push @{ $self->{rc0}{$match->[0]} }, [$cb]; 443 : push @{ $self->{rc0}{$match->[0]} }, [$cb];
234 } else { 444 } else {
235 push @{ $self->{any} }, [$cb, $match]; 445 push @{ $self->{any} }, [$cb, $match];
446 }
236 } 447 }
237} 448}
238 449
239=item $port->register ($name) 450=item $closure = psub { BLOCK }
240 451
241Registers the given port under the well known name C<$name>. If the name 452Remembers C<$SELF> and creates a closure out of the BLOCK. When the
242already exists it is replaced. 453closure is executed, sets up the environment in the same way as in C<rcv>
454callbacks, i.e. runtime errors will cause the port to get C<kil>ed.
243 455
244A port can only be registered under one well known name. 456This is useful when you register callbacks from C<rcv> callbacks:
245 457
246=cut 458 rcv delayed_reply => sub {
459 my ($delay, @reply) = @_;
460 my $timer = AE::timer $delay, 0, psub {
461 snd @reply, $SELF;
462 };
463 };
247 464
248sub register { 465=cut
249 my ($self, $name) = @_;
250 466
251 $self->{wkname} = $name; 467sub psub(&) {
252 $AnyEvent::MP::Base::WKP{$name} = "$self"; 468 my $cb = shift;
469
470 my $port = $SELF
471 or Carp::croak "psub can only be called from within rcv or psub callbacks, not";
472
473 sub {
474 local $SELF = $port;
475
476 if (wantarray) {
477 my @res = eval { &$cb };
478 _self_die if $@;
479 @res
480 } else {
481 my $res = eval { &$cb };
482 _self_die if $@;
483 $res
484 }
485 }
253} 486}
254 487
255=item $port->destroy
256
257Explicitly destroy/remove/nuke/vaporise the port.
258
259Ports are normally kept alive by there mere existance alone, and need to
260be destroyed explicitly.
261
262=cut
263
264sub destroy {
265 my ($self) = @_;
266
267 delete $AnyEvent::MP::Base::WKP{ $self->{wkname} };
268
269 delete $AnyEvent::MP::Base::PORT{$_}
270 for @{ $self->{names} };
271}
272
273=back 488=back
274 489
275=head1 FUNCTIONS FOR NODES 490=head1 FUNCTIONS FOR NODES
276 491
277=over 4 492=over 4
278 493
279=item mon $noderef, $callback->($noderef, $status, $) 494=item become_public $noderef
280
281Monitors the given noderef.
282
283=item become_public endpoint...
284 495
285Tells the node to become a public node, i.e. reachable from other nodes. 496Tells the node to become a public node, i.e. reachable from other nodes.
286 497
287If no arguments are given, or the first argument is C<undef>, then 498The first argument is the (unresolved) node reference of the local node
288AnyEvent::MP tries to bind on port C<4040> on all IP addresses that the 499(if missing then the empty string is used).
289local nodename resolves to.
290 500
291Otherwise the first argument must be an array-reference with transport 501It is quite common to not specify anything, in which case the local node
292endpoints ("ip:port", "hostname:port") or port numbers (in which case the 502tries to listen on the default port, or to only specify a port number, in
293local nodename is used as hostname). The endpoints are all resolved and 503which case AnyEvent::MP tries to guess the local addresses.
294will become the node reference.
295 504
296=cut 505=cut
297 506
298=back 507=back
299 508
302Nodes understand the following messages sent to them. Many of them take 511Nodes understand the following messages sent to them. Many of them take
303arguments called C<@reply>, which will simply be used to compose a reply 512arguments called C<@reply>, which will simply be used to compose a reply
304message - C<$reply[0]> is the port to reply to, C<$reply[1]> the type and 513message - C<$reply[0]> is the port to reply to, C<$reply[1]> the type and
305the remaining arguments are simply the message data. 514the remaining arguments are simply the message data.
306 515
516While other messages exist, they are not public and subject to change.
517
307=over 4 518=over 4
308 519
309=cut 520=cut
310 521
311=item wkp => $name, @reply 522=item lookup => $name, @reply
312 523
313Replies with the port ID of the specified well-known port, or C<undef>. 524Replies with the port ID of the specified well-known port, or C<undef>.
314 525
315=item devnull => ... 526=item devnull => ...
316 527
339 snd $NODE, time => $myport, timereply => 1, 2; 550 snd $NODE, time => $myport, timereply => 1, 2;
340 # => snd $myport, timereply => 1, 2, <time> 551 # => snd $myport, timereply => 1, 2, <time>
341 552
342=back 553=back
343 554
555=head1 AnyEvent::MP vs. Distributed Erlang
556
557AnyEvent::MP got lots of its ideas from distributed erlang (erlang node
558== aemp node, erlang process == aemp port), so many of the documents and
559programming techniques employed by erlang apply to AnyEvent::MP. Here is a
560sample:
561
562 http://www.erlang.se/doc/programming_rules.shtml
563 http://erlang.org/doc/getting_started/part_frame.html # chapters 3 and 4
564 http://erlang.org/download/erlang-book-part1.pdf # chapters 5 and 6
565 http://erlang.org/download/armstrong_thesis_2003.pdf # chapters 4 and 5
566
567Despite the similarities, there are also some important differences:
568
569=over 4
570
571=item * Node references contain the recipe on how to contact them.
572
573Erlang relies on special naming and DNS to work everywhere in the
574same way. AEMP relies on each node knowing it's own address(es), with
575convenience functionality.
576
577This means that AEMP requires a less tightly controlled environment at the
578cost of longer node references and a slightly higher management overhead.
579
580=item * Erlang uses processes and a mailbox, AEMP does not queue.
581
582Erlang uses processes that selctively receive messages, and therefore
583needs a queue. AEMP is event based, queuing messages would serve no useful
584purpose.
585
586(But see L<Coro::MP> for a more erlang-like process model on top of AEMP).
587
588=item * Erlang sends are synchronous, AEMP sends are asynchronous.
589
590Sending messages in erlang is synchronous and blocks the process. AEMP
591sends are immediate, connection establishment is handled in the
592background.
593
594=item * Erlang can silently lose messages, AEMP cannot.
595
596Erlang makes few guarantees on messages delivery - messages can get lost
597without any of the processes realising it (i.e. you send messages a, b,
598and c, and the other side only receives messages a and c).
599
600AEMP guarantees correct ordering, and the guarantee that there are no
601holes in the message sequence.
602
603=item * In erlang, processes can be declared dead and later be found to be
604alive.
605
606In erlang it can happen that a monitored process is declared dead and
607linked processes get killed, but later it turns out that the process is
608still alive - and can receive messages.
609
610In AEMP, when port monitoring detects a port as dead, then that port will
611eventually be killed - it cannot happen that a node detects a port as dead
612and then later sends messages to it, finding it is still alive.
613
614=item * Erlang can send messages to the wrong port, AEMP does not.
615
616In erlang it is quite possible that a node that restarts reuses a process
617ID known to other nodes for a completely different process, causing
618messages destined for that process to end up in an unrelated process.
619
620AEMP never reuses port IDs, so old messages or old port IDs floating
621around in the network will not be sent to an unrelated port.
622
623=item * Erlang uses unprotected connections, AEMP uses secure
624authentication and can use TLS.
625
626AEMP can use a proven protocol - SSL/TLS - to protect connections and
627securely authenticate nodes.
628
629=item * The AEMP protocol is optimised for both text-based and binary
630communications.
631
632The AEMP protocol, unlike the erlang protocol, supports both
633language-independent text-only protocols (good for debugging) and binary,
634language-specific serialisers (e.g. Storable).
635
636It has also been carefully designed to be implementable in other languages
637with a minimum of work while gracefully degrading fucntionality to make the
638protocol simple.
639
640=back
641
344=head1 SEE ALSO 642=head1 SEE ALSO
345 643
346L<AnyEvent>. 644L<AnyEvent>.
347 645
348=head1 AUTHOR 646=head1 AUTHOR

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines