ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP.pm
(Generate patch)

Comparing AnyEvent-MP/MP.pm (file contents):
Revision 1.10 by root, Sun Aug 2 18:05:43 2009 UTC vs.
Revision 1.29 by root, Tue Aug 4 23:16:57 2009 UTC

4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use AnyEvent::MP; 7 use AnyEvent::MP;
8 8
9 NODE # returns this node identifier
10 $NODE # contains this node identifier 9 $NODE # contains this node's noderef
10 NODE # returns this node's noderef
11 NODE $port # returns the noderef of the port
11 12
12 snd $port, type => data...; 13 snd $port, type => data...;
14
15 $SELF # receiving/own port id in rcv callbacks
13 16
14 rcv $port, smartmatch => $cb->($port, @msg); 17 rcv $port, smartmatch => $cb->($port, @msg);
15 18
16 # examples: 19 # examples:
17 rcv $port2, ping => sub { snd $_[0], "pong"; 0 }; 20 rcv $port2, ping => sub { snd $_[0], "pong"; 0 };
27This module (-family) implements a simple message passing framework. 30This module (-family) implements a simple message passing framework.
28 31
29Despite its simplicity, you can securely message other processes running 32Despite its simplicity, you can securely message other processes running
30on the same or other hosts. 33on the same or other hosts.
31 34
35For an introduction to this module family, see the L<AnyEvent::MP::Intro>
36manual page.
37
32At the moment, this module family is severly brokena nd underdocumented, 38At the moment, this module family is severly broken and underdocumented,
33so do not use. This was uploaded mainly to resreve the CPAN namespace - 39so do not use. This was uploaded mainly to reserve the CPAN namespace -
34stay tuned! 40stay tuned! The basic API should be finished, however.
35 41
36=head1 CONCEPTS 42=head1 CONCEPTS
37 43
38=over 4 44=over 4
39 45
40=item port 46=item port
41 47
42A port is something you can send messages to with the C<snd> function, and 48A port is something you can send messages to (with the C<snd> function).
43you can register C<rcv> handlers with. All C<rcv> handlers will receive 49
44messages they match, messages will not be queued. 50Some ports allow you to register C<rcv> handlers that can match specific
51messages. All C<rcv> handlers will receive messages they match, messages
52will not be queued.
45 53
46=item port id - C<noderef#portname> 54=item port id - C<noderef#portname>
47 55
48A port id is always the noderef, a hash-mark (C<#>) as separator, followed 56A port id is normaly the concatenation of a noderef, a hash-mark (C<#>) as
49by a port name (a printable string of unspecified format). 57separator, and a port name (a printable string of unspecified format). An
58exception is the the node port, whose ID is identical to it's node
59reference.
50 60
51=item node 61=item node
52 62
53A node is a single process containing at least one port - the node 63A node is a single process containing at least one port - the node
54port. You can send messages to node ports to let them create new ports, 64port. You can send messages to node ports to find existing ports or to
55among other things. 65create new ports, among other things.
56 66
57Initially, nodes are either private (single-process only) or hidden 67Nodes are either private (single-process only), slaves (connected to a
58(connected to a master node only). Only when they epxlicitly "become 68master node only) or public nodes (connectable from unrelated nodes).
59public" can you send them messages from unrelated other nodes.
60 69
61=item noderef - C<host:port,host:port...>, C<id@noderef>, C<id> 70=item noderef - C<host:port,host:port...>, C<id@noderef>, C<id>
62 71
63A noderef is a string that either uniquely identifies a given node (for 72A node reference is a string that either simply identifies the node (for
64private and hidden nodes), or contains a recipe on how to reach a given 73private and slave nodes), or contains a recipe on how to reach a given
65node (for public nodes). 74node (for public nodes).
66 75
76This recipe is simply a comma-separated list of C<address:port> pairs (for
77TCP/IP, other protocols might look different).
78
79Node references come in two flavours: resolved (containing only numerical
80addresses) or unresolved (where hostnames are used instead of addresses).
81
82Before using an unresolved node reference in a message you first have to
83resolve it.
84
67=back 85=back
68 86
69=head1 VARIABLES/FUNCTIONS 87=head1 VARIABLES/FUNCTIONS
70 88
71=over 4 89=over 4
82 100
83use AE (); 101use AE ();
84 102
85use base "Exporter"; 103use base "Exporter";
86 104
87our $VERSION = '0.02'; 105our $VERSION = '0.1';
88our @EXPORT = qw( 106our @EXPORT = qw(
89 NODE $NODE $PORT snd rcv _any_ 107 NODE $NODE *SELF node_of _any_
90 create_port create_port_on 108 resolve_node
91 become_slave become_public 109 become_slave become_public
110 snd rcv mon kil reg psub
111 port
92); 112);
93 113
114our $SELF;
115
116sub _self_die() {
117 my $msg = $@;
118 $msg =~ s/\n+$// unless ref $msg;
119 kil $SELF, die => $msg;
120}
121
94=item NODE / $NODE 122=item $thisnode = NODE / $NODE
95 123
96The C<NODE ()> function and the C<$NODE> variable contain the noderef of 124The C<NODE> function returns, and the C<$NODE> variable contains
97the local node. The value is initialised by a call to C<become_public> or 125the noderef of the local node. The value is initialised by a call
98C<become_slave>, after which all local port identifiers become invalid. 126to C<become_public> or C<become_slave>, after which all local port
127identifiers become invalid.
128
129=item $noderef = node_of $portid
130
131Extracts and returns the noderef from a portid or a noderef.
132
133=item $cv = resolve_node $noderef
134
135Takes an unresolved node reference that may contain hostnames and
136abbreviated IDs, resolves all of them and returns a resolved node
137reference.
138
139In addition to C<address:port> pairs allowed in resolved noderefs, the
140following forms are supported:
141
142=over 4
143
144=item the empty string
145
146An empty-string component gets resolved as if the default port (4040) was
147specified.
148
149=item naked port numbers (e.g. C<1234>)
150
151These are resolved by prepending the local nodename and a colon, to be
152further resolved.
153
154=item hostnames (e.g. C<localhost:1234>, C<localhost>)
155
156These are resolved by using AnyEvent::DNS to resolve them, optionally
157looking up SRV records for the C<aemp=4040> port, if no port was
158specified.
159
160=back
161
162=item $SELF
163
164Contains the current port id while executing C<rcv> callbacks or C<psub>
165blocks.
166
167=item SELF, %SELF, @SELF...
168
169Due to some quirks in how perl exports variables, it is impossible to
170just export C<$SELF>, all the symbols called C<SELF> are exported by this
171module, but only C<$SELF> is currently used.
99 172
100=item snd $portid, type => @data 173=item snd $portid, type => @data
101 174
102=item snd $portid, @msg 175=item snd $portid, @msg
103 176
117JSON is used, then only strings, numbers and arrays and hashes consisting 190JSON is used, then only strings, numbers and arrays and hashes consisting
118of those are allowed (no objects). When Storable is used, then anything 191of those are allowed (no objects). When Storable is used, then anything
119that Storable can serialise and deserialise is allowed, and for the local 192that Storable can serialise and deserialise is allowed, and for the local
120node, anything can be passed. 193node, anything can be passed.
121 194
122=item $local_port = create_port 195=item kil $portid[, @reason]
123 196
124Create a new local port object. See the next section for allowed methods. 197Kill the specified port with the given C<@reason>.
125 198
126=cut 199If no C<@reason> is specified, then the port is killed "normally" (linked
200ports will not be kileld, or even notified).
127 201
128sub create_port { 202Otherwise, linked ports get killed with the same reason (second form of
129 my $id = "$AnyEvent::MP::Base::UNIQ." . ++$AnyEvent::MP::Base::ID; 203C<mon>, see below).
130 204
131 my $self = bless { 205Runtime errors while evaluating C<rcv> callbacks or inside C<psub> blocks
132 id => "$NODE#$id", 206will be reported as reason C<< die => $@ >>.
133 names => [$id],
134 }, "AnyEvent::MP::Port";
135 207
136 $AnyEvent::MP::Base::PORT{$id} = sub { 208Transport/communication errors are reported as C<< transport_error =>
137 unshift @_, $self; 209$message >>.
138 210
139 for (@{ $self->{rc0}{$_[1]} }) { 211=item $guard = mon $portid, $cb->(@reason)
140 $_ && &{$_->[0]} 212
141 && undef $_; 213=item $guard = mon $portid, $otherport
214
215=item $guard = mon $portid, $otherport, @msg
216
217Monitor the given port and do something when the port is killed.
218
219In the first form, the callback is simply called with any number
220of C<@reason> elements (no @reason means that the port was deleted
221"normally"). Note also that I<< the callback B<must> never die >>, so use
222C<eval> if unsure.
223
224In the second form, the other port will be C<kil>'ed with C<@reason>, iff
225a @reason was specified, i.e. on "normal" kils nothing happens, while
226under all other conditions, the other port is killed with the same reason.
227
228In the last form, a message of the form C<@msg, @reason> will be C<snd>.
229
230Example: call a given callback when C<$port> is killed.
231
232 mon $port, sub { warn "port died because of <@_>\n" };
233
234Example: kill ourselves when C<$port> is killed abnormally.
235
236 mon $port, $self;
237
238Example: send us a restart message another C<$port> is killed.
239
240 mon $port, $self => "restart";
241
242=cut
243
244sub mon {
245 my ($noderef, $port, $cb) = ((split /#/, shift, 2), shift);
246
247 my $node = $NODE{$noderef} || add_node $noderef;
248
249 #TODO: ports must not be references
250 if (!ref $cb or "AnyEvent::MP::Port" eq ref $cb) {
251 if (@_) {
252 # send a kill info message
253 my (@msg) = ($cb, @_);
254 $cb = sub { snd @msg, @_ };
255 } else {
256 # simply kill other port
257 my $port = $cb;
258 $cb = sub { kil $port, @_ if @_ };
142 } 259 }
260 }
143 261
144 for (@{ $self->{rcv}{$_[1]} }) { 262 $node->monitor ($port, $cb);
145 $_ && [@_[1 .. @{$_->[1]}]] ~~ $_->[1] 263
146 && &{$_->[0]} 264 defined wantarray
147 && undef $_; 265 and AnyEvent::Util::guard { $node->unmonitor ($port, $cb) }
266}
267
268=item $guard = mon_guard $port, $ref, $ref...
269
270Monitors the given C<$port> and keeps the passed references. When the port
271is killed, the references will be freed.
272
273Optionally returns a guard that will stop the monitoring.
274
275This function is useful when you create e.g. timers or other watchers and
276want to free them when the port gets killed:
277
278 $port->rcv (start => sub {
279 my $timer; $timer = mon_guard $port, AE::timer 1, 1, sub {
280 undef $timer if 0.9 < rand;
148 } 281 });
282 });
149 283
150 for (@{ $self->{any} }) { 284=cut
151 $_ && [@_[0 .. $#{$_->[1]}]] ~~ $_->[1] 285
152 && &{$_->[0]} 286sub mon_guard {
153 && undef $_; 287 my ($port, @refs) = @_;
154 } 288
289 mon $port, sub { 0 && @refs }
290}
291
292=item lnk $port1, $port2
293
294Link two ports. This is simply a shorthand for:
295
296 mon $port1, $port2;
297 mon $port2, $port1;
298
299It means that if either one is killed abnormally, the other one gets
300killed as well.
301
302=item $local_port = port
303
304Create a new local port object that supports message matching.
305
306=item $portid = port { my @msg = @_; $finished }
307
308Creates a "mini port", that is, a very lightweight port without any
309pattern matching behind it, and returns its ID.
310
311The block will be called for every message received on the port. When the
312callback returns a true value its job is considered "done" and the port
313will be destroyed. Otherwise it will stay alive.
314
315The message will be passed as-is, no extra argument (i.e. no port id) will
316be passed to the callback.
317
318If you need the local port id in the callback, this works nicely:
319
320 my $port; $port = miniport {
321 snd $otherport, reply => $port;
155 }; 322 };
156 323
157 $self 324=cut
325
326sub port(;&) {
327 my $id = "$UNIQ." . $ID++;
328 my $port = "$NODE#$id";
329
330 if (@_) {
331 my $cb = shift;
332 $PORT{$id} = sub {
333 local $SELF = $port;
334 eval {
335 &$cb
336 and kil $id;
337 };
338 _self_die if $@;
339 };
340 } else {
341 my $self = bless {
342 id => "$NODE#$id",
343 }, "AnyEvent::MP::Port";
344
345 $PORT_DATA{$id} = $self;
346 $PORT{$id} = sub {
347 local $SELF = $port;
348
349 eval {
350 for (@{ $self->{rc0}{$_[0]} }) {
351 $_ && &{$_->[0]}
352 && undef $_;
353 }
354
355 for (@{ $self->{rcv}{$_[0]} }) {
356 $_ && [@_[1 .. @{$_->[1]}]] ~~ $_->[1]
357 && &{$_->[0]}
358 && undef $_;
359 }
360
361 for (@{ $self->{any} }) {
362 $_ && [@_[0 .. $#{$_->[1]}]] ~~ $_->[1]
363 && &{$_->[0]}
364 && undef $_;
365 }
366 };
367 _self_die if $@;
368 };
369 }
370
371 $port
158} 372}
159 373
160=item $portid = create_miniport { } 374=item reg $portid, $name
161 375
162Creates a "mini port", that is, a port without much #TODO 376Registers the given port under the name C<$name>. If the name already
377exists it is replaced.
163 378
164=cut 379A port can only be registered under one well known name.
165 380
166sub create_miniport(&) { 381A port automatically becomes unregistered when it is killed.
167 my $cb = shift;
168 my $id = "$AnyEvent::MP::Base::UNIQ." . ++$AnyEvent::MP::Base::ID;
169 382
170 $AnyEvent::MP::Base::PORT{$id} = sub { 383=cut
171 &$cb
172 and delete $AnyEvent::MP::Base::PORT{$id};
173 };
174 384
175 "$NODE#$id" 385sub reg(@) {
386 my ($portid, $name) = @_;
387
388 $REG{$name} = $portid;
176} 389}
177 390
178package AnyEvent::MP::Port; 391=item rcv $portid, tagstring => $callback->(@msg), ...
179 392
180=back 393=item rcv $portid, $smartmatch => $callback->(@msg), ...
181 394
182=head1 METHODS FOR PORT OBJECTS
183
184=over 4
185
186=item "$port"
187
188A port object stringifies to its port ID, so can be used directly for
189C<snd> operations.
190
191=cut
192
193use overload
194 '""' => sub { $_[0]{id} },
195 fallback => 1;
196
197=item $port->rcv (type => $callback->($port, @msg))
198
199=item $port->rcv ($smartmatch => $callback->($port, @msg))
200
201=item $port->rcv ([$smartmatch...] => $callback->($port, @msg)) 395=item rcv $portid, [$smartmatch...] => $callback->(@msg), ...
202 396
203Register a callback on the given port. 397Register callbacks to be called on matching messages on the given port.
204 398
205The callback has to return a true value when its work is done, after 399The callback has to return a true value when its work is done, after
206which is will be removed, or a false value in which case it will stay 400which is will be removed, or a false value in which case it will stay
207registered. 401registered.
208 402
403The global C<$SELF> (exported by this module) contains C<$portid> while
404executing the callback.
405
406Runtime errors wdurign callback execution will result in the port being
407C<kil>ed.
408
209If the match is an array reference, then it will be matched against the 409If the match is an array reference, then it will be matched against the
210first elements of the message, otherwise only the first element is being 410first elements of the message, otherwise only the first element is being
211matched. 411matched.
212 412
213Any element in the match that is specified as C<_any_> (a function 413Any element in the match that is specified as C<_any_> (a function
218also the most efficient match (by far). 418also the most efficient match (by far).
219 419
220=cut 420=cut
221 421
222sub rcv($@) { 422sub rcv($@) {
223 my ($self, $match, $cb) = @_; 423 my ($noderef, $port) = split /#/, shift, 2;
224 424
425 ($NODE{$noderef} || add_node $noderef) == $NODE{""}
426 or Carp::croak "$noderef#$port: rcv can only be called on local ports, caught";
427
428 my $self = $PORT_DATA{$port}
429 or Carp::croak "$noderef#$port: rcv can only be called on message matching ports, caught";
430
431 "AnyEvent::MP::Port" eq ref $self
432 or Carp::croak "$noderef#$port: rcv can only be called on message matching ports, caught";
433
434 while (@_) {
435 my ($match, $cb) = splice @_, 0, 2;
436
225 if (!ref $match) { 437 if (!ref $match) {
226 push @{ $self->{rc0}{$match} }, [$cb]; 438 push @{ $self->{rc0}{$match} }, [$cb];
227 } elsif (("ARRAY" eq ref $match && !ref $match->[0])) { 439 } elsif (("ARRAY" eq ref $match && !ref $match->[0])) {
228 my ($type, @match) = @$match; 440 my ($type, @match) = @$match;
229 @match 441 @match
230 ? push @{ $self->{rcv}{$match->[0]} }, [$cb, \@match] 442 ? push @{ $self->{rcv}{$match->[0]} }, [$cb, \@match]
231 : push @{ $self->{rc0}{$match->[0]} }, [$cb]; 443 : push @{ $self->{rc0}{$match->[0]} }, [$cb];
232 } else { 444 } else {
233 push @{ $self->{any} }, [$cb, $match]; 445 push @{ $self->{any} }, [$cb, $match];
446 }
234 } 447 }
235} 448}
236 449
237=item $port->register ($name) 450=item $closure = psub { BLOCK }
238 451
239Registers the given port under the well known name C<$name>. If the name 452Remembers C<$SELF> and creates a closure out of the BLOCK. When the
240already exists it is replaced. 453closure is executed, sets up the environment in the same way as in C<rcv>
454callbacks, i.e. runtime errors will cause the port to get C<kil>ed.
241 455
242A port can only be registered under one well known name. 456This is useful when you register callbacks from C<rcv> callbacks:
243 457
244=cut 458 rcv delayed_reply => sub {
459 my ($delay, @reply) = @_;
460 my $timer = AE::timer $delay, 0, psub {
461 snd @reply, $SELF;
462 };
463 };
245 464
246sub register { 465=cut
247 my ($self, $name) = @_;
248 466
249 $self->{wkname} = $name; 467sub psub(&) {
250 $AnyEvent::MP::Base::WKP{$name} = "$self"; 468 my $cb = shift;
469
470 my $port = $SELF
471 or Carp::croak "psub can only be called from within rcv or psub callbacks, not";
472
473 sub {
474 local $SELF = $port;
475
476 if (wantarray) {
477 my @res = eval { &$cb };
478 _self_die if $@;
479 @res
480 } else {
481 my $res = eval { &$cb };
482 _self_die if $@;
483 $res
484 }
485 }
251} 486}
252 487
253=item $port->destroy
254
255Explicitly destroy/remove/nuke/vaporise the port.
256
257Ports are normally kept alive by there mere existance alone, and need to
258be destroyed explicitly.
259
260=cut
261
262sub destroy {
263 my ($self) = @_;
264
265 delete $AnyEvent::MP::Base::WKP{ $self->{wkname} };
266
267 delete $AnyEvent::MP::Base::PORT{$_}
268 for @{ $self->{names} };
269}
270
271=back 488=back
272 489
273=head1 FUNCTIONS FOR NODES 490=head1 FUNCTIONS FOR NODES
274 491
275=over 4 492=over 4
276 493
277=item mon $noderef, $callback->($noderef, $status, $) 494=item become_public $noderef
278
279Monitors the given noderef.
280
281=item become_public endpoint...
282 495
283Tells the node to become a public node, i.e. reachable from other nodes. 496Tells the node to become a public node, i.e. reachable from other nodes.
284 497
285If no arguments are given, or the first argument is C<undef>, then 498The first argument is the (unresolved) node reference of the local node
286AnyEvent::MP tries to bind on port C<4040> on all IP addresses that the 499(if missing then the empty string is used).
287local nodename resolves to.
288 500
289Otherwise the first argument must be an array-reference with transport 501It is quite common to not specify anything, in which case the local node
290endpoints ("ip:port", "hostname:port") or port numbers (in which case the 502tries to listen on the default port, or to only specify a port number, in
291local nodename is used as hostname). The endpoints are all resolved and 503which case AnyEvent::MP tries to guess the local addresses.
292will become the node reference.
293 504
294=cut 505=cut
295 506
296=back 507=back
297 508
300Nodes understand the following messages sent to them. Many of them take 511Nodes understand the following messages sent to them. Many of them take
301arguments called C<@reply>, which will simply be used to compose a reply 512arguments called C<@reply>, which will simply be used to compose a reply
302message - C<$reply[0]> is the port to reply to, C<$reply[1]> the type and 513message - C<$reply[0]> is the port to reply to, C<$reply[1]> the type and
303the remaining arguments are simply the message data. 514the remaining arguments are simply the message data.
304 515
516While other messages exist, they are not public and subject to change.
517
305=over 4 518=over 4
306 519
307=cut 520=cut
308 521
309=item wkp => $name, @reply 522=item lookup => $name, @reply
310 523
311Replies with the port ID of the specified well-known port, or C<undef>. 524Replies with the port ID of the specified well-known port, or C<undef>.
312 525
313=item devnull => ... 526=item devnull => ...
314 527
337 snd $NODE, time => $myport, timereply => 1, 2; 550 snd $NODE, time => $myport, timereply => 1, 2;
338 # => snd $myport, timereply => 1, 2, <time> 551 # => snd $myport, timereply => 1, 2, <time>
339 552
340=back 553=back
341 554
555=head1 AnyEvent::MP vs. Distributed Erlang
556
557AnyEvent::MP got lots of its ideas from distributed erlang (erlang node
558== aemp node, erlang process == aemp port), so many of the documents and
559programming techniques employed by erlang apply to AnyEvent::MP. Here is a
560sample:
561
562 http://www.erlang.se/doc/programming_rules.shtml
563 http://erlang.org/doc/getting_started/part_frame.html # chapters 3 and 4
564 http://erlang.org/download/erlang-book-part1.pdf # chapters 5 and 6
565 http://erlang.org/download/armstrong_thesis_2003.pdf # chapters 4 and 5
566
567Despite the similarities, there are also some important differences:
568
569=over 4
570
571=item * Node references contain the recipe on how to contact them.
572
573Erlang relies on special naming and DNS to work everywhere in the
574same way. AEMP relies on each node knowing it's own address(es), with
575convenience functionality.
576
577This means that AEMP requires a less tightly controlled environment at the
578cost of longer node references and a slightly higher management overhead.
579
580=item * Erlang uses processes and a mailbox, AEMP does not queue.
581
582Erlang uses processes that selctively receive messages, and therefore
583needs a queue. AEMP is event based, queuing messages would serve no useful
584purpose.
585
586(But see L<Coro::MP> for a more erlang-like process model on top of AEMP).
587
588=item * Erlang sends are synchronous, AEMP sends are asynchronous.
589
590Sending messages in erlang is synchronous and blocks the process. AEMP
591sends are immediate, connection establishment is handled in the
592background.
593
594=item * Erlang can silently lose messages, AEMP cannot.
595
596Erlang makes few guarantees on messages delivery - messages can get lost
597without any of the processes realising it (i.e. you send messages a, b,
598and c, and the other side only receives messages a and c).
599
600AEMP guarantees correct ordering, and the guarantee that there are no
601holes in the message sequence.
602
603=item * In erlang, processes can be declared dead and later be found to be
604alive.
605
606In erlang it can happen that a monitored process is declared dead and
607linked processes get killed, but later it turns out that the process is
608still alive - and can receive messages.
609
610In AEMP, when port monitoring detects a port as dead, then that port will
611eventually be killed - it cannot happen that a node detects a port as dead
612and then later sends messages to it, finding it is still alive.
613
614=item * Erlang can send messages to the wrong port, AEMP does not.
615
616In erlang it is quite possible that a node that restarts reuses a process
617ID known to other nodes for a completely different process, causing
618messages destined for that process to end up in an unrelated process.
619
620AEMP never reuses port IDs, so old messages or old port IDs floating
621around in the network will not be sent to an unrelated port.
622
623=item * Erlang uses unprotected connections, AEMP uses secure
624authentication and can use TLS.
625
626AEMP can use a proven protocol - SSL/TLS - to protect connections and
627securely authenticate nodes.
628
629=item * The AEMP protocol is optimised for both text-based and binary
630communications.
631
632The AEMP protocol, unlike the erlang protocol, supports both
633language-independent text-only protocols (good for debugging) and binary,
634language-specific serialisers (e.g. Storable).
635
636It has also been carefully designed to be implementable in other languages
637with a minimum of work while gracefully degrading fucntionality to make the
638protocol simple.
639
640=back
641
342=head1 SEE ALSO 642=head1 SEE ALSO
343 643
344L<AnyEvent>. 644L<AnyEvent>.
345 645
346=head1 AUTHOR 646=head1 AUTHOR

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines