ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP.pm
(Generate patch)

Comparing AnyEvent-MP/MP.pm (file contents):
Revision 1.30 by root, Tue Aug 4 23:35:51 2009 UTC vs.
Revision 1.35 by root, Thu Aug 6 10:21:48 2009 UTC

23 23
24 # more, smarter, matches (_any_ is exported by this module) 24 # more, smarter, matches (_any_ is exported by this module)
25 rcv $port, [child_died => $pid] => sub { ... 25 rcv $port, [child_died => $pid] => sub { ...
26 rcv $port, [_any_, _any_, 3] => sub { .. $_[2] is 3 26 rcv $port, [_any_, _any_, 3] => sub { .. $_[2] is 3
27 27
28 # linking two ports, so they both crash together
29 lnk $port1, $port2;
30
31 # monitoring
32 mon $port, $cb->(@msg) # callback is invoked on death
33 mon $port, $otherport # kill otherport on abnormal death
34 mon $port, $otherport, @msg # send message on death
35
28=head1 DESCRIPTION 36=head1 DESCRIPTION
29 37
30This module (-family) implements a simple message passing framework. 38This module (-family) implements a simple message passing framework.
31 39
32Despite its simplicity, you can securely message other processes running 40Despite its simplicity, you can securely message other processes running
103use base "Exporter"; 111use base "Exporter";
104 112
105our $VERSION = '0.1'; 113our $VERSION = '0.1';
106our @EXPORT = qw( 114our @EXPORT = qw(
107 NODE $NODE *SELF node_of _any_ 115 NODE $NODE *SELF node_of _any_
108 resolve_node 116 resolve_node initialise_node
109 become_slave become_public
110 snd rcv mon kil reg psub 117 snd rcv mon kil reg psub
111 port 118 port
112); 119);
113 120
114our $SELF; 121our $SELF;
124The C<NODE> function returns, and the C<$NODE> variable contains 131The C<NODE> function returns, and the C<$NODE> variable contains
125the noderef of the local node. The value is initialised by a call 132the noderef of the local node. The value is initialised by a call
126to C<become_public> or C<become_slave>, after which all local port 133to C<become_public> or C<become_slave>, after which all local port
127identifiers become invalid. 134identifiers become invalid.
128 135
129=item $noderef = node_of $portid 136=item $noderef = node_of $port
130 137
131Extracts and returns the noderef from a portid or a noderef. 138Extracts and returns the noderef from a portid or a noderef.
139
140=item initialise_node $noderef, $seednode, $seednode...
141
142=item initialise_node "slave/", $master, $master...
143
144Before a node can talk to other nodes on the network it has to initialise
145itself - the minimum a node needs to know is it's own name, and optionally
146it should know the noderefs of some other nodes in the network.
147
148This function initialises a node - it must be called exactly once (or
149never) before calling other AnyEvent::MP functions.
150
151All arguments are noderefs, which can be either resolved or unresolved.
152
153There are two types of networked nodes, public nodes and slave nodes:
154
155=over 4
156
157=item public nodes
158
159For public nodes, C<$noderef> must either be a (possibly unresolved)
160noderef, in which case it will be resolved, or C<undef> (or missing), in
161which case the noderef will be guessed.
162
163Afterwards, the node will bind itself on all endpoints and try to connect
164to all additional C<$seednodes> that are specified. Seednodes are optional
165and can be used to quickly bootstrap the node into an existing network.
166
167=item slave nodes
168
169When the C<$noderef> is the special string C<slave/>, then the node will
170become a slave node. Slave nodes cannot be contacted from outside and will
171route most of their traffic to the master node that they attach to.
172
173At least one additional noderef is required: The node will try to connect
174to all of them and will become a slave attached to the first node it can
175successfully connect to.
176
177=back
178
179This function will block until all nodes have been resolved and, for slave
180nodes, until it has successfully established a connection to a master
181server.
182
183Example: become a public node listening on the default node.
184
185 initialise_node;
186
187Example: become a public node, and try to contact some well-known master
188servers to become part of the network.
189
190 initialise_node undef, "master1", "master2";
191
192Example: become a public node listening on port C<4041>.
193
194 initialise_node 4041;
195
196Example: become a public node, only visible on localhost port 4044.
197
198 initialise_node "locahost:4044";
199
200Example: become a slave node to any of the specified master servers.
201
202 initialise_node "slave/", "master1", "192.168.13.17", "mp.example.net";
132 203
133=item $cv = resolve_node $noderef 204=item $cv = resolve_node $noderef
134 205
135Takes an unresolved node reference that may contain hostnames and 206Takes an unresolved node reference that may contain hostnames and
136abbreviated IDs, resolves all of them and returns a resolved node 207abbreviated IDs, resolves all of them and returns a resolved node
168 239
169Due to some quirks in how perl exports variables, it is impossible to 240Due to some quirks in how perl exports variables, it is impossible to
170just export C<$SELF>, all the symbols called C<SELF> are exported by this 241just export C<$SELF>, all the symbols called C<SELF> are exported by this
171module, but only C<$SELF> is currently used. 242module, but only C<$SELF> is currently used.
172 243
173=item snd $portid, type => @data 244=item snd $port, type => @data
174 245
175=item snd $portid, @msg 246=item snd $port, @msg
176 247
177Send the given message to the given port ID, which can identify either 248Send the given message to the given port ID, which can identify either
178a local or a remote port, and can be either a string or soemthignt hat 249a local or a remote port, and can be either a string or soemthignt hat
179stringifies a sa port ID (such as a port object :). 250stringifies a sa port ID (such as a port object :).
180 251
190JSON is used, then only strings, numbers and arrays and hashes consisting 261JSON is used, then only strings, numbers and arrays and hashes consisting
191of those are allowed (no objects). When Storable is used, then anything 262of those are allowed (no objects). When Storable is used, then anything
192that Storable can serialise and deserialise is allowed, and for the local 263that Storable can serialise and deserialise is allowed, and for the local
193node, anything can be passed. 264node, anything can be passed.
194 265
195=item kil $portid[, @reason] 266=item $local_port = port
196 267
197Kill the specified port with the given C<@reason>. 268Create a new local port object that can be used either as a pattern
269matching port ("full port") or a single-callback port ("miniport"),
270depending on how C<rcv> callbacks are bound to the object.
198 271
199If no C<@reason> is specified, then the port is killed "normally" (linked 272=item $port = port { my @msg = @_; $finished }
200ports will not be kileld, or even notified).
201 273
202Otherwise, linked ports get killed with the same reason (second form of 274Creates a "miniport", that is, a very lightweight port without any pattern
203C<mon>, see below). 275matching behind it, and returns its ID. Semantically the same as creating
276a port and calling C<rcv $port, $callback> on it.
204 277
205Runtime errors while evaluating C<rcv> callbacks or inside C<psub> blocks 278The block will be called for every message received on the port. When the
206will be reported as reason C<< die => $@ >>. 279callback returns a true value its job is considered "done" and the port
280will be destroyed. Otherwise it will stay alive.
207 281
208Transport/communication errors are reported as C<< transport_error => 282The message will be passed as-is, no extra argument (i.e. no port id) will
209$message >>. 283be passed to the callback.
210 284
285If you need the local port id in the callback, this works nicely:
286
287 my $port; $port = port {
288 snd $otherport, reply => $port;
289 };
290
291=cut
292
293sub rcv($@);
294
295sub port(;&) {
296 my $id = "$UNIQ." . $ID++;
297 my $port = "$NODE#$id";
298
299 if (@_) {
300 rcv $port, shift;
301 } else {
302 $PORT{$id} = sub { }; # nop
303 }
304
305 $port
306}
307
308=item reg $port, $name
309
310Registers the given port under the name C<$name>. If the name already
311exists it is replaced.
312
313A port can only be registered under one well known name.
314
315A port automatically becomes unregistered when it is killed.
316
317=cut
318
319sub reg(@) {
320 my ($port, $name) = @_;
321
322 $REG{$name} = $port;
323}
324
325=item rcv $port, $callback->(@msg)
326
327Replaces the callback on the specified miniport (after converting it to
328one if required).
329
330=item rcv $port, tagstring => $callback->(@msg), ...
331
332=item rcv $port, $smartmatch => $callback->(@msg), ...
333
334=item rcv $port, [$smartmatch...] => $callback->(@msg), ...
335
336Register callbacks to be called on matching messages on the given full
337port (after converting it to one if required).
338
339The callback has to return a true value when its work is done, after
340which is will be removed, or a false value in which case it will stay
341registered.
342
343The global C<$SELF> (exported by this module) contains C<$port> while
344executing the callback.
345
346Runtime errors wdurign callback execution will result in the port being
347C<kil>ed.
348
349If the match is an array reference, then it will be matched against the
350first elements of the message, otherwise only the first element is being
351matched.
352
353Any element in the match that is specified as C<_any_> (a function
354exported by this module) matches any single element of the message.
355
356While not required, it is highly recommended that the first matching
357element is a string identifying the message. The one-string-only match is
358also the most efficient match (by far).
359
360=cut
361
362sub rcv($@) {
363 my $port = shift;
364 my ($noderef, $portid) = split /#/, $port, 2;
365
366 ($NODE{$noderef} || add_node $noderef) == $NODE{""}
367 or Carp::croak "$port: rcv can only be called on local ports, caught";
368
369 if (@_ == 1) {
370 my $cb = shift;
371 delete $PORT_DATA{$portid};
372 $PORT{$portid} = sub {
373 local $SELF = $port;
374 eval {
375 &$cb
376 and kil $port;
377 };
378 _self_die if $@;
379 };
380 } else {
381 my $self = $PORT_DATA{$portid} ||= do {
382 my $self = bless {
383 id => $port,
384 }, "AnyEvent::MP::Port";
385
386 $PORT{$portid} = sub {
387 local $SELF = $port;
388
389 eval {
390 for (@{ $self->{rc0}{$_[0]} }) {
391 $_ && &{$_->[0]}
392 && undef $_;
393 }
394
395 for (@{ $self->{rcv}{$_[0]} }) {
396 $_ && [@_[1 .. @{$_->[1]}]] ~~ $_->[1]
397 && &{$_->[0]}
398 && undef $_;
399 }
400
401 for (@{ $self->{any} }) {
402 $_ && [@_[0 .. $#{$_->[1]}]] ~~ $_->[1]
403 && &{$_->[0]}
404 && undef $_;
405 }
406 };
407 _self_die if $@;
408 };
409
410 $self
411 };
412
413 "AnyEvent::MP::Port" eq ref $self
414 or Carp::croak "$port: rcv can only be called on message matching ports, caught";
415
416 while (@_) {
417 my ($match, $cb) = splice @_, 0, 2;
418
419 if (!ref $match) {
420 push @{ $self->{rc0}{$match} }, [$cb];
421 } elsif (("ARRAY" eq ref $match && !ref $match->[0])) {
422 my ($type, @match) = @$match;
423 @match
424 ? push @{ $self->{rcv}{$match->[0]} }, [$cb, \@match]
425 : push @{ $self->{rc0}{$match->[0]} }, [$cb];
426 } else {
427 push @{ $self->{any} }, [$cb, $match];
428 }
429 }
430 }
431
432 $port
433}
434
435=item $closure = psub { BLOCK }
436
437Remembers C<$SELF> and creates a closure out of the BLOCK. When the
438closure is executed, sets up the environment in the same way as in C<rcv>
439callbacks, i.e. runtime errors will cause the port to get C<kil>ed.
440
441This is useful when you register callbacks from C<rcv> callbacks:
442
443 rcv delayed_reply => sub {
444 my ($delay, @reply) = @_;
445 my $timer = AE::timer $delay, 0, psub {
446 snd @reply, $SELF;
447 };
448 };
449
450=cut
451
452sub psub(&) {
453 my $cb = shift;
454
455 my $port = $SELF
456 or Carp::croak "psub can only be called from within rcv or psub callbacks, not";
457
458 sub {
459 local $SELF = $port;
460
461 if (wantarray) {
462 my @res = eval { &$cb };
463 _self_die if $@;
464 @res
465 } else {
466 my $res = eval { &$cb };
467 _self_die if $@;
468 $res
469 }
470 }
471}
472
211=item $guard = mon $portid, $cb->(@reason) 473=item $guard = mon $port, $cb->(@reason)
212 474
213=item $guard = mon $portid, $otherport 475=item $guard = mon $port, $otherport
214 476
215=item $guard = mon $portid, $otherport, @msg 477=item $guard = mon $port, $otherport, @msg
216 478
217Monitor the given port and do something when the port is killed. 479Monitor the given port and do something when the port is killed.
218 480
219In the first form, the callback is simply called with any number 481In the first form, the callback is simply called with any number
220of C<@reason> elements (no @reason means that the port was deleted 482of C<@reason> elements (no @reason means that the port was deleted
298 mon $port2, $port1; 560 mon $port2, $port1;
299 561
300It means that if either one is killed abnormally, the other one gets 562It means that if either one is killed abnormally, the other one gets
301killed as well. 563killed as well.
302 564
303=item $local_port = port 565=item kil $port[, @reason]
304 566
305Create a new local port object that supports message matching. 567Kill the specified port with the given C<@reason>.
306 568
307=item $portid = port { my @msg = @_; $finished } 569If no C<@reason> is specified, then the port is killed "normally" (linked
570ports will not be kileld, or even notified).
308 571
309Creates a "mini port", that is, a very lightweight port without any 572Otherwise, linked ports get killed with the same reason (second form of
310pattern matching behind it, and returns its ID. 573C<mon>, see below).
311 574
312The block will be called for every message received on the port. When the 575Runtime errors while evaluating C<rcv> callbacks or inside C<psub> blocks
313callback returns a true value its job is considered "done" and the port 576will be reported as reason C<< die => $@ >>.
314will be destroyed. Otherwise it will stay alive.
315 577
316The message will be passed as-is, no extra argument (i.e. no port id) will 578Transport/communication errors are reported as C<< transport_error =>
317be passed to the callback. 579$message >>.
318
319If you need the local port id in the callback, this works nicely:
320
321 my $port; $port = miniport {
322 snd $otherport, reply => $port;
323 };
324
325=cut
326
327sub port(;&) {
328 my $id = "$UNIQ." . $ID++;
329 my $port = "$NODE#$id";
330
331 if (@_) {
332 my $cb = shift;
333 $PORT{$id} = sub {
334 local $SELF = $port;
335 eval {
336 &$cb
337 and kil $id;
338 };
339 _self_die if $@;
340 };
341 } else {
342 my $self = bless {
343 id => "$NODE#$id",
344 }, "AnyEvent::MP::Port";
345
346 $PORT_DATA{$id} = $self;
347 $PORT{$id} = sub {
348 local $SELF = $port;
349
350 eval {
351 for (@{ $self->{rc0}{$_[0]} }) {
352 $_ && &{$_->[0]}
353 && undef $_;
354 }
355
356 for (@{ $self->{rcv}{$_[0]} }) {
357 $_ && [@_[1 .. @{$_->[1]}]] ~~ $_->[1]
358 && &{$_->[0]}
359 && undef $_;
360 }
361
362 for (@{ $self->{any} }) {
363 $_ && [@_[0 .. $#{$_->[1]}]] ~~ $_->[1]
364 && &{$_->[0]}
365 && undef $_;
366 }
367 };
368 _self_die if $@;
369 };
370 }
371
372 $port
373}
374
375=item reg $portid, $name
376
377Registers the given port under the name C<$name>. If the name already
378exists it is replaced.
379
380A port can only be registered under one well known name.
381
382A port automatically becomes unregistered when it is killed.
383
384=cut
385
386sub reg(@) {
387 my ($portid, $name) = @_;
388
389 $REG{$name} = $portid;
390}
391
392=item rcv $portid, tagstring => $callback->(@msg), ...
393
394=item rcv $portid, $smartmatch => $callback->(@msg), ...
395
396=item rcv $portid, [$smartmatch...] => $callback->(@msg), ...
397
398Register callbacks to be called on matching messages on the given port.
399
400The callback has to return a true value when its work is done, after
401which is will be removed, or a false value in which case it will stay
402registered.
403
404The global C<$SELF> (exported by this module) contains C<$portid> while
405executing the callback.
406
407Runtime errors wdurign callback execution will result in the port being
408C<kil>ed.
409
410If the match is an array reference, then it will be matched against the
411first elements of the message, otherwise only the first element is being
412matched.
413
414Any element in the match that is specified as C<_any_> (a function
415exported by this module) matches any single element of the message.
416
417While not required, it is highly recommended that the first matching
418element is a string identifying the message. The one-string-only match is
419also the most efficient match (by far).
420
421=cut
422
423sub rcv($@) {
424 my ($noderef, $port) = split /#/, shift, 2;
425
426 ($NODE{$noderef} || add_node $noderef) == $NODE{""}
427 or Carp::croak "$noderef#$port: rcv can only be called on local ports, caught";
428
429 my $self = $PORT_DATA{$port}
430 or Carp::croak "$noderef#$port: rcv can only be called on message matching ports, caught";
431
432 "AnyEvent::MP::Port" eq ref $self
433 or Carp::croak "$noderef#$port: rcv can only be called on message matching ports, caught";
434
435 while (@_) {
436 my ($match, $cb) = splice @_, 0, 2;
437
438 if (!ref $match) {
439 push @{ $self->{rc0}{$match} }, [$cb];
440 } elsif (("ARRAY" eq ref $match && !ref $match->[0])) {
441 my ($type, @match) = @$match;
442 @match
443 ? push @{ $self->{rcv}{$match->[0]} }, [$cb, \@match]
444 : push @{ $self->{rc0}{$match->[0]} }, [$cb];
445 } else {
446 push @{ $self->{any} }, [$cb, $match];
447 }
448 }
449}
450
451=item $closure = psub { BLOCK }
452
453Remembers C<$SELF> and creates a closure out of the BLOCK. When the
454closure is executed, sets up the environment in the same way as in C<rcv>
455callbacks, i.e. runtime errors will cause the port to get C<kil>ed.
456
457This is useful when you register callbacks from C<rcv> callbacks:
458
459 rcv delayed_reply => sub {
460 my ($delay, @reply) = @_;
461 my $timer = AE::timer $delay, 0, psub {
462 snd @reply, $SELF;
463 };
464 };
465
466=cut
467
468sub psub(&) {
469 my $cb = shift;
470
471 my $port = $SELF
472 or Carp::croak "psub can only be called from within rcv or psub callbacks, not";
473
474 sub {
475 local $SELF = $port;
476
477 if (wantarray) {
478 my @res = eval { &$cb };
479 _self_die if $@;
480 @res
481 } else {
482 my $res = eval { &$cb };
483 _self_die if $@;
484 $res
485 }
486 }
487}
488
489=back
490
491=head1 FUNCTIONS FOR NODES
492
493=over 4
494
495=item become_public $noderef
496
497Tells the node to become a public node, i.e. reachable from other nodes.
498
499The first argument is the (unresolved) node reference of the local node
500(if missing then the empty string is used).
501
502It is quite common to not specify anything, in which case the local node
503tries to listen on the default port, or to only specify a port number, in
504which case AnyEvent::MP tries to guess the local addresses.
505
506=cut
507 580
508=back 581=back
509 582
510=head1 NODE MESSAGES 583=head1 NODE MESSAGES
511 584
553 626
554=back 627=back
555 628
556=head1 AnyEvent::MP vs. Distributed Erlang 629=head1 AnyEvent::MP vs. Distributed Erlang
557 630
558AnyEvent::MP got lots of its ideas from distributed erlang (erlang node 631AnyEvent::MP got lots of its ideas from distributed Erlang (Erlang node
559== aemp node, erlang process == aemp port), so many of the documents and 632== aemp node, Erlang process == aemp port), so many of the documents and
560programming techniques employed by erlang apply to AnyEvent::MP. Here is a 633programming techniques employed by Erlang apply to AnyEvent::MP. Here is a
561sample: 634sample:
562 635
563 http://www.erlang.se/doc/programming_rules.shtml 636 http://www.Erlang.se/doc/programming_rules.shtml
564 http://erlang.org/doc/getting_started/part_frame.html # chapters 3 and 4 637 http://Erlang.org/doc/getting_started/part_frame.html # chapters 3 and 4
565 http://erlang.org/download/erlang-book-part1.pdf # chapters 5 and 6 638 http://Erlang.org/download/Erlang-book-part1.pdf # chapters 5 and 6
566 http://erlang.org/download/armstrong_thesis_2003.pdf # chapters 4 and 5 639 http://Erlang.org/download/armstrong_thesis_2003.pdf # chapters 4 and 5
567 640
568Despite the similarities, there are also some important differences: 641Despite the similarities, there are also some important differences:
569 642
570=over 4 643=over 4
571 644
582 655
583Erlang uses processes that selctively receive messages, and therefore 656Erlang uses processes that selctively receive messages, and therefore
584needs a queue. AEMP is event based, queuing messages would serve no useful 657needs a queue. AEMP is event based, queuing messages would serve no useful
585purpose. 658purpose.
586 659
587(But see L<Coro::MP> for a more erlang-like process model on top of AEMP). 660(But see L<Coro::MP> for a more Erlang-like process model on top of AEMP).
588 661
589=item * Erlang sends are synchronous, AEMP sends are asynchronous. 662=item * Erlang sends are synchronous, AEMP sends are asynchronous.
590 663
591Sending messages in erlang is synchronous and blocks the process. AEMP 664Sending messages in Erlang is synchronous and blocks the process. AEMP
592sends are immediate, connection establishment is handled in the 665sends are immediate, connection establishment is handled in the
593background. 666background.
594 667
595=item * Erlang can silently lose messages, AEMP cannot. 668=item * Erlang can silently lose messages, AEMP cannot.
596 669
599and c, and the other side only receives messages a and c). 672and c, and the other side only receives messages a and c).
600 673
601AEMP guarantees correct ordering, and the guarantee that there are no 674AEMP guarantees correct ordering, and the guarantee that there are no
602holes in the message sequence. 675holes in the message sequence.
603 676
604=item * In erlang, processes can be declared dead and later be found to be 677=item * In Erlang, processes can be declared dead and later be found to be
605alive. 678alive.
606 679
607In erlang it can happen that a monitored process is declared dead and 680In Erlang it can happen that a monitored process is declared dead and
608linked processes get killed, but later it turns out that the process is 681linked processes get killed, but later it turns out that the process is
609still alive - and can receive messages. 682still alive - and can receive messages.
610 683
611In AEMP, when port monitoring detects a port as dead, then that port will 684In AEMP, when port monitoring detects a port as dead, then that port will
612eventually be killed - it cannot happen that a node detects a port as dead 685eventually be killed - it cannot happen that a node detects a port as dead
613and then later sends messages to it, finding it is still alive. 686and then later sends messages to it, finding it is still alive.
614 687
615=item * Erlang can send messages to the wrong port, AEMP does not. 688=item * Erlang can send messages to the wrong port, AEMP does not.
616 689
617In erlang it is quite possible that a node that restarts reuses a process 690In Erlang it is quite possible that a node that restarts reuses a process
618ID known to other nodes for a completely different process, causing 691ID known to other nodes for a completely different process, causing
619messages destined for that process to end up in an unrelated process. 692messages destined for that process to end up in an unrelated process.
620 693
621AEMP never reuses port IDs, so old messages or old port IDs floating 694AEMP never reuses port IDs, so old messages or old port IDs floating
622around in the network will not be sent to an unrelated port. 695around in the network will not be sent to an unrelated port.
628securely authenticate nodes. 701securely authenticate nodes.
629 702
630=item * The AEMP protocol is optimised for both text-based and binary 703=item * The AEMP protocol is optimised for both text-based and binary
631communications. 704communications.
632 705
633The AEMP protocol, unlike the erlang protocol, supports both 706The AEMP protocol, unlike the Erlang protocol, supports both
634language-independent text-only protocols (good for debugging) and binary, 707language-independent text-only protocols (good for debugging) and binary,
635language-specific serialisers (e.g. Storable). 708language-specific serialisers (e.g. Storable).
636 709
637It has also been carefully designed to be implementable in other languages 710It has also been carefully designed to be implementable in other languages
638with a minimum of work while gracefully degrading fucntionality to make the 711with a minimum of work while gracefully degrading fucntionality to make the
639protocol simple. 712protocol simple.
640 713
714=item * AEMP has more flexible monitoring options than Erlang.
715
716In Erlang, you can chose to receive I<all> exit signals as messages
717or I<none>, there is no in-between, so monitoring single processes is
718difficult to implement. Monitoring in AEMP is more flexible than in
719Erlang, as one can choose between automatic kill, exit message or callback
720on a per-process basis.
721
722=item * Erlang has different semantics for monitoring and linking, AEMP has the same.
723
724Monitoring in Erlang is not an indicator of process death/crashes,
725as linking is (except linking is unreliable in Erlang). In AEMP, the
726semantics of monitoring and linking are identical, linking is simply
727two-way monitoring with automatic kill.
728
641=back 729=back
642 730
643=head1 SEE ALSO 731=head1 SEE ALSO
644 732
645L<AnyEvent>. 733L<AnyEvent>.

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines