ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP.pm
(Generate patch)

Comparing AnyEvent-MP/MP.pm (file contents):
Revision 1.85 by root, Tue Sep 8 01:54:13 2009 UTC vs.
Revision 1.114 by root, Thu Apr 22 16:06:19 2010 UTC

1=head1 NAME 1=head1 NAME
2 2
3AnyEvent::MP - multi-processing/message-passing framework 3AnyEvent::MP - erlang-style multi-processing/message-passing framework
4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use AnyEvent::MP; 7 use AnyEvent::MP;
8 8
30 rcv $port, pong => sub { warn "pong received\n" }; 30 rcv $port, pong => sub { warn "pong received\n" };
31 31
32 # create a port on another node 32 # create a port on another node
33 my $port = spawn $node, $initfunc, @initdata; 33 my $port = spawn $node, $initfunc, @initdata;
34 34
35 # destroy a prot again
36 kil $port; # "normal" kill
37 kil $port, my_error => "everything is broken"; # error kill
38
35 # monitoring 39 # monitoring
36 mon $port, $cb->(@msg) # callback is invoked on death 40 mon $localport, $cb->(@msg) # callback is invoked on death
37 mon $port, $otherport # kill otherport on abnormal death 41 mon $localport, $otherport # kill otherport on abnormal death
38 mon $port, $otherport, @msg # send message on death 42 mon $localport, $otherport, @msg # send message on death
43
44 # temporarily execute code in port context
45 peval $port, sub { die "kill the port!" };
46
47 # execute callbacks in $SELF port context
48 my $timer = AE::timer 1, 0, psub {
49 die "kill the port, delayed";
50 };
39 51
40=head1 CURRENT STATUS 52=head1 CURRENT STATUS
41 53
42 bin/aemp - stable. 54 bin/aemp - stable.
43 AnyEvent::MP - stable API, should work. 55 AnyEvent::MP - stable API, should work.
44 AnyEvent::MP::Intro - explains most concepts. 56 AnyEvent::MP::Intro - explains most concepts.
45 AnyEvent::MP::Kernel - mostly stable. 57 AnyEvent::MP::Kernel - mostly stable API.
46 AnyEvent::MP::Global - stable but incomplete, protocol not yet final. 58 AnyEvent::MP::Global - stable API.
47
48stay tuned.
49 59
50=head1 DESCRIPTION 60=head1 DESCRIPTION
51 61
52This module (-family) implements a simple message passing framework. 62This module (-family) implements a simple message passing framework.
53 63
109to have fixed listening addresses, seed nodes are perfectly normal nodes - 119to have fixed listening addresses, seed nodes are perfectly normal nodes -
110any node can function as a seed node for others. 120any node can function as a seed node for others.
111 121
112In addition to discovering the network, seed nodes are also used to 122In addition to discovering the network, seed nodes are also used to
113maintain the network and to connect nodes that otherwise would have 123maintain the network and to connect nodes that otherwise would have
114trouble connecting. They form the backbone of the AnyEvent::MP network. 124trouble connecting. They form the backbone of an AnyEvent::MP network.
115 125
116Seed nodes are expected to be long-running, and at least one seed node 126Seed nodes are expected to be long-running, and at least one seed node
117should always be available. They should also be relatively responsive - a 127should always be available. They should also be relatively responsive - a
118seed node that blocks for long periods will slow down everybody else. 128seed node that blocks for long periods will slow down everybody else.
119 129
120=item seeds - C<host:port> 130=item seeds - C<host:port>
121 131
122Seeds are transport endpoint(s) (usually a hostname/IP address and a 132Seeds are transport endpoint(s) (usually a hostname/IP address and a
123TCP port) of nodes thta should be used as seed nodes. 133TCP port) of nodes that should be used as seed nodes.
124 134
125The nodes listening on those endpoints are expected to be long-running, 135The nodes listening on those endpoints are expected to be long-running,
126and at least one of those should always be available. When nodes run out 136and at least one of those should always be available. When nodes run out
127of connections (e.g. due to a network error), they try to re-establish 137of connections (e.g. due to a network error), they try to re-establish
128connections to some seednodes again to join the network. 138connections to some seednodes again to join the network.
145 155
146use AE (); 156use AE ();
147 157
148use base "Exporter"; 158use base "Exporter";
149 159
150our $VERSION = $AnyEvent::MP::Kernel::VERSION; 160our $VERSION = 1.28;
151 161
152our @EXPORT = qw( 162our @EXPORT = qw(
153 NODE $NODE *SELF node_of after 163 NODE $NODE *SELF node_of after
154 configure 164 configure
155 snd rcv mon mon_guard kil reg psub spawn 165 snd rcv mon mon_guard kil psub peval spawn cal
156 port 166 port
157); 167);
158 168
159our $SELF; 169our $SELF;
160 170
180 190
181Before a node can talk to other nodes on the network (i.e. enter 191Before a node can talk to other nodes on the network (i.e. enter
182"distributed mode") it has to configure itself - the minimum a node needs 192"distributed mode") it has to configure itself - the minimum a node needs
183to know is its own name, and optionally it should know the addresses of 193to know is its own name, and optionally it should know the addresses of
184some other nodes in the network to discover other nodes. 194some other nodes in the network to discover other nodes.
195
196The key/value pairs are basically the same ones as documented for the
197F<aemp> command line utility (sans the set/del prefix).
185 198
186This function configures a node - it must be called exactly once (or 199This function configures a node - it must be called exactly once (or
187never) before calling other AnyEvent::MP functions. 200never) before calling other AnyEvent::MP functions.
188 201
189=over 4 202=over 4
229L<AnyEvent::MP::Global> module, which will then use it to keep 242L<AnyEvent::MP::Global> module, which will then use it to keep
230connectivity with at least one node at any point in time. 243connectivity with at least one node at any point in time.
231 244
232=back 245=back
233 246
234Example: become a distributed node using the locla node name as profile. 247Example: become a distributed node using the local node name as profile.
235This should be the most common form of invocation for "daemon"-type nodes. 248This should be the most common form of invocation for "daemon"-type nodes.
236 249
237 configure 250 configure
238 251
239Example: become an anonymous node. This form is often used for commandline 252Example: become an anonymous node. This form is often used for commandline
373 msg1 => sub { ... }, 386 msg1 => sub { ... },
374 ... 387 ...
375 ; 388 ;
376 389
377Example: temporarily register a rcv callback for a tag matching some port 390Example: temporarily register a rcv callback for a tag matching some port
378(e.g. for a rpc reply) and unregister it after a message was received. 391(e.g. for an rpc reply) and unregister it after a message was received.
379 392
380 rcv $port, $otherport => sub { 393 rcv $port, $otherport => sub {
381 my @reply = @_; 394 my @reply = @_;
382 395
383 rcv $SELF, $otherport; 396 rcv $SELF, $otherport;
396 if (ref $_[0]) { 409 if (ref $_[0]) {
397 if (my $self = $PORT_DATA{$portid}) { 410 if (my $self = $PORT_DATA{$portid}) {
398 "AnyEvent::MP::Port" eq ref $self 411 "AnyEvent::MP::Port" eq ref $self
399 or Carp::croak "$port: rcv can only be called on message matching ports, caught"; 412 or Carp::croak "$port: rcv can only be called on message matching ports, caught";
400 413
401 $self->[2] = shift; 414 $self->[0] = shift;
402 } else { 415 } else {
403 my $cb = shift; 416 my $cb = shift;
404 $PORT{$portid} = sub { 417 $PORT{$portid} = sub {
405 local $SELF = $port; 418 local $SELF = $port;
406 eval { &$cb }; _self_die if $@; 419 eval { &$cb }; _self_die if $@;
407 }; 420 };
408 } 421 }
409 } elsif (defined $_[0]) { 422 } elsif (defined $_[0]) {
410 my $self = $PORT_DATA{$portid} ||= do { 423 my $self = $PORT_DATA{$portid} ||= do {
411 my $self = bless [$PORT{$port} || sub { }, { }, $port], "AnyEvent::MP::Port"; 424 my $self = bless [$PORT{$portid} || sub { }, { }, $port], "AnyEvent::MP::Port";
412 425
413 $PORT{$portid} = sub { 426 $PORT{$portid} = sub {
414 local $SELF = $port; 427 local $SELF = $port;
415 428
416 if (my $cb = $self->[1]{$_[0]}) { 429 if (my $cb = $self->[1]{$_[0]}) {
438 } 451 }
439 452
440 $port 453 $port
441} 454}
442 455
456=item peval $port, $coderef[, @args]
457
458Evaluates the given C<$codref> within the contetx of C<$port>, that is,
459when the code throews an exception the C<$port> will be killed.
460
461Any remaining args will be passed to the callback. Any return values will
462be returned to the caller.
463
464This is useful when you temporarily want to execute code in the context of
465a port.
466
467Example: create a port and run some initialisation code in it's context.
468
469 my $port = port { ... };
470
471 peval $port, sub {
472 init
473 or die "unable to init";
474 };
475
476=cut
477
478sub peval($$) {
479 local $SELF = shift;
480 my $cb = shift;
481
482 if (wantarray) {
483 my @res = eval { &$cb };
484 _self_die if $@;
485 @res
486 } else {
487 my $res = eval { &$cb };
488 _self_die if $@;
489 $res
490 }
491}
492
443=item $closure = psub { BLOCK } 493=item $closure = psub { BLOCK }
444 494
445Remembers C<$SELF> and creates a closure out of the BLOCK. When the 495Remembers C<$SELF> and creates a closure out of the BLOCK. When the
446closure is executed, sets up the environment in the same way as in C<rcv> 496closure is executed, sets up the environment in the same way as in C<rcv>
447callbacks, i.e. runtime errors will cause the port to get C<kil>ed. 497callbacks, i.e. runtime errors will cause the port to get C<kil>ed.
498
499The effect is basically as if it returned C<< sub { peval $SELF, sub {
500BLOCK }, @_ } >>.
448 501
449This is useful when you register callbacks from C<rcv> callbacks: 502This is useful when you register callbacks from C<rcv> callbacks:
450 503
451 rcv delayed_reply => sub { 504 rcv delayed_reply => sub {
452 my ($delay, @reply) = @_; 505 my ($delay, @reply) = @_;
525delivered again. 578delivered again.
526 579
527Inter-host-connection timeouts and monitoring depend on the transport 580Inter-host-connection timeouts and monitoring depend on the transport
528used. The only transport currently implemented is TCP, and AnyEvent::MP 581used. The only transport currently implemented is TCP, and AnyEvent::MP
529relies on TCP to detect node-downs (this can take 10-15 minutes on a 582relies on TCP to detect node-downs (this can take 10-15 minutes on a
530non-idle connection, and usually around two hours for idle conenctions). 583non-idle connection, and usually around two hours for idle connections).
531 584
532This means that monitoring is good for program errors and cleaning up 585This means that monitoring is good for program errors and cleaning up
533stuff eventually, but they are no replacement for a timeout when you need 586stuff eventually, but they are no replacement for a timeout when you need
534to ensure some maximum latency. 587to ensure some maximum latency.
535 588
567 } 620 }
568 621
569 $node->monitor ($port, $cb); 622 $node->monitor ($port, $cb);
570 623
571 defined wantarray 624 defined wantarray
572 and AnyEvent::Util::guard { $node->unmonitor ($port, $cb) } 625 and ($cb += 0, AnyEvent::Util::guard { $node->unmonitor ($port, $cb) })
573} 626}
574 627
575=item $guard = mon_guard $port, $ref, $ref... 628=item $guard = mon_guard $port, $ref, $ref...
576 629
577Monitors the given C<$port> and keeps the passed references. When the port 630Monitors the given C<$port> and keeps the passed references. When the port
600 653
601=item kil $port[, @reason] 654=item kil $port[, @reason]
602 655
603Kill the specified port with the given C<@reason>. 656Kill the specified port with the given C<@reason>.
604 657
605If no C<@reason> is specified, then the port is killed "normally" (ports 658If no C<@reason> is specified, then the port is killed "normally" -
606monitoring other ports will not necessarily die because a port dies 659monitor callback will be invoked, but the kil will not cause linked ports
607"normally"). 660(C<mon $mport, $lport> form) to get killed.
608 661
609Otherwise, linked ports get killed with the same reason (second form of 662If a C<@reason> is specified, then linked ports (C<mon $mport, $lport>
610C<mon>, see above). 663form) get killed with the same reason.
611 664
612Runtime errors while evaluating C<rcv> callbacks or inside C<psub> blocks 665Runtime errors while evaluating C<rcv> callbacks or inside C<psub> blocks
613will be reported as reason C<< die => $@ >>. 666will be reported as reason C<< die => $@ >>.
614 667
615Transport/communication errors are reported as C<< transport_error => 668Transport/communication errors are reported as C<< transport_error =>
715 ? $action[0]() 768 ? $action[0]()
716 : snd @action; 769 : snd @action;
717 }; 770 };
718} 771}
719 772
773=item cal $port, @msg, $callback[, $timeout]
774
775A simple form of RPC - sends a message to the given C<$port> with the
776given contents (C<@msg>), but adds a reply port to the message.
777
778The reply port is created temporarily just for the purpose of receiving
779the reply, and will be C<kil>ed when no longer needed.
780
781A reply message sent to the port is passed to the C<$callback> as-is.
782
783If an optional time-out (in seconds) is given and it is not C<undef>,
784then the callback will be called without any arguments after the time-out
785elapsed and the port is C<kil>ed.
786
787If no time-out is given (or it is C<undef>), then the local port will
788monitor the remote port instead, so it eventually gets cleaned-up.
789
790Currently this function returns the temporary port, but this "feature"
791might go in future versions unless you can make a convincing case that
792this is indeed useful for something.
793
794=cut
795
796sub cal(@) {
797 my $timeout = ref $_[-1] ? undef : pop;
798 my $cb = pop;
799
800 my $port = port {
801 undef $timeout;
802 kil $SELF;
803 &$cb;
804 };
805
806 if (defined $timeout) {
807 $timeout = AE::timer $timeout, 0, sub {
808 undef $timeout;
809 kil $port;
810 $cb->();
811 };
812 } else {
813 mon $_[0], sub {
814 kil $port;
815 $cb->();
816 };
817 }
818
819 push @_, $port;
820 &snd;
821
822 $port
823}
824
720=back 825=back
721 826
722=head1 AnyEvent::MP vs. Distributed Erlang 827=head1 AnyEvent::MP vs. Distributed Erlang
723 828
724AnyEvent::MP got lots of its ideas from distributed Erlang (Erlang node 829AnyEvent::MP got lots of its ideas from distributed Erlang (Erlang node
725== aemp node, Erlang process == aemp port), so many of the documents and 830== aemp node, Erlang process == aemp port), so many of the documents and
726programming techniques employed by Erlang apply to AnyEvent::MP. Here is a 831programming techniques employed by Erlang apply to AnyEvent::MP. Here is a
727sample: 832sample:
728 833
729 http://www.Erlang.se/doc/programming_rules.shtml 834 http://www.erlang.se/doc/programming_rules.shtml
730 http://Erlang.org/doc/getting_started/part_frame.html # chapters 3 and 4 835 http://erlang.org/doc/getting_started/part_frame.html # chapters 3 and 4
731 http://Erlang.org/download/Erlang-book-part1.pdf # chapters 5 and 6 836 http://erlang.org/download/erlang-book-part1.pdf # chapters 5 and 6
732 http://Erlang.org/download/armstrong_thesis_2003.pdf # chapters 4 and 5 837 http://erlang.org/download/armstrong_thesis_2003.pdf # chapters 4 and 5
733 838
734Despite the similarities, there are also some important differences: 839Despite the similarities, there are also some important differences:
735 840
736=over 4 841=over 4
737 842
738=item * Node IDs are arbitrary strings in AEMP. 843=item * Node IDs are arbitrary strings in AEMP.
739 844
740Erlang relies on special naming and DNS to work everywhere in the same 845Erlang relies on special naming and DNS to work everywhere in the same
741way. AEMP relies on each node somehow knowing its own address(es) (e.g. by 846way. AEMP relies on each node somehow knowing its own address(es) (e.g. by
742configuration or DNS), but will otherwise discover other odes itself. 847configuration or DNS), and possibly the addresses of some seed nodes, but
848will otherwise discover other nodes (and their IDs) itself.
743 849
744=item * Erlang has a "remote ports are like local ports" philosophy, AEMP 850=item * Erlang has a "remote ports are like local ports" philosophy, AEMP
745uses "local ports are like remote ports". 851uses "local ports are like remote ports".
746 852
747The failure modes for local ports are quite different (runtime errors 853The failure modes for local ports are quite different (runtime errors
772so does not need a queue that can overflow). AEMP sends are immediate, 878so does not need a queue that can overflow). AEMP sends are immediate,
773connection establishment is handled in the background. 879connection establishment is handled in the background.
774 880
775=item * Erlang suffers from silent message loss, AEMP does not. 881=item * Erlang suffers from silent message loss, AEMP does not.
776 882
777Erlang makes few guarantees on messages delivery - messages can get lost 883Erlang implements few guarantees on messages delivery - messages can get
778without any of the processes realising it (i.e. you send messages a, b, 884lost without any of the processes realising it (i.e. you send messages a,
779and c, and the other side only receives messages a and c). 885b, and c, and the other side only receives messages a and c).
780 886
781AEMP guarantees correct ordering, and the guarantee that after one message 887AEMP guarantees correct ordering, and the guarantee that after one message
782is lost, all following ones sent to the same port are lost as well, until 888is lost, all following ones sent to the same port are lost as well, until
783monitoring raises an error, so there are no silent "holes" in the message 889monitoring raises an error, so there are no silent "holes" in the message
784sequence. 890sequence.
846overhead, as well as having to keep a proxy object everywhere. 952overhead, as well as having to keep a proxy object everywhere.
847 953
848Strings can easily be printed, easily serialised etc. and need no special 954Strings can easily be printed, easily serialised etc. and need no special
849procedures to be "valid". 955procedures to be "valid".
850 956
851And as a result, a miniport consists of a single closure stored in a 957And as a result, a port with just a default receiver consists of a single
852global hash - it can't become much cheaper. 958closure stored in a global hash - it can't become much cheaper.
853 959
854=item Why favour JSON, why not a real serialising format such as Storable? 960=item Why favour JSON, why not a real serialising format such as Storable?
855 961
856In fact, any AnyEvent::MP node will happily accept Storable as framing 962In fact, any AnyEvent::MP node will happily accept Storable as framing
857format, but currently there is no way to make a node use Storable by 963format, but currently there is no way to make a node use Storable by
873 979
874L<AnyEvent::MP::Intro> - a gentle introduction. 980L<AnyEvent::MP::Intro> - a gentle introduction.
875 981
876L<AnyEvent::MP::Kernel> - more, lower-level, stuff. 982L<AnyEvent::MP::Kernel> - more, lower-level, stuff.
877 983
878L<AnyEvent::MP::Global> - network maintainance and port groups, to find 984L<AnyEvent::MP::Global> - network maintenance and port groups, to find
879your applications. 985your applications.
986
987L<AnyEvent::MP::DataConn> - establish data connections between nodes.
880 988
881L<AnyEvent::MP::LogCatcher> - simple service to display log messages from 989L<AnyEvent::MP::LogCatcher> - simple service to display log messages from
882all nodes. 990all nodes.
883 991
884L<AnyEvent>. 992L<AnyEvent>.

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines