--- AnyEvent-MP/MP.pm 2009/08/04 22:05:43 1.26 +++ AnyEvent-MP/MP.pm 2009/08/05 22:40:51 1.33 @@ -45,31 +45,43 @@ =item port -A port is something you can send messages to with the C function, and -you can register C handlers with. All C handlers will receive -messages they match, messages will not be queued. +A port is something you can send messages to (with the C function). + +Some ports allow you to register C handlers that can match specific +messages. All C handlers will receive messages they match, messages +will not be queued. =item port id - C -A port id is always the noderef, a hash-mark (C<#>) as separator, followed -by a port name (a printable string of unspecified format). +A port id is normaly the concatenation of a noderef, a hash-mark (C<#>) as +separator, and a port name (a printable string of unspecified format). An +exception is the the node port, whose ID is identical to its node +reference. =item node A node is a single process containing at least one port - the node -port. You can send messages to node ports to let them create new ports, -among other things. +port. You can send messages to node ports to find existing ports or to +create new ports, among other things. -Initially, nodes are either private (single-process only) or hidden -(connected to a master node only). Only when they epxlicitly "become -public" can you send them messages from unrelated other nodes. +Nodes are either private (single-process only), slaves (connected to a +master node only) or public nodes (connectable from unrelated nodes). =item noderef - C, C, C -A noderef is a string that either uniquely identifies a given node (for -private and hidden nodes), or contains a recipe on how to reach a given +A node reference is a string that either simply identifies the node (for +private and slave nodes), or contains a recipe on how to reach a given node (for public nodes). +This recipe is simply a comma-separated list of C pairs (for +TCP/IP, other protocols might look different). + +Node references come in two flavours: resolved (containing only numerical +addresses) or unresolved (where hostnames are used instead of addresses). + +Before using an unresolved node reference in a message you first have to +resolve it. + =back =head1 VARIABLES/FUNCTIONS @@ -93,7 +105,7 @@ our $VERSION = '0.1'; our @EXPORT = qw( NODE $NODE *SELF node_of _any_ - become_slave become_public + resolve_node initialise_node snd rcv mon kil reg psub port ); @@ -113,10 +125,39 @@ to C or C, after which all local port identifiers become invalid. -=item $noderef = node_of $portid +=item $noderef = node_of $port Extracts and returns the noderef from a portid or a noderef. +=item $cv = resolve_node $noderef + +Takes an unresolved node reference that may contain hostnames and +abbreviated IDs, resolves all of them and returns a resolved node +reference. + +In addition to C pairs allowed in resolved noderefs, the +following forms are supported: + +=over 4 + +=item the empty string + +An empty-string component gets resolved as if the default port (4040) was +specified. + +=item naked port numbers (e.g. C<1234>) + +These are resolved by prepending the local nodename and a colon, to be +further resolved. + +=item hostnames (e.g. C, C) + +These are resolved by using AnyEvent::DNS to resolve them, optionally +looking up SRV records for the C port, if no port was +specified. + +=back + =item $SELF Contains the current port id while executing C callbacks or C @@ -128,9 +169,9 @@ just export C<$SELF>, all the symbols called C are exported by this module, but only C<$SELF> is currently used. -=item snd $portid, type => @data +=item snd $port, type => @data -=item snd $portid, @msg +=item snd $port, @msg Send the given message to the given port ID, which can identify either a local or a remote port, and can be either a string or soemthignt hat @@ -150,121 +191,17 @@ that Storable can serialise and deserialise is allowed, and for the local node, anything can be passed. -=item kil $portid[, @reason] - -Kill the specified port with the given C<@reason>. - -If no C<@reason> is specified, then the port is killed "normally" (linked -ports will not be kileld, or even notified). - -Otherwise, linked ports get killed with the same reason (second form of -C, see below). - -Runtime errors while evaluating C callbacks or inside C blocks -will be reported as reason C<< die => $@ >>. - -Transport/communication errors are reported as C<< transport_error => -$message >>. - -=item $guard = mon $portid, $cb->(@reason) - -=item $guard = mon $portid, $otherport - -=item $guard = mon $portid, $otherport, @msg - -Monitor the given port and do something when the port is killed. - -In the first form, the callback is simply called with any number -of C<@reason> elements (no @reason means that the port was deleted -"normally"). Note also that I<< the callback B never die >>, so use -C if unsure. - -In the second form, the other port will be C'ed with C<@reason>, iff -a @reason was specified, i.e. on "normal" kils nothing happens, while -under all other conditions, the other port is killed with the same reason. - -In the last form, a message of the form C<@msg, @reason> will be C. - -Example: call a given callback when C<$port> is killed. - - mon $port, sub { warn "port died because of <@_>\n" }; - -Example: kill ourselves when C<$port> is killed abnormally. - - mon $port, $self; - -Example: send us a restart message another C<$port> is killed. - - mon $port, $self => "restart"; - -=cut - -sub mon { - my ($noderef, $port, $cb) = ((split /#/, shift, 2), shift); - - my $node = $NODE{$noderef} || add_node $noderef; - - #TODO: ports must not be references - if (!ref $cb or "AnyEvent::MP::Port" eq ref $cb) { - if (@_) { - # send a kill info message - my (@msg) = ($cb, @_); - $cb = sub { snd @msg, @_ }; - } else { - # simply kill other port - my $port = $cb; - $cb = sub { kil $port, @_ if @_ }; - } - } - - $node->monitor ($port, $cb); - - defined wantarray - and AnyEvent::Util::guard { $node->unmonitor ($port, $cb) } -} - -=item $guard = mon_guard $port, $ref, $ref... - -Monitors the given C<$port> and keeps the passed references. When the port -is killed, the references will be freed. - -Optionally returns a guard that will stop the monitoring. - -This function is useful when you create e.g. timers or other watchers and -want to free them when the port gets killed: - - $port->rcv (start => sub { - my $timer; $timer = mon_guard $port, AE::timer 1, 1, sub { - undef $timer if 0.9 < rand; - }); - }); - -=cut - -sub mon_guard { - my ($port, @refs) = @_; - - mon $port, sub { 0 && @refs } -} - -=item lnk $port1, $port2 - -Link two ports. This is simply a shorthand for: - - mon $port1, $port2; - mon $port2, $port1; - -It means that if either one is killed abnormally, the other one gets -killed as well. - =item $local_port = port -Create a new local port object that supports message matching. - -=item $portid = port { my @msg = @_; $finished } - -Creates a "mini port", that is, a very lightweight port without any -pattern matching behind it, and returns its ID. +Create a new local port object that can be used either as a pattern +matching port ("full port") or a single-callback port ("miniport"), +depending on how C callbacks are bound to the object. + +=item $port = port { my @msg = @_; $finished } + +Creates a "miniport", that is, a very lightweight port without any pattern +matching behind it, and returns its ID. Semantically the same as creating +a port and calling C on it. The block will be called for every message received on the port. When the callback returns a true value its job is considered "done" and the port @@ -275,61 +212,28 @@ If you need the local port id in the callback, this works nicely: - my $port; $port = miniport { + my $port; $port = port { snd $otherport, reply => $port; }; =cut +sub rcv($@); + sub port(;&) { my $id = "$UNIQ." . $ID++; my $port = "$NODE#$id"; if (@_) { - my $cb = shift; - $PORT{$id} = sub { - local $SELF = $port; - eval { - &$cb - and kil $id; - }; - _self_die if $@; - }; + rcv $port, shift; } else { - my $self = bless { - id => "$NODE#$id", - }, "AnyEvent::MP::Port"; - - $PORT_DATA{$id} = $self; - $PORT{$id} = sub { - local $SELF = $port; - - eval { - for (@{ $self->{rc0}{$_[0]} }) { - $_ && &{$_->[0]} - && undef $_; - } - - for (@{ $self->{rcv}{$_[0]} }) { - $_ && [@_[1 .. @{$_->[1]}]] ~~ $_->[1] - && &{$_->[0]} - && undef $_; - } - - for (@{ $self->{any} }) { - $_ && [@_[0 .. $#{$_->[1]}]] ~~ $_->[1] - && &{$_->[0]} - && undef $_; - } - }; - _self_die if $@; - }; + $PORT{$id} = sub { }; # nop } $port } -=item reg $portid, $name +=item reg $port, $name Registers the given port under the name C<$name>. If the name already exists it is replaced. @@ -341,24 +245,30 @@ =cut sub reg(@) { - my ($portid, $name) = @_; + my ($port, $name) = @_; - $REG{$name} = $portid; + $REG{$name} = $port; } -=item rcv $portid, tagstring => $callback->(@msg), ... +=item rcv $port, $callback->(@msg) -=item rcv $portid, $smartmatch => $callback->(@msg), ... +Replaces the callback on the specified miniport (after converting it to +one if required). -=item rcv $portid, [$smartmatch...] => $callback->(@msg), ... +=item rcv $port, tagstring => $callback->(@msg), ... -Register callbacks to be called on matching messages on the given port. +=item rcv $port, $smartmatch => $callback->(@msg), ... + +=item rcv $port, [$smartmatch...] => $callback->(@msg), ... + +Register callbacks to be called on matching messages on the given full +port (after converting it to one if required). The callback has to return a true value when its work is done, after which is will be removed, or a false value in which case it will stay registered. -The global C<$SELF> (exported by this module) contains C<$portid> while +The global C<$SELF> (exported by this module) contains C<$port> while executing the callback. Runtime errors wdurign callback execution will result in the port being @@ -378,31 +288,76 @@ =cut sub rcv($@) { - my ($noderef, $port) = split /#/, shift, 2; + my $port = shift; + my ($noderef, $portid) = split /#/, $port, 2; ($NODE{$noderef} || add_node $noderef) == $NODE{""} - or Carp::croak "$noderef#$port: rcv can only be called on local ports, caught"; + or Carp::croak "$port: rcv can only be called on local ports, caught"; - my $self = $PORT_DATA{$port} - or Carp::croak "$noderef#$port: rcv can only be called on message matching ports, caught"; + if (@_ == 1) { + my $cb = shift; + delete $PORT_DATA{$portid}; + $PORT{$portid} = sub { + local $SELF = $port; + eval { + &$cb + and kil $port; + }; + _self_die if $@; + }; + } else { + my $self = $PORT_DATA{$portid} ||= do { + my $self = bless { + id => $port, + }, "AnyEvent::MP::Port"; + + $PORT{$portid} = sub { + local $SELF = $port; + + eval { + for (@{ $self->{rc0}{$_[0]} }) { + $_ && &{$_->[0]} + && undef $_; + } + + for (@{ $self->{rcv}{$_[0]} }) { + $_ && [@_[1 .. @{$_->[1]}]] ~~ $_->[1] + && &{$_->[0]} + && undef $_; + } + + for (@{ $self->{any} }) { + $_ && [@_[0 .. $#{$_->[1]}]] ~~ $_->[1] + && &{$_->[0]} + && undef $_; + } + }; + _self_die if $@; + }; - "AnyEvent::MP::Port" eq ref $self - or Carp::croak "$noderef#$port: rcv can only be called on message matching ports, caught"; + $self + }; - while (@_) { - my ($match, $cb) = splice @_, 0, 2; - - if (!ref $match) { - push @{ $self->{rc0}{$match} }, [$cb]; - } elsif (("ARRAY" eq ref $match && !ref $match->[0])) { - my ($type, @match) = @$match; - @match - ? push @{ $self->{rcv}{$match->[0]} }, [$cb, \@match] - : push @{ $self->{rc0}{$match->[0]} }, [$cb]; - } else { - push @{ $self->{any} }, [$cb, $match]; + "AnyEvent::MP::Port" eq ref $self + or Carp::croak "$port: rcv can only be called on message matching ports, caught"; + + while (@_) { + my ($match, $cb) = splice @_, 0, 2; + + if (!ref $match) { + push @{ $self->{rc0}{$match} }, [$cb]; + } elsif (("ARRAY" eq ref $match && !ref $match->[0])) { + my ($type, @match) = @$match; + @match + ? push @{ $self->{rcv}{$match->[0]} }, [$cb, \@match] + : push @{ $self->{rc0}{$match->[0]} }, [$cb]; + } else { + push @{ $self->{any} }, [$cb, $match]; + } } } + + $port } =item $closure = psub { BLOCK } @@ -443,24 +398,179 @@ } } +=item $guard = mon $port, $cb->(@reason) + +=item $guard = mon $port, $otherport + +=item $guard = mon $port, $otherport, @msg + +Monitor the given port and do something when the port is killed. + +In the first form, the callback is simply called with any number +of C<@reason> elements (no @reason means that the port was deleted +"normally"). Note also that I<< the callback B never die >>, so use +C if unsure. + +In the second form, the other port will be C'ed with C<@reason>, iff +a @reason was specified, i.e. on "normal" kils nothing happens, while +under all other conditions, the other port is killed with the same reason. + +In the last form, a message of the form C<@msg, @reason> will be C. + +Example: call a given callback when C<$port> is killed. + + mon $port, sub { warn "port died because of <@_>\n" }; + +Example: kill ourselves when C<$port> is killed abnormally. + + mon $port, $self; + +Example: send us a restart message another C<$port> is killed. + + mon $port, $self => "restart"; + +=cut + +sub mon { + my ($noderef, $port) = split /#/, shift, 2; + + my $node = $NODE{$noderef} || add_node $noderef; + + my $cb = shift; + + unless (ref $cb) { + if (@_) { + # send a kill info message + my (@msg) = ($cb, @_); + $cb = sub { snd @msg, @_ }; + } else { + # simply kill other port + my $port = $cb; + $cb = sub { kil $port, @_ if @_ }; + } + } + + $node->monitor ($port, $cb); + + defined wantarray + and AnyEvent::Util::guard { $node->unmonitor ($port, $cb) } +} + +=item $guard = mon_guard $port, $ref, $ref... + +Monitors the given C<$port> and keeps the passed references. When the port +is killed, the references will be freed. + +Optionally returns a guard that will stop the monitoring. + +This function is useful when you create e.g. timers or other watchers and +want to free them when the port gets killed: + + $port->rcv (start => sub { + my $timer; $timer = mon_guard $port, AE::timer 1, 1, sub { + undef $timer if 0.9 < rand; + }); + }); + +=cut + +sub mon_guard { + my ($port, @refs) = @_; + + mon $port, sub { 0 && @refs } +} + +=item lnk $port1, $port2 + +Link two ports. This is simply a shorthand for: + + mon $port1, $port2; + mon $port2, $port1; + +It means that if either one is killed abnormally, the other one gets +killed as well. + +=item kil $port[, @reason] + +Kill the specified port with the given C<@reason>. + +If no C<@reason> is specified, then the port is killed "normally" (linked +ports will not be kileld, or even notified). + +Otherwise, linked ports get killed with the same reason (second form of +C, see below). + +Runtime errors while evaluating C callbacks or inside C blocks +will be reported as reason C<< die => $@ >>. + +Transport/communication errors are reported as C<< transport_error => +$message >>. + =back =head1 FUNCTIONS FOR NODES =over 4 -=item become_public endpoint... +=item initialise_node $noderef, $seednode, $seednode... + +=item initialise_node "slave/", $master, $master... + +Initialises a node - must be called exactly once before calling other +AnyEvent::MP functions when talking to other nodes is required. + +All arguments are noderefs, which can be either resolved or unresolved. + +There are two types of networked nodes, public nodes and slave nodes: + +=over 4 + +=item public nodes -Tells the node to become a public node, i.e. reachable from other nodes. +For public nodes, C<$noderef> must either be a (possibly unresolved) +noderef, in which case it will be resolved, or C (or missing), in +which case the noderef will be guessed. -If no arguments are given, or the first argument is C, then -AnyEvent::MP tries to bind on port C<4040> on all IP addresses that the -local nodename resolves to. - -Otherwise the first argument must be an array-reference with transport -endpoints ("ip:port", "hostname:port") or port numbers (in which case the -local nodename is used as hostname). The endpoints are all resolved and -will become the node reference. +Afterwards, the node will bind itself on all endpoints and try to connect +to all additional C<$seednodes> that are specified. Seednodes are optional +and can be used to quickly bootstrap the node into an existing network. + +=item slave nodes + +When the C<$noderef> is the special string C, then the node will +become a slave node. Slave nodes cannot be contacted from outside and will +route most of their traffic to the master node that they attach to. + +At least one additional noderef is required: The node will try to connect +to all of them and will become a slave attached to the first node it can +successfully connect to. + +=back + +This function will block until all nodes have been resolved and, for slave +nodes, until it has successfully established a connection to a master +server. + +Example: become a public node listening on the default node. + + initialise_node; + +Example: become a public node, and try to contact some well-known master +servers to become part of the network. + + initialise_node undef, "master1", "master2"; + +Example: become a public node listening on port C<4041>. + + initialise_node 4041; + +Example: become a public node, only visible on localhost port 4044. + + initialise_node "locahost:4044"; + +Example: become a slave node to any of the specified master servers. + + initialise_node "slave/", "master1", "192.168.13.17", "mp.example.net"; =cut @@ -473,6 +583,8 @@ message - C<$reply[0]> is the port to reply to, C<$reply[1]> the type and the remaining arguments are simply the message data. +While other messages exist, they are not public and subject to change. + =over 4 =cut @@ -512,9 +624,17 @@ =head1 AnyEvent::MP vs. Distributed Erlang -AnyEvent::MP got lots of its ideas from distributed erlang. Despite the -similarities (erlang node == aemp node, erlang process == aemp port and so -on), there are also some important differences: +AnyEvent::MP got lots of its ideas from distributed erlang (erlang node +== aemp node, erlang process == aemp port), so many of the documents and +programming techniques employed by erlang apply to AnyEvent::MP. Here is a +sample: + + http://www.erlang.se/doc/programming_rules.shtml + http://erlang.org/doc/getting_started/part_frame.html # chapters 3 and 4 + http://erlang.org/download/erlang-book-part1.pdf # chapters 5 and 6 + http://erlang.org/download/armstrong_thesis_2003.pdf # chapters 4 and 5 + +Despite the similarities, there are also some important differences: =over 4 @@ -524,6 +644,9 @@ same way. AEMP relies on each node knowing it's own address(es), with convenience functionality. +This means that AEMP requires a less tightly controlled environment at the +cost of longer node references and a slightly higher management overhead. + =item * Erlang uses processes and a mailbox, AEMP does not queue. Erlang uses processes that selctively receive messages, and therefore @@ -573,6 +696,17 @@ AEMP can use a proven protocol - SSL/TLS - to protect connections and securely authenticate nodes. +=item * The AEMP protocol is optimised for both text-based and binary +communications. + +The AEMP protocol, unlike the erlang protocol, supports both +language-independent text-only protocols (good for debugging) and binary, +language-specific serialisers (e.g. Storable). + +It has also been carefully designed to be implementable in other languages +with a minimum of work while gracefully degrading fucntionality to make the +protocol simple. + =back =head1 SEE ALSO