[ViewVC] Diff of: cvs/AnyEvent-MP/MP.pm

Comparing AnyEvent-MP/MP.pm (file contents):
Revision 1.128 by root, Sun Mar 4 14:28:44 2012 UTC vs.
Revision 1.140 by root, Thu Mar 22 23:47:01 2012 UTC

    # destroy a port again
    kil $port;  # "normal" kill
    kil $port, my_error => "everything is broken"; # error kill
    # monitoring
-   mon $localport, $cb->(@msg)      # callback is invoked on death
+   mon $port, $cb->(@msg)      # callback is invoked on death
-   mon $localport, $otherport       # kill otherport on abnormal death
+   mon $port, $localport       # kill localport on abnormal death
-   mon $localport, $otherport, @msg # send message on death
+   mon $port, $localport, @msg # send message on death
    # temporarily execute code in port context
    peval $port, sub { die "kill the port!" };
    # execute callbacks in $SELF port context
    my $timer = AE::timer 1, 0, psub {
       die "kill the port, delayed";
    };
-=head1 CURRENT STATUS
-   bin/aemp                - stable.
-   AnyEvent::MP            - stable API, should work.
-   AnyEvent::MP::Intro     - explains most concepts.
-   AnyEvent::MP::Kernel    - mostly stable API.
-   AnyEvent::MP::Global    - stable API.
 =head1 DESCRIPTION
 This module (-family) implements a simple message passing framework.
 IF true, then the values specified in the C<configure> will take
 precedence over any values configured via the rc file. The default is for
 the rc file to override any options specified in the program.
-=item secure => $pass->($nodeid)
+=item secure => $pass->(@msg)
 In addition to specifying a boolean, you can specify a code reference that
-is called for every remote execution attempt - the execution request is
+is called for every code execution attempt - the execution request is
 granted iff the callback returns a true value.
+Most of the time the callback should look only at
+C<$AnyEvent::MP::Kernel::SRCNODE> to make a decision, and not at the
+actual message (which can be about anything, and is mostly provided for
+diagnostic purposes).
 See F<semp setsecure> for more info.
 =back
 =cut
 sub rcv($@);
-sub _kilme {
+my $KILME = sub {
-   die "received message on port without callback";
+   (my $tag = substr $_[0], 0, 30) =~ s/([\x20-\x7e])/./g;
-}
+   kil $SELF, unhandled_message => "no callback found for message '$tag'";
+};
 sub port(;&) {
    my $id = $UNIQ . ++$ID;
    my $port = "$NODE#$id";
-   rcv $port, shift || \&_kilme;
+   rcv $port, shift || $KILME;
    $port
 }
 =item rcv $local_port, $callback->(@msg)
 The global C<$SELF> (exported by this module) contains C<$port> while
 executing the callback. Runtime errors during callback execution will
 result in the port being C<kil>ed.
-The default callback received all messages not matched by a more specific
+The default callback receives all messages not matched by a more specific
 C<tag> match.
 =item rcv $local_port, tag => $callback->(@msg_without_tag), ...
 Register (or replace) callbacks to be called on messages starting with the
          $res
       }
    }
 }
+=item $guard = mon $port, $rcvport          # kill $rcvport when $port dies
+=item $guard = mon $port                    # kill $SELF when $port dies
 =item $guard = mon $port, $cb->(@reason)    # call $cb when $port dies
-=item $guard = mon $port, $rcvport          # kill $rcvport when $port dies
-=item $guard = mon $port                    # kill $SELF when $port dies
 =item $guard = mon $port, $rcvport, @msg    # send a message when $port dies
 Monitor the given port and do something when the port is killed or
 messages to it were lost, and optionally return a guard that can be used
 to stop monitoring again.
+The first two forms distinguish between "normal" and "abnormal" kil's:
+In the first form (another port given), if the C<$port> is C<kil>'ed with
+a non-empty reason, the other port (C<$rcvport>) will be kil'ed with the
+same reason. That is, on "normal" kil's nothing happens, while under all
+other conditions, the other port is killed with the same reason.
+The second form (kill self) is the same as the first form, except that
+C<$rvport> defaults to C<$SELF>.
+The remaining forms don't distinguish between "normal" and "abnormal" kil's
+- it's up to the callback or receiver to check whether the C<@reason> is
+empty and act accordingly.
-In the first form (callback), the callback is simply called with any
+In the third form (callback), the callback is simply called with any
-number of C<@reason> elements (no @reason means that the port was deleted
+number of C<@reason> elements (empty @reason means that the port was deleted
 "normally"). Note also that I<< the callback B<must> never die >>, so use
 C<eval> if unsure.
-In the second form (another port given), the other port (C<$rcvport>)
-will be C<kil>'ed with C<@reason>, if a @reason was specified, i.e. on
-"normal" kils nothing happens, while under all other conditions, the other
-port is killed with the same reason.
-The third form (kill self) is the same as the second form, except that
-C<$rvport> defaults to C<$SELF>.
-In the last form (message), a message of the form C<@msg, @reason> will be
+In the last form (message), a message of the form C<$rcvport, @msg,
-C<snd>.
+@reason> will be C<snd>.
 Monitoring-actions are one-shot: once messages are lost (and a monitoring
-alert was raised), they are removed and will not trigger again.
+alert was raised), they are removed and will not trigger again, even if it
+turns out that the port is still alive.
-As a rule of thumb, monitoring requests should always monitor a port from
+As a rule of thumb, monitoring requests should always monitor a remote
-a local port (or callback). The reason is that kill messages might get
+port locally (using a local C<$rcvport> or a callback). The reason is that
-lost, just like any other message. Another less obvious reason is that
+kill messages might get lost, just like any other message. Another less
-even monitoring requests can get lost (for example, when the connection
+obvious reason is that even monitoring requests can get lost (for example,
-to the other node goes down permanently). When monitoring a port locally
+when the connection to the other node goes down permanently). When
-these problems do not exist.
+monitoring a port locally these problems do not exist.
 C<mon> effectively guarantees that, in the absence of hardware failures,
 after starting the monitor, either all messages sent to the port will
 arrive, or the monitoring action will be invoked after possible message
 loss has been detected. No messages will be lost "in between" (after
 will be reported as reason C<< die => $@ >>.
 Transport/communication errors are reported as C<< transport_error =>
 $message >>.
-=cut
+Common idioms:
+   # silently remove yourself, do not kill linked ports
+   kil $SELF;
+   # report a failure in some detail
+   kil $SELF, failure_mode_1 => "it failed with too high temperature";
+   # do not waste much time with killing, just die when something goes wrong
+   open my $fh, "<file"
+      or die "file: $!";
 =item $port = spawn $node, $initfunc[, @initdata]
 Creates a port on the node C<$node> (which can also be a port ID, in which
 case it's the node where that port resides).
       ref $action[0]
          ? $action[0]()
          : snd @action;
    };
 }
+#=item $cb2 = timeout $seconds, $cb[, @args]
 =item cal $port, @msg, $callback[, $timeout]
 A simple form of RPC - sends a message to the given C<$port> with the
 given contents (C<@msg>), but adds a reply port to the message.
 =back
 =head1 DISTRIBUTED DATABASE
 AnyEvent::MP comes with a simple distributed database. The database will
-be mirrored asynchronously at all global nodes. Other nodes bind to one of
+be mirrored asynchronously on all global nodes. Other nodes bind to one
-the global nodes for their needs.
+of the global nodes for their needs. Every node has a "local database"
+which contains all the values that are set locally. All local databases
+are merged together to form the global database, which can be queried.
-The database consists of a two-level hash - a hash contains a hash which
+The database structure is that of a two-level hash - the database hash
-contains values.
+contains hashes which contain values, similarly to a perl hash of hashes,
+i.e.:
+  $DATABASE{$family}{$subkey} = $value
 The top level hash key is called "family", and the second-level hash key
 is called "subkey" or simply "key".
 The family must be alphanumeric, i.e. start with a letter and consist
 with the name of the application or module using it.
 The subkeys must be non-empty strings, with no further restrictions.
 The values should preferably be strings, but other perl scalars should
-work as well (such as undef, arrays and hashes).
+work as well (such as C<undef>, arrays and hashes).
 Every database entry is owned by one node - adding the same family/subkey
 combination on multiple nodes will not cause discomfort for AnyEvent::MP,
 but the result might be nondeterministic, i.e. the key might have
 different values on different nodes.
 pools. For example, a worker port for image scaling might do this:
    db_set my_image_scalers => $port;
 And clients looking for an image scaler will want to get the
-C<my_image_scalers> keys:
+C<my_image_scalers> keys from time to time:
-   db_keys "my_image_scalers" => 60 => sub {
+   db_keys my_image_scalers => sub {
-   #d##TODO#
+      @ports = @{ $_[0] };
+   };
+Or better yet, they want to monitor the database family, so they always
+have a reasonable up-to-date copy:
+   db_mon my_image_scalers => sub {
+      @ports = keys %{ $_[0] };
+   };
+In general, you can set or delete single subkeys, but query and monitor
+whole families only.
+If you feel the need to monitor or query a single subkey, try giving it
+it's own family.
 =over
-=item db_set $family => $subkey [=> $value]
+=item $guard = db_set $family => $subkey [=> $value]
 Sets (or replaces) a key to the database - if C<$value> is omitted,
 C<undef> is used instead.
+When called in non-void context, C<db_set> returns a guard that
+automatically calls C<db_del> when it is destroyed.
-=item db_del $family => $subkey
+=item db_del $family => $subkey...
-Deletes a key from the database.
+Deletes one or more subkeys from the database family.
-=item $guard = db_reg $family => $subkey [=> $value]
+=item $guard = db_reg $family => $port => $value
-Sets the key on the database and returns a guard. When the guard is
+=item $guard = db_reg $family => $port
-destroyed, the key is deleted from the database. If C<$value> is missing,
-then C<undef> is used.
+=item $guard = db_reg $family
+Registers a port in the given family and optionally returns a guard to
+remove it.
+This function basically does the same as:
+   db_set $family => $port => $value
+Except that the port is monitored and automatically removed from the
+database family when it is kil'ed.
+If C<$value> is missing, C<undef> is used. If C<$port> is missing, then
+C<$SELF> is used.
+This function is most useful to register a port in some port group (which
+is just another name for a database family), and have it removed when the
+port is gone. This works best when the port is a local port.
+=cut
+sub db_reg($$;$) {
+   my $family = shift;
+   my $port = @_ ? shift : $SELF;
+   my $clr = sub { db_del $family => $port };
+   mon $port, $clr;
+   db_set $family => $port => $_[0];
+   defined wantarray
+      and &Guard::guard ($clr)
+}
+=item db_family $family => $cb->(\%familyhash)
+Queries the named database C<$family> and call the callback with the
+family represented as a hash. You can keep and freely modify the hash.
+=item db_keys $family => $cb->(\@keys)
+Same as C<db_family>, except it only queries the family I<subkeys> and passes
+them as array reference to the callback.
+=item db_values $family => $cb->(\@values)
+Same as C<db_family>, except it only queries the family I<values> and passes them
+as array reference to the callback.
-=item $guard = db_mon $family => $cb->($familyhash, \@subkeys...)
+=item $guard = db_mon $family => $cb->($familyhash, \@added, \@changed, \@deleted)
-Creates a monitor on the given database family. Each time a key is set or
+Creates a monitor on the given database family. Each time a key is set
-or is deleted the callback is called with a hash containing the database
+or or is deleted the callback is called with a hash containing the
-family and an arrayref with subkeys that have changed.
+database family and three lists of added, changed and deleted subkeys,
+respectively. If no keys have changed then the array reference might be
+C<undef> or even missing.
-Specifically, if one of the passed subkeys exists in the $familyhash, then
+If not called in void context, a guard object is returned that, when
-it is currently set to the value in the $familyhash. Otherwise, it has
+destroyed, stops the monitor.
-been deleted.
-The first call will be with the current contents of the family and all
+The family hash reference and the key arrays belong to AnyEvent::MP and
-keys, as if they were just added.
+B<must not be modified or stored> by the callback. When in doubt, make a
+copy.
+As soon as possible after the monitoring starts, the callback will be
+called with the intiial contents of the family, even if it is empty,
+i.e. there will always be a timely call to the callback with the current
+contents.
 It is possible that the callback is called with a change event even though
 the subkey is already present and the value has not changed.
 The monitoring stops when the guard object is destroyed.
 Example: on every change to the family "mygroup", print out all keys.
    my $guard = db_mon mygroup => sub {
-      my ($family, $keys) = @_;
+      my ($family, $a, $c, $d) = @_;
       print "mygroup members: ", (join " ", keys %$family), "\n";
    };
 Exmaple: wait until the family "My::Module::workers" is non-empty.
    my $guard; $guard = db_mon My::Module::workers => sub {
-      my ($family, $keys) = @_;
+      my ($family, $a, $c, $d) = @_;
       return unless %$family;
       undef $guard;
       print "My::Module::workers now nonempty\n";
    };
 Example: print all changes to the family "AnyRvent::Fantasy::Module".
    my $guard = db_mon AnyRvent::Fantasy::Module => sub {
-      my ($family, $keys) = @_;
+      my ($family, $a, $c, $d) = @_;
-      for (@$keys) {
+      print "+$_=$family->{$_}\n" for @$a;
-         print "$_: ",
+      print "*$_=$family->{$_}\n" for @$c;
-               (exists $family->{$_}
+      print "-$_=$family->{$_}\n" for @$d;
-                  ? $family->{$_}
-                  : "(deleted)"),
-               "\n";
-      }
    };
 =cut
 =back
 Keeping your messages simple, concentrating on data structures rather than
 objects, will keep your messages clean, tidy and efficient.
 =back
+=head1 PORTING FROM AnyEvent::MP VERSION 1.X
+AEMP version 2 has a few major incompatible changes compared to version 1:
+=over 4
+=item AnyEvent::MP::Global no longer has group management functions.
+At least not officially - the grp_* functions are still exported and might
+work, but they will be removed in some later release.
+AnyEvent::MP now comes with a distributed database that is more
+powerful. Its database families map closely to port groups, but the API
+has changed (the functions are also now exported by AnyEvent::MP). Here is
+a rough porting guide:
+  grp_reg $group, $port                      # old
+  db_reg $group, $port                       # new
+  $list = grp_get $group                     # old
+  db_keys $group, sub { my $list = shift }   # new
+  grp_mon $group, $cb->(\@ports, $add, $del) # old
+  db_mon $group, $cb->(\%ports, $add, $change, $del) # new
+C<grp_reg> is a no-brainer (just replace by C<db_reg>), but C<grp_get> is
+no longer instant, because the local node might not have a copy of the
+group. You can either modify your code to allow for a callback, or use
+C<db_mon> to keep an updated copy of the group:
+  my $local_group_copy;
+  db_mon $group => sub { $local_group_copy = $_[0] };
+  # now "keys %$local_group_copy" always returns the most up-to-date
+  # list of ports in the group.
+C<grp_mon> can be replaced by C<db_mon> with minor changes - C<db_mon>
+passes a hash as first argument, and an extra C<$chg> argument that can be
+ignored:
+  db_mon $group => sub {
+     my ($ports, $add, $chg, $lde) = @_;
+     $ports = [keys %$ports];
+     # now $ports, $add and $del are the same as
+     # were originally passed by grp_mon.
+     ...
+  };
+=item Nodes not longer connect to all other nodes.
+In AEMP 1.x, every node automatically loads the L<AnyEvent::MP::Global>
+module, which in turn would create connections to all other nodes in the
+network (helped by the seed nodes).
+In version 2.x, global nodes still connect to all other global nodes, but
+other nodes don't - now every node either is a global node itself, or
+attaches itself to another global node.
+If a node isn't a global node itself, then it attaches itself to one
+of its seed nodes. If that seed node isn't a global node yet, it will
+automatically be upgraded to a global node.
+So in many cases, nothing needs to be changed - one just has to make sure
+that all seed nodes are meshed together with the other seed nodes (as with
+AEMP 1.x), and other nodes specify them as seed nodes. This is most easily
+achieved by specifying the same set of seed nodes for all nodes in the
+network.
+Not opening a connection to every other node is usually an advantage,
+except when you need the lower latency of an already established
+connection. To ensure a node establishes a connection to another node,
+you can monitor the node port (C<mon $node, ...>), which will attempt to
+create the connection (and notify you when the connection fails).
+=item Listener-less nodes (nodes without binds) are gone.
+And are not coming back, at least not in their old form. If no C<binds>
+are specified for a node, AnyEvent::MP assumes a default of C<*:*>.
+There are vague plans to implement some form of routing domains, which
+might or might not bring back listener-less nodes, but don't count on it.
+The fact that most connections are now optional somewhat mitigates this,
+as a node can be effectively unreachable from the outside without any
+problems, as long as it isn't a global node and only reaches out to other
+nodes (as opposed to being contacted from other nodes).
+=item $AnyEvent::MP::Kernel::WARN has gone.
+AnyEvent has acquired a logging framework (L<AnyEvent::Log>), and AEMP now
+uses this, and so should your programs.
+Every module now documents what kinds of messages it generates, with
+AnyEvent::MP acting as a catch all.
+On the positive side, this means that instead of setting
+C<PERL_ANYEVENT_MP_WARNLEVEL>, you can get away by setting C<AE_VERBOSE> -
+much less to type.
+=back
+=head1 LOGGING
+AnyEvent::MP does not normally log anything by itself, but sinc eit is the
+root of the contetx hierarchy for AnyEvent::MP modules, it will receive
+all log messages by submodules.
 =head1 SEE ALSO
 L<AnyEvent::MP::Intro> - a gentle introduction.
 L<AnyEvent::MP::Kernel> - more, lower-level, stuff.

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent-MP/MP.pm (file contents): Revision 1.128 by root, Sun Mar 4 14:28:44 2012 UTC vs. Revision 1.140 by root, Thu Mar 22 23:47:01 2012 UTC

Diff Legend

Comparing AnyEvent-MP/MP.pm (file contents):
Revision 1.128 by root, Sun Mar 4 14:28:44 2012 UTC vs.
Revision 1.140 by root, Thu Mar 22 23:47:01 2012 UTC