ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/cvsroot/AnyEvent-MP/MP.pm
(Generate patch)

Comparing cvsroot/AnyEvent-MP/MP.pm (file contents):
Revision 1.127 by root, Sat Mar 3 20:35:10 2012 UTC vs.
Revision 1.143 by root, Fri Mar 23 17:54:36 2012 UTC

35 # destroy a port again 35 # destroy a port again
36 kil $port; # "normal" kill 36 kil $port; # "normal" kill
37 kil $port, my_error => "everything is broken"; # error kill 37 kil $port, my_error => "everything is broken"; # error kill
38 38
39 # monitoring 39 # monitoring
40 mon $localport, $cb->(@msg) # callback is invoked on death 40 mon $port, $cb->(@msg) # callback is invoked on death
41 mon $localport, $otherport # kill otherport on abnormal death 41 mon $port, $localport # kill localport on abnormal death
42 mon $localport, $otherport, @msg # send message on death 42 mon $port, $localport, @msg # send message on death
43 43
44 # temporarily execute code in port context 44 # temporarily execute code in port context
45 peval $port, sub { die "kill the port!" }; 45 peval $port, sub { die "kill the port!" };
46 46
47 # execute callbacks in $SELF port context 47 # execute callbacks in $SELF port context
48 my $timer = AE::timer 1, 0, psub { 48 my $timer = AE::timer 1, 0, psub {
49 die "kill the port, delayed"; 49 die "kill the port, delayed";
50 }; 50 };
51 51
52=head1 CURRENT STATUS 52 # distributed database - modification
53 db_set $family => $subkey [=> $value] # add a subkey
54 db_del $family => $subkey... # delete one or more subkeys
55 db_reg $family => $port [=> $value] # register a port
53 56
54 bin/aemp - stable. 57 # distributed database - queries
55 AnyEvent::MP - stable API, should work. 58 db_family $family => $cb->(\%familyhash)
56 AnyEvent::MP::Intro - explains most concepts. 59 db_keys $family => $cb->(\@keys)
57 AnyEvent::MP::Kernel - mostly stable API. 60 db_values $family => $cb->(\@values)
58 AnyEvent::MP::Global - stable API. 61
62 # distributed database - monitoring a family
63 db_mon $family => $cb->(\%familyhash, \@added, \@changed, \@deleted)
59 64
60=head1 DESCRIPTION 65=head1 DESCRIPTION
61 66
62This module (-family) implements a simple message passing framework. 67This module (-family) implements a simple message passing framework.
63 68
184 189
185use common::sense; 190use common::sense;
186 191
187use Carp (); 192use Carp ();
188 193
189use AE (); 194use AnyEvent ();
190use Guard (); 195use Guard ();
191 196
192use base "Exporter"; 197use base "Exporter";
193 198
194our $VERSION = $AnyEvent::MP::Config::VERSION; 199our $VERSION = $AnyEvent::MP::Config::VERSION;
197 NODE $NODE *SELF node_of after 202 NODE $NODE *SELF node_of after
198 configure 203 configure
199 snd rcv mon mon_guard kil psub peval spawn cal 204 snd rcv mon mon_guard kil psub peval spawn cal
200 port 205 port
201 db_set db_del db_reg 206 db_set db_del db_reg
207 db_mon db_family db_keys db_values
202); 208);
203 209
204our $SELF; 210our $SELF;
205 211
206sub _self_die() { 212sub _self_die() {
245=item force => $boolean (default false) 251=item force => $boolean (default false)
246 252
247IF true, then the values specified in the C<configure> will take 253IF true, then the values specified in the C<configure> will take
248precedence over any values configured via the rc file. The default is for 254precedence over any values configured via the rc file. The default is for
249the rc file to override any options specified in the program. 255the rc file to override any options specified in the program.
250
251=item secure => $pass->($nodeid)
252
253In addition to specifying a boolean, you can specify a code reference that
254is called for every remote execution attempt - the execution request is
255granted iff the callback returns a true value.
256
257See F<semp setsecure> for more info.
258 256
259=back 257=back
260 258
261=over 4 259=over 4
262 260
397 395
398=cut 396=cut
399 397
400sub rcv($@); 398sub rcv($@);
401 399
402sub _kilme { 400my $KILME = sub {
403 die "received message on port without callback"; 401 (my $tag = substr $_[0], 0, 30) =~ s/([\x20-\x7e])/./g;
404} 402 kil $SELF, unhandled_message => "no callback found for message '$tag'";
403};
405 404
406sub port(;&) { 405sub port(;&) {
407 my $id = $UNIQ . ++$ID; 406 my $id = $UNIQ . ++$ID;
408 my $port = "$NODE#$id"; 407 my $port = "$NODE#$id";
409 408
410 rcv $port, shift || \&_kilme; 409 rcv $port, shift || $KILME;
411 410
412 $port 411 $port
413} 412}
414 413
415=item rcv $local_port, $callback->(@msg) 414=item rcv $local_port, $callback->(@msg)
420 419
421The global C<$SELF> (exported by this module) contains C<$port> while 420The global C<$SELF> (exported by this module) contains C<$port> while
422executing the callback. Runtime errors during callback execution will 421executing the callback. Runtime errors during callback execution will
423result in the port being C<kil>ed. 422result in the port being C<kil>ed.
424 423
425The default callback received all messages not matched by a more specific 424The default callback receives all messages not matched by a more specific
426C<tag> match. 425C<tag> match.
427 426
428=item rcv $local_port, tag => $callback->(@msg_without_tag), ... 427=item rcv $local_port, tag => $callback->(@msg_without_tag), ...
429 428
430Register (or replace) callbacks to be called on messages starting with the 429Register (or replace) callbacks to be called on messages starting with the
594 $res 593 $res
595 } 594 }
596 } 595 }
597} 596}
598 597
598=item $guard = mon $port, $rcvport # kill $rcvport when $port dies
599
600=item $guard = mon $port # kill $SELF when $port dies
601
599=item $guard = mon $port, $cb->(@reason) # call $cb when $port dies 602=item $guard = mon $port, $cb->(@reason) # call $cb when $port dies
600
601=item $guard = mon $port, $rcvport # kill $rcvport when $port dies
602
603=item $guard = mon $port # kill $SELF when $port dies
604 603
605=item $guard = mon $port, $rcvport, @msg # send a message when $port dies 604=item $guard = mon $port, $rcvport, @msg # send a message when $port dies
606 605
607Monitor the given port and do something when the port is killed or 606Monitor the given port and do something when the port is killed or
608messages to it were lost, and optionally return a guard that can be used 607messages to it were lost, and optionally return a guard that can be used
609to stop monitoring again. 608to stop monitoring again.
610 609
610The first two forms distinguish between "normal" and "abnormal" kil's:
611
612In the first form (another port given), if the C<$port> is C<kil>'ed with
613a non-empty reason, the other port (C<$rcvport>) will be kil'ed with the
614same reason. That is, on "normal" kil's nothing happens, while under all
615other conditions, the other port is killed with the same reason.
616
617The second form (kill self) is the same as the first form, except that
618C<$rvport> defaults to C<$SELF>.
619
620The remaining forms don't distinguish between "normal" and "abnormal" kil's
621- it's up to the callback or receiver to check whether the C<@reason> is
622empty and act accordingly.
623
611In the first form (callback), the callback is simply called with any 624In the third form (callback), the callback is simply called with any
612number of C<@reason> elements (no @reason means that the port was deleted 625number of C<@reason> elements (empty @reason means that the port was deleted
613"normally"). Note also that I<< the callback B<must> never die >>, so use 626"normally"). Note also that I<< the callback B<must> never die >>, so use
614C<eval> if unsure. 627C<eval> if unsure.
615 628
616In the second form (another port given), the other port (C<$rcvport>)
617will be C<kil>'ed with C<@reason>, if a @reason was specified, i.e. on
618"normal" kils nothing happens, while under all other conditions, the other
619port is killed with the same reason.
620
621The third form (kill self) is the same as the second form, except that
622C<$rvport> defaults to C<$SELF>.
623
624In the last form (message), a message of the form C<@msg, @reason> will be 629In the last form (message), a message of the form C<$rcvport, @msg,
625C<snd>. 630@reason> will be C<snd>.
626 631
627Monitoring-actions are one-shot: once messages are lost (and a monitoring 632Monitoring-actions are one-shot: once messages are lost (and a monitoring
628alert was raised), they are removed and will not trigger again. 633alert was raised), they are removed and will not trigger again, even if it
634turns out that the port is still alive.
629 635
630As a rule of thumb, monitoring requests should always monitor a port from 636As a rule of thumb, monitoring requests should always monitor a remote
631a local port (or callback). The reason is that kill messages might get 637port locally (using a local C<$rcvport> or a callback). The reason is that
632lost, just like any other message. Another less obvious reason is that 638kill messages might get lost, just like any other message. Another less
633even monitoring requests can get lost (for example, when the connection 639obvious reason is that even monitoring requests can get lost (for example,
634to the other node goes down permanently). When monitoring a port locally 640when the connection to the other node goes down permanently). When
635these problems do not exist. 641monitoring a port locally these problems do not exist.
636 642
637C<mon> effectively guarantees that, in the absence of hardware failures, 643C<mon> effectively guarantees that, in the absence of hardware failures,
638after starting the monitor, either all messages sent to the port will 644after starting the monitor, either all messages sent to the port will
639arrive, or the monitoring action will be invoked after possible message 645arrive, or the monitoring action will be invoked after possible message
640loss has been detected. No messages will be lost "in between" (after 646loss has been detected. No messages will be lost "in between" (after
731will be reported as reason C<< die => $@ >>. 737will be reported as reason C<< die => $@ >>.
732 738
733Transport/communication errors are reported as C<< transport_error => 739Transport/communication errors are reported as C<< transport_error =>
734$message >>. 740$message >>.
735 741
736=cut 742Common idioms:
743
744 # silently remove yourself, do not kill linked ports
745 kil $SELF;
746
747 # report a failure in some detail
748 kil $SELF, failure_mode_1 => "it failed with too high temperature";
749
750 # do not waste much time with killing, just die when something goes wrong
751 open my $fh, "<file"
752 or die "file: $!";
737 753
738=item $port = spawn $node, $initfunc[, @initdata] 754=item $port = spawn $node, $initfunc[, @initdata]
739 755
740Creates a port on the node C<$node> (which can also be a port ID, in which 756Creates a port on the node C<$node> (which can also be a port ID, in which
741case it's the node where that port resides). 757case it's the node where that port resides).
833 ref $action[0] 849 ref $action[0]
834 ? $action[0]() 850 ? $action[0]()
835 : snd @action; 851 : snd @action;
836 }; 852 };
837} 853}
854
855#=item $cb2 = timeout $seconds, $cb[, @args]
838 856
839=item cal $port, @msg, $callback[, $timeout] 857=item cal $port, @msg, $callback[, $timeout]
840 858
841A simple form of RPC - sends a message to the given C<$port> with the 859A simple form of RPC - sends a message to the given C<$port> with the
842given contents (C<@msg>), but adds a reply port to the message. 860given contents (C<@msg>), but adds a reply port to the message.
891=back 909=back
892 910
893=head1 DISTRIBUTED DATABASE 911=head1 DISTRIBUTED DATABASE
894 912
895AnyEvent::MP comes with a simple distributed database. The database will 913AnyEvent::MP comes with a simple distributed database. The database will
896be mirrored asynchronously at all global nodes. Other nodes bind to one of 914be mirrored asynchronously on all global nodes. Other nodes bind to one
897the global nodes for their needs. 915of the global nodes for their needs. Every node has a "local database"
916which contains all the values that are set locally. All local databases
917are merged together to form the global database, which can be queried.
898 918
899The database consists of a two-level hash - a hash contains a hash which 919The database structure is that of a two-level hash - the database hash
900contains values. 920contains hashes which contain values, similarly to a perl hash of hashes,
921i.e.:
922
923 $DATABASE{$family}{$subkey} = $value
901 924
902The top level hash key is called "family", and the second-level hash key 925The top level hash key is called "family", and the second-level hash key
903is called "subkey" or simply "key". 926is called "subkey" or simply "key".
904 927
905The family must be alphanumeric, i.e. start with a letter and consist 928The family must be alphanumeric, i.e. start with a letter and consist
910with the name of the application or module using it. 933with the name of the application or module using it.
911 934
912The subkeys must be non-empty strings, with no further restrictions. 935The subkeys must be non-empty strings, with no further restrictions.
913 936
914The values should preferably be strings, but other perl scalars should 937The values should preferably be strings, but other perl scalars should
915work as well (such as undef, arrays and hashes). 938work as well (such as C<undef>, arrays and hashes).
916 939
917Every database entry is owned by one node - adding the same family/subkey 940Every database entry is owned by one node - adding the same family/subkey
918combination on multiple nodes will not cause discomfort for AnyEvent::MP, 941combination on multiple nodes will not cause discomfort for AnyEvent::MP,
919but the result might be nondeterministic, i.e. the key might have 942but the result might be nondeterministic, i.e. the key might have
920different values on different nodes. 943different values on different nodes.
924pools. For example, a worker port for image scaling might do this: 947pools. For example, a worker port for image scaling might do this:
925 948
926 db_set my_image_scalers => $port; 949 db_set my_image_scalers => $port;
927 950
928And clients looking for an image scaler will want to get the 951And clients looking for an image scaler will want to get the
929C<my_image_scalers> keys: 952C<my_image_scalers> keys from time to time:
930 953
931 db_keys "my_image_scalers" => 60 => sub { 954 db_keys my_image_scalers => sub {
932 #d##TODO# 955 @ports = @{ $_[0] };
956 };
957
958Or better yet, they want to monitor the database family, so they always
959have a reasonable up-to-date copy:
960
961 db_mon my_image_scalers => sub {
962 @ports = keys %{ $_[0] };
963 };
964
965In general, you can set or delete single subkeys, but query and monitor
966whole families only.
967
968If you feel the need to monitor or query a single subkey, try giving it
969it's own family.
933 970
934=over 971=over
935 972
936=item db_set $family => $subkey [=> $value] 973=item $guard = db_set $family => $subkey [=> $value]
937 974
938Sets (or replaces) a key to the database - if C<$value> is omitted, 975Sets (or replaces) a key to the database - if C<$value> is omitted,
939C<undef> is used instead. 976C<undef> is used instead.
940 977
978When called in non-void context, C<db_set> returns a guard that
979automatically calls C<db_del> when it is destroyed.
980
941=item db_del $family => $subkey 981=item db_del $family => $subkey...
942 982
943Deletes a key from the database. 983Deletes one or more subkeys from the database family.
944 984
945=item $guard = db_reg $family => $subkey [=> $value] 985=item $guard = db_reg $family => $port => $value
946 986
947Sets the key on the database and returns a guard. When the guard is 987=item $guard = db_reg $family => $port
948destroyed, the key is deleted from the database. If C<$value> is missing, 988
949then C<undef> is used. 989=item $guard = db_reg $family
990
991Registers a port in the given family and optionally returns a guard to
992remove it.
993
994This function basically does the same as:
995
996 db_set $family => $port => $value
997
998Except that the port is monitored and automatically removed from the
999database family when it is kil'ed.
1000
1001If C<$value> is missing, C<undef> is used. If C<$port> is missing, then
1002C<$SELF> is used.
1003
1004This function is most useful to register a port in some port group (which
1005is just another name for a database family), and have it removed when the
1006port is gone. This works best when the port is a local port.
1007
1008=cut
1009
1010sub db_reg($$;$) {
1011 my $family = shift;
1012 my $port = @_ ? shift : $SELF;
1013
1014 my $clr = sub { db_del $family => $port };
1015 mon $port, $clr;
1016
1017 db_set $family => $port => $_[0];
1018
1019 defined wantarray
1020 and &Guard::guard ($clr)
1021}
1022
1023=item db_family $family => $cb->(\%familyhash)
1024
1025Queries the named database C<$family> and call the callback with the
1026family represented as a hash. You can keep and freely modify the hash.
1027
1028=item db_keys $family => $cb->(\@keys)
1029
1030Same as C<db_family>, except it only queries the family I<subkeys> and passes
1031them as array reference to the callback.
1032
1033=item db_values $family => $cb->(\@values)
1034
1035Same as C<db_family>, except it only queries the family I<values> and passes them
1036as array reference to the callback.
1037
1038=item $guard = db_mon $family => $cb->(\%familyhash, \@added, \@changed, \@deleted)
1039
1040Creates a monitor on the given database family. Each time a key is set
1041or or is deleted the callback is called with a hash containing the
1042database family and three lists of added, changed and deleted subkeys,
1043respectively. If no keys have changed then the array reference might be
1044C<undef> or even missing.
1045
1046If not called in void context, a guard object is returned that, when
1047destroyed, stops the monitor.
1048
1049The family hash reference and the key arrays belong to AnyEvent::MP and
1050B<must not be modified or stored> by the callback. When in doubt, make a
1051copy.
1052
1053As soon as possible after the monitoring starts, the callback will be
1054called with the intiial contents of the family, even if it is empty,
1055i.e. there will always be a timely call to the callback with the current
1056contents.
1057
1058It is possible that the callback is called with a change event even though
1059the subkey is already present and the value has not changed.
1060
1061The monitoring stops when the guard object is destroyed.
1062
1063Example: on every change to the family "mygroup", print out all keys.
1064
1065 my $guard = db_mon mygroup => sub {
1066 my ($family, $a, $c, $d) = @_;
1067 print "mygroup members: ", (join " ", keys %$family), "\n";
1068 };
1069
1070Exmaple: wait until the family "My::Module::workers" is non-empty.
1071
1072 my $guard; $guard = db_mon My::Module::workers => sub {
1073 my ($family, $a, $c, $d) = @_;
1074 return unless %$family;
1075 undef $guard;
1076 print "My::Module::workers now nonempty\n";
1077 };
1078
1079Example: print all changes to the family "AnyRvent::Fantasy::Module".
1080
1081 my $guard = db_mon AnyRvent::Fantasy::Module => sub {
1082 my ($family, $a, $c, $d) = @_;
1083
1084 print "+$_=$family->{$_}\n" for @$a;
1085 print "*$_=$family->{$_}\n" for @$c;
1086 print "-$_=$family->{$_}\n" for @$d;
1087 };
950 1088
951=cut 1089=cut
952 1090
953=back 1091=back
954 1092
1113Keeping your messages simple, concentrating on data structures rather than 1251Keeping your messages simple, concentrating on data structures rather than
1114objects, will keep your messages clean, tidy and efficient. 1252objects, will keep your messages clean, tidy and efficient.
1115 1253
1116=back 1254=back
1117 1255
1256=head1 PORTING FROM AnyEvent::MP VERSION 1.X
1257
1258AEMP version 2 has a few major incompatible changes compared to version 1:
1259
1260=over 4
1261
1262=item AnyEvent::MP::Global no longer has group management functions.
1263
1264At least not officially - the grp_* functions are still exported and might
1265work, but they will be removed in some later release.
1266
1267AnyEvent::MP now comes with a distributed database that is more
1268powerful. Its database families map closely to port groups, but the API
1269has changed (the functions are also now exported by AnyEvent::MP). Here is
1270a rough porting guide:
1271
1272 grp_reg $group, $port # old
1273 db_reg $group, $port # new
1274
1275 $list = grp_get $group # old
1276 db_keys $group, sub { my $list = shift } # new
1277
1278 grp_mon $group, $cb->(\@ports, $add, $del) # old
1279 db_mon $group, $cb->(\%ports, $add, $change, $del) # new
1280
1281C<grp_reg> is a no-brainer (just replace by C<db_reg>), but C<grp_get> is
1282no longer instant, because the local node might not have a copy of the
1283group. You can either modify your code to allow for a callback, or use
1284C<db_mon> to keep an updated copy of the group:
1285
1286 my $local_group_copy;
1287 db_mon $group => sub { $local_group_copy = $_[0] };
1288
1289 # now "keys %$local_group_copy" always returns the most up-to-date
1290 # list of ports in the group.
1291
1292C<grp_mon> can be replaced by C<db_mon> with minor changes - C<db_mon>
1293passes a hash as first argument, and an extra C<$chg> argument that can be
1294ignored:
1295
1296 db_mon $group => sub {
1297 my ($ports, $add, $chg, $lde) = @_;
1298 $ports = [keys %$ports];
1299
1300 # now $ports, $add and $del are the same as
1301 # were originally passed by grp_mon.
1302 ...
1303 };
1304
1305=item Nodes not longer connect to all other nodes.
1306
1307In AEMP 1.x, every node automatically loads the L<AnyEvent::MP::Global>
1308module, which in turn would create connections to all other nodes in the
1309network (helped by the seed nodes).
1310
1311In version 2.x, global nodes still connect to all other global nodes, but
1312other nodes don't - now every node either is a global node itself, or
1313attaches itself to another global node.
1314
1315If a node isn't a global node itself, then it attaches itself to one
1316of its seed nodes. If that seed node isn't a global node yet, it will
1317automatically be upgraded to a global node.
1318
1319So in many cases, nothing needs to be changed - one just has to make sure
1320that all seed nodes are meshed together with the other seed nodes (as with
1321AEMP 1.x), and other nodes specify them as seed nodes. This is most easily
1322achieved by specifying the same set of seed nodes for all nodes in the
1323network.
1324
1325Not opening a connection to every other node is usually an advantage,
1326except when you need the lower latency of an already established
1327connection. To ensure a node establishes a connection to another node,
1328you can monitor the node port (C<mon $node, ...>), which will attempt to
1329create the connection (and notify you when the connection fails).
1330
1331=item Listener-less nodes (nodes without binds) are gone.
1332
1333And are not coming back, at least not in their old form. If no C<binds>
1334are specified for a node, AnyEvent::MP assumes a default of C<*:*>.
1335
1336There are vague plans to implement some form of routing domains, which
1337might or might not bring back listener-less nodes, but don't count on it.
1338
1339The fact that most connections are now optional somewhat mitigates this,
1340as a node can be effectively unreachable from the outside without any
1341problems, as long as it isn't a global node and only reaches out to other
1342nodes (as opposed to being contacted from other nodes).
1343
1344=item $AnyEvent::MP::Kernel::WARN has gone.
1345
1346AnyEvent has acquired a logging framework (L<AnyEvent::Log>), and AEMP now
1347uses this, and so should your programs.
1348
1349Every module now documents what kinds of messages it generates, with
1350AnyEvent::MP acting as a catch all.
1351
1352On the positive side, this means that instead of setting
1353C<PERL_ANYEVENT_MP_WARNLEVEL>, you can get away by setting C<AE_VERBOSE> -
1354much less to type.
1355
1356=back
1357
1358=head1 LOGGING
1359
1360AnyEvent::MP does not normally log anything by itself, but sinc eit is the
1361root of the contetx hierarchy for AnyEvent::MP modules, it will receive
1362all log messages by submodules.
1363
1118=head1 SEE ALSO 1364=head1 SEE ALSO
1119 1365
1120L<AnyEvent::MP::Intro> - a gentle introduction. 1366L<AnyEvent::MP::Intro> - a gentle introduction.
1121 1367
1122L<AnyEvent::MP::Kernel> - more, lower-level, stuff. 1368L<AnyEvent::MP::Kernel> - more, lower-level, stuff.

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines