ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP.pm
(Generate patch)

Comparing AnyEvent-MP/MP.pm (file contents):
Revision 1.126 by root, Sat Mar 3 19:43:41 2012 UTC vs.
Revision 1.141 by root, Fri Mar 23 03:24:41 2012 UTC

35 # destroy a port again 35 # destroy a port again
36 kil $port; # "normal" kill 36 kil $port; # "normal" kill
37 kil $port, my_error => "everything is broken"; # error kill 37 kil $port, my_error => "everything is broken"; # error kill
38 38
39 # monitoring 39 # monitoring
40 mon $localport, $cb->(@msg) # callback is invoked on death 40 mon $port, $cb->(@msg) # callback is invoked on death
41 mon $localport, $otherport # kill otherport on abnormal death 41 mon $port, $localport # kill localport on abnormal death
42 mon $localport, $otherport, @msg # send message on death 42 mon $port, $localport, @msg # send message on death
43 43
44 # temporarily execute code in port context 44 # temporarily execute code in port context
45 peval $port, sub { die "kill the port!" }; 45 peval $port, sub { die "kill the port!" };
46 46
47 # execute callbacks in $SELF port context 47 # execute callbacks in $SELF port context
48 my $timer = AE::timer 1, 0, psub { 48 my $timer = AE::timer 1, 0, psub {
49 die "kill the port, delayed"; 49 die "kill the port, delayed";
50 }; 50 };
51
52=head1 CURRENT STATUS
53
54 bin/aemp - stable.
55 AnyEvent::MP - stable API, should work.
56 AnyEvent::MP::Intro - explains most concepts.
57 AnyEvent::MP::Kernel - mostly stable API.
58 AnyEvent::MP::Global - stable API.
59 51
60=head1 DESCRIPTION 52=head1 DESCRIPTION
61 53
62This module (-family) implements a simple message passing framework. 54This module (-family) implements a simple message passing framework.
63 55
184 176
185use common::sense; 177use common::sense;
186 178
187use Carp (); 179use Carp ();
188 180
189use AE (); 181use AnyEvent ();
190use Guard (); 182use Guard ();
191 183
192use base "Exporter"; 184use base "Exporter";
193 185
194our $VERSION = $AnyEvent::MP::Config::VERSION; 186our $VERSION = $AnyEvent::MP::Config::VERSION;
197 NODE $NODE *SELF node_of after 189 NODE $NODE *SELF node_of after
198 configure 190 configure
199 snd rcv mon mon_guard kil psub peval spawn cal 191 snd rcv mon mon_guard kil psub peval spawn cal
200 port 192 port
201 db_set db_del db_reg 193 db_set db_del db_reg
194 db_mon db_family db_keys db_values
202); 195);
203 196
204our $SELF; 197our $SELF;
205 198
206sub _self_die() { 199sub _self_die() {
230 223
231This function configures a node - it must be called exactly once (or 224This function configures a node - it must be called exactly once (or
232never) before calling other AnyEvent::MP functions. 225never) before calling other AnyEvent::MP functions.
233 226
234The key/value pairs are basically the same ones as documented for the 227The key/value pairs are basically the same ones as documented for the
235F<aemp> command line utility (sans the set/del prefix), with two additions: 228F<aemp> command line utility (sans the set/del prefix), with these additions:
236 229
237=over 4 230=over 4
238 231
239=item norc => $boolean (default false) 232=item norc => $boolean (default false)
240 233
245=item force => $boolean (default false) 238=item force => $boolean (default false)
246 239
247IF true, then the values specified in the C<configure> will take 240IF true, then the values specified in the C<configure> will take
248precedence over any values configured via the rc file. The default is for 241precedence over any values configured via the rc file. The default is for
249the rc file to override any options specified in the program. 242the rc file to override any options specified in the program.
243
244=item secure => $pass->(@msg)
245
246In addition to specifying a boolean, you can specify a code reference that
247is called for every code execution attempt - the execution request is
248granted iff the callback returns a true value.
249
250Most of the time the callback should look only at
251C<$AnyEvent::MP::Kernel::SRCNODE> to make a decision, and not at the
252actual message (which can be about anything, and is mostly provided for
253diagnostic purposes).
254
255See F<semp setsecure> for more info.
250 256
251=back 257=back
252 258
253=over 4 259=over 4
254 260
389 395
390=cut 396=cut
391 397
392sub rcv($@); 398sub rcv($@);
393 399
394sub _kilme { 400my $KILME = sub {
395 die "received message on port without callback"; 401 (my $tag = substr $_[0], 0, 30) =~ s/([\x20-\x7e])/./g;
396} 402 kil $SELF, unhandled_message => "no callback found for message '$tag'";
403};
397 404
398sub port(;&) { 405sub port(;&) {
399 my $id = $UNIQ . ++$ID; 406 my $id = $UNIQ . ++$ID;
400 my $port = "$NODE#$id"; 407 my $port = "$NODE#$id";
401 408
402 rcv $port, shift || \&_kilme; 409 rcv $port, shift || $KILME;
403 410
404 $port 411 $port
405} 412}
406 413
407=item rcv $local_port, $callback->(@msg) 414=item rcv $local_port, $callback->(@msg)
412 419
413The global C<$SELF> (exported by this module) contains C<$port> while 420The global C<$SELF> (exported by this module) contains C<$port> while
414executing the callback. Runtime errors during callback execution will 421executing the callback. Runtime errors during callback execution will
415result in the port being C<kil>ed. 422result in the port being C<kil>ed.
416 423
417The default callback received all messages not matched by a more specific 424The default callback receives all messages not matched by a more specific
418C<tag> match. 425C<tag> match.
419 426
420=item rcv $local_port, tag => $callback->(@msg_without_tag), ... 427=item rcv $local_port, tag => $callback->(@msg_without_tag), ...
421 428
422Register (or replace) callbacks to be called on messages starting with the 429Register (or replace) callbacks to be called on messages starting with the
586 $res 593 $res
587 } 594 }
588 } 595 }
589} 596}
590 597
598=item $guard = mon $port, $rcvport # kill $rcvport when $port dies
599
600=item $guard = mon $port # kill $SELF when $port dies
601
591=item $guard = mon $port, $cb->(@reason) # call $cb when $port dies 602=item $guard = mon $port, $cb->(@reason) # call $cb when $port dies
592
593=item $guard = mon $port, $rcvport # kill $rcvport when $port dies
594
595=item $guard = mon $port # kill $SELF when $port dies
596 603
597=item $guard = mon $port, $rcvport, @msg # send a message when $port dies 604=item $guard = mon $port, $rcvport, @msg # send a message when $port dies
598 605
599Monitor the given port and do something when the port is killed or 606Monitor the given port and do something when the port is killed or
600messages to it were lost, and optionally return a guard that can be used 607messages to it were lost, and optionally return a guard that can be used
601to stop monitoring again. 608to stop monitoring again.
602 609
610The first two forms distinguish between "normal" and "abnormal" kil's:
611
612In the first form (another port given), if the C<$port> is C<kil>'ed with
613a non-empty reason, the other port (C<$rcvport>) will be kil'ed with the
614same reason. That is, on "normal" kil's nothing happens, while under all
615other conditions, the other port is killed with the same reason.
616
617The second form (kill self) is the same as the first form, except that
618C<$rvport> defaults to C<$SELF>.
619
620The remaining forms don't distinguish between "normal" and "abnormal" kil's
621- it's up to the callback or receiver to check whether the C<@reason> is
622empty and act accordingly.
623
603In the first form (callback), the callback is simply called with any 624In the third form (callback), the callback is simply called with any
604number of C<@reason> elements (no @reason means that the port was deleted 625number of C<@reason> elements (empty @reason means that the port was deleted
605"normally"). Note also that I<< the callback B<must> never die >>, so use 626"normally"). Note also that I<< the callback B<must> never die >>, so use
606C<eval> if unsure. 627C<eval> if unsure.
607 628
608In the second form (another port given), the other port (C<$rcvport>)
609will be C<kil>'ed with C<@reason>, if a @reason was specified, i.e. on
610"normal" kils nothing happens, while under all other conditions, the other
611port is killed with the same reason.
612
613The third form (kill self) is the same as the second form, except that
614C<$rvport> defaults to C<$SELF>.
615
616In the last form (message), a message of the form C<@msg, @reason> will be 629In the last form (message), a message of the form C<$rcvport, @msg,
617C<snd>. 630@reason> will be C<snd>.
618 631
619Monitoring-actions are one-shot: once messages are lost (and a monitoring 632Monitoring-actions are one-shot: once messages are lost (and a monitoring
620alert was raised), they are removed and will not trigger again. 633alert was raised), they are removed and will not trigger again, even if it
634turns out that the port is still alive.
621 635
622As a rule of thumb, monitoring requests should always monitor a port from 636As a rule of thumb, monitoring requests should always monitor a remote
623a local port (or callback). The reason is that kill messages might get 637port locally (using a local C<$rcvport> or a callback). The reason is that
624lost, just like any other message. Another less obvious reason is that 638kill messages might get lost, just like any other message. Another less
625even monitoring requests can get lost (for example, when the connection 639obvious reason is that even monitoring requests can get lost (for example,
626to the other node goes down permanently). When monitoring a port locally 640when the connection to the other node goes down permanently). When
627these problems do not exist. 641monitoring a port locally these problems do not exist.
628 642
629C<mon> effectively guarantees that, in the absence of hardware failures, 643C<mon> effectively guarantees that, in the absence of hardware failures,
630after starting the monitor, either all messages sent to the port will 644after starting the monitor, either all messages sent to the port will
631arrive, or the monitoring action will be invoked after possible message 645arrive, or the monitoring action will be invoked after possible message
632loss has been detected. No messages will be lost "in between" (after 646loss has been detected. No messages will be lost "in between" (after
723will be reported as reason C<< die => $@ >>. 737will be reported as reason C<< die => $@ >>.
724 738
725Transport/communication errors are reported as C<< transport_error => 739Transport/communication errors are reported as C<< transport_error =>
726$message >>. 740$message >>.
727 741
728=cut 742Common idioms:
743
744 # silently remove yourself, do not kill linked ports
745 kil $SELF;
746
747 # report a failure in some detail
748 kil $SELF, failure_mode_1 => "it failed with too high temperature";
749
750 # do not waste much time with killing, just die when something goes wrong
751 open my $fh, "<file"
752 or die "file: $!";
729 753
730=item $port = spawn $node, $initfunc[, @initdata] 754=item $port = spawn $node, $initfunc[, @initdata]
731 755
732Creates a port on the node C<$node> (which can also be a port ID, in which 756Creates a port on the node C<$node> (which can also be a port ID, in which
733case it's the node where that port resides). 757case it's the node where that port resides).
825 ref $action[0] 849 ref $action[0]
826 ? $action[0]() 850 ? $action[0]()
827 : snd @action; 851 : snd @action;
828 }; 852 };
829} 853}
854
855#=item $cb2 = timeout $seconds, $cb[, @args]
830 856
831=item cal $port, @msg, $callback[, $timeout] 857=item cal $port, @msg, $callback[, $timeout]
832 858
833A simple form of RPC - sends a message to the given C<$port> with the 859A simple form of RPC - sends a message to the given C<$port> with the
834given contents (C<@msg>), but adds a reply port to the message. 860given contents (C<@msg>), but adds a reply port to the message.
883=back 909=back
884 910
885=head1 DISTRIBUTED DATABASE 911=head1 DISTRIBUTED DATABASE
886 912
887AnyEvent::MP comes with a simple distributed database. The database will 913AnyEvent::MP comes with a simple distributed database. The database will
888be mirrored asynchronously at all global nodes. Other nodes bind to one of 914be mirrored asynchronously on all global nodes. Other nodes bind to one
889the global nodes for their needs. 915of the global nodes for their needs. Every node has a "local database"
916which contains all the values that are set locally. All local databases
917are merged together to form the global database, which can be queried.
890 918
891The database consists of a two-level hash - a hash contains a hash which 919The database structure is that of a two-level hash - the database hash
892contains values. 920contains hashes which contain values, similarly to a perl hash of hashes,
921i.e.:
922
923 $DATABASE{$family}{$subkey} = $value
893 924
894The top level hash key is called "family", and the second-level hash key 925The top level hash key is called "family", and the second-level hash key
895is called "subkey" or simply "key". 926is called "subkey" or simply "key".
896 927
897The family must be alphanumeric, i.e. start with a letter and consist 928The family must be alphanumeric, i.e. start with a letter and consist
902with the name of the application or module using it. 933with the name of the application or module using it.
903 934
904The subkeys must be non-empty strings, with no further restrictions. 935The subkeys must be non-empty strings, with no further restrictions.
905 936
906The values should preferably be strings, but other perl scalars should 937The values should preferably be strings, but other perl scalars should
907work as well (such as undef, arrays and hashes). 938work as well (such as C<undef>, arrays and hashes).
908 939
909Every database entry is owned by one node - adding the same family/subkey 940Every database entry is owned by one node - adding the same family/subkey
910combination on multiple nodes will not cause discomfort for AnyEvent::MP, 941combination on multiple nodes will not cause discomfort for AnyEvent::MP,
911but the result might be nondeterministic, i.e. the key might have 942but the result might be nondeterministic, i.e. the key might have
912different values on different nodes. 943different values on different nodes.
916pools. For example, a worker port for image scaling might do this: 947pools. For example, a worker port for image scaling might do this:
917 948
918 db_set my_image_scalers => $port; 949 db_set my_image_scalers => $port;
919 950
920And clients looking for an image scaler will want to get the 951And clients looking for an image scaler will want to get the
921C<my_image_scalers> keys: 952C<my_image_scalers> keys from time to time:
922 953
923 db_keys "my_image_scalers" => 60 => sub { 954 db_keys my_image_scalers => sub {
924 #d##TODO# 955 @ports = @{ $_[0] };
956 };
957
958Or better yet, they want to monitor the database family, so they always
959have a reasonable up-to-date copy:
960
961 db_mon my_image_scalers => sub {
962 @ports = keys %{ $_[0] };
963 };
964
965In general, you can set or delete single subkeys, but query and monitor
966whole families only.
967
968If you feel the need to monitor or query a single subkey, try giving it
969it's own family.
925 970
926=over 971=over
927 972
928=item db_set $family => $subkey [=> $value] 973=item $guard = db_set $family => $subkey [=> $value]
929 974
930Sets (or replaces) a key to the database - if C<$value> is omitted, 975Sets (or replaces) a key to the database - if C<$value> is omitted,
931C<undef> is used instead. 976C<undef> is used instead.
932 977
978When called in non-void context, C<db_set> returns a guard that
979automatically calls C<db_del> when it is destroyed.
980
933=item db_del $family => $subkey 981=item db_del $family => $subkey...
934 982
935Deletes a key from the database. 983Deletes one or more subkeys from the database family.
936 984
937=item $guard = db_reg $family => $subkey [=> $value] 985=item $guard = db_reg $family => $port => $value
938 986
939Sets the key on the database and returns a guard. When the guard is 987=item $guard = db_reg $family => $port
940destroyed, the key is deleted from the database. If C<$value> is missing, 988
941then C<undef> is used. 989=item $guard = db_reg $family
990
991Registers a port in the given family and optionally returns a guard to
992remove it.
993
994This function basically does the same as:
995
996 db_set $family => $port => $value
997
998Except that the port is monitored and automatically removed from the
999database family when it is kil'ed.
1000
1001If C<$value> is missing, C<undef> is used. If C<$port> is missing, then
1002C<$SELF> is used.
1003
1004This function is most useful to register a port in some port group (which
1005is just another name for a database family), and have it removed when the
1006port is gone. This works best when the port is a local port.
1007
1008=cut
1009
1010sub db_reg($$;$) {
1011 my $family = shift;
1012 my $port = @_ ? shift : $SELF;
1013
1014 my $clr = sub { db_del $family => $port };
1015 mon $port, $clr;
1016
1017 db_set $family => $port => $_[0];
1018
1019 defined wantarray
1020 and &Guard::guard ($clr)
1021}
1022
1023=item db_family $family => $cb->(\%familyhash)
1024
1025Queries the named database C<$family> and call the callback with the
1026family represented as a hash. You can keep and freely modify the hash.
1027
1028=item db_keys $family => $cb->(\@keys)
1029
1030Same as C<db_family>, except it only queries the family I<subkeys> and passes
1031them as array reference to the callback.
1032
1033=item db_values $family => $cb->(\@values)
1034
1035Same as C<db_family>, except it only queries the family I<values> and passes them
1036as array reference to the callback.
1037
1038=item $guard = db_mon $family => $cb->($familyhash, \@added, \@changed, \@deleted)
1039
1040Creates a monitor on the given database family. Each time a key is set
1041or or is deleted the callback is called with a hash containing the
1042database family and three lists of added, changed and deleted subkeys,
1043respectively. If no keys have changed then the array reference might be
1044C<undef> or even missing.
1045
1046If not called in void context, a guard object is returned that, when
1047destroyed, stops the monitor.
1048
1049The family hash reference and the key arrays belong to AnyEvent::MP and
1050B<must not be modified or stored> by the callback. When in doubt, make a
1051copy.
1052
1053As soon as possible after the monitoring starts, the callback will be
1054called with the intiial contents of the family, even if it is empty,
1055i.e. there will always be a timely call to the callback with the current
1056contents.
1057
1058It is possible that the callback is called with a change event even though
1059the subkey is already present and the value has not changed.
1060
1061The monitoring stops when the guard object is destroyed.
1062
1063Example: on every change to the family "mygroup", print out all keys.
1064
1065 my $guard = db_mon mygroup => sub {
1066 my ($family, $a, $c, $d) = @_;
1067 print "mygroup members: ", (join " ", keys %$family), "\n";
1068 };
1069
1070Exmaple: wait until the family "My::Module::workers" is non-empty.
1071
1072 my $guard; $guard = db_mon My::Module::workers => sub {
1073 my ($family, $a, $c, $d) = @_;
1074 return unless %$family;
1075 undef $guard;
1076 print "My::Module::workers now nonempty\n";
1077 };
1078
1079Example: print all changes to the family "AnyRvent::Fantasy::Module".
1080
1081 my $guard = db_mon AnyRvent::Fantasy::Module => sub {
1082 my ($family, $a, $c, $d) = @_;
1083
1084 print "+$_=$family->{$_}\n" for @$a;
1085 print "*$_=$family->{$_}\n" for @$c;
1086 print "-$_=$family->{$_}\n" for @$d;
1087 };
942 1088
943=cut 1089=cut
944 1090
945=back 1091=back
946 1092
1105Keeping your messages simple, concentrating on data structures rather than 1251Keeping your messages simple, concentrating on data structures rather than
1106objects, will keep your messages clean, tidy and efficient. 1252objects, will keep your messages clean, tidy and efficient.
1107 1253
1108=back 1254=back
1109 1255
1256=head1 PORTING FROM AnyEvent::MP VERSION 1.X
1257
1258AEMP version 2 has a few major incompatible changes compared to version 1:
1259
1260=over 4
1261
1262=item AnyEvent::MP::Global no longer has group management functions.
1263
1264At least not officially - the grp_* functions are still exported and might
1265work, but they will be removed in some later release.
1266
1267AnyEvent::MP now comes with a distributed database that is more
1268powerful. Its database families map closely to port groups, but the API
1269has changed (the functions are also now exported by AnyEvent::MP). Here is
1270a rough porting guide:
1271
1272 grp_reg $group, $port # old
1273 db_reg $group, $port # new
1274
1275 $list = grp_get $group # old
1276 db_keys $group, sub { my $list = shift } # new
1277
1278 grp_mon $group, $cb->(\@ports, $add, $del) # old
1279 db_mon $group, $cb->(\%ports, $add, $change, $del) # new
1280
1281C<grp_reg> is a no-brainer (just replace by C<db_reg>), but C<grp_get> is
1282no longer instant, because the local node might not have a copy of the
1283group. You can either modify your code to allow for a callback, or use
1284C<db_mon> to keep an updated copy of the group:
1285
1286 my $local_group_copy;
1287 db_mon $group => sub { $local_group_copy = $_[0] };
1288
1289 # now "keys %$local_group_copy" always returns the most up-to-date
1290 # list of ports in the group.
1291
1292C<grp_mon> can be replaced by C<db_mon> with minor changes - C<db_mon>
1293passes a hash as first argument, and an extra C<$chg> argument that can be
1294ignored:
1295
1296 db_mon $group => sub {
1297 my ($ports, $add, $chg, $lde) = @_;
1298 $ports = [keys %$ports];
1299
1300 # now $ports, $add and $del are the same as
1301 # were originally passed by grp_mon.
1302 ...
1303 };
1304
1305=item Nodes not longer connect to all other nodes.
1306
1307In AEMP 1.x, every node automatically loads the L<AnyEvent::MP::Global>
1308module, which in turn would create connections to all other nodes in the
1309network (helped by the seed nodes).
1310
1311In version 2.x, global nodes still connect to all other global nodes, but
1312other nodes don't - now every node either is a global node itself, or
1313attaches itself to another global node.
1314
1315If a node isn't a global node itself, then it attaches itself to one
1316of its seed nodes. If that seed node isn't a global node yet, it will
1317automatically be upgraded to a global node.
1318
1319So in many cases, nothing needs to be changed - one just has to make sure
1320that all seed nodes are meshed together with the other seed nodes (as with
1321AEMP 1.x), and other nodes specify them as seed nodes. This is most easily
1322achieved by specifying the same set of seed nodes for all nodes in the
1323network.
1324
1325Not opening a connection to every other node is usually an advantage,
1326except when you need the lower latency of an already established
1327connection. To ensure a node establishes a connection to another node,
1328you can monitor the node port (C<mon $node, ...>), which will attempt to
1329create the connection (and notify you when the connection fails).
1330
1331=item Listener-less nodes (nodes without binds) are gone.
1332
1333And are not coming back, at least not in their old form. If no C<binds>
1334are specified for a node, AnyEvent::MP assumes a default of C<*:*>.
1335
1336There are vague plans to implement some form of routing domains, which
1337might or might not bring back listener-less nodes, but don't count on it.
1338
1339The fact that most connections are now optional somewhat mitigates this,
1340as a node can be effectively unreachable from the outside without any
1341problems, as long as it isn't a global node and only reaches out to other
1342nodes (as opposed to being contacted from other nodes).
1343
1344=item $AnyEvent::MP::Kernel::WARN has gone.
1345
1346AnyEvent has acquired a logging framework (L<AnyEvent::Log>), and AEMP now
1347uses this, and so should your programs.
1348
1349Every module now documents what kinds of messages it generates, with
1350AnyEvent::MP acting as a catch all.
1351
1352On the positive side, this means that instead of setting
1353C<PERL_ANYEVENT_MP_WARNLEVEL>, you can get away by setting C<AE_VERBOSE> -
1354much less to type.
1355
1356=back
1357
1358=head1 LOGGING
1359
1360AnyEvent::MP does not normally log anything by itself, but sinc eit is the
1361root of the contetx hierarchy for AnyEvent::MP modules, it will receive
1362all log messages by submodules.
1363
1110=head1 SEE ALSO 1364=head1 SEE ALSO
1111 1365
1112L<AnyEvent::MP::Intro> - a gentle introduction. 1366L<AnyEvent::MP::Intro> - a gentle introduction.
1113 1367
1114L<AnyEvent::MP::Kernel> - more, lower-level, stuff. 1368L<AnyEvent::MP::Kernel> - more, lower-level, stuff.

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines