ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP.pm
(Generate patch)

Comparing AnyEvent-MP/MP.pm (file contents):
Revision 1.99 by root, Fri Oct 2 14:12:16 2009 UTC vs.
Revision 1.153 by root, Sat Nov 2 01:30:49 2019 UTC

30 rcv $port, pong => sub { warn "pong received\n" }; 30 rcv $port, pong => sub { warn "pong received\n" };
31 31
32 # create a port on another node 32 # create a port on another node
33 my $port = spawn $node, $initfunc, @initdata; 33 my $port = spawn $node, $initfunc, @initdata;
34 34
35 # destroy a port again
36 kil $port; # "normal" kill
37 kil $port, my_error => "everything is broken"; # error kill
38
35 # monitoring 39 # monitoring
36 mon $localport, $cb->(@msg) # callback is invoked on death 40 mon $port, $cb->(@msg) # callback is invoked on death
37 mon $localport, $otherport # kill otherport on abnormal death 41 mon $port, $localport # kill localport on abnormal death
38 mon $localport, $otherport, @msg # send message on death 42 mon $port, $localport, @msg # send message on death
39 43
40=head1 CURRENT STATUS 44 # temporarily execute code in port context
45 peval $port, sub { die "kill the port!" };
41 46
42 bin/aemp - stable. 47 # execute callbacks in $SELF port context
43 AnyEvent::MP - stable API, should work. 48 my $timer = AE::timer 1, 0, psub {
44 AnyEvent::MP::Intro - explains most concepts. 49 die "kill the port, delayed";
45 AnyEvent::MP::Kernel - mostly stable API. 50 };
46 AnyEvent::MP::Global - stable API. 51
52 # distributed database - modification
53 db_set $family => $subkey [=> $value] # add a subkey
54 db_del $family => $subkey... # delete one or more subkeys
55 db_reg $family => $port [=> $value] # register a port
56
57 # distributed database - queries
58 db_family $family => $cb->(\%familyhash)
59 db_keys $family => $cb->(\@keys)
60 db_values $family => $cb->(\@values)
61
62 # distributed database - monitoring a family
63 db_mon $family => $cb->(\%familyhash, \@added, \@changed, \@deleted)
47 64
48=head1 DESCRIPTION 65=head1 DESCRIPTION
49 66
50This module (-family) implements a simple message passing framework. 67This module (-family) implements a simple message passing framework.
51 68
66 83
67Ports allow you to register C<rcv> handlers that can match all or just 84Ports allow you to register C<rcv> handlers that can match all or just
68some messages. Messages send to ports will not be queued, regardless of 85some messages. Messages send to ports will not be queued, regardless of
69anything was listening for them or not. 86anything was listening for them or not.
70 87
88Ports are represented by (printable) strings called "port IDs".
89
71=item port ID - C<nodeid#portname> 90=item port ID - C<nodeid#portname>
72 91
73A port ID is the concatenation of a node ID, a hash-mark (C<#>) as 92A port ID is the concatenation of a node ID, a hash-mark (C<#>)
74separator, and a port name (a printable string of unspecified format). 93as separator, and a port name (a printable string of unspecified
94format created by AnyEvent::MP).
75 95
76=item node 96=item node
77 97
78A node is a single process containing at least one port - the node port, 98A node is a single process containing at least one port - the node port,
79which enables nodes to manage each other remotely, and to create new 99which enables nodes to manage each other remotely, and to create new
80ports. 100ports.
81 101
82Nodes are either public (have one or more listening ports) or private 102Nodes are either public (have one or more listening ports) or private
83(no listening ports). Private nodes cannot talk to other private nodes 103(no listening ports). Private nodes cannot talk to other private nodes
84currently. 104currently, but all nodes can talk to public nodes.
85 105
106Nodes is represented by (printable) strings called "node IDs".
107
86=item node ID - C<[A-Z_][a-zA-Z0-9_\-.:]*> 108=item node ID - C<[A-Za-z0-9_\-.:]*>
87 109
88A node ID is a string that uniquely identifies the node within a 110A node ID is a string that uniquely identifies the node within a
89network. Depending on the configuration used, node IDs can look like a 111network. Depending on the configuration used, node IDs can look like a
90hostname, a hostname and a port, or a random string. AnyEvent::MP itself 112hostname, a hostname and a port, or a random string. AnyEvent::MP itself
91doesn't interpret node IDs in any way. 113doesn't interpret node IDs in any way except to uniquely identify a node.
92 114
93=item binds - C<ip:port> 115=item binds - C<ip:port>
94 116
95Nodes can only talk to each other by creating some kind of connection to 117Nodes can only talk to each other by creating some kind of connection to
96each other. To do this, nodes should listen on one or more local transport 118each other. To do this, nodes should listen on one or more local transport
119endpoints - binds.
120
97endpoints - binds. Currently, only standard C<ip:port> specifications can 121Currently, only standard C<ip:port> specifications can be used, which
98be used, which specify TCP ports to listen on. 122specify TCP ports to listen on. So a bind is basically just a tcp socket
123in listening mode that accepts connections from other nodes.
99 124
100=item seed nodes 125=item seed nodes
101 126
102When a node starts, it knows nothing about the network. To teach the node 127When a node starts, it knows nothing about the network it is in - it
103about the network it first has to contact some other node within the 128needs to connect to at least one other node that is already in the
104network. This node is called a seed. 129network. These other nodes are called "seed nodes".
105 130
106Apart from the fact that other nodes know them as seed nodes and they have 131Seed nodes themselves are not special - they are seed nodes only because
107to have fixed listening addresses, seed nodes are perfectly normal nodes - 132some other node I<uses> them as such, but any node can be used as seed
108any node can function as a seed node for others. 133node for other nodes, and eahc node can use a different set of seed nodes.
109 134
110In addition to discovering the network, seed nodes are also used to 135In addition to discovering the network, seed nodes are also used to
111maintain the network and to connect nodes that otherwise would have 136maintain the network - all nodes using the same seed node are part of the
112trouble connecting. They form the backbone of an AnyEvent::MP network. 137same network. If a network is split into multiple subnets because e.g. the
138network link between the parts goes down, then using the same seed nodes
139for all nodes ensures that eventually the subnets get merged again.
113 140
114Seed nodes are expected to be long-running, and at least one seed node 141Seed nodes are expected to be long-running, and at least one seed node
115should always be available. They should also be relatively responsive - a 142should always be available. They should also be relatively responsive - a
116seed node that blocks for long periods will slow down everybody else. 143seed node that blocks for long periods will slow down everybody else.
117 144
145For small networks, it's best if every node uses the same set of seed
146nodes. For large networks, it can be useful to specify "regional" seed
147nodes for most nodes in an area, and use all seed nodes as seed nodes for
148each other. What's important is that all seed nodes connections form a
149complete graph, so that the network cannot split into separate subnets
150forever.
151
152Seed nodes are represented by seed IDs.
153
118=item seeds - C<host:port> 154=item seed IDs - C<host:port>
119 155
120Seeds are transport endpoint(s) (usually a hostname/IP address and a 156Seed IDs are transport endpoint(s) (usually a hostname/IP address and a
121TCP port) of nodes that should be used as seed nodes. 157TCP port) of nodes that should be used as seed nodes.
122 158
123The nodes listening on those endpoints are expected to be long-running, 159=item global nodes
124and at least one of those should always be available. When nodes run out 160
125of connections (e.g. due to a network error), they try to re-establish 161An AEMP network needs a discovery service - nodes need to know how to
126connections to some seednodes again to join the network. 162connect to other nodes they only know by name. In addition, AEMP offers a
163distributed "group database", which maps group names to a list of strings
164- for example, to register worker ports.
165
166A network needs at least one global node to work, and allows every node to
167be a global node.
168
169Any node that loads the L<AnyEvent::MP::Global> module becomes a global
170node and tries to keep connections to all other nodes. So while it can
171make sense to make every node "global" in small networks, it usually makes
172sense to only make seed nodes into global nodes in large networks (nodes
173keep connections to seed nodes and global nodes, so making them the same
174reduces overhead).
127 175
128=back 176=back
129 177
130=head1 VARIABLES/FUNCTIONS 178=head1 VARIABLES/FUNCTIONS
131 179
133 181
134=cut 182=cut
135 183
136package AnyEvent::MP; 184package AnyEvent::MP;
137 185
186use AnyEvent::MP::Config ();
138use AnyEvent::MP::Kernel; 187use AnyEvent::MP::Kernel;
188use AnyEvent::MP::Kernel qw(
189 %NODE %PORT %PORT_DATA $UNIQ $RUNIQ $ID
190 add_node load_func
191
192 NODE $NODE
193 configure
194 node_of port_is_local
195 snd kil
196 db_set db_del
197 db_mon db_family db_keys db_values
198);
139 199
140use common::sense; 200use common::sense;
141 201
142use Carp (); 202use Carp ();
143 203
144use AE (); 204use AnyEvent ();
205use Guard ();
145 206
146use base "Exporter"; 207use base "Exporter";
147 208
148our $VERSION = 1.2; 209our $VERSION = '2.02'; # also in MP/Config.pm
149 210
150our @EXPORT = qw( 211our @EXPORT = qw(
151 NODE $NODE *SELF node_of after
152 configure 212 configure
213
214 NODE $NODE
215 *SELF
216
217 node_of port_is_local
218
219 snd kil
153 snd rcv mon mon_guard kil psub spawn cal 220 port rcv mon mon_guard psub peval spawn cal
154 port 221 db_set db_del db_reg
222 db_mon db_family db_keys db_values
223
224 after
155); 225);
156 226
157our $SELF; 227our $SELF;
158 228
159sub _self_die() { 229sub _self_die() {
170 240
171=item $nodeid = node_of $port 241=item $nodeid = node_of $port
172 242
173Extracts and returns the node ID from a port ID or a node ID. 243Extracts and returns the node ID from a port ID or a node ID.
174 244
245=item $is_local = port_is_local $port
246
247Returns true iff the port is a local port.
248
175=item configure $profile, key => value... 249=item configure $profile, key => value...
176 250
177=item configure key => value... 251=item configure key => value...
178 252
179Before a node can talk to other nodes on the network (i.e. enter 253Before a node can talk to other nodes on the network (i.e. enter
181to know is its own name, and optionally it should know the addresses of 255to know is its own name, and optionally it should know the addresses of
182some other nodes in the network to discover other nodes. 256some other nodes in the network to discover other nodes.
183 257
184This function configures a node - it must be called exactly once (or 258This function configures a node - it must be called exactly once (or
185never) before calling other AnyEvent::MP functions. 259never) before calling other AnyEvent::MP functions.
260
261The key/value pairs are basically the same ones as documented for the
262F<aemp> command line utility (sans the set/del prefix), with these additions:
263
264=over 4
265
266=item norc => $boolean (default false)
267
268If true, then the rc file (e.g. F<~/.perl-anyevent-mp>) will I<not>
269be consulted - all configuration options must be specified in the
270C<configure> call.
271
272=item force => $boolean (default false)
273
274IF true, then the values specified in the C<configure> will take
275precedence over any values configured via the rc file. The default is for
276the rc file to override any options specified in the program.
277
278=back
186 279
187=over 4 280=over 4
188 281
189=item step 1, gathering configuration from profiles 282=item step 1, gathering configuration from profiles
190 283
204That means that the values specified in the profile have highest priority 297That means that the values specified in the profile have highest priority
205and the values specified directly via C<configure> have lowest priority, 298and the values specified directly via C<configure> have lowest priority,
206and can only be used to specify defaults. 299and can only be used to specify defaults.
207 300
208If the profile specifies a node ID, then this will become the node ID of 301If the profile specifies a node ID, then this will become the node ID of
209this process. If not, then the profile name will be used as node ID. The 302this process. If not, then the profile name will be used as node ID, with
210special node ID of C<anon/> will be replaced by a random node ID. 303a unique randoms tring (C</%u>) appended.
304
305The node ID can contain some C<%> sequences that are expanded: C<%n>
306is expanded to the local nodename, C<%u> is replaced by a random
307strign to make the node unique. For example, the F<aemp> commandline
308utility uses C<aemp/%n/%u> as nodename, which might expand to
309C<aemp/cerebro/ZQDGSIkRhEZQDGSIkRhE>.
211 310
212=item step 2, bind listener sockets 311=item step 2, bind listener sockets
213 312
214The next step is to look up the binds in the profile, followed by binding 313The next step is to look up the binds in the profile, followed by binding
215aemp protocol listeners on all binds specified (it is possible and valid 314aemp protocol listeners on all binds specified (it is possible and valid
216to have no binds, meaning that the node cannot be contacted form the 315to have no binds, meaning that the node cannot be contacted from the
217outside. This means the node cannot talk to other nodes that also have no 316outside. This means the node cannot talk to other nodes that also have no
218binds, but it can still talk to all "normal" nodes). 317binds, but it can still talk to all "normal" nodes).
219 318
220If the profile does not specify a binds list, then a default of C<*> is 319If the profile does not specify a binds list, then a default of C<*> is
221used, meaning the node will bind on a dynamically-assigned port on every 320used, meaning the node will bind on a dynamically-assigned port on every
222local IP address it finds. 321local IP address it finds.
223 322
224=item step 3, connect to seed nodes 323=item step 3, connect to seed nodes
225 324
226As the last step, the seeds list from the profile is passed to the 325As the last step, the seed ID list from the profile is passed to the
227L<AnyEvent::MP::Global> module, which will then use it to keep 326L<AnyEvent::MP::Global> module, which will then use it to keep
228connectivity with at least one node at any point in time. 327connectivity with at least one node at any point in time.
229 328
230=back 329=back
231 330
232Example: become a distributed node using the local node name as profile. 331Example: become a distributed node using the local node name as profile.
233This should be the most common form of invocation for "daemon"-type nodes. 332This should be the most common form of invocation for "daemon"-type nodes.
234 333
235 configure 334 configure
236 335
237Example: become an anonymous node. This form is often used for commandline 336Example: become a semi-anonymous node. This form is often used for
238clients. 337commandline clients.
239 338
240 configure nodeid => "anon/"; 339 configure nodeid => "myscript/%n/%u";
241 340
242Example: configure a node using a profile called seed, which si suitable 341Example: configure a node using a profile called seed, which is suitable
243for a seed node as it binds on all local addresses on a fixed port (4040, 342for a seed node as it binds on all local addresses on a fixed port (4040,
244customary for aemp). 343customary for aemp).
245 344
246 # use the aemp commandline utility 345 # use the aemp commandline utility
247 # aemp profile seed nodeid anon/ binds '*:4040' 346 # aemp profile seed binds '*:4040'
248 347
249 # then use it 348 # then use it
250 configure profile => "seed"; 349 configure profile => "seed";
251 350
252 # or simply use aemp from the shell again: 351 # or simply use aemp from the shell again:
317 416
318=cut 417=cut
319 418
320sub rcv($@); 419sub rcv($@);
321 420
322sub _kilme { 421my $KILME = sub {
323 die "received message on port without callback"; 422 (my $tag = substr $_[0], 0, 30) =~ s/([^\x20-\x7e])/./g;
324} 423 kil $SELF, unhandled_message => "no callback found for message '$tag'";
424};
325 425
326sub port(;&) { 426sub port(;&) {
327 my $id = "$UNIQ." . $ID++; 427 my $id = $UNIQ . ++$ID;
328 my $port = "$NODE#$id"; 428 my $port = "$NODE#$id";
329 429
330 rcv $port, shift || \&_kilme; 430 rcv $port, shift || $KILME;
331 431
332 $port 432 $port
333} 433}
334 434
335=item rcv $local_port, $callback->(@msg) 435=item rcv $local_port, $callback->(@msg)
340 440
341The global C<$SELF> (exported by this module) contains C<$port> while 441The global C<$SELF> (exported by this module) contains C<$port> while
342executing the callback. Runtime errors during callback execution will 442executing the callback. Runtime errors during callback execution will
343result in the port being C<kil>ed. 443result in the port being C<kil>ed.
344 444
345The default callback received all messages not matched by a more specific 445The default callback receives all messages not matched by a more specific
346C<tag> match. 446C<tag> match.
347 447
348=item rcv $local_port, tag => $callback->(@msg_without_tag), ... 448=item rcv $local_port, tag => $callback->(@msg_without_tag), ...
349 449
350Register (or replace) callbacks to be called on messages starting with the 450Register (or replace) callbacks to be called on messages starting with the
371 msg1 => sub { ... }, 471 msg1 => sub { ... },
372 ... 472 ...
373 ; 473 ;
374 474
375Example: temporarily register a rcv callback for a tag matching some port 475Example: temporarily register a rcv callback for a tag matching some port
376(e.g. for a rpc reply) and unregister it after a message was received. 476(e.g. for an rpc reply) and unregister it after a message was received.
377 477
378 rcv $port, $otherport => sub { 478 rcv $port, $otherport => sub {
379 my @reply = @_; 479 my @reply = @_;
380 480
381 rcv $SELF, $otherport; 481 rcv $SELF, $otherport;
385 485
386sub rcv($@) { 486sub rcv($@) {
387 my $port = shift; 487 my $port = shift;
388 my ($nodeid, $portid) = split /#/, $port, 2; 488 my ($nodeid, $portid) = split /#/, $port, 2;
389 489
390 $NODE{$nodeid} == $NODE{""} 490 $nodeid eq $NODE
391 or Carp::croak "$port: rcv can only be called on local ports, caught"; 491 or Carp::croak "$port: rcv can only be called on local ports, caught";
392 492
393 while (@_) { 493 while (@_) {
394 if (ref $_[0]) { 494 if (ref $_[0]) {
395 if (my $self = $PORT_DATA{$portid}) { 495 if (my $self = $PORT_DATA{$portid}) {
396 "AnyEvent::MP::Port" eq ref $self 496 "AnyEvent::MP::Port" eq ref $self
397 or Carp::croak "$port: rcv can only be called on message matching ports, caught"; 497 or Carp::croak "$port: rcv can only be called on message matching ports, caught";
398 498
399 $self->[2] = shift; 499 $self->[0] = shift;
400 } else { 500 } else {
401 my $cb = shift; 501 my $cb = shift;
402 $PORT{$portid} = sub { 502 $PORT{$portid} = sub {
403 local $SELF = $port; 503 local $SELF = $port;
404 eval { &$cb }; _self_die if $@; 504 eval { &$cb }; _self_die if $@;
405 }; 505 };
406 } 506 }
407 } elsif (defined $_[0]) { 507 } elsif (defined $_[0]) {
408 my $self = $PORT_DATA{$portid} ||= do { 508 my $self = $PORT_DATA{$portid} ||= do {
409 my $self = bless [$PORT{$port} || sub { }, { }, $port], "AnyEvent::MP::Port"; 509 my $self = bless [$PORT{$portid} || sub { }, { }, $port], "AnyEvent::MP::Port";
410 510
411 $PORT{$portid} = sub { 511 $PORT{$portid} = sub {
412 local $SELF = $port; 512 local $SELF = $port;
413 513
414 if (my $cb = $self->[1]{$_[0]}) { 514 if (my $cb = $self->[1]{$_[0]}) {
436 } 536 }
437 537
438 $port 538 $port
439} 539}
440 540
541=item peval $port, $coderef[, @args]
542
543Evaluates the given C<$codref> within the context of C<$port>, that is,
544when the code throws an exception the C<$port> will be killed.
545
546Any remaining args will be passed to the callback. Any return values will
547be returned to the caller.
548
549This is useful when you temporarily want to execute code in the context of
550a port.
551
552Example: create a port and run some initialisation code in it's context.
553
554 my $port = port { ... };
555
556 peval $port, sub {
557 init
558 or die "unable to init";
559 };
560
561=cut
562
563sub peval($$) {
564 local $SELF = shift;
565 my $cb = shift;
566
567 if (wantarray) {
568 my @res = eval { &$cb };
569 _self_die if $@;
570 @res
571 } else {
572 my $res = eval { &$cb };
573 _self_die if $@;
574 $res
575 }
576}
577
441=item $closure = psub { BLOCK } 578=item $closure = psub { BLOCK }
442 579
443Remembers C<$SELF> and creates a closure out of the BLOCK. When the 580Remembers C<$SELF> and creates a closure out of the BLOCK. When the
444closure is executed, sets up the environment in the same way as in C<rcv> 581closure is executed, sets up the environment in the same way as in C<rcv>
445callbacks, i.e. runtime errors will cause the port to get C<kil>ed. 582callbacks, i.e. runtime errors will cause the port to get C<kil>ed.
583
584The effect is basically as if it returned C<< sub { peval $SELF, sub {
585BLOCK }, @_ } >>.
446 586
447This is useful when you register callbacks from C<rcv> callbacks: 587This is useful when you register callbacks from C<rcv> callbacks:
448 588
449 rcv delayed_reply => sub { 589 rcv delayed_reply => sub {
450 my ($delay, @reply) = @_; 590 my ($delay, @reply) = @_;
474 $res 614 $res
475 } 615 }
476 } 616 }
477} 617}
478 618
619=item $guard = mon $port, $rcvport # kill $rcvport when $port dies
620
621=item $guard = mon $port # kill $SELF when $port dies
622
479=item $guard = mon $port, $cb->(@reason) # call $cb when $port dies 623=item $guard = mon $port, $cb->(@reason) # call $cb when $port dies
480
481=item $guard = mon $port, $rcvport # kill $rcvport when $port dies
482
483=item $guard = mon $port # kill $SELF when $port dies
484 624
485=item $guard = mon $port, $rcvport, @msg # send a message when $port dies 625=item $guard = mon $port, $rcvport, @msg # send a message when $port dies
486 626
487Monitor the given port and do something when the port is killed or 627Monitor the given port and do something when the port is killed or
488messages to it were lost, and optionally return a guard that can be used 628messages to it were lost, and optionally return a guard that can be used
489to stop monitoring again. 629to stop monitoring again.
490 630
631The first two forms distinguish between "normal" and "abnormal" kil's:
632
633In the first form (another port given), if the C<$port> is C<kil>'ed with
634a non-empty reason, the other port (C<$rcvport>) will be kil'ed with the
635same reason. That is, on "normal" kil's nothing happens, while under all
636other conditions, the other port is killed with the same reason.
637
638The second form (kill self) is the same as the first form, except that
639C<$rvport> defaults to C<$SELF>.
640
641The remaining forms don't distinguish between "normal" and "abnormal" kil's
642- it's up to the callback or receiver to check whether the C<@reason> is
643empty and act accordingly.
644
491In the first form (callback), the callback is simply called with any 645In the third form (callback), the callback is simply called with any
492number of C<@reason> elements (no @reason means that the port was deleted 646number of C<@reason> elements (empty @reason means that the port was deleted
493"normally"). Note also that I<< the callback B<must> never die >>, so use 647"normally"). Note also that I<< the callback B<must> never die >>, so use
494C<eval> if unsure. 648C<eval> if unsure.
495 649
496In the second form (another port given), the other port (C<$rcvport>)
497will be C<kil>'ed with C<@reason>, if a @reason was specified, i.e. on
498"normal" kils nothing happens, while under all other conditions, the other
499port is killed with the same reason.
500
501The third form (kill self) is the same as the second form, except that
502C<$rvport> defaults to C<$SELF>.
503
504In the last form (message), a message of the form C<@msg, @reason> will be 650In the last form (message), a message of the form C<$rcvport, @msg,
505C<snd>. 651@reason> will be C<snd>.
506 652
507Monitoring-actions are one-shot: once messages are lost (and a monitoring 653Monitoring-actions are one-shot: once messages are lost (and a monitoring
508alert was raised), they are removed and will not trigger again. 654alert was raised), they are removed and will not trigger again, even if it
655turns out that the port is still alive.
509 656
510As a rule of thumb, monitoring requests should always monitor a port from 657As a rule of thumb, monitoring requests should always monitor a remote
511a local port (or callback). The reason is that kill messages might get 658port locally (using a local C<$rcvport> or a callback). The reason is that
512lost, just like any other message. Another less obvious reason is that 659kill messages might get lost, just like any other message. Another less
513even monitoring requests can get lost (for example, when the connection 660obvious reason is that even monitoring requests can get lost (for example,
514to the other node goes down permanently). When monitoring a port locally 661when the connection to the other node goes down permanently). When
515these problems do not exist. 662monitoring a port locally these problems do not exist.
516 663
517C<mon> effectively guarantees that, in the absence of hardware failures, 664C<mon> effectively guarantees that, in the absence of hardware failures,
518after starting the monitor, either all messages sent to the port will 665after starting the monitor, either all messages sent to the port will
519arrive, or the monitoring action will be invoked after possible message 666arrive, or the monitoring action will be invoked after possible message
520loss has been detected. No messages will be lost "in between" (after 667loss has been detected. No messages will be lost "in between" (after
565 } 712 }
566 713
567 $node->monitor ($port, $cb); 714 $node->monitor ($port, $cb);
568 715
569 defined wantarray 716 defined wantarray
570 and ($cb += 0, AnyEvent::Util::guard { $node->unmonitor ($port, $cb) }) 717 and ($cb += 0, Guard::guard { $node->unmonitor ($port, $cb) })
571} 718}
572 719
573=item $guard = mon_guard $port, $ref, $ref... 720=item $guard = mon_guard $port, $ref, $ref...
574 721
575Monitors the given C<$port> and keeps the passed references. When the port 722Monitors the given C<$port> and keeps the passed references. When the port
598 745
599=item kil $port[, @reason] 746=item kil $port[, @reason]
600 747
601Kill the specified port with the given C<@reason>. 748Kill the specified port with the given C<@reason>.
602 749
603If no C<@reason> is specified, then the port is killed "normally" (ports 750If no C<@reason> is specified, then the port is killed "normally" -
604monitoring other ports will not necessarily die because a port dies 751monitor callback will be invoked, but the kil will not cause linked ports
605"normally"). 752(C<mon $mport, $lport> form) to get killed.
606 753
607Otherwise, linked ports get killed with the same reason (second form of 754If a C<@reason> is specified, then linked ports (C<mon $mport, $lport>
608C<mon>, see above). 755form) get killed with the same reason.
609 756
610Runtime errors while evaluating C<rcv> callbacks or inside C<psub> blocks 757Runtime errors while evaluating C<rcv> callbacks or inside C<psub> blocks
611will be reported as reason C<< die => $@ >>. 758will be reported as reason C<< die => $@ >>.
612 759
613Transport/communication errors are reported as C<< transport_error => 760Transport/communication errors are reported as C<< transport_error =>
614$message >>. 761$message >>.
615 762
616=cut 763Common idioms:
764
765 # silently remove yourself, do not kill linked ports
766 kil $SELF;
767
768 # report a failure in some detail
769 kil $SELF, failure_mode_1 => "it failed with too high temperature";
770
771 # do not waste much time with killing, just die when something goes wrong
772 open my $fh, "<file"
773 or die "file: $!";
617 774
618=item $port = spawn $node, $initfunc[, @initdata] 775=item $port = spawn $node, $initfunc[, @initdata]
619 776
620Creates a port on the node C<$node> (which can also be a port ID, in which 777Creates a port on the node C<$node> (which can also be a port ID, in which
621case it's the node where that port resides). 778case it's the node where that port resides).
679} 836}
680 837
681sub spawn(@) { 838sub spawn(@) {
682 my ($nodeid, undef) = split /#/, shift, 2; 839 my ($nodeid, undef) = split /#/, shift, 2;
683 840
684 my $id = "$RUNIQ." . $ID++; 841 my $id = $RUNIQ . ++$ID;
685 842
686 $_[0] =~ /::/ 843 $_[0] =~ /::/
687 or Carp::croak "spawn init function must be a fully-qualified name, caught"; 844 or Carp::croak "spawn init function must be a fully-qualified name, caught";
688 845
689 snd_to_func $nodeid, "AnyEvent::MP::_spawn" => $id, @_; 846 snd_to_func $nodeid, "AnyEvent::MP::_spawn" => $id, @_;
690 847
691 "$nodeid#$id" 848 "$nodeid#$id"
692} 849}
850
693 851
694=item after $timeout, @msg 852=item after $timeout, @msg
695 853
696=item after $timeout, $callback 854=item after $timeout, $callback
697 855
712 ref $action[0] 870 ref $action[0]
713 ? $action[0]() 871 ? $action[0]()
714 : snd @action; 872 : snd @action;
715 }; 873 };
716} 874}
875
876#=item $cb2 = timeout $seconds, $cb[, @args]
717 877
718=item cal $port, @msg, $callback[, $timeout] 878=item cal $port, @msg, $callback[, $timeout]
719 879
720A simple form of RPC - sends a message to the given C<$port> with the 880A simple form of RPC - sends a message to the given C<$port> with the
721given contents (C<@msg>), but adds a reply port to the message. 881given contents (C<@msg>), but adds a reply port to the message.
767 $port 927 $port
768} 928}
769 929
770=back 930=back
771 931
932=head1 DISTRIBUTED DATABASE
933
934AnyEvent::MP comes with a simple distributed database. The database will
935be mirrored asynchronously on all global nodes. Other nodes bind to one
936of the global nodes for their needs. Every node has a "local database"
937which contains all the values that are set locally. All local databases
938are merged together to form the global database, which can be queried.
939
940The database structure is that of a two-level hash - the database hash
941contains hashes which contain values, similarly to a perl hash of hashes,
942i.e.:
943
944 $DATABASE{$family}{$subkey} = $value
945
946The top level hash key is called "family", and the second-level hash key
947is called "subkey" or simply "key".
948
949The family must be alphanumeric, i.e. start with a letter and consist
950of letters, digits, underscores and colons (C<[A-Za-z][A-Za-z0-9_:]*>,
951pretty much like Perl module names.
952
953As the family namespace is global, it is recommended to prefix family names
954with the name of the application or module using it.
955
956The subkeys must be non-empty strings, with no further restrictions.
957
958The values should preferably be strings, but other perl scalars should
959work as well (such as C<undef>, arrays and hashes).
960
961Every database entry is owned by one node - adding the same family/subkey
962combination on multiple nodes will not cause discomfort for AnyEvent::MP,
963but the result might be nondeterministic, i.e. the key might have
964different values on different nodes.
965
966Different subkeys in the same family can be owned by different nodes
967without problems, and in fact, this is the common method to create worker
968pools. For example, a worker port for image scaling might do this:
969
970 db_set my_image_scalers => $port;
971
972And clients looking for an image scaler will want to get the
973C<my_image_scalers> keys from time to time:
974
975 db_keys my_image_scalers => sub {
976 @ports = @{ $_[0] };
977 };
978
979Or better yet, they want to monitor the database family, so they always
980have a reasonable up-to-date copy:
981
982 db_mon my_image_scalers => sub {
983 @ports = keys %{ $_[0] };
984 };
985
986In general, you can set or delete single subkeys, but query and monitor
987whole families only.
988
989If you feel the need to monitor or query a single subkey, try giving it
990it's own family.
991
992=over
993
994=item $guard = db_set $family => $subkey [=> $value]
995
996Sets (or replaces) a key to the database - if C<$value> is omitted,
997C<undef> is used instead.
998
999When called in non-void context, C<db_set> returns a guard that
1000automatically calls C<db_del> when it is destroyed.
1001
1002=item db_del $family => $subkey...
1003
1004Deletes one or more subkeys from the database family.
1005
1006=item $guard = db_reg $family => $port => $value
1007
1008=item $guard = db_reg $family => $port
1009
1010=item $guard = db_reg $family
1011
1012Registers a port in the given family and optionally returns a guard to
1013remove it.
1014
1015This function basically does the same as:
1016
1017 db_set $family => $port => $value
1018
1019Except that the port is monitored and automatically removed from the
1020database family when it is kil'ed.
1021
1022If C<$value> is missing, C<undef> is used. If C<$port> is missing, then
1023C<$SELF> is used.
1024
1025This function is most useful to register a port in some port group (which
1026is just another name for a database family), and have it removed when the
1027port is gone. This works best when the port is a local port.
1028
1029=cut
1030
1031sub db_reg($$;$) {
1032 my $family = shift;
1033 my $port = @_ ? shift : $SELF;
1034
1035 my $clr = sub { db_del $family => $port };
1036 mon $port, $clr;
1037
1038 db_set $family => $port => $_[0];
1039
1040 defined wantarray
1041 and &Guard::guard ($clr)
1042}
1043
1044=item db_family $family => $cb->(\%familyhash)
1045
1046Queries the named database C<$family> and call the callback with the
1047family represented as a hash. You can keep and freely modify the hash.
1048
1049=item db_keys $family => $cb->(\@keys)
1050
1051Same as C<db_family>, except it only queries the family I<subkeys> and passes
1052them as array reference to the callback.
1053
1054=item db_values $family => $cb->(\@values)
1055
1056Same as C<db_family>, except it only queries the family I<values> and passes them
1057as array reference to the callback.
1058
1059=item $guard = db_mon $family => $cb->(\%familyhash, \@added, \@changed, \@deleted)
1060
1061Creates a monitor on the given database family. Each time a key is
1062set or is deleted the callback is called with a hash containing the
1063database family and three lists of added, changed and deleted subkeys,
1064respectively. If no keys have changed then the array reference might be
1065C<undef> or even missing.
1066
1067If not called in void context, a guard object is returned that, when
1068destroyed, stops the monitor.
1069
1070The family hash reference and the key arrays belong to AnyEvent::MP and
1071B<must not be modified or stored> by the callback. When in doubt, make a
1072copy.
1073
1074As soon as possible after the monitoring starts, the callback will be
1075called with the intiial contents of the family, even if it is empty,
1076i.e. there will always be a timely call to the callback with the current
1077contents.
1078
1079It is possible that the callback is called with a change event even though
1080the subkey is already present and the value has not changed.
1081
1082The monitoring stops when the guard object is destroyed.
1083
1084Example: on every change to the family "mygroup", print out all keys.
1085
1086 my $guard = db_mon mygroup => sub {
1087 my ($family, $a, $c, $d) = @_;
1088 print "mygroup members: ", (join " ", keys %$family), "\n";
1089 };
1090
1091Exmaple: wait until the family "My::Module::workers" is non-empty.
1092
1093 my $guard; $guard = db_mon My::Module::workers => sub {
1094 my ($family, $a, $c, $d) = @_;
1095 return unless %$family;
1096 undef $guard;
1097 print "My::Module::workers now nonempty\n";
1098 };
1099
1100Example: print all changes to the family "AnyEvent::Fantasy::Module".
1101
1102 my $guard = db_mon AnyEvent::Fantasy::Module => sub {
1103 my ($family, $a, $c, $d) = @_;
1104
1105 print "+$_=$family->{$_}\n" for @$a;
1106 print "*$_=$family->{$_}\n" for @$c;
1107 print "-$_=$family->{$_}\n" for @$d;
1108 };
1109
1110=cut
1111
1112=back
1113
772=head1 AnyEvent::MP vs. Distributed Erlang 1114=head1 AnyEvent::MP vs. Distributed Erlang
773 1115
774AnyEvent::MP got lots of its ideas from distributed Erlang (Erlang node 1116AnyEvent::MP got lots of its ideas from distributed Erlang (Erlang node
775== aemp node, Erlang process == aemp port), so many of the documents and 1117== aemp node, Erlang process == aemp port), so many of the documents and
776programming techniques employed by Erlang apply to AnyEvent::MP. Here is a 1118programming techniques employed by Erlang apply to AnyEvent::MP. Here is a
807ports being the special case/exception, where transport errors cannot 1149ports being the special case/exception, where transport errors cannot
808occur. 1150occur.
809 1151
810=item * Erlang uses processes and a mailbox, AEMP does not queue. 1152=item * Erlang uses processes and a mailbox, AEMP does not queue.
811 1153
812Erlang uses processes that selectively receive messages, and therefore 1154Erlang uses processes that selectively receive messages out of order, and
813needs a queue. AEMP is event based, queuing messages would serve no 1155therefore needs a queue. AEMP is event based, queuing messages would serve
814useful purpose. For the same reason the pattern-matching abilities of 1156no useful purpose. For the same reason the pattern-matching abilities
815AnyEvent::MP are more limited, as there is little need to be able to 1157of AnyEvent::MP are more limited, as there is little need to be able to
816filter messages without dequeuing them. 1158filter messages without dequeuing them.
817 1159
818(But see L<Coro::MP> for a more Erlang-like process model on top of AEMP). 1160This is not a philosophical difference, but simply stems from AnyEvent::MP
1161being event-based, while Erlang is process-based.
1162
1163You can have a look at L<Coro::MP> for a more Erlang-like process model on
1164top of AEMP and Coro threads.
819 1165
820=item * Erlang sends are synchronous, AEMP sends are asynchronous. 1166=item * Erlang sends are synchronous, AEMP sends are asynchronous.
821 1167
822Sending messages in Erlang is synchronous and blocks the process (and 1168Sending messages in Erlang is synchronous and blocks the process until
1169a connection has been established and the message sent (and so does not
823so does not need a queue that can overflow). AEMP sends are immediate, 1170need a queue that can overflow). AEMP sends return immediately, connection
824connection establishment is handled in the background. 1171establishment is handled in the background.
825 1172
826=item * Erlang suffers from silent message loss, AEMP does not. 1173=item * Erlang suffers from silent message loss, AEMP does not.
827 1174
828Erlang implements few guarantees on messages delivery - messages can get 1175Erlang implements few guarantees on messages delivery - messages can get
829lost without any of the processes realising it (i.e. you send messages a, 1176lost without any of the processes realising it (i.e. you send messages a,
830b, and c, and the other side only receives messages a and c). 1177b, and c, and the other side only receives messages a and c).
831 1178
832AEMP guarantees correct ordering, and the guarantee that after one message 1179AEMP guarantees (modulo hardware errors) correct ordering, and the
833is lost, all following ones sent to the same port are lost as well, until 1180guarantee that after one message is lost, all following ones sent to the
834monitoring raises an error, so there are no silent "holes" in the message 1181same port are lost as well, until monitoring raises an error, so there are
835sequence. 1182no silent "holes" in the message sequence.
1183
1184If you want your software to be very reliable, you have to cope with
1185corrupted and even out-of-order messages in both Erlang and AEMP. AEMP
1186simply tries to work better in common error cases, such as when a network
1187link goes down.
836 1188
837=item * Erlang can send messages to the wrong port, AEMP does not. 1189=item * Erlang can send messages to the wrong port, AEMP does not.
838 1190
839In Erlang it is quite likely that a node that restarts reuses a process ID 1191In Erlang it is quite likely that a node that restarts reuses an Erlang
840known to other nodes for a completely different process, causing messages 1192process ID known to other nodes for a completely different process,
841destined for that process to end up in an unrelated process. 1193causing messages destined for that process to end up in an unrelated
1194process.
842 1195
843AEMP never reuses port IDs, so old messages or old port IDs floating 1196AEMP does not reuse port IDs, so old messages or old port IDs floating
844around in the network will not be sent to an unrelated port. 1197around in the network will not be sent to an unrelated port.
845 1198
846=item * Erlang uses unprotected connections, AEMP uses secure 1199=item * Erlang uses unprotected connections, AEMP uses secure
847authentication and can use TLS. 1200authentication and can use TLS.
848 1201
851 1204
852=item * The AEMP protocol is optimised for both text-based and binary 1205=item * The AEMP protocol is optimised for both text-based and binary
853communications. 1206communications.
854 1207
855The AEMP protocol, unlike the Erlang protocol, supports both programming 1208The AEMP protocol, unlike the Erlang protocol, supports both programming
856language independent text-only protocols (good for debugging) and binary, 1209language independent text-only protocols (good for debugging), and binary,
857language-specific serialisers (e.g. Storable). By default, unless TLS is 1210language-specific serialisers (e.g. Storable). By default, unless TLS is
858used, the protocol is actually completely text-based. 1211used, the protocol is actually completely text-based.
859 1212
860It has also been carefully designed to be implementable in other languages 1213It has also been carefully designed to be implementable in other languages
861with a minimum of work while gracefully degrading functionality to make the 1214with a minimum of work while gracefully degrading functionality to make the
862protocol simple. 1215protocol simple.
863 1216
864=item * AEMP has more flexible monitoring options than Erlang. 1217=item * AEMP has more flexible monitoring options than Erlang.
865 1218
866In Erlang, you can chose to receive I<all> exit signals as messages 1219In Erlang, you can chose to receive I<all> exit signals as messages or
867or I<none>, there is no in-between, so monitoring single processes is 1220I<none>, there is no in-between, so monitoring single Erlang processes is
868difficult to implement. Monitoring in AEMP is more flexible than in 1221difficult to implement.
869Erlang, as one can choose between automatic kill, exit message or callback 1222
870on a per-process basis. 1223Monitoring in AEMP is more flexible than in Erlang, as one can choose
1224between automatic kill, exit message or callback on a per-port basis.
871 1225
872=item * Erlang tries to hide remote/local connections, AEMP does not. 1226=item * Erlang tries to hide remote/local connections, AEMP does not.
873 1227
874Monitoring in Erlang is not an indicator of process death/crashes, in the 1228Monitoring in Erlang is not an indicator of process death/crashes, in the
875same way as linking is (except linking is unreliable in Erlang). 1229same way as linking is (except linking is unreliable in Erlang).
897overhead, as well as having to keep a proxy object everywhere. 1251overhead, as well as having to keep a proxy object everywhere.
898 1252
899Strings can easily be printed, easily serialised etc. and need no special 1253Strings can easily be printed, easily serialised etc. and need no special
900procedures to be "valid". 1254procedures to be "valid".
901 1255
902And as a result, a miniport consists of a single closure stored in a 1256And as a result, a port with just a default receiver consists of a single
903global hash - it can't become much cheaper. 1257code reference stored in a global hash - it can't become much cheaper.
904 1258
905=item Why favour JSON, why not a real serialising format such as Storable? 1259=item Why favour JSON, why not a real serialising format such as Storable?
906 1260
907In fact, any AnyEvent::MP node will happily accept Storable as framing 1261In fact, any AnyEvent::MP node will happily accept Storable as framing
908format, but currently there is no way to make a node use Storable by 1262format, but currently there is no way to make a node use Storable by
918Keeping your messages simple, concentrating on data structures rather than 1272Keeping your messages simple, concentrating on data structures rather than
919objects, will keep your messages clean, tidy and efficient. 1273objects, will keep your messages clean, tidy and efficient.
920 1274
921=back 1275=back
922 1276
1277=head1 PORTING FROM AnyEvent::MP VERSION 1.X
1278
1279AEMP version 2 has a few major incompatible changes compared to version 1:
1280
1281=over 4
1282
1283=item AnyEvent::MP::Global no longer has group management functions.
1284
1285At least not officially - the grp_* functions are still exported and might
1286work, but they will be removed in some later release.
1287
1288AnyEvent::MP now comes with a distributed database that is more
1289powerful. Its database families map closely to port groups, but the API
1290has changed (the functions are also now exported by AnyEvent::MP). Here is
1291a rough porting guide:
1292
1293 grp_reg $group, $port # old
1294 db_reg $group, $port # new
1295
1296 $list = grp_get $group # old
1297 db_keys $group, sub { my $list = shift } # new
1298
1299 grp_mon $group, $cb->(\@ports, $add, $del) # old
1300 db_mon $group, $cb->(\%ports, $add, $change, $del) # new
1301
1302C<grp_reg> is a no-brainer (just replace by C<db_reg>), but C<grp_get> is
1303no longer instant, because the local node might not have a copy of the
1304group. You can either modify your code to allow for a callback, or use
1305C<db_mon> to keep an updated copy of the group:
1306
1307 my $local_group_copy;
1308 db_mon $group => sub { $local_group_copy = $_[0] };
1309
1310 # now "keys %$local_group_copy" always returns the most up-to-date
1311 # list of ports in the group.
1312
1313C<grp_mon> can be replaced by C<db_mon> with minor changes - C<db_mon>
1314passes a hash as first argument, and an extra C<$chg> argument that can be
1315ignored:
1316
1317 db_mon $group => sub {
1318 my ($ports, $add, $chg, $del) = @_;
1319 $ports = [keys %$ports];
1320
1321 # now $ports, $add and $del are the same as
1322 # were originally passed by grp_mon.
1323 ...
1324 };
1325
1326=item Nodes not longer connect to all other nodes.
1327
1328In AEMP 1.x, every node automatically loads the L<AnyEvent::MP::Global>
1329module, which in turn would create connections to all other nodes in the
1330network (helped by the seed nodes).
1331
1332In version 2.x, global nodes still connect to all other global nodes, but
1333other nodes don't - now every node either is a global node itself, or
1334attaches itself to another global node.
1335
1336If a node isn't a global node itself, then it attaches itself to one
1337of its seed nodes. If that seed node isn't a global node yet, it will
1338automatically be upgraded to a global node.
1339
1340So in many cases, nothing needs to be changed - one just has to make sure
1341that all seed nodes are meshed together with the other seed nodes (as with
1342AEMP 1.x), and other nodes specify them as seed nodes. This is most easily
1343achieved by specifying the same set of seed nodes for all nodes in the
1344network.
1345
1346Not opening a connection to every other node is usually an advantage,
1347except when you need the lower latency of an already established
1348connection. To ensure a node establishes a connection to another node,
1349you can monitor the node port (C<mon $node, ...>), which will attempt to
1350create the connection (and notify you when the connection fails).
1351
1352=item Listener-less nodes (nodes without binds) are gone.
1353
1354And are not coming back, at least not in their old form. If no C<binds>
1355are specified for a node, AnyEvent::MP assumes a default of C<*:*>.
1356
1357There are vague plans to implement some form of routing domains, which
1358might or might not bring back listener-less nodes, but don't count on it.
1359
1360The fact that most connections are now optional somewhat mitigates this,
1361as a node can be effectively unreachable from the outside without any
1362problems, as long as it isn't a global node and only reaches out to other
1363nodes (as opposed to being contacted from other nodes).
1364
1365=item $AnyEvent::MP::Kernel::WARN has gone.
1366
1367AnyEvent has acquired a logging framework (L<AnyEvent::Log>), and AEMP now
1368uses this, and so should your programs.
1369
1370Every module now documents what kinds of messages it generates, with
1371AnyEvent::MP acting as a catch all.
1372
1373On the positive side, this means that instead of setting
1374C<PERL_ANYEVENT_MP_WARNLEVEL>, you can get away by setting C<AE_VERBOSE> -
1375much less to type.
1376
1377=back
1378
1379=head1 LOGGING
1380
1381AnyEvent::MP does not normally log anything by itself, but since it is the
1382root of the context hierarchy for AnyEvent::MP modules, it will receive
1383all log messages by submodules.
1384
923=head1 SEE ALSO 1385=head1 SEE ALSO
924 1386
925L<AnyEvent::MP::Intro> - a gentle introduction. 1387L<AnyEvent::MP::Intro> - a gentle introduction.
926 1388
927L<AnyEvent::MP::Kernel> - more, lower-level, stuff. 1389L<AnyEvent::MP::Kernel> - more, lower-level, stuff.
928 1390
929L<AnyEvent::MP::Global> - network maintainance and port groups, to find 1391L<AnyEvent::MP::Global> - network maintenance and port groups, to find
930your applications. 1392your applications.
1393
1394L<AnyEvent::MP::DataConn> - establish data connections between nodes.
931 1395
932L<AnyEvent::MP::LogCatcher> - simple service to display log messages from 1396L<AnyEvent::MP::LogCatcher> - simple service to display log messages from
933all nodes. 1397all nodes.
934 1398
935L<AnyEvent>. 1399L<AnyEvent>.

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines