ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.22
Committed: Fri Aug 28 17:03:02 2009 UTC (14 years, 9 months ago) by elmex
Branch: MAIN
Changes since 1.21: +2 -1 lines
Log Message:
small fix.

File Contents

# User Rev Content
1 root 1.4 =head1 Message Passing for the Non-Blocked Mind
2 elmex 1.1
3 root 1.8 =head1 Introduction and Terminology
4 elmex 1.1
5 root 1.4 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 root 1.14 module, which allows us to transparently pass messages to our own process
7 root 1.8 and to other processes on another or the same host.
8 elmex 1.1
9 root 1.15 What kind of messages? Well, basically a message here means a list of Perl
10     strings, numbers, hashes and arrays, anything that can be expressed as a
11 elmex 1.21 L<JSON> text (as JSON is used by default in the protocol). Here is an example:
12 elmex 1.1
13 elmex 1.21 [123, ["a", "b", "c"], { foo => 'bar' }]
14    
15     When using L<AnyEvent::MP> you should usually use a string as first element
16     of a message, which describes the type of the
17     message. That element is called a I<tag> in L<AnyEvent::MP>, as some
18 elmex 1.17 API functions (C<rcv>) support matching it directly. So supposedly you want to
19     send a ping message with your current time to something, this is how such a
20     message might look like (in Perl syntax):
21 elmex 1.1
22 elmex 1.17 ['ping', 1251381636]
23 elmex 1.7
24 elmex 1.21 The next interesting question is: Between which entities are those messages
25     being I<passed>? They are I<passed> between I<ports>. I<ports> are just sources
26     and destinations for messages. How do these ports relate to things you know?
27     Well, each I<port> belongs to a I<node>, and a I<node> is just the UNIX process
28     that runs your L<AnyEvent::MP> application.
29 elmex 1.17
30 elmex 1.21 Each I<node> is distinguished from other I<nodes>, that run on the same host or
31     multiple hosts in a network, by it's I<node ID>. A I<node ID> can be manually
32 elmex 1.17 assigned or L<AnyEvent::MP> will assign one it self for you.
33    
34 elmex 1.21 Here is a visualisation about how I<nodes>, I<ports> and UNIX processes
35     relate to each other. The setup consists of two nodes (more are of
36     course possible): Node C<A> (in UNIX process 7066) with the ports C<ABC> and C<DEF>.
37     And the node C<B> (in UNIX process 8321) with the ports C<FOO> and C<BAR>.
38 elmex 1.17
39    
40     |- PID: 7066 -| |- PID: 8321 -|
41     | | | |
42     | Node ID: A | | Node ID: B |
43     | | | |
44     | Port ABC =|= <----\ /-----> =|= Port FOO |
45     | | X | |
46     | Port DEF =|= <----/ \-----> =|= Port BAR |
47     | | | |
48     |-------------| |-------------|
49    
50 elmex 1.21 The strings for the I<port ids> here are just for illustrative purposes. Even
51     if in reality I<ports> in L<AnyEvent::MP> are also identified by strings they
52     can't be choosen manually and are assigned randomly. These I<port ids> should
53     also not be used directly for other purposes than referring to an endpoint for
54 elmex 1.17 messages.
55    
56 elmex 1.21 The next sections will explain the API of L<AnyEvent::MP> by going through a
57     few simple examples. Later some more complex idioms are introduced, which are
58     maybe useful to solve some real world problems.
59 root 1.8
60 elmex 1.16 =head1 Passing Your First Message
61    
62     As start lets have a look at the messaging API. The next example is just a
63     demo to show the basic elements of message passing with L<AnyEvent::MP>.
64     It shout just print: "Ending with: 123". So here the code:
65    
66     use AnyEvent;
67     use AnyEvent::MP;
68    
69     my $end_cv = AnyEvent->condvar;
70    
71     my $port = port;
72    
73     rcv $port, test => sub {
74     my ($data) = @_;
75     $end_cv->send ($data);
76     };
77    
78     snd $port, test => 123;
79    
80     print "Ending with: " . $end_cv->recv . "\n";
81    
82 elmex 1.17 It already contains most functions of the essential L<AnyEvent::MP> API.
83    
84     First there is the C<port> function which will create a I<port> and will return
85     it's I<port id>.
86    
87     That I<port id> can be used to send and receive messages. That I<port id> is a
88     simple string and can be safely passed to other I<nodes> in the network to
89     refer to that specific port (usually used for RPC, where you need to
90     tell the other end which I<port> to send the reply to).
91    
92     Next function is C<rcv>:
93 elmex 1.16
94 elmex 1.17 rcv $port, test => sub { ... };
95 elmex 1.16
96 elmex 1.17 It sets up a receiver callback on a specific I<port> which needs to be
97     specified as the first argument. The next argument, in this example C<test>, is
98     a I<tag> match. This means that whenever a message, with the first element
99     being the string C<tag>, is received the callback is called with the remaining
100     parts of that message.
101    
102     Messages can be send with the C<snd> function, which looks like this in the
103     example above:
104    
105     snd $port, test => 123;
106    
107     This will send the message C<['test', 123]> to the I<port> with the I<port id>
108     in C<$port>. The receiver got a I<tag> match on C<test> and will call the
109     callback with the first argument being the number C<123>.
110    
111     That callback then just passes that number on to the I<condition variable>
112     C<$end_cv> which will then pass the value to the print. But I<condition
113     variables> are out of the scope of this tutorial. So please consult the
114     L<AnyEvent::Intro> about them.
115    
116     But passing messages inside one process is boring, but before we can continue
117     and take the next step to interprocess message passing we first have to make
118     sure some things have been setup.
119    
120     =head1 System Requirements and System Setup
121    
122     Before we can start with real IPC we have to make sure some things work on your
123     system.
124    
125     First we have to setup a I<shared secret>: for two L<AnyEvent::MP> I<nodes> to
126     be able to communicate with each other and authenticate each other it is
127     necessary to setup the same I<shared secret> for both of them (or use TLS
128     certificates).
129    
130     The easiest way is to set this up is to use the F<aemp> utility:
131    
132     aemp gensecret
133    
134     This creates a F<$HOME/.perl-anyevent-mp> config file and generates a random
135     shared secret. You can copy this file to any other system and then communicate
136     over the network (via TCP) with it. You can also select your own shared secret
137     (F<aemp setsecret>) and for increased security requirements you can even create
138     a TLS certificate (F<aemp gencert>), causing connections to not just be
139     authenticated, but also to be encrypted.
140    
141     Connections will only be successful when the I<nodes> that want to connect to
142     each other have the same I<shared secret> (or successfully verify the TLS
143     certificate of the other side).
144    
145     B<If something does not work as expected, and for example tcpdump shows
146     that the connections are closed almost immediately, you should make sure
147     that F<~/.perl-anyevent-mp> is the same on all hosts/user accounts that
148     you try to connect with each other!>
149 elmex 1.16
150 elmex 1.18 Thats all for now, there is more fiddling around with the C<aemp> utility
151     later.
152    
153     =head1 Passing Messages Between Processes
154    
155     =head2 The Receiver
156    
157     Lets split the previous example up into two small programs. First the
158     receiver application:
159    
160     #!/opt/perl/bin/perl
161     use AnyEvent;
162     use AnyEvent::MP;
163     use AnyEvent::MP::Global;
164    
165     initialise_node "eg_simple_receiver";
166    
167     my $port = port;
168    
169     AnyEvent::MP::Global::register $port, "eg_receivers";
170    
171     rcv $port, test => sub {
172     my ($data, $reply_port) = @_;
173    
174     print "Received data: " . $data . "\n";
175     };
176    
177     AnyEvent->condvar->recv;
178    
179     =head3 AnyEvent::MP::Global
180    
181     Now, that wasn't too bad, was it? Ok, lets step through the new functions
182     and modules that have been used. For starters there is now an additional
183     module loaded: L<AnyEvent::MP::Global>.
184    
185     That module provides us with a I<global registry>, which lets us share data
186     among all I<nodes> in a network. Why do we need it you might ask?
187    
188 elmex 1.20 The thing is, that the I<port ids> are just random strings, assigned by
189     L<AnyEvent::MP>. We can't know those I<port ids> in advance, so we don't know
190     which I<port id> to send messages to if the message is to be passed between
191     I<nodes> (or UNIX processes). To find the right I<port> of another I<node> in
192     the network we will need to communicate that somehow to the sender. And
193     exactly that is what L<AnyEvent::MP::Global> provides.
194 elmex 1.18
195     =head3 initialise_node And The Network
196    
197     Now, lets have a look at the next new thing, the C<initialise_node>:
198    
199     initialise_node "eg_simple_receiver";
200    
201     Before we are able to send messages to other nodes we have to initialise
202     ourself. The first argument, the string C<"eg_simple_receiver">, is called the
203     I<profile> of this node. A profile holds some information about the application
204     that is going to be a node in an L<AnyEvent::MP> network.
205    
206     Most importantly the profile allows you to set the I<node id> that your
207     application will use. You can also set I<binds> in the profile, meaning that
208     you can define TCP ports that the application will listen on for incoming
209     connections from other nodes of the network.
210    
211     Next you can configure I<seeds> in profile. A I<seed> is just a TCP endpoint
212     which tells the application where to find other nodes of it's network. To
213     explain this a bit more detailed we have to look at the topology of an
214     L<AnyEvent::MP> network. The topology is called a I<fully connected mesh>, here
215     an example with 4 nodes:
216    
217     N1--N2
218     | \/ |
219     | /\ |
220     N3--N4
221    
222     Now imagine another I<node> C<N5>. wants to connect itself to that network:
223    
224     N1--N2
225     | \/ | N5
226     | /\ |
227     N3--N4
228    
229     The new node needs to know the I<binds> of all of those 4 already connected
230     nodes. And exactly this is what the I<seeds> are for. Now lets assume that
231     the new node C<N5> has as I<seed> the TCP endpoint of the node C<N2>.
232     It then connects to C<N2>:
233    
234     N1--N2____
235     | \/ | N5
236     | /\ |
237     N3--N4
238    
239     C<N2> then tells C<N5> the I<binds> of the other nodes it is connected to,
240     and C<N5> builds up the rest of the connections:
241    
242     /--------\
243     N1--N2____|
244     | \/ | N5
245     | /\ | /|
246     N3--N4--- |
247     \________/
248    
249     Finished. C<N5> is now happily connected to the rest of the network.
250    
251 elmex 1.19 =head3 Setting Up The Profiles
252    
253     Ok, so much to the profile. Now lets setup the C<eg_simple_receiver> I<profile>
254     for later. For the receiver we just give the receiver a I<bind>:
255    
256     aemp profile eg_simple_receiver setbinds localhost:12266
257    
258     And while we are at it, just setup the I<profile> for the sender in the second
259     part of this example too. We will call the sender I<profile>
260     C<eg_simple_sender>. For the sender we will just setup a I<seed> to the
261     receiver:
262    
263     aemp profile eg_simple_sender setseeds localhost:12266
264 elmex 1.22 aemp profile eg_simple_sender setbinds
265 elmex 1.19
266 elmex 1.22 You might wonder why we setup I<binds> to be empty here. Well, there can be
267 elmex 1.19 exceptions to the I<fully> in the I<fully connected mesh> in L<AnyEvent::MP>.
268     If you don't configure a I<bind> for a node's profile it won't bind itself
269     somewhere. These kinds of I<nodes> will not be able to send messages to other
270     I<nodes> that also didn't I<bind> them self to some TCP address. For this
271     example, as well as some cases in the real world, we can live with this
272     limitation.
273    
274     =head3 Registering The Receiver
275    
276     Ok, where were we. We now discussed the basic purpose of L<AnyEvent::MP::Global>
277     and initialise_node with it's relations to profiles. We also setup our profiles
278     for later use and now have to continue talking about the receiver example.
279    
280     Lets look at the next undiscussed line(s) of code:
281    
282     my $port = port;
283     AnyEvent::MP::Global::register $port, "eg_receivers";
284    
285     The C<port> function already has been discussed. It just creates a new I<port>
286     and gives us the I<port id>. Now to the C<register> function of
287     L<AnyEvent::MP::Global>: The first argument is a I<port id> that we want to add
288     to a I<global group>, and it's second argument is the name of that I<global
289     group>.
290    
291     You can choose that name of such a I<global group> freely, and it's purpose is
292     to store a set of I<port ids>. That set is made available throughout the whole
293     L<AnyEvent::MP> network, so that each node can see which ports belong to that
294     group.
295    
296     The sender will later look for the ports in that I<global group> and send
297     messages to them.
298    
299     Last step in the example is to setup a receiver callback for those messages
300     like we have discussed in the first example. We again match for the I<tag>
301     C<test>. The difference is just that we don't end the application after
302     receiving the first message. We just infinitely continue to look out for new
303     messages.
304    
305 elmex 1.20 =head2 The Sender
306 root 1.8
307 elmex 1.20 Ok, now lets take a look at the sender:
308 root 1.4
309 elmex 1.20 #!/opt/perl/bin/perl
310 elmex 1.1 use AnyEvent;
311     use AnyEvent::MP;
312 elmex 1.20 use AnyEvent::MP::Global;
313 elmex 1.1
314 elmex 1.20 initialise_node "eg_simple_sender";
315 elmex 1.1
316 elmex 1.20 my $find_timer =
317     AnyEvent->timer (after => 0, interval => 1, cb => sub {
318     my $ports = AnyEvent::MP::Global::find "eg_receivers"
319     or return;
320    
321     snd $_, test => time
322     for @$ports;
323     });
324 elmex 1.1
325     AnyEvent->condvar->recv;
326    
327 elmex 1.20 It's even less code. The C<initialise_node> is known now from the receiver
328     above. As discussed in the section where we setup the profiles we configure
329     this application to use the I<profile> C<eg_simple_sender>.
330 root 1.10
331 elmex 1.20 Next we setup a timer that repeatedly calls this chunk of code:
332 elmex 1.1
333 elmex 1.20 my $ports = AnyEvent::MP::Global::find "eg_receivers"
334     or return;
335 elmex 1.2
336 elmex 1.20 snd $_, test => time
337     for @$ports;
338 elmex 1.1
339 elmex 1.20 The new function here is the C<find> function of L<AnyEvent::MP::Global>. It
340     searches in the I<global group> named C<eg_receivers> for ports. If none are
341     found C<undef> is returned and we wait for the next time the timer fires.
342 elmex 1.1
343 elmex 1.20 In case the receiver application has been connected and the newly added port by
344     the receiver has propagated to the sender C<find> returns an array reference
345     that contains the I<port id> of the receiver I<port(s)>.
346 elmex 1.1
347 elmex 1.20 We then just send to every I<port> in the I<global group> a message consisting
348     of the I<tag> C<test> and the current time in form of a UNIX timestamp.
349 elmex 1.7
350 elmex 1.20 And thats all.
351 elmex 1.7
352 elmex 1.1 =head1 SEE ALSO
353    
354     L<AnyEvent>
355    
356     L<AnyEvent::Handle>
357    
358     L<AnyEvent::MP>
359    
360 elmex 1.20 L<AnyEvent::MP::Global>
361    
362 elmex 1.1 =head1 AUTHOR
363    
364     Robin Redeker <elmex@ta-sa.org>
365 root 1.4