ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.22
Committed: Fri Aug 28 17:03:02 2009 UTC (14 years, 9 months ago) by elmex
Branch: MAIN
Changes since 1.21: +2 -1 lines
Log Message:
small fix.

File Contents

# Content
1 =head1 Message Passing for the Non-Blocked Mind
2
3 =head1 Introduction and Terminology
4
5 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 module, which allows us to transparently pass messages to our own process
7 and to other processes on another or the same host.
8
9 What kind of messages? Well, basically a message here means a list of Perl
10 strings, numbers, hashes and arrays, anything that can be expressed as a
11 L<JSON> text (as JSON is used by default in the protocol). Here is an example:
12
13 [123, ["a", "b", "c"], { foo => 'bar' }]
14
15 When using L<AnyEvent::MP> you should usually use a string as first element
16 of a message, which describes the type of the
17 message. That element is called a I<tag> in L<AnyEvent::MP>, as some
18 API functions (C<rcv>) support matching it directly. So supposedly you want to
19 send a ping message with your current time to something, this is how such a
20 message might look like (in Perl syntax):
21
22 ['ping', 1251381636]
23
24 The next interesting question is: Between which entities are those messages
25 being I<passed>? They are I<passed> between I<ports>. I<ports> are just sources
26 and destinations for messages. How do these ports relate to things you know?
27 Well, each I<port> belongs to a I<node>, and a I<node> is just the UNIX process
28 that runs your L<AnyEvent::MP> application.
29
30 Each I<node> is distinguished from other I<nodes>, that run on the same host or
31 multiple hosts in a network, by it's I<node ID>. A I<node ID> can be manually
32 assigned or L<AnyEvent::MP> will assign one it self for you.
33
34 Here is a visualisation about how I<nodes>, I<ports> and UNIX processes
35 relate to each other. The setup consists of two nodes (more are of
36 course possible): Node C<A> (in UNIX process 7066) with the ports C<ABC> and C<DEF>.
37 And the node C<B> (in UNIX process 8321) with the ports C<FOO> and C<BAR>.
38
39
40 |- PID: 7066 -| |- PID: 8321 -|
41 | | | |
42 | Node ID: A | | Node ID: B |
43 | | | |
44 | Port ABC =|= <----\ /-----> =|= Port FOO |
45 | | X | |
46 | Port DEF =|= <----/ \-----> =|= Port BAR |
47 | | | |
48 |-------------| |-------------|
49
50 The strings for the I<port ids> here are just for illustrative purposes. Even
51 if in reality I<ports> in L<AnyEvent::MP> are also identified by strings they
52 can't be choosen manually and are assigned randomly. These I<port ids> should
53 also not be used directly for other purposes than referring to an endpoint for
54 messages.
55
56 The next sections will explain the API of L<AnyEvent::MP> by going through a
57 few simple examples. Later some more complex idioms are introduced, which are
58 maybe useful to solve some real world problems.
59
60 =head1 Passing Your First Message
61
62 As start lets have a look at the messaging API. The next example is just a
63 demo to show the basic elements of message passing with L<AnyEvent::MP>.
64 It shout just print: "Ending with: 123". So here the code:
65
66 use AnyEvent;
67 use AnyEvent::MP;
68
69 my $end_cv = AnyEvent->condvar;
70
71 my $port = port;
72
73 rcv $port, test => sub {
74 my ($data) = @_;
75 $end_cv->send ($data);
76 };
77
78 snd $port, test => 123;
79
80 print "Ending with: " . $end_cv->recv . "\n";
81
82 It already contains most functions of the essential L<AnyEvent::MP> API.
83
84 First there is the C<port> function which will create a I<port> and will return
85 it's I<port id>.
86
87 That I<port id> can be used to send and receive messages. That I<port id> is a
88 simple string and can be safely passed to other I<nodes> in the network to
89 refer to that specific port (usually used for RPC, where you need to
90 tell the other end which I<port> to send the reply to).
91
92 Next function is C<rcv>:
93
94 rcv $port, test => sub { ... };
95
96 It sets up a receiver callback on a specific I<port> which needs to be
97 specified as the first argument. The next argument, in this example C<test>, is
98 a I<tag> match. This means that whenever a message, with the first element
99 being the string C<tag>, is received the callback is called with the remaining
100 parts of that message.
101
102 Messages can be send with the C<snd> function, which looks like this in the
103 example above:
104
105 snd $port, test => 123;
106
107 This will send the message C<['test', 123]> to the I<port> with the I<port id>
108 in C<$port>. The receiver got a I<tag> match on C<test> and will call the
109 callback with the first argument being the number C<123>.
110
111 That callback then just passes that number on to the I<condition variable>
112 C<$end_cv> which will then pass the value to the print. But I<condition
113 variables> are out of the scope of this tutorial. So please consult the
114 L<AnyEvent::Intro> about them.
115
116 But passing messages inside one process is boring, but before we can continue
117 and take the next step to interprocess message passing we first have to make
118 sure some things have been setup.
119
120 =head1 System Requirements and System Setup
121
122 Before we can start with real IPC we have to make sure some things work on your
123 system.
124
125 First we have to setup a I<shared secret>: for two L<AnyEvent::MP> I<nodes> to
126 be able to communicate with each other and authenticate each other it is
127 necessary to setup the same I<shared secret> for both of them (or use TLS
128 certificates).
129
130 The easiest way is to set this up is to use the F<aemp> utility:
131
132 aemp gensecret
133
134 This creates a F<$HOME/.perl-anyevent-mp> config file and generates a random
135 shared secret. You can copy this file to any other system and then communicate
136 over the network (via TCP) with it. You can also select your own shared secret
137 (F<aemp setsecret>) and for increased security requirements you can even create
138 a TLS certificate (F<aemp gencert>), causing connections to not just be
139 authenticated, but also to be encrypted.
140
141 Connections will only be successful when the I<nodes> that want to connect to
142 each other have the same I<shared secret> (or successfully verify the TLS
143 certificate of the other side).
144
145 B<If something does not work as expected, and for example tcpdump shows
146 that the connections are closed almost immediately, you should make sure
147 that F<~/.perl-anyevent-mp> is the same on all hosts/user accounts that
148 you try to connect with each other!>
149
150 Thats all for now, there is more fiddling around with the C<aemp> utility
151 later.
152
153 =head1 Passing Messages Between Processes
154
155 =head2 The Receiver
156
157 Lets split the previous example up into two small programs. First the
158 receiver application:
159
160 #!/opt/perl/bin/perl
161 use AnyEvent;
162 use AnyEvent::MP;
163 use AnyEvent::MP::Global;
164
165 initialise_node "eg_simple_receiver";
166
167 my $port = port;
168
169 AnyEvent::MP::Global::register $port, "eg_receivers";
170
171 rcv $port, test => sub {
172 my ($data, $reply_port) = @_;
173
174 print "Received data: " . $data . "\n";
175 };
176
177 AnyEvent->condvar->recv;
178
179 =head3 AnyEvent::MP::Global
180
181 Now, that wasn't too bad, was it? Ok, lets step through the new functions
182 and modules that have been used. For starters there is now an additional
183 module loaded: L<AnyEvent::MP::Global>.
184
185 That module provides us with a I<global registry>, which lets us share data
186 among all I<nodes> in a network. Why do we need it you might ask?
187
188 The thing is, that the I<port ids> are just random strings, assigned by
189 L<AnyEvent::MP>. We can't know those I<port ids> in advance, so we don't know
190 which I<port id> to send messages to if the message is to be passed between
191 I<nodes> (or UNIX processes). To find the right I<port> of another I<node> in
192 the network we will need to communicate that somehow to the sender. And
193 exactly that is what L<AnyEvent::MP::Global> provides.
194
195 =head3 initialise_node And The Network
196
197 Now, lets have a look at the next new thing, the C<initialise_node>:
198
199 initialise_node "eg_simple_receiver";
200
201 Before we are able to send messages to other nodes we have to initialise
202 ourself. The first argument, the string C<"eg_simple_receiver">, is called the
203 I<profile> of this node. A profile holds some information about the application
204 that is going to be a node in an L<AnyEvent::MP> network.
205
206 Most importantly the profile allows you to set the I<node id> that your
207 application will use. You can also set I<binds> in the profile, meaning that
208 you can define TCP ports that the application will listen on for incoming
209 connections from other nodes of the network.
210
211 Next you can configure I<seeds> in profile. A I<seed> is just a TCP endpoint
212 which tells the application where to find other nodes of it's network. To
213 explain this a bit more detailed we have to look at the topology of an
214 L<AnyEvent::MP> network. The topology is called a I<fully connected mesh>, here
215 an example with 4 nodes:
216
217 N1--N2
218 | \/ |
219 | /\ |
220 N3--N4
221
222 Now imagine another I<node> C<N5>. wants to connect itself to that network:
223
224 N1--N2
225 | \/ | N5
226 | /\ |
227 N3--N4
228
229 The new node needs to know the I<binds> of all of those 4 already connected
230 nodes. And exactly this is what the I<seeds> are for. Now lets assume that
231 the new node C<N5> has as I<seed> the TCP endpoint of the node C<N2>.
232 It then connects to C<N2>:
233
234 N1--N2____
235 | \/ | N5
236 | /\ |
237 N3--N4
238
239 C<N2> then tells C<N5> the I<binds> of the other nodes it is connected to,
240 and C<N5> builds up the rest of the connections:
241
242 /--------\
243 N1--N2____|
244 | \/ | N5
245 | /\ | /|
246 N3--N4--- |
247 \________/
248
249 Finished. C<N5> is now happily connected to the rest of the network.
250
251 =head3 Setting Up The Profiles
252
253 Ok, so much to the profile. Now lets setup the C<eg_simple_receiver> I<profile>
254 for later. For the receiver we just give the receiver a I<bind>:
255
256 aemp profile eg_simple_receiver setbinds localhost:12266
257
258 And while we are at it, just setup the I<profile> for the sender in the second
259 part of this example too. We will call the sender I<profile>
260 C<eg_simple_sender>. For the sender we will just setup a I<seed> to the
261 receiver:
262
263 aemp profile eg_simple_sender setseeds localhost:12266
264 aemp profile eg_simple_sender setbinds
265
266 You might wonder why we setup I<binds> to be empty here. Well, there can be
267 exceptions to the I<fully> in the I<fully connected mesh> in L<AnyEvent::MP>.
268 If you don't configure a I<bind> for a node's profile it won't bind itself
269 somewhere. These kinds of I<nodes> will not be able to send messages to other
270 I<nodes> that also didn't I<bind> them self to some TCP address. For this
271 example, as well as some cases in the real world, we can live with this
272 limitation.
273
274 =head3 Registering The Receiver
275
276 Ok, where were we. We now discussed the basic purpose of L<AnyEvent::MP::Global>
277 and initialise_node with it's relations to profiles. We also setup our profiles
278 for later use and now have to continue talking about the receiver example.
279
280 Lets look at the next undiscussed line(s) of code:
281
282 my $port = port;
283 AnyEvent::MP::Global::register $port, "eg_receivers";
284
285 The C<port> function already has been discussed. It just creates a new I<port>
286 and gives us the I<port id>. Now to the C<register> function of
287 L<AnyEvent::MP::Global>: The first argument is a I<port id> that we want to add
288 to a I<global group>, and it's second argument is the name of that I<global
289 group>.
290
291 You can choose that name of such a I<global group> freely, and it's purpose is
292 to store a set of I<port ids>. That set is made available throughout the whole
293 L<AnyEvent::MP> network, so that each node can see which ports belong to that
294 group.
295
296 The sender will later look for the ports in that I<global group> and send
297 messages to them.
298
299 Last step in the example is to setup a receiver callback for those messages
300 like we have discussed in the first example. We again match for the I<tag>
301 C<test>. The difference is just that we don't end the application after
302 receiving the first message. We just infinitely continue to look out for new
303 messages.
304
305 =head2 The Sender
306
307 Ok, now lets take a look at the sender:
308
309 #!/opt/perl/bin/perl
310 use AnyEvent;
311 use AnyEvent::MP;
312 use AnyEvent::MP::Global;
313
314 initialise_node "eg_simple_sender";
315
316 my $find_timer =
317 AnyEvent->timer (after => 0, interval => 1, cb => sub {
318 my $ports = AnyEvent::MP::Global::find "eg_receivers"
319 or return;
320
321 snd $_, test => time
322 for @$ports;
323 });
324
325 AnyEvent->condvar->recv;
326
327 It's even less code. The C<initialise_node> is known now from the receiver
328 above. As discussed in the section where we setup the profiles we configure
329 this application to use the I<profile> C<eg_simple_sender>.
330
331 Next we setup a timer that repeatedly calls this chunk of code:
332
333 my $ports = AnyEvent::MP::Global::find "eg_receivers"
334 or return;
335
336 snd $_, test => time
337 for @$ports;
338
339 The new function here is the C<find> function of L<AnyEvent::MP::Global>. It
340 searches in the I<global group> named C<eg_receivers> for ports. If none are
341 found C<undef> is returned and we wait for the next time the timer fires.
342
343 In case the receiver application has been connected and the newly added port by
344 the receiver has propagated to the sender C<find> returns an array reference
345 that contains the I<port id> of the receiver I<port(s)>.
346
347 We then just send to every I<port> in the I<global group> a message consisting
348 of the I<tag> C<test> and the current time in form of a UNIX timestamp.
349
350 And thats all.
351
352 =head1 SEE ALSO
353
354 L<AnyEvent>
355
356 L<AnyEvent::Handle>
357
358 L<AnyEvent::MP>
359
360 L<AnyEvent::MP::Global>
361
362 =head1 AUTHOR
363
364 Robin Redeker <elmex@ta-sa.org>
365