ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.20
Committed: Fri Aug 28 15:44:39 2009 UTC (14 years, 9 months ago) by elmex
Branch: MAIN
Changes since 1.19: +38 -407 lines
Log Message:
finished intro of second example.

File Contents

# Content
1 =head1 Message Passing for the Non-Blocked Mind
2
3 =head1 Introduction and Terminology
4
5 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 module, which allows us to transparently pass messages to our own process
7 and to other processes on another or the same host.
8
9 What kind of messages? Well, basically a message here means a list of Perl
10 strings, numbers, hashes and arrays, anything that can be expressed as a
11 L<JSON> text (as JSON is used by default in the protocol).
12
13 It's custom in L<AnyEvent::MP> to have a string which describes the type of the
14 message as first element (this is called a I<tag> in L<AnyEvent::MP>), as some
15 API functions (C<rcv>) support matching it directly. So supposedly you want to
16 send a ping message with your current time to something, this is how such a
17 message might look like (in Perl syntax):
18
19 ['ping', 1251381636]
20
21 And next you might ask: between which entities are those messages being
22 I<passed>? They are I<passed> between I<ports>. I<ports> are just sources and
23 destinations for messages. How do these ports relate to things you know? Well,
24 each I<port> belongs to a I<node>, and a I<node> is just the UNIX process that
25 runs your L<AnyEvent::MP> application.
26
27 Each I<node> is distinguished from other I<nodes> running on the same host or
28 multiple hosts in a network by it's I<node ID>. A I<node ID> can be manually
29 assigned or L<AnyEvent::MP> will assign one it self for you.
30
31 So, you might want to visualize it like this (setup is two nodes (more are of
32 course possible): Node C<A> (in UNIX process 7066) with ports C<ABC> and C<DEF>
33 and C<B> (in UNIX process 8321) with ports C<FOO> and C<BAR>).
34
35
36 |- PID: 7066 -| |- PID: 8321 -|
37 | | | |
38 | Node ID: A | | Node ID: B |
39 | | | |
40 | Port ABC =|= <----\ /-----> =|= Port FOO |
41 | | X | |
42 | Port DEF =|= <----/ \-----> =|= Port BAR |
43 | | | |
44 |-------------| |-------------|
45
46 The strings for the ports here are just for illustrative purposes. Even if in
47 reality I<ports> in L<AnyEvent::MP> are also identified by strings they can't
48 be choosen manually and are assigned randomly. These I<port ids> should also
49 not be used directly for other purposes than referring to an endpoint for
50 messages.
51
52 The next sections will explain the API of L<AnyEvent::MP>. First the API is
53 laid out by simple examples. Later some more complex idioms are introduced,
54 which are maybe useful to solve some real world purposes.
55
56 # In this tutorial I'll show you how to write a simple chat server based on
57 # L<AnyEvent::MP>. This example is used because it nicely shows how to organise a
58 # simple application, but keep in mind that every node trusts any other, so this
59 # chat cannot be used to implement a real public chat server and client system,
60 # but it can be used to implement a distributed chat server for example.
61
62 =head1 Passing Your First Message
63
64 As start lets have a look at the messaging API. The next example is just a
65 demo to show the basic elements of message passing with L<AnyEvent::MP>.
66 It shout just print: "Ending with: 123". So here the code:
67
68 use AnyEvent;
69 use AnyEvent::MP;
70
71 my $end_cv = AnyEvent->condvar;
72
73 my $port = port;
74
75 rcv $port, test => sub {
76 my ($data) = @_;
77 $end_cv->send ($data);
78 };
79
80 snd $port, test => 123;
81
82 print "Ending with: " . $end_cv->recv . "\n";
83
84 It already contains most functions of the essential L<AnyEvent::MP> API.
85
86 First there is the C<port> function which will create a I<port> and will return
87 it's I<port id>.
88
89 That I<port id> can be used to send and receive messages. That I<port id> is a
90 simple string and can be safely passed to other I<nodes> in the network to
91 refer to that specific port (usually used for RPC, where you need to
92 tell the other end which I<port> to send the reply to).
93
94 Next function is C<rcv>:
95
96 rcv $port, test => sub { ... };
97
98 It sets up a receiver callback on a specific I<port> which needs to be
99 specified as the first argument. The next argument, in this example C<test>, is
100 a I<tag> match. This means that whenever a message, with the first element
101 being the string C<tag>, is received the callback is called with the remaining
102 parts of that message.
103
104 Messages can be send with the C<snd> function, which looks like this in the
105 example above:
106
107 snd $port, test => 123;
108
109 This will send the message C<['test', 123]> to the I<port> with the I<port id>
110 in C<$port>. The receiver got a I<tag> match on C<test> and will call the
111 callback with the first argument being the number C<123>.
112
113 That callback then just passes that number on to the I<condition variable>
114 C<$end_cv> which will then pass the value to the print. But I<condition
115 variables> are out of the scope of this tutorial. So please consult the
116 L<AnyEvent::Intro> about them.
117
118 But passing messages inside one process is boring, but before we can continue
119 and take the next step to interprocess message passing we first have to make
120 sure some things have been setup.
121
122 =head1 System Requirements and System Setup
123
124 Before we can start with real IPC we have to make sure some things work on your
125 system.
126
127 First we have to setup a I<shared secret>: for two L<AnyEvent::MP> I<nodes> to
128 be able to communicate with each other and authenticate each other it is
129 necessary to setup the same I<shared secret> for both of them (or use TLS
130 certificates).
131
132 The easiest way is to set this up is to use the F<aemp> utility:
133
134 aemp gensecret
135
136 This creates a F<$HOME/.perl-anyevent-mp> config file and generates a random
137 shared secret. You can copy this file to any other system and then communicate
138 over the network (via TCP) with it. You can also select your own shared secret
139 (F<aemp setsecret>) and for increased security requirements you can even create
140 a TLS certificate (F<aemp gencert>), causing connections to not just be
141 authenticated, but also to be encrypted.
142
143 Connections will only be successful when the I<nodes> that want to connect to
144 each other have the same I<shared secret> (or successfully verify the TLS
145 certificate of the other side).
146
147 B<If something does not work as expected, and for example tcpdump shows
148 that the connections are closed almost immediately, you should make sure
149 that F<~/.perl-anyevent-mp> is the same on all hosts/user accounts that
150 you try to connect with each other!>
151
152 Thats all for now, there is more fiddling around with the C<aemp> utility
153 later.
154
155 =head1 Passing Messages Between Processes
156
157 =head2 The Receiver
158
159 Lets split the previous example up into two small programs. First the
160 receiver application:
161
162 #!/opt/perl/bin/perl
163 use AnyEvent;
164 use AnyEvent::MP;
165 use AnyEvent::MP::Global;
166
167 initialise_node "eg_simple_receiver";
168
169 my $port = port;
170
171 AnyEvent::MP::Global::register $port, "eg_receivers";
172
173 rcv $port, test => sub {
174 my ($data, $reply_port) = @_;
175
176 print "Received data: " . $data . "\n";
177 };
178
179 AnyEvent->condvar->recv;
180
181 =head3 AnyEvent::MP::Global
182
183 Now, that wasn't too bad, was it? Ok, lets step through the new functions
184 and modules that have been used. For starters there is now an additional
185 module loaded: L<AnyEvent::MP::Global>.
186
187 That module provides us with a I<global registry>, which lets us share data
188 among all I<nodes> in a network. Why do we need it you might ask?
189
190 The thing is, that the I<port ids> are just random strings, assigned by
191 L<AnyEvent::MP>. We can't know those I<port ids> in advance, so we don't know
192 which I<port id> to send messages to if the message is to be passed between
193 I<nodes> (or UNIX processes). To find the right I<port> of another I<node> in
194 the network we will need to communicate that somehow to the sender. And
195 exactly that is what L<AnyEvent::MP::Global> provides.
196
197 =head3 initialise_node And The Network
198
199 Now, lets have a look at the next new thing, the C<initialise_node>:
200
201 initialise_node "eg_simple_receiver";
202
203 Before we are able to send messages to other nodes we have to initialise
204 ourself. The first argument, the string C<"eg_simple_receiver">, is called the
205 I<profile> of this node. A profile holds some information about the application
206 that is going to be a node in an L<AnyEvent::MP> network.
207
208 Most importantly the profile allows you to set the I<node id> that your
209 application will use. You can also set I<binds> in the profile, meaning that
210 you can define TCP ports that the application will listen on for incoming
211 connections from other nodes of the network.
212
213 Next you can configure I<seeds> in profile. A I<seed> is just a TCP endpoint
214 which tells the application where to find other nodes of it's network. To
215 explain this a bit more detailed we have to look at the topology of an
216 L<AnyEvent::MP> network. The topology is called a I<fully connected mesh>, here
217 an example with 4 nodes:
218
219 N1--N2
220 | \/ |
221 | /\ |
222 N3--N4
223
224 Now imagine another I<node> C<N5>. wants to connect itself to that network:
225
226 N1--N2
227 | \/ | N5
228 | /\ |
229 N3--N4
230
231 The new node needs to know the I<binds> of all of those 4 already connected
232 nodes. And exactly this is what the I<seeds> are for. Now lets assume that
233 the new node C<N5> has as I<seed> the TCP endpoint of the node C<N2>.
234 It then connects to C<N2>:
235
236 N1--N2____
237 | \/ | N5
238 | /\ |
239 N3--N4
240
241 C<N2> then tells C<N5> the I<binds> of the other nodes it is connected to,
242 and C<N5> builds up the rest of the connections:
243
244 /--------\
245 N1--N2____|
246 | \/ | N5
247 | /\ | /|
248 N3--N4--- |
249 \________/
250
251 Finished. C<N5> is now happily connected to the rest of the network.
252
253 =head3 Setting Up The Profiles
254
255 Ok, so much to the profile. Now lets setup the C<eg_simple_receiver> I<profile>
256 for later. For the receiver we just give the receiver a I<bind>:
257
258 aemp profile eg_simple_receiver setbinds localhost:12266
259
260 And while we are at it, just setup the I<profile> for the sender in the second
261 part of this example too. We will call the sender I<profile>
262 C<eg_simple_sender>. For the sender we will just setup a I<seed> to the
263 receiver:
264
265 aemp profile eg_simple_sender setseeds localhost:12266
266
267 You might wonder why we don't setup a I<bind> here. Well, there can be
268 exceptions to the I<fully> in the I<fully connected mesh> in L<AnyEvent::MP>.
269 If you don't configure a I<bind> for a node's profile it won't bind itself
270 somewhere. These kinds of I<nodes> will not be able to send messages to other
271 I<nodes> that also didn't I<bind> them self to some TCP address. For this
272 example, as well as some cases in the real world, we can live with this
273 limitation.
274
275 =head3 Registering The Receiver
276
277 Ok, where were we. We now discussed the basic purpose of L<AnyEvent::MP::Global>
278 and initialise_node with it's relations to profiles. We also setup our profiles
279 for later use and now have to continue talking about the receiver example.
280
281 Lets look at the next undiscussed line(s) of code:
282
283 my $port = port;
284 AnyEvent::MP::Global::register $port, "eg_receivers";
285
286 The C<port> function already has been discussed. It just creates a new I<port>
287 and gives us the I<port id>. Now to the C<register> function of
288 L<AnyEvent::MP::Global>: The first argument is a I<port id> that we want to add
289 to a I<global group>, and it's second argument is the name of that I<global
290 group>.
291
292 You can choose that name of such a I<global group> freely, and it's purpose is
293 to store a set of I<port ids>. That set is made available throughout the whole
294 L<AnyEvent::MP> network, so that each node can see which ports belong to that
295 group.
296
297 The sender will later look for the ports in that I<global group> and send
298 messages to them.
299
300 Last step in the example is to setup a receiver callback for those messages
301 like we have discussed in the first example. We again match for the I<tag>
302 C<test>. The difference is just that we don't end the application after
303 receiving the first message. We just infinitely continue to look out for new
304 messages.
305
306 =head2 The Sender
307
308 Ok, now lets take a look at the sender:
309
310 #!/opt/perl/bin/perl
311 use AnyEvent;
312 use AnyEvent::MP;
313 use AnyEvent::MP::Global;
314
315 initialise_node "eg_simple_sender";
316
317 my $find_timer =
318 AnyEvent->timer (after => 0, interval => 1, cb => sub {
319 my $ports = AnyEvent::MP::Global::find "eg_receivers"
320 or return;
321
322 snd $_, test => time
323 for @$ports;
324 });
325
326 AnyEvent->condvar->recv;
327
328 It's even less code. The C<initialise_node> is known now from the receiver
329 above. As discussed in the section where we setup the profiles we configure
330 this application to use the I<profile> C<eg_simple_sender>.
331
332 Next we setup a timer that repeatedly calls this chunk of code:
333
334 my $ports = AnyEvent::MP::Global::find "eg_receivers"
335 or return;
336
337 snd $_, test => time
338 for @$ports;
339
340 The new function here is the C<find> function of L<AnyEvent::MP::Global>. It
341 searches in the I<global group> named C<eg_receivers> for ports. If none are
342 found C<undef> is returned and we wait for the next time the timer fires.
343
344 In case the receiver application has been connected and the newly added port by
345 the receiver has propagated to the sender C<find> returns an array reference
346 that contains the I<port id> of the receiver I<port(s)>.
347
348 We then just send to every I<port> in the I<global group> a message consisting
349 of the I<tag> C<test> and the current time in form of a UNIX timestamp.
350
351 And thats all.
352
353 =head1 SEE ALSO
354
355 L<AnyEvent>
356
357 L<AnyEvent::Handle>
358
359 L<AnyEvent::MP>
360
361 L<AnyEvent::MP::Global>
362
363 =head1 AUTHOR
364
365 Robin Redeker <elmex@ta-sa.org>
366