ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.11
Committed: Tue Aug 4 21:06:47 2009 UTC (14 years, 10 months ago) by root
Branch: MAIN
CVS Tags: rel-0_1, rel-0_4
Changes since 1.10: +1 -1 lines
Log Message:
*** empty log message ***

File Contents

# Content
1 =head1 Message Passing for the Non-Blocked Mind
2
3 =head1 Introduction and Terminology
4
5 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 module. Which allows us to transparently pass messages to our own process
7 and to other processes on another or the same host.
8
9 What kind of messages? Well, basically a message here means a list of
10 Perl strings, numbers, hashes and arrays, mostly everything that can be
11 expressed as a L<JSON> text (as JSON is used by default in the protocol).
12
13 And next you might ask: between which entities are those messages
14 being "passed"? Effectively between I<nodes>: a nodes is basically a
15 process/program that use L<AnyEvent::MP> and can run either on the same or
16 different hosts.
17
18 To make this more managable, every node can contain any number of
19 I<ports>: Ports are ultimately the receivers of your messages.
20
21 In this tutorial I'll show you how to write a simple chat server based on
22 L<AnyEvent::MP>.
23
24 =head1 System Requirements
25
26 Before we can start we have to make sure some things work on your
27 system.
28
29 You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
30 are installed. But how to do that is out of scope of this tutorial.
31
32 Then we have to setup a I<shared secret>. For two L<AnyEvent::MP> nodes to
33 be able to communicate with each other and authenticate each other it is
34 necessary to setup the same I<shared secret> for both of them. For testing
35 you can write a random string into the file C<.aemp-secret> in your home
36 directory:
37
38 mcookie > ~/.aemp-secret
39
40 # or something more predictable
41 echo "secret123#4blabla_please_pick_your_own" > ~/.aemp-secret
42
43 Connections will only be successful when the nodes that want to connect
44 to each other have the same I<shared secret>. For more security, you can
45 put a self-signed SSL/TLS key/certificate pair into the file (or a normal
46 key/certificate and it's CA certificate).
47
48 B<If something does not work as expected, and for example tcpdump shows
49 that the connections are closed almost immediatly, you should make sure
50 that F<~/.aemp-secret> is the same on all hosts/user accounts that you try
51 to connect with each other!>
52
53 =head1 The Chat Client
54
55 OK, lets start by implementing the "frontend" of the client. We will
56 develop the client first and postpone the server for later, as the most
57 complex things actually happen in the client.
58
59 We will use L<AnyEvent::Handle> to do non-blocking IO read on standard
60 input (all of this code deals with actually handling user input, no
61 message passing yet):
62
63 #!perl
64
65 use AnyEvent;
66 use AnyEvent::Handle;
67
68 sub send_message {
69 die "This is where we will send the messages to the server"
70 . "in the next step of this tutorial.\n"
71 }
72
73 # make an AnyEvent condition variable for the 'quit' condition
74 # (when we want to exit the client).
75 my $quit_cv = AnyEvent->condvar;
76
77 my $stdin_hdl = AnyEvent::Handle->new (
78 fh => *STDIN,
79 on_error => sub { $quit_cv->send },
80 on_read => sub {
81 my ($hdl) = @_;
82
83 $hdl->push_read (line => sub {
84 my ($hdl, $line) = @_;
85
86 if ($line =~ /^\/quit/) { # /quit will end the client
87 $quit_cv->send;
88 } else {
89 send_message ($line);
90 }
91 });
92 }
93 );
94
95 $quit_cv->recv;
96
97 This is now a very basic client. Explaining explicitly what
98 L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
99 of this document, please consult L<AnyEvent::Intro> or the manual pages
100 for L<AnyEvent> and L<AnyEvent::Handle>.
101
102 =head1 First Steps Into Messaging
103
104 To supply the C<send_message> function we now take a look at
105 L<AnyEvent::MP>. This is an example of how it might look like:
106
107 ... # the use lines from the above snippet
108
109 use AnyEvent::MP;
110
111 sub send_message {
112 my ($msg) = @_;
113
114 snd $server_port, message => $msg;
115 }
116
117 ... # the rest of the above script
118
119 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send
120 a message'. The first argument is the I<port> (a I<port> is something
121 that can receive messages, represented by a printable string) of the
122 server which will receive the message. How we get this port will be
123 explained in the next step.
124
125 The remaining arguments of C<snd> are C<message> and C<$msg>, the first
126 two elements of the I<message> (a I<message> in L<AnyEvent::MP> is a
127 simple list of values, which can be sent to a I<port>).
128
129 So all the function does is send the two values C<message> (a constant
130 string to tell the server what to expect) and the actual message string.
131
132 Thats all fine and simple so far, but where do we get the
133 C<$server_port>? Well, we need to get the unique I<port id> of the
134 server's port where it wants to receive all the incoming chat messages. A
135 I<port id> is unfortunately a very unique string, which we are unable to
136 know in advance. But L<AnyEvent::MP> supports the concept of 'registered
137 ports', which is basically a port on the server side registered under
138 a well known name.
139
140 For example, the server has a port for receiving chat messages with a
141 unique I<port id> and registers it under the name C<chatter>.
142
143 BTW, these "registered port names" should follow similar rules as Perl
144 identifiers, so you should prefix them with your package/module name to
145 make them unique, unless you use them in the main program.
146
147 As I<messages> can only be sent to a I<port id> and not just to some name
148 we have to ask the server node for the I<port id> of the port registered
149 as C<chatter>.
150
151 =head1 Finding The Chatter Port
152
153 Ok, lots of talk, now some code. Now we will actually get the
154 C<$server_port> from the backend:
155
156 ...
157
158 use AnyEvent::MP;
159
160 my $server_node = "127.0.0.1:1299";
161
162 my $client_port = port;
163
164 snd $server_node, lookup => "chatter", $client_port, "resolved";
165
166 my $resolved_cv = AnyEvent->condvar;
167 my $server_port;
168
169 # setup a receiver callback for the 'resolved' message:
170 rcv $client_port, resolved => sub {
171 my ($tag, $chatter_port_id) = @_;
172
173 print "Resolved the server port 'chatter' to $chatter_port_id\n";
174 $server_port = $chatter_port_id;
175
176 $resolved_cv->send;
177 1
178 };
179
180 # lets block the client until we have resolved the server port.
181 $resolved_cv->recv;
182
183 # now setup another receiver callback for the chat messages:
184 rcv $client_port, message => sub {
185 my ($tag, $msg) = @_;
186
187 print "chat> $msg\n";
188 0
189 };
190
191 # send a 'join' message to the server:
192 snd $server_port, join => "$client_port";
193
194 sub send_message { ...
195
196 Now that was a lot of new stuff:
197
198 First we define the C<$server_node>: In order to refer to another node
199 we need some kind of string to reference it - the node reference. The
200 I<noderef> is basically a comma separated list of C<address:port>
201 pairs. We assume in this tutorial that the server runs on C<127.0.0.1>
202 (localhost) on port 1299, which results in the noderef C<127.0.0.1:1299>.
203
204 Next, in order to receive a reply from the other node or the server we
205 need to have a I<port> that messages can be sent to. This is what the
206 C<port> function will do for us, it just creates a new local port and
207 returns it's I<port ID> that can then be used to receive messages.
208
209 When you look carefully, you will see that the first C<snd> uses the
210 C<$server_node> (a noderef) as destination port. Well, what I didn't
211 tell you yet is that each I<node> has a default I<port> to receive
212 messages. The ID of this port is the same as the noderef.
213
214 This I<default port> provides some special services for us, for example
215 resolving a registered name to a I<port id> (a-ha! finally!).
216
217 This is exactly what this line does:
218
219 snd $server_node, lookup => "chatter", $client_port, "resolved";
220
221 This sends a message with first element being C<lookup>, followed by the
222 (hopefully) registered port name that we want to resolve to a I<port
223 id>: C<chatter>. And in order for the server node to be able to send us
224 back the resolved I<port ID> we have to tell it where to send it: The
225 result message will be sent to C<$client_port> (the I<port id> of the
226 port we just created), and will have the string C<resolved> as the first
227 element.
228
229 When the node receives this message, it will look up the name, gobble up
230 all the extra arguments we passed, append the resolved name, and send the
231 resulting list as a message.
232
233 Next we register a receiver for this C<lookup>-request.
234
235 rcv $client_port, resolved => sub {
236 my ($tag, $chatter_port_id) = @_;
237 ...
238 1
239 };
240
241 This sets up a receiver on our own port for messages with the first
242 element being the string C<resolved>. Receivers can match the contents of
243 the messages before actually executing the specified callback.
244
245 B<Please note> that the every C<rcv> callback has to return either a true
246 or a false value, indicating whether it is B<successful>/B<done> (true) or
247 still wants to B<continue> (false) receiving messages.
248
249 In this case we tell the C<$client_port> to look into all the messages
250 it receives and look for the string C<resolved> in the first element of
251 the message. If it is found, the given callback will be called with the
252 message elements as arguments.
253
254 Using a string as the first element of the message is called I<tagging>
255 the message. It's common practise to code the 'type' of a message into
256 it's first element, as this allows for simple matching.
257
258 The result message will contain the I<port ID> of the well known port
259 C<chatter> as second element, which will be stored in C<$chatter_port_id>.
260
261 This port ID will then be stored in C<$server_port>, followed by calling
262 C<send> on $resolved_cv> so the program will continue.
263
264 The callback then returns a C<1> (a true value), to indicate that it has
265 done it's job and doesn't want to receive further C<resolved> messages.
266
267 After this the chat message receiver callback is registered with the port:
268
269 rcv $client_port, message => sub {
270 my ($tag, $msg) = @_;
271
272 print "chat> $msg\n";
273
274 0
275 };
276
277 We assume that all messages that are broadcast to the clients by the
278 server contain the string tag C<message> as first element, and the actual
279 message as second element. The callback returns a false value this time,
280 to indicate that it is not yet done and wants to receive further messages.
281
282 The last thing to do is to tell the server to send us new chat messages
283 from other clients. We do so by sending the message C<join> followed by
284 our own I<port ID>.
285
286 # send the server a 'join' message:
287 snd $server_port, join => $client_port;
288
289 This way the server knows where to send all the new messages to.
290
291 =head1 The Completed Client
292
293 This is the complete client script:
294
295 #!perl
296
297 use AnyEvent;
298 use AnyEvent::Handle;
299 use AnyEvent::MP;
300
301 my $server_node = "127.0.0.1:1299";
302
303 my $client_port = port;
304
305 snd $server_node, lookup => "chatter", $client_port, "resolved";
306
307 my $resolved_cv = AnyEvent->condvar;
308 my $server_port;
309
310 # setup a receiver callback for the 'resolved' message:
311 rcv $client_port, resolved => sub {
312 my ($tag, $chatter_port_id) = @_;
313
314 print "Resolved the server port 'chatter' to $chatter_port_id\n";
315 $server_port = $chatter_port_id;
316
317 $resolved_cv->send;
318 1
319 };
320
321 # lets block the client until we have resolved the server port.
322 $resolved_cv->recv;
323
324 # now setup another receiver callback for the chat messages:
325 rcv $client_port, message => sub {
326 my ($tag, $msg) = @_;
327
328 print "chat> $msg\n";
329 0
330 };
331
332 # send a 'join' message to the server:
333 snd $server_port, join => "$client_port";
334
335 sub send_message {
336 my ($msg) = @_;
337
338 snd $server_port, message => $msg;
339 }
340
341 # make an AnyEvent condition variable for the 'quit' condition
342 # (when we want to exit the client).
343 my $quit_cv = AnyEvent->condvar;
344
345 my $stdin_hdl = AnyEvent::Handle->new (
346 fh => *STDIN,
347 on_error => sub { $quit_cv->send },
348 on_read => sub {
349 my ($hdl) = @_;
350
351 $hdl->push_read (line => sub {
352 my ($hdl, $line) = @_;
353
354 if ($line =~ /^\/quit/) { # /quit will end the client
355 $quit_cv->send;
356 } else {
357 send_message ($line);
358 }
359 });
360 }
361 );
362
363 $quit_cv->recv;
364
365 =head1 The Server
366
367 Ok, we finally come to the server.
368
369 The server of course also needs to set up a port, and in addition needs to
370 I<register> it, so the clients can find it.
371
372 Again, let's jump directly into the code:
373
374 #!perl
375
376 use AnyEvent;
377 use AnyEvent::MP;
378
379 become_public "127.0.0.1:1299";
380
381 my $chatter_port = port;
382
383 reg $chatter_port, "chatter";
384
385 my %client_ports;
386
387 rcv $chatter_port,
388 join => sub {
389 my ($tag, $client_port) = @_;
390
391 print "got new client port: $client_port\n";
392 $client_ports{$client_port} = 1;
393
394 0
395 },
396 message => sub {
397 my ($tag, $msg) = @_;
398
399 print "message> $msg\n";
400
401 snd $_, message => $msg
402 for keys %client_ports;
403
404 0
405 };
406
407 AnyEvent->condvar->recv;
408
409 That is all. Looks much simpler than the client, doesn't it?
410
411 Let's quickly look over it, as C<rcv> has already been discussed in the
412 client part of this tutorial above.
413
414 First this:
415
416 become_public "127.0.0.1:1299";
417
418 This will tell our I<node> to become a I<public> node, which means that it
419 can be contacted via TCP. The first argument should be the I<noderef> the
420 server wants to be reachable at. In this case it's the TCP port 1299 on
421 C<127.0.0.1>.
422
423 Next we set up two receivers, one for the C<join> messages and another one
424 for the actual messages of type C<messsage>. This is done with a single
425 call to C<rcv>, which allows multiple C<< match => $callback >> pairs.
426
427 In the C<join> callback we receive the client port, which is simply
428 remembered in the C<%client_ports> hash. In the C<message> callback we
429 just iterate through all known C<%client_ports> and relay the message to
430 them.
431
432 That concludes the server.
433
434 =head1 The Remaining Problems
435
436 The implementation as shown still has some bugs. For instance: How does
437 the server know that the client isn't there anymore, so it can clean up
438 the C<%client_ports> hash? Also, the chat messages have no originator, so
439 we don't know who actually sent the message (which would be quite useful
440 for human-to-human interaction: to know who the other one is :).
441
442 But aside from these issues I hope this tutorial showed you the basics of
443 L<AnyEvent::MP> and explained some common idioms.
444
445 How to solve the reliability and C<%client_ports> cleanup problem will
446 be explained later in this tutorial (TODO).
447
448 =head1 Inside The Protocol
449
450 Now, for the interested parties, let me explain some details about the protocol
451 that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
452 interested you can skip this section.
453
454 Usually TCP is used for communication. Each I<node>, if configured to be a
455 I<public> node with the C<become_public> function will listen on the configured
456 TCP port (default is 4040).
457
458 If then one I<node> wants to send a message to another I<node> it will
459 connect to the host and port given in the I<port ID>.
460
461 Then some handshaking occurs to check whether both I<nodes> know the
462 I<shared secret>. Optionally, TLS can be enabled (about how to do this
463 exactly please consult the L<AnyEvent::MP> man page, just a hint: It
464 should be enough to put the private key and (self signed) certificate in
465 the C<~/.aemp-secret> file of all nodes).
466
467 After the handshake, messages will be exchanged using a serialiser
468 (usually L<JSON> is used for this, but it is also possible to use other
469 serialization formats such as L<Storable>).
470
471 =head1 SEE ALSO
472
473 L<AnyEvent>
474
475 L<AnyEvent::Handle>
476
477 L<AnyEvent::MP>
478
479 =head1 AUTHOR
480
481 Robin Redeker <elmex@ta-sa.org>
482