ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.14
Committed: Fri Aug 14 03:27:59 2009 UTC (14 years, 9 months ago) by root
Branch: MAIN
Changes since 1.13: +18 -13 lines
Log Message:
*** empty log message ***

File Contents

# Content
1 =head1 Message Passing for the Non-Blocked Mind
2
3 =head1 Introduction and Terminology
4
5 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 module, which allows us to transparently pass messages to our own process
7 and to other processes on another or the same host.
8
9 What kind of messages? Well, basically a message here means a list of
10 Perl strings, numbers, hashes and arrays, anythin everything that can be
11 expressed as a L<JSON> text (as JSON is used by default in the protocol).
12
13 And next you might ask: between which entities are those messages
14 being "passed"? Effectively between I<nodes>: a nodes is basically a
15 process/program that use L<AnyEvent::MP> and can run either on the same or
16 different hosts.
17
18 To make this more managable, every node can contain any number of
19 I<ports>: Ports are ultimately the receivers of your messages.
20
21 In this tutorial I'll show you how to write a simple chat server based
22 on L<AnyEvent::MP>. This example is used because it nicely shows how to
23 organise a simple application, but keep in mind that every node trusts any
24 other, so this chat cannot be used to implement a real chat server and
25 client system, but it can be used to implement a distributed chat system
26 for example.
27
28 =head1 System Requirements and System Setup
29
30 Before we can start we have to make sure some things work on your
31 system.
32
33 You should of course first make sure that L<AnyEvent> and L<AnyEvent::MP>
34 are installed. But how to do that is out of scope of this tutorial.
35
36 Then we have to setup a I<shared secret>: for two L<AnyEvent::MP> nodes to
37 be able to communicate with each other and authenticate each other it is
38 necessary to setup the same I<shared secret> for both of them (use use TLS
39 certificates).
40
41 The easiest way is to set this up is to use the F<aemp> utility:
42
43 aemp gensecret
44
45 This creates a F<$HOME/.perl-anyevent-mp> config file and generates a
46 random shared secret. You can copy this file to any other system and then
47 communicate with it. You can also select your own shared secret (F<aemp
48 setsecret>) and for increased security requirements you can even create
49 a TLS certificate (F<aemp gencert>), causing connections to not just be
50 authenticated, but also to be encrypted.
51
52 Connections will only be successful when the nodes that want to connect to
53 each other have the same I<shared secret> (or successfully verify the TLS
54 certificate of the other side).
55
56 B<If something does not work as expected, and for example tcpdump shows
57 that the connections are closed almost immediatly, you should make sure
58 that F<~/.perl-anyevent-mp> is the same on all hosts/user accounts that
59 you try to connect with each other!>
60
61 =head1 The Chat Client
62
63 OK, lets start by implementing the "frontend" of the client. We will
64 develop the client first and postpone the server for later, as the most
65 complex things actually happen in the client.
66
67 We will use L<AnyEvent::Handle> to do non-blocking IO read on standard
68 input (all of this code deals with actually handling user input, no
69 message passing yet):
70
71 #!perl
72
73 use AnyEvent;
74 use AnyEvent::Handle;
75
76 sub send_message {
77 die "This is where we will send the messages to the server"
78 . "in the next step of this tutorial.\n"
79 }
80
81 # make an AnyEvent condition variable for the 'quit' condition
82 # (when we want to exit the client).
83 my $quit_cv = AnyEvent->condvar;
84
85 my $stdin_hdl = AnyEvent::Handle->new (
86 fh => *STDIN,
87 on_error => sub { $quit_cv->send },
88 on_read => sub {
89 my ($hdl) = @_;
90
91 $hdl->push_read (line => sub {
92 my ($hdl, $line) = @_;
93
94 if ($line =~ /^\/quit/) { # /quit will end the client
95 $quit_cv->send;
96 } else {
97 send_message ($line);
98 }
99 });
100 }
101 );
102
103 $quit_cv->recv;
104
105 This is now a very basic client. Explaining explicitly what
106 L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
107 of this document, please consult L<AnyEvent::Intro> or the manual pages
108 for L<AnyEvent> and L<AnyEvent::Handle>.
109
110 =head1 First Steps Into Messaging
111
112 To supply the C<send_message> function we now take a look at
113 L<AnyEvent::MP>. This is an example of how it might look like:
114
115 ... # the use lines from the above snippet
116
117 use AnyEvent::MP;
118
119 sub send_message {
120 my ($msg) = @_;
121
122 snd $server_port, message => $msg;
123 }
124
125 ... # the rest of the above script
126
127 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send
128 a message'. The first argument is the I<port> (a I<port> is something
129 that can receive messages, represented by a printable string) of the
130 server which will receive the message. How we get this port will be
131 explained in the next step.
132
133 The remaining arguments of C<snd> are C<message> and C<$msg>, the first
134 two elements of the I<message> (a I<message> in L<AnyEvent::MP> is a
135 simple list of values, which can be sent to a I<port>).
136
137 So all the function does is send the two values C<message> (a constant
138 string to tell the server what to expect) and the actual message string.
139
140 Thats all fine and simple so far, but where do we get the
141 C<$server_port>? Well, we need to get the unique I<port id> of the
142 server's port where it wants to receive all the incoming chat messages. A
143 I<port id> is unfortunately a very unique string, which we are unable to
144 know in advance. But L<AnyEvent::MP> supports the concept of 'registered
145 ports', which is basically a port on the server side registered under
146 a well known name.
147
148 For example, the server has a port for receiving chat messages with a
149 unique I<port id> and registers it under the name C<chatter>.
150
151 BTW, these "registered port names" should follow similar rules as Perl
152 identifiers, so you should prefix them with your package/module name to
153 make them unique, unless you use them in the main program.
154
155 As I<messages> can only be sent to a I<port id> and not just to some name
156 we have to ask the server node for the I<port id> of the port registered
157 as C<chatter>.
158
159 =head1 Finding The Chatter Port
160
161 Ok, lots of talk, now some code. Now we will actually get the
162 C<$server_port> from the backend:
163
164 ...
165
166 use AnyEvent::MP;
167
168 my $server_node = "127.0.0.1:1299";
169
170 my $client_port = port;
171
172 snd $server_node, lookup => "chatter", $client_port, "resolved";
173
174 my $resolved_cv = AnyEvent->condvar;
175 my $server_port;
176
177 # setup a receiver callback for the 'resolved' message:
178 rcv $client_port, resolved => sub {
179 my ($tag, $chatter_port_id) = @_;
180
181 print "Resolved the server port 'chatter' to $chatter_port_id\n";
182 $server_port = $chatter_port_id;
183
184 $resolved_cv->send;
185 1
186 };
187
188 # lets block the client until we have resolved the server port.
189 $resolved_cv->recv;
190
191 # now setup another receiver callback for the chat messages:
192 rcv $client_port, message => sub {
193 my ($tag, $msg) = @_;
194
195 print "chat> $msg\n";
196 0
197 };
198
199 # send a 'join' message to the server:
200 snd $server_port, join => "$client_port";
201
202 sub send_message { ...
203
204 Now that was a lot of new stuff:
205
206 First we define the C<$server_node>: In order to refer to another node
207 we need some kind of string to reference it - the node reference. The
208 I<noderef> is basically a comma separated list of C<address:port>
209 pairs. We assume in this tutorial that the server runs on C<127.0.0.1>
210 (localhost) on port 1299, which results in the noderef C<127.0.0.1:1299>.
211
212 Next, in order to receive a reply from the other node or the server we
213 need to have a I<port> that messages can be sent to. This is what the
214 C<port> function will do for us, it just creates a new local port and
215 returns it's I<port ID> that can then be used to receive messages.
216
217 When you look carefully, you will see that the first C<snd> uses the
218 C<$server_node> (a noderef) as destination port. Well, what I didn't
219 tell you yet is that each I<node> has a default I<port> to receive
220 messages. The ID of this port is the same as the noderef.
221
222 This I<default port> provides some special services for us, for example
223 resolving a registered name to a I<port id> (a-ha! finally!).
224
225 This is exactly what this line does:
226
227 snd $server_node, lookup => "chatter", $client_port, "resolved";
228
229 This sends a message with first element being C<lookup>, followed by the
230 (hopefully) registered port name that we want to resolve to a I<port
231 id>: C<chatter>. And in order for the server node to be able to send us
232 back the resolved I<port ID> we have to tell it where to send it: The
233 result message will be sent to C<$client_port> (the I<port id> of the
234 port we just created), and will have the string C<resolved> as the first
235 element.
236
237 When the node receives this message, it will look up the name, gobble up
238 all the extra arguments we passed, append the resolved name, and send the
239 resulting list as a message.
240
241 Next we register a receiver for this C<lookup>-request.
242
243 rcv $client_port, resolved => sub {
244 my ($tag, $chatter_port_id) = @_;
245 ...
246 1
247 };
248
249 This sets up a receiver on our own port for messages with the first
250 element being the string C<resolved>. Receivers can match the contents of
251 the messages before actually executing the specified callback.
252
253 B<Please note> that the every C<rcv> callback has to return either a true
254 or a false value, indicating whether it is B<successful>/B<done> (true) or
255 still wants to B<continue> (false) receiving messages.
256
257 In this case we tell the C<$client_port> to look into all the messages
258 it receives and look for the string C<resolved> in the first element of
259 the message. If it is found, the given callback will be called with the
260 message elements as arguments.
261
262 Using a string as the first element of the message is called I<tagging>
263 the message. It's common practise to code the 'type' of a message into
264 it's first element, as this allows for simple matching.
265
266 The result message will contain the I<port ID> of the well known port
267 C<chatter> as second element, which will be stored in C<$chatter_port_id>.
268
269 This port ID will then be stored in C<$server_port>, followed by calling
270 C<send> on $resolved_cv> so the program will continue.
271
272 The callback then returns a C<1> (a true value), to indicate that it has
273 done it's job and doesn't want to receive further C<resolved> messages.
274
275 After this the chat message receiver callback is registered with the port:
276
277 rcv $client_port, message => sub {
278 my ($tag, $msg) = @_;
279
280 print "chat> $msg\n";
281
282 0
283 };
284
285 We assume that all messages that are broadcast to the clients by the
286 server contain the string tag C<message> as first element, and the actual
287 message as second element. The callback returns a false value this time,
288 to indicate that it is not yet done and wants to receive further messages.
289
290 The last thing to do is to tell the server to send us new chat messages
291 from other clients. We do so by sending the message C<join> followed by
292 our own I<port ID>.
293
294 # send the server a 'join' message:
295 snd $server_port, join => $client_port;
296
297 This way the server knows where to send all the new messages to.
298
299 =head1 The Completed Client
300
301 This is the complete client script:
302
303 #!perl
304
305 use AnyEvent;
306 use AnyEvent::Handle;
307 use AnyEvent::MP;
308
309 my $server_node = "127.0.0.1:1299";
310
311 my $client_port = port;
312
313 snd $server_node, lookup => "chatter", $client_port, "resolved";
314
315 my $resolved_cv = AnyEvent->condvar;
316 my $server_port;
317
318 # setup a receiver callback for the 'resolved' message:
319 rcv $client_port, resolved => sub {
320 my ($tag, $chatter_port_id) = @_;
321
322 print "Resolved the server port 'chatter' to $chatter_port_id\n";
323 $server_port = $chatter_port_id;
324
325 $resolved_cv->send;
326 1
327 };
328
329 # lets block the client until we have resolved the server port.
330 $resolved_cv->recv;
331
332 # now setup another receiver callback for the chat messages:
333 rcv $client_port, message => sub {
334 my ($tag, $msg) = @_;
335
336 print "chat> $msg\n";
337 0
338 };
339
340 # send a 'join' message to the server:
341 snd $server_port, join => "$client_port";
342
343 sub send_message {
344 my ($msg) = @_;
345
346 snd $server_port, message => $msg;
347 }
348
349 # make an AnyEvent condition variable for the 'quit' condition
350 # (when we want to exit the client).
351 my $quit_cv = AnyEvent->condvar;
352
353 my $stdin_hdl = AnyEvent::Handle->new (
354 fh => *STDIN,
355 on_error => sub { $quit_cv->send },
356 on_read => sub {
357 my ($hdl) = @_;
358
359 $hdl->push_read (line => sub {
360 my ($hdl, $line) = @_;
361
362 if ($line =~ /^\/quit/) { # /quit will end the client
363 $quit_cv->send;
364 } else {
365 send_message ($line);
366 }
367 });
368 }
369 );
370
371 $quit_cv->recv;
372
373 =head1 The Server
374
375 Ok, we finally come to the server.
376
377 The server of course also needs to set up a port, and in addition needs to
378 I<register> it, so the clients can find it.
379
380 Again, let's jump directly into the code:
381
382 #!perl
383
384 use AnyEvent;
385 use AnyEvent::MP;
386
387 become_public "127.0.0.1:1299";
388
389 my $chatter_port = port;
390
391 reg $chatter_port, "chatter";
392
393 my %client_ports;
394
395 rcv $chatter_port,
396 join => sub {
397 my ($tag, $client_port) = @_;
398
399 print "got new client port: $client_port\n";
400 $client_ports{$client_port} = 1;
401
402 0
403 },
404 message => sub {
405 my ($tag, $msg) = @_;
406
407 print "message> $msg\n";
408
409 snd $_, message => $msg
410 for keys %client_ports;
411
412 0
413 };
414
415 AnyEvent->condvar->recv;
416
417 That is all. Looks much simpler than the client, doesn't it?
418
419 Let's quickly look over it, as C<rcv> has already been discussed in the
420 client part of this tutorial above.
421
422 First this:
423
424 become_public "127.0.0.1:1299";
425
426 This will tell our I<node> to become a I<public> node, which means that it
427 can be contacted via TCP. The first argument should be the I<noderef> the
428 server wants to be reachable at. In this case it's the TCP port 1299 on
429 C<127.0.0.1>.
430
431 Next we set up two receivers, one for the C<join> messages and another one
432 for the actual messages of type C<messsage>. This is done with a single
433 call to C<rcv>, which allows multiple C<< match => $callback >> pairs.
434
435 In the C<join> callback we receive the client port, which is simply
436 remembered in the C<%client_ports> hash. In the C<message> callback we
437 just iterate through all known C<%client_ports> and relay the message to
438 them.
439
440 That concludes the server.
441
442 =head1 The Remaining Problems
443
444 The implementation as shown still has some bugs. For instance: How does
445 the server know that the client isn't there anymore, so it can clean up
446 the C<%client_ports> hash? Also, the chat messages have no originator, so
447 we don't know who actually sent the message (which would be quite useful
448 for human-to-human interaction: to know who the other one is :).
449
450 But aside from these issues I hope this tutorial showed you the basics of
451 L<AnyEvent::MP> and explained some common idioms.
452
453 How to solve the reliability and C<%client_ports> cleanup problem will
454 be explained later in this tutorial (TODO).
455
456 =head1 Inside The Protocol
457
458 Now, for the interested parties, let me explain some details about the protocol
459 that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
460 interested you can skip this section.
461
462 Usually TCP is used for communication. Each I<node>, if configured to be
463 a I<public> node with the C<initialise_node> function will listen on the
464 configured TCP port (default is 4040).
465
466 If then one I<node> wants to send a message to another I<node> it will
467 connect to the host and port given in the I<port ID>.
468
469 Then some handshaking occurs to check whether both I<nodes> know the
470 I<shared secret>. Optionally, TLS can be enabled (about how to do this
471 exactly please consult the L<AnyEvent::MP> man page, just a hint: It
472 should be enough to put the private key and (self signed) certificate in
473 the C<~/.aemp-secret> file of all nodes).
474
475 After the handshake, messages will be exchanged using a serialiser
476 (usually L<JSON> is used for this, but it is also possible to use other
477 serialization formats such as L<Storable>).
478
479 =head1 SEE ALSO
480
481 L<AnyEvent>
482
483 L<AnyEvent::Handle>
484
485 L<AnyEvent::MP>
486
487 =head1 AUTHOR
488
489 Robin Redeker <elmex@ta-sa.org>
490