ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.13
Committed: Thu Aug 13 20:47:54 2009 UTC (14 years, 9 months ago) by root
Branch: MAIN
Changes since 1.12: +6 -5 lines
Log Message:
*** empty log message ***

File Contents

# Content
1 =head1 Message Passing for the Non-Blocked Mind
2
3 =head1 Introduction and Terminology
4
5 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 module. Which allows us to transparently pass messages to our own process
7 and to other processes on another or the same host.
8
9 What kind of messages? Well, basically a message here means a list of
10 Perl strings, numbers, hashes and arrays, mostly everything that can be
11 expressed as a L<JSON> text (as JSON is used by default in the protocol).
12
13 And next you might ask: between which entities are those messages
14 being "passed"? Effectively between I<nodes>: a nodes is basically a
15 process/program that use L<AnyEvent::MP> and can run either on the same or
16 different hosts.
17
18 To make this more managable, every node can contain any number of
19 I<ports>: Ports are ultimately the receivers of your messages.
20
21 In this tutorial I'll show you how to write a simple chat server based on
22 L<AnyEvent::MP>.
23
24 =head1 System Requirements and System Setup
25
26 Before we can start we have to make sure some things work on your
27 system.
28
29 You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
30 are installed. But how to do that is out of scope of this tutorial.
31
32 Then we have to setup a I<shared secret>: for two L<AnyEvent::MP> nodes to
33 be able to communicate with each other and authenticate each other it is
34 necessary to setup the same I<shared secret> for both of them.
35
36 The easiest way is to use the F<aemp> utility:
37
38 aemp gensecret
39
40 This creates the F<$HOME/.perl-anyevent-mp> config file and generates a
41 random shared secret. You can copy this file to any other system and then
42 communicate with it. You can also select your own shared secret (F<aemp
43 setsecret>) and for increased security requirements you can even create
44 a TLS certificate (F<aemp gencert>), causing conenctions to not just be
45 authenticated, but also to be encrypted.
46
47 Connections will only be successful when the nodes that want to connect to
48 each other have the same I<shared secret> (or successfully verify the TLS
49 certificate).
50
51 B<If something does not work as expected, and for example tcpdump shows
52 that the connections are closed almost immediatly, you should make sure
53 that F<~/.perl-anyevent-mp> is the same on all hosts/user accounts that
54 you try to connect with each other!>
55
56 =head1 The Chat Client
57
58 OK, lets start by implementing the "frontend" of the client. We will
59 develop the client first and postpone the server for later, as the most
60 complex things actually happen in the client.
61
62 We will use L<AnyEvent::Handle> to do non-blocking IO read on standard
63 input (all of this code deals with actually handling user input, no
64 message passing yet):
65
66 #!perl
67
68 use AnyEvent;
69 use AnyEvent::Handle;
70
71 sub send_message {
72 die "This is where we will send the messages to the server"
73 . "in the next step of this tutorial.\n"
74 }
75
76 # make an AnyEvent condition variable for the 'quit' condition
77 # (when we want to exit the client).
78 my $quit_cv = AnyEvent->condvar;
79
80 my $stdin_hdl = AnyEvent::Handle->new (
81 fh => *STDIN,
82 on_error => sub { $quit_cv->send },
83 on_read => sub {
84 my ($hdl) = @_;
85
86 $hdl->push_read (line => sub {
87 my ($hdl, $line) = @_;
88
89 if ($line =~ /^\/quit/) { # /quit will end the client
90 $quit_cv->send;
91 } else {
92 send_message ($line);
93 }
94 });
95 }
96 );
97
98 $quit_cv->recv;
99
100 This is now a very basic client. Explaining explicitly what
101 L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
102 of this document, please consult L<AnyEvent::Intro> or the manual pages
103 for L<AnyEvent> and L<AnyEvent::Handle>.
104
105 =head1 First Steps Into Messaging
106
107 To supply the C<send_message> function we now take a look at
108 L<AnyEvent::MP>. This is an example of how it might look like:
109
110 ... # the use lines from the above snippet
111
112 use AnyEvent::MP;
113
114 sub send_message {
115 my ($msg) = @_;
116
117 snd $server_port, message => $msg;
118 }
119
120 ... # the rest of the above script
121
122 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send
123 a message'. The first argument is the I<port> (a I<port> is something
124 that can receive messages, represented by a printable string) of the
125 server which will receive the message. How we get this port will be
126 explained in the next step.
127
128 The remaining arguments of C<snd> are C<message> and C<$msg>, the first
129 two elements of the I<message> (a I<message> in L<AnyEvent::MP> is a
130 simple list of values, which can be sent to a I<port>).
131
132 So all the function does is send the two values C<message> (a constant
133 string to tell the server what to expect) and the actual message string.
134
135 Thats all fine and simple so far, but where do we get the
136 C<$server_port>? Well, we need to get the unique I<port id> of the
137 server's port where it wants to receive all the incoming chat messages. A
138 I<port id> is unfortunately a very unique string, which we are unable to
139 know in advance. But L<AnyEvent::MP> supports the concept of 'registered
140 ports', which is basically a port on the server side registered under
141 a well known name.
142
143 For example, the server has a port for receiving chat messages with a
144 unique I<port id> and registers it under the name C<chatter>.
145
146 BTW, these "registered port names" should follow similar rules as Perl
147 identifiers, so you should prefix them with your package/module name to
148 make them unique, unless you use them in the main program.
149
150 As I<messages> can only be sent to a I<port id> and not just to some name
151 we have to ask the server node for the I<port id> of the port registered
152 as C<chatter>.
153
154 =head1 Finding The Chatter Port
155
156 Ok, lots of talk, now some code. Now we will actually get the
157 C<$server_port> from the backend:
158
159 ...
160
161 use AnyEvent::MP;
162
163 my $server_node = "127.0.0.1:1299";
164
165 my $client_port = port;
166
167 snd $server_node, lookup => "chatter", $client_port, "resolved";
168
169 my $resolved_cv = AnyEvent->condvar;
170 my $server_port;
171
172 # setup a receiver callback for the 'resolved' message:
173 rcv $client_port, resolved => sub {
174 my ($tag, $chatter_port_id) = @_;
175
176 print "Resolved the server port 'chatter' to $chatter_port_id\n";
177 $server_port = $chatter_port_id;
178
179 $resolved_cv->send;
180 1
181 };
182
183 # lets block the client until we have resolved the server port.
184 $resolved_cv->recv;
185
186 # now setup another receiver callback for the chat messages:
187 rcv $client_port, message => sub {
188 my ($tag, $msg) = @_;
189
190 print "chat> $msg\n";
191 0
192 };
193
194 # send a 'join' message to the server:
195 snd $server_port, join => "$client_port";
196
197 sub send_message { ...
198
199 Now that was a lot of new stuff:
200
201 First we define the C<$server_node>: In order to refer to another node
202 we need some kind of string to reference it - the node reference. The
203 I<noderef> is basically a comma separated list of C<address:port>
204 pairs. We assume in this tutorial that the server runs on C<127.0.0.1>
205 (localhost) on port 1299, which results in the noderef C<127.0.0.1:1299>.
206
207 Next, in order to receive a reply from the other node or the server we
208 need to have a I<port> that messages can be sent to. This is what the
209 C<port> function will do for us, it just creates a new local port and
210 returns it's I<port ID> that can then be used to receive messages.
211
212 When you look carefully, you will see that the first C<snd> uses the
213 C<$server_node> (a noderef) as destination port. Well, what I didn't
214 tell you yet is that each I<node> has a default I<port> to receive
215 messages. The ID of this port is the same as the noderef.
216
217 This I<default port> provides some special services for us, for example
218 resolving a registered name to a I<port id> (a-ha! finally!).
219
220 This is exactly what this line does:
221
222 snd $server_node, lookup => "chatter", $client_port, "resolved";
223
224 This sends a message with first element being C<lookup>, followed by the
225 (hopefully) registered port name that we want to resolve to a I<port
226 id>: C<chatter>. And in order for the server node to be able to send us
227 back the resolved I<port ID> we have to tell it where to send it: The
228 result message will be sent to C<$client_port> (the I<port id> of the
229 port we just created), and will have the string C<resolved> as the first
230 element.
231
232 When the node receives this message, it will look up the name, gobble up
233 all the extra arguments we passed, append the resolved name, and send the
234 resulting list as a message.
235
236 Next we register a receiver for this C<lookup>-request.
237
238 rcv $client_port, resolved => sub {
239 my ($tag, $chatter_port_id) = @_;
240 ...
241 1
242 };
243
244 This sets up a receiver on our own port for messages with the first
245 element being the string C<resolved>. Receivers can match the contents of
246 the messages before actually executing the specified callback.
247
248 B<Please note> that the every C<rcv> callback has to return either a true
249 or a false value, indicating whether it is B<successful>/B<done> (true) or
250 still wants to B<continue> (false) receiving messages.
251
252 In this case we tell the C<$client_port> to look into all the messages
253 it receives and look for the string C<resolved> in the first element of
254 the message. If it is found, the given callback will be called with the
255 message elements as arguments.
256
257 Using a string as the first element of the message is called I<tagging>
258 the message. It's common practise to code the 'type' of a message into
259 it's first element, as this allows for simple matching.
260
261 The result message will contain the I<port ID> of the well known port
262 C<chatter> as second element, which will be stored in C<$chatter_port_id>.
263
264 This port ID will then be stored in C<$server_port>, followed by calling
265 C<send> on $resolved_cv> so the program will continue.
266
267 The callback then returns a C<1> (a true value), to indicate that it has
268 done it's job and doesn't want to receive further C<resolved> messages.
269
270 After this the chat message receiver callback is registered with the port:
271
272 rcv $client_port, message => sub {
273 my ($tag, $msg) = @_;
274
275 print "chat> $msg\n";
276
277 0
278 };
279
280 We assume that all messages that are broadcast to the clients by the
281 server contain the string tag C<message> as first element, and the actual
282 message as second element. The callback returns a false value this time,
283 to indicate that it is not yet done and wants to receive further messages.
284
285 The last thing to do is to tell the server to send us new chat messages
286 from other clients. We do so by sending the message C<join> followed by
287 our own I<port ID>.
288
289 # send the server a 'join' message:
290 snd $server_port, join => $client_port;
291
292 This way the server knows where to send all the new messages to.
293
294 =head1 The Completed Client
295
296 This is the complete client script:
297
298 #!perl
299
300 use AnyEvent;
301 use AnyEvent::Handle;
302 use AnyEvent::MP;
303
304 my $server_node = "127.0.0.1:1299";
305
306 my $client_port = port;
307
308 snd $server_node, lookup => "chatter", $client_port, "resolved";
309
310 my $resolved_cv = AnyEvent->condvar;
311 my $server_port;
312
313 # setup a receiver callback for the 'resolved' message:
314 rcv $client_port, resolved => sub {
315 my ($tag, $chatter_port_id) = @_;
316
317 print "Resolved the server port 'chatter' to $chatter_port_id\n";
318 $server_port = $chatter_port_id;
319
320 $resolved_cv->send;
321 1
322 };
323
324 # lets block the client until we have resolved the server port.
325 $resolved_cv->recv;
326
327 # now setup another receiver callback for the chat messages:
328 rcv $client_port, message => sub {
329 my ($tag, $msg) = @_;
330
331 print "chat> $msg\n";
332 0
333 };
334
335 # send a 'join' message to the server:
336 snd $server_port, join => "$client_port";
337
338 sub send_message {
339 my ($msg) = @_;
340
341 snd $server_port, message => $msg;
342 }
343
344 # make an AnyEvent condition variable for the 'quit' condition
345 # (when we want to exit the client).
346 my $quit_cv = AnyEvent->condvar;
347
348 my $stdin_hdl = AnyEvent::Handle->new (
349 fh => *STDIN,
350 on_error => sub { $quit_cv->send },
351 on_read => sub {
352 my ($hdl) = @_;
353
354 $hdl->push_read (line => sub {
355 my ($hdl, $line) = @_;
356
357 if ($line =~ /^\/quit/) { # /quit will end the client
358 $quit_cv->send;
359 } else {
360 send_message ($line);
361 }
362 });
363 }
364 );
365
366 $quit_cv->recv;
367
368 =head1 The Server
369
370 Ok, we finally come to the server.
371
372 The server of course also needs to set up a port, and in addition needs to
373 I<register> it, so the clients can find it.
374
375 Again, let's jump directly into the code:
376
377 #!perl
378
379 use AnyEvent;
380 use AnyEvent::MP;
381
382 become_public "127.0.0.1:1299";
383
384 my $chatter_port = port;
385
386 reg $chatter_port, "chatter";
387
388 my %client_ports;
389
390 rcv $chatter_port,
391 join => sub {
392 my ($tag, $client_port) = @_;
393
394 print "got new client port: $client_port\n";
395 $client_ports{$client_port} = 1;
396
397 0
398 },
399 message => sub {
400 my ($tag, $msg) = @_;
401
402 print "message> $msg\n";
403
404 snd $_, message => $msg
405 for keys %client_ports;
406
407 0
408 };
409
410 AnyEvent->condvar->recv;
411
412 That is all. Looks much simpler than the client, doesn't it?
413
414 Let's quickly look over it, as C<rcv> has already been discussed in the
415 client part of this tutorial above.
416
417 First this:
418
419 become_public "127.0.0.1:1299";
420
421 This will tell our I<node> to become a I<public> node, which means that it
422 can be contacted via TCP. The first argument should be the I<noderef> the
423 server wants to be reachable at. In this case it's the TCP port 1299 on
424 C<127.0.0.1>.
425
426 Next we set up two receivers, one for the C<join> messages and another one
427 for the actual messages of type C<messsage>. This is done with a single
428 call to C<rcv>, which allows multiple C<< match => $callback >> pairs.
429
430 In the C<join> callback we receive the client port, which is simply
431 remembered in the C<%client_ports> hash. In the C<message> callback we
432 just iterate through all known C<%client_ports> and relay the message to
433 them.
434
435 That concludes the server.
436
437 =head1 The Remaining Problems
438
439 The implementation as shown still has some bugs. For instance: How does
440 the server know that the client isn't there anymore, so it can clean up
441 the C<%client_ports> hash? Also, the chat messages have no originator, so
442 we don't know who actually sent the message (which would be quite useful
443 for human-to-human interaction: to know who the other one is :).
444
445 But aside from these issues I hope this tutorial showed you the basics of
446 L<AnyEvent::MP> and explained some common idioms.
447
448 How to solve the reliability and C<%client_ports> cleanup problem will
449 be explained later in this tutorial (TODO).
450
451 =head1 Inside The Protocol
452
453 Now, for the interested parties, let me explain some details about the protocol
454 that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
455 interested you can skip this section.
456
457 Usually TCP is used for communication. Each I<node>, if configured to be a
458 I<public> node with the C<become_public> function will listen on the configured
459 TCP port (default is 4040).
460
461 If then one I<node> wants to send a message to another I<node> it will
462 connect to the host and port given in the I<port ID>.
463
464 Then some handshaking occurs to check whether both I<nodes> know the
465 I<shared secret>. Optionally, TLS can be enabled (about how to do this
466 exactly please consult the L<AnyEvent::MP> man page, just a hint: It
467 should be enough to put the private key and (self signed) certificate in
468 the C<~/.aemp-secret> file of all nodes).
469
470 After the handshake, messages will be exchanged using a serialiser
471 (usually L<JSON> is used for this, but it is also possible to use other
472 serialization formats such as L<Storable>).
473
474 =head1 SEE ALSO
475
476 L<AnyEvent>
477
478 L<AnyEvent::Handle>
479
480 L<AnyEvent::MP>
481
482 =head1 AUTHOR
483
484 Robin Redeker <elmex@ta-sa.org>
485