ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.12
Committed: Mon Aug 10 19:23:43 2009 UTC (14 years, 9 months ago) by root
Branch: MAIN
CVS Tags: rel-0_6
Changes since 1.11: +17 -15 lines
Log Message:
*** empty log message ***

File Contents

# Content
1 =head1 Message Passing for the Non-Blocked Mind
2
3 =head1 Introduction and Terminology
4
5 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 module. Which allows us to transparently pass messages to our own process
7 and to other processes on another or the same host.
8
9 What kind of messages? Well, basically a message here means a list of
10 Perl strings, numbers, hashes and arrays, mostly everything that can be
11 expressed as a L<JSON> text (as JSON is used by default in the protocol).
12
13 And next you might ask: between which entities are those messages
14 being "passed"? Effectively between I<nodes>: a nodes is basically a
15 process/program that use L<AnyEvent::MP> and can run either on the same or
16 different hosts.
17
18 To make this more managable, every node can contain any number of
19 I<ports>: Ports are ultimately the receivers of your messages.
20
21 In this tutorial I'll show you how to write a simple chat server based on
22 L<AnyEvent::MP>.
23
24 =head1 System Requirements
25
26 Before we can start we have to make sure some things work on your
27 system.
28
29 You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
30 are installed. But how to do that is out of scope of this tutorial.
31
32 Then we have to setup a I<shared secret>. For two L<AnyEvent::MP> nodes to
33 be able to communicate with each other and authenticate each other it is
34 necessary to setup the same I<shared secret> for both of them.
35
36 The easiest way is to use the F<aemp> utility:
37
38 aemp gensecret
39
40 This creates the F<$HOME/.perl-anyevent-mp> config file and generates a
41 random shared secret. You can copy this file to any other system and then
42 communicate with it. You cna also select your own shared secret (F<aemp
43 setsecret>) and for increased security requirements you can even create a
44 TLS certificate (F<aemp gencert>).
45
46 Connections will only be successful when the nodes that want to connect to
47 each other have the same I<shared secret> (or successfully verify the TLS
48 certificate).
49
50 B<If something does not work as expected, and for example tcpdump shows
51 that the connections are closed almost immediatly, you should make sure
52 that F<~/.perl-anyevent-mp> is the same on all hosts/user accounts that
53 you try to connect with each other!>
54
55 =head1 The Chat Client
56
57 OK, lets start by implementing the "frontend" of the client. We will
58 develop the client first and postpone the server for later, as the most
59 complex things actually happen in the client.
60
61 We will use L<AnyEvent::Handle> to do non-blocking IO read on standard
62 input (all of this code deals with actually handling user input, no
63 message passing yet):
64
65 #!perl
66
67 use AnyEvent;
68 use AnyEvent::Handle;
69
70 sub send_message {
71 die "This is where we will send the messages to the server"
72 . "in the next step of this tutorial.\n"
73 }
74
75 # make an AnyEvent condition variable for the 'quit' condition
76 # (when we want to exit the client).
77 my $quit_cv = AnyEvent->condvar;
78
79 my $stdin_hdl = AnyEvent::Handle->new (
80 fh => *STDIN,
81 on_error => sub { $quit_cv->send },
82 on_read => sub {
83 my ($hdl) = @_;
84
85 $hdl->push_read (line => sub {
86 my ($hdl, $line) = @_;
87
88 if ($line =~ /^\/quit/) { # /quit will end the client
89 $quit_cv->send;
90 } else {
91 send_message ($line);
92 }
93 });
94 }
95 );
96
97 $quit_cv->recv;
98
99 This is now a very basic client. Explaining explicitly what
100 L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
101 of this document, please consult L<AnyEvent::Intro> or the manual pages
102 for L<AnyEvent> and L<AnyEvent::Handle>.
103
104 =head1 First Steps Into Messaging
105
106 To supply the C<send_message> function we now take a look at
107 L<AnyEvent::MP>. This is an example of how it might look like:
108
109 ... # the use lines from the above snippet
110
111 use AnyEvent::MP;
112
113 sub send_message {
114 my ($msg) = @_;
115
116 snd $server_port, message => $msg;
117 }
118
119 ... # the rest of the above script
120
121 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send
122 a message'. The first argument is the I<port> (a I<port> is something
123 that can receive messages, represented by a printable string) of the
124 server which will receive the message. How we get this port will be
125 explained in the next step.
126
127 The remaining arguments of C<snd> are C<message> and C<$msg>, the first
128 two elements of the I<message> (a I<message> in L<AnyEvent::MP> is a
129 simple list of values, which can be sent to a I<port>).
130
131 So all the function does is send the two values C<message> (a constant
132 string to tell the server what to expect) and the actual message string.
133
134 Thats all fine and simple so far, but where do we get the
135 C<$server_port>? Well, we need to get the unique I<port id> of the
136 server's port where it wants to receive all the incoming chat messages. A
137 I<port id> is unfortunately a very unique string, which we are unable to
138 know in advance. But L<AnyEvent::MP> supports the concept of 'registered
139 ports', which is basically a port on the server side registered under
140 a well known name.
141
142 For example, the server has a port for receiving chat messages with a
143 unique I<port id> and registers it under the name C<chatter>.
144
145 BTW, these "registered port names" should follow similar rules as Perl
146 identifiers, so you should prefix them with your package/module name to
147 make them unique, unless you use them in the main program.
148
149 As I<messages> can only be sent to a I<port id> and not just to some name
150 we have to ask the server node for the I<port id> of the port registered
151 as C<chatter>.
152
153 =head1 Finding The Chatter Port
154
155 Ok, lots of talk, now some code. Now we will actually get the
156 C<$server_port> from the backend:
157
158 ...
159
160 use AnyEvent::MP;
161
162 my $server_node = "127.0.0.1:1299";
163
164 my $client_port = port;
165
166 snd $server_node, lookup => "chatter", $client_port, "resolved";
167
168 my $resolved_cv = AnyEvent->condvar;
169 my $server_port;
170
171 # setup a receiver callback for the 'resolved' message:
172 rcv $client_port, resolved => sub {
173 my ($tag, $chatter_port_id) = @_;
174
175 print "Resolved the server port 'chatter' to $chatter_port_id\n";
176 $server_port = $chatter_port_id;
177
178 $resolved_cv->send;
179 1
180 };
181
182 # lets block the client until we have resolved the server port.
183 $resolved_cv->recv;
184
185 # now setup another receiver callback for the chat messages:
186 rcv $client_port, message => sub {
187 my ($tag, $msg) = @_;
188
189 print "chat> $msg\n";
190 0
191 };
192
193 # send a 'join' message to the server:
194 snd $server_port, join => "$client_port";
195
196 sub send_message { ...
197
198 Now that was a lot of new stuff:
199
200 First we define the C<$server_node>: In order to refer to another node
201 we need some kind of string to reference it - the node reference. The
202 I<noderef> is basically a comma separated list of C<address:port>
203 pairs. We assume in this tutorial that the server runs on C<127.0.0.1>
204 (localhost) on port 1299, which results in the noderef C<127.0.0.1:1299>.
205
206 Next, in order to receive a reply from the other node or the server we
207 need to have a I<port> that messages can be sent to. This is what the
208 C<port> function will do for us, it just creates a new local port and
209 returns it's I<port ID> that can then be used to receive messages.
210
211 When you look carefully, you will see that the first C<snd> uses the
212 C<$server_node> (a noderef) as destination port. Well, what I didn't
213 tell you yet is that each I<node> has a default I<port> to receive
214 messages. The ID of this port is the same as the noderef.
215
216 This I<default port> provides some special services for us, for example
217 resolving a registered name to a I<port id> (a-ha! finally!).
218
219 This is exactly what this line does:
220
221 snd $server_node, lookup => "chatter", $client_port, "resolved";
222
223 This sends a message with first element being C<lookup>, followed by the
224 (hopefully) registered port name that we want to resolve to a I<port
225 id>: C<chatter>. And in order for the server node to be able to send us
226 back the resolved I<port ID> we have to tell it where to send it: The
227 result message will be sent to C<$client_port> (the I<port id> of the
228 port we just created), and will have the string C<resolved> as the first
229 element.
230
231 When the node receives this message, it will look up the name, gobble up
232 all the extra arguments we passed, append the resolved name, and send the
233 resulting list as a message.
234
235 Next we register a receiver for this C<lookup>-request.
236
237 rcv $client_port, resolved => sub {
238 my ($tag, $chatter_port_id) = @_;
239 ...
240 1
241 };
242
243 This sets up a receiver on our own port for messages with the first
244 element being the string C<resolved>. Receivers can match the contents of
245 the messages before actually executing the specified callback.
246
247 B<Please note> that the every C<rcv> callback has to return either a true
248 or a false value, indicating whether it is B<successful>/B<done> (true) or
249 still wants to B<continue> (false) receiving messages.
250
251 In this case we tell the C<$client_port> to look into all the messages
252 it receives and look for the string C<resolved> in the first element of
253 the message. If it is found, the given callback will be called with the
254 message elements as arguments.
255
256 Using a string as the first element of the message is called I<tagging>
257 the message. It's common practise to code the 'type' of a message into
258 it's first element, as this allows for simple matching.
259
260 The result message will contain the I<port ID> of the well known port
261 C<chatter> as second element, which will be stored in C<$chatter_port_id>.
262
263 This port ID will then be stored in C<$server_port>, followed by calling
264 C<send> on $resolved_cv> so the program will continue.
265
266 The callback then returns a C<1> (a true value), to indicate that it has
267 done it's job and doesn't want to receive further C<resolved> messages.
268
269 After this the chat message receiver callback is registered with the port:
270
271 rcv $client_port, message => sub {
272 my ($tag, $msg) = @_;
273
274 print "chat> $msg\n";
275
276 0
277 };
278
279 We assume that all messages that are broadcast to the clients by the
280 server contain the string tag C<message> as first element, and the actual
281 message as second element. The callback returns a false value this time,
282 to indicate that it is not yet done and wants to receive further messages.
283
284 The last thing to do is to tell the server to send us new chat messages
285 from other clients. We do so by sending the message C<join> followed by
286 our own I<port ID>.
287
288 # send the server a 'join' message:
289 snd $server_port, join => $client_port;
290
291 This way the server knows where to send all the new messages to.
292
293 =head1 The Completed Client
294
295 This is the complete client script:
296
297 #!perl
298
299 use AnyEvent;
300 use AnyEvent::Handle;
301 use AnyEvent::MP;
302
303 my $server_node = "127.0.0.1:1299";
304
305 my $client_port = port;
306
307 snd $server_node, lookup => "chatter", $client_port, "resolved";
308
309 my $resolved_cv = AnyEvent->condvar;
310 my $server_port;
311
312 # setup a receiver callback for the 'resolved' message:
313 rcv $client_port, resolved => sub {
314 my ($tag, $chatter_port_id) = @_;
315
316 print "Resolved the server port 'chatter' to $chatter_port_id\n";
317 $server_port = $chatter_port_id;
318
319 $resolved_cv->send;
320 1
321 };
322
323 # lets block the client until we have resolved the server port.
324 $resolved_cv->recv;
325
326 # now setup another receiver callback for the chat messages:
327 rcv $client_port, message => sub {
328 my ($tag, $msg) = @_;
329
330 print "chat> $msg\n";
331 0
332 };
333
334 # send a 'join' message to the server:
335 snd $server_port, join => "$client_port";
336
337 sub send_message {
338 my ($msg) = @_;
339
340 snd $server_port, message => $msg;
341 }
342
343 # make an AnyEvent condition variable for the 'quit' condition
344 # (when we want to exit the client).
345 my $quit_cv = AnyEvent->condvar;
346
347 my $stdin_hdl = AnyEvent::Handle->new (
348 fh => *STDIN,
349 on_error => sub { $quit_cv->send },
350 on_read => sub {
351 my ($hdl) = @_;
352
353 $hdl->push_read (line => sub {
354 my ($hdl, $line) = @_;
355
356 if ($line =~ /^\/quit/) { # /quit will end the client
357 $quit_cv->send;
358 } else {
359 send_message ($line);
360 }
361 });
362 }
363 );
364
365 $quit_cv->recv;
366
367 =head1 The Server
368
369 Ok, we finally come to the server.
370
371 The server of course also needs to set up a port, and in addition needs to
372 I<register> it, so the clients can find it.
373
374 Again, let's jump directly into the code:
375
376 #!perl
377
378 use AnyEvent;
379 use AnyEvent::MP;
380
381 become_public "127.0.0.1:1299";
382
383 my $chatter_port = port;
384
385 reg $chatter_port, "chatter";
386
387 my %client_ports;
388
389 rcv $chatter_port,
390 join => sub {
391 my ($tag, $client_port) = @_;
392
393 print "got new client port: $client_port\n";
394 $client_ports{$client_port} = 1;
395
396 0
397 },
398 message => sub {
399 my ($tag, $msg) = @_;
400
401 print "message> $msg\n";
402
403 snd $_, message => $msg
404 for keys %client_ports;
405
406 0
407 };
408
409 AnyEvent->condvar->recv;
410
411 That is all. Looks much simpler than the client, doesn't it?
412
413 Let's quickly look over it, as C<rcv> has already been discussed in the
414 client part of this tutorial above.
415
416 First this:
417
418 become_public "127.0.0.1:1299";
419
420 This will tell our I<node> to become a I<public> node, which means that it
421 can be contacted via TCP. The first argument should be the I<noderef> the
422 server wants to be reachable at. In this case it's the TCP port 1299 on
423 C<127.0.0.1>.
424
425 Next we set up two receivers, one for the C<join> messages and another one
426 for the actual messages of type C<messsage>. This is done with a single
427 call to C<rcv>, which allows multiple C<< match => $callback >> pairs.
428
429 In the C<join> callback we receive the client port, which is simply
430 remembered in the C<%client_ports> hash. In the C<message> callback we
431 just iterate through all known C<%client_ports> and relay the message to
432 them.
433
434 That concludes the server.
435
436 =head1 The Remaining Problems
437
438 The implementation as shown still has some bugs. For instance: How does
439 the server know that the client isn't there anymore, so it can clean up
440 the C<%client_ports> hash? Also, the chat messages have no originator, so
441 we don't know who actually sent the message (which would be quite useful
442 for human-to-human interaction: to know who the other one is :).
443
444 But aside from these issues I hope this tutorial showed you the basics of
445 L<AnyEvent::MP> and explained some common idioms.
446
447 How to solve the reliability and C<%client_ports> cleanup problem will
448 be explained later in this tutorial (TODO).
449
450 =head1 Inside The Protocol
451
452 Now, for the interested parties, let me explain some details about the protocol
453 that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
454 interested you can skip this section.
455
456 Usually TCP is used for communication. Each I<node>, if configured to be a
457 I<public> node with the C<become_public> function will listen on the configured
458 TCP port (default is 4040).
459
460 If then one I<node> wants to send a message to another I<node> it will
461 connect to the host and port given in the I<port ID>.
462
463 Then some handshaking occurs to check whether both I<nodes> know the
464 I<shared secret>. Optionally, TLS can be enabled (about how to do this
465 exactly please consult the L<AnyEvent::MP> man page, just a hint: It
466 should be enough to put the private key and (self signed) certificate in
467 the C<~/.aemp-secret> file of all nodes).
468
469 After the handshake, messages will be exchanged using a serialiser
470 (usually L<JSON> is used for this, but it is also possible to use other
471 serialization formats such as L<Storable>).
472
473 =head1 SEE ALSO
474
475 L<AnyEvent>
476
477 L<AnyEvent::Handle>
478
479 L<AnyEvent::MP>
480
481 =head1 AUTHOR
482
483 Robin Redeker <elmex@ta-sa.org>
484