ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.13
Committed: Thu Aug 13 20:47:54 2009 UTC (14 years, 9 months ago) by root
Branch: MAIN
Changes since 1.12: +6 -5 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.4 =head1 Message Passing for the Non-Blocked Mind
2 elmex 1.1
3 root 1.8 =head1 Introduction and Terminology
4 elmex 1.1
5 root 1.4 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 root 1.8 module. Which allows us to transparently pass messages to our own process
7     and to other processes on another or the same host.
8 elmex 1.1
9 root 1.4 What kind of messages? Well, basically a message here means a list of
10     Perl strings, numbers, hashes and arrays, mostly everything that can be
11 root 1.8 expressed as a L<JSON> text (as JSON is used by default in the protocol).
12 elmex 1.1
13 root 1.8 And next you might ask: between which entities are those messages
14 root 1.9 being "passed"? Effectively between I<nodes>: a nodes is basically a
15 root 1.8 process/program that use L<AnyEvent::MP> and can run either on the same or
16     different hosts.
17 elmex 1.1
18 root 1.9 To make this more managable, every node can contain any number of
19     I<ports>: Ports are ultimately the receivers of your messages.
20    
21 root 1.4 In this tutorial I'll show you how to write a simple chat server based on
22 elmex 1.1 L<AnyEvent::MP>.
23    
24 root 1.13 =head1 System Requirements and System Setup
25 elmex 1.7
26     Before we can start we have to make sure some things work on your
27 root 1.8 system.
28 elmex 1.7
29     You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
30     are installed. But how to do that is out of scope of this tutorial.
31    
32 root 1.13 Then we have to setup a I<shared secret>: for two L<AnyEvent::MP> nodes to
33 root 1.8 be able to communicate with each other and authenticate each other it is
34 root 1.12 necessary to setup the same I<shared secret> for both of them.
35    
36     The easiest way is to use the F<aemp> utility:
37    
38     aemp gensecret
39    
40     This creates the F<$HOME/.perl-anyevent-mp> config file and generates a
41     random shared secret. You can copy this file to any other system and then
42 root 1.13 communicate with it. You can also select your own shared secret (F<aemp
43     setsecret>) and for increased security requirements you can even create
44     a TLS certificate (F<aemp gencert>), causing conenctions to not just be
45     authenticated, but also to be encrypted.
46 root 1.12
47     Connections will only be successful when the nodes that want to connect to
48     each other have the same I<shared secret> (or successfully verify the TLS
49     certificate).
50 elmex 1.7
51     B<If something does not work as expected, and for example tcpdump shows
52 root 1.8 that the connections are closed almost immediatly, you should make sure
53 root 1.12 that F<~/.perl-anyevent-mp> is the same on all hosts/user accounts that
54     you try to connect with each other!>
55 root 1.8
56     =head1 The Chat Client
57    
58     OK, lets start by implementing the "frontend" of the client. We will
59     develop the client first and postpone the server for later, as the most
60     complex things actually happen in the client.
61    
62     We will use L<AnyEvent::Handle> to do non-blocking IO read on standard
63     input (all of this code deals with actually handling user input, no
64     message passing yet):
65 elmex 1.7
66 root 1.8 #!perl
67 elmex 1.1
68     use AnyEvent;
69     use AnyEvent::Handle;
70    
71     sub send_message {
72     die "This is where we will send the messages to the server"
73     . "in the next step of this tutorial.\n"
74     }
75    
76     # make an AnyEvent condition variable for the 'quit' condition
77     # (when we want to exit the client).
78     my $quit_cv = AnyEvent->condvar;
79    
80     my $stdin_hdl = AnyEvent::Handle->new (
81 root 1.8 fh => *STDIN,
82     on_error => sub { $quit_cv->send },
83     on_read => sub {
84 elmex 1.1 my ($hdl) = @_;
85    
86     $hdl->push_read (line => sub {
87     my ($hdl, $line) = @_;
88    
89     if ($line =~ /^\/quit/) { # /quit will end the client
90     $quit_cv->send;
91     } else {
92     send_message ($line);
93     }
94     });
95     }
96     );
97    
98     $quit_cv->recv;
99    
100 root 1.4 This is now a very basic client. Explaining explicitly what
101     L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
102     of this document, please consult L<AnyEvent::Intro> or the manual pages
103     for L<AnyEvent> and L<AnyEvent::Handle>.
104 elmex 1.1
105 root 1.9 =head1 First Steps Into Messaging
106 elmex 1.1
107 root 1.8 To supply the C<send_message> function we now take a look at
108     L<AnyEvent::MP>. This is an example of how it might look like:
109 elmex 1.1
110     ... # the use lines from the above snippet
111    
112     use AnyEvent::MP;
113    
114     sub send_message {
115     my ($msg) = @_;
116    
117     snd $server_port, message => $msg;
118     }
119    
120     ... # the rest of the above script
121    
122 root 1.8 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send
123     a message'. The first argument is the I<port> (a I<port> is something
124     that can receive messages, represented by a printable string) of the
125     server which will receive the message. How we get this port will be
126     explained in the next step.
127    
128     The remaining arguments of C<snd> are C<message> and C<$msg>, the first
129     two elements of the I<message> (a I<message> in L<AnyEvent::MP> is a
130     simple list of values, which can be sent to a I<port>).
131    
132     So all the function does is send the two values C<message> (a constant
133     string to tell the server what to expect) and the actual message string.
134    
135     Thats all fine and simple so far, but where do we get the
136     C<$server_port>? Well, we need to get the unique I<port id> of the
137     server's port where it wants to receive all the incoming chat messages. A
138 root 1.9 I<port id> is unfortunately a very unique string, which we are unable to
139     know in advance. But L<AnyEvent::MP> supports the concept of 'registered
140     ports', which is basically a port on the server side registered under
141     a well known name.
142    
143     For example, the server has a port for receiving chat messages with a
144     unique I<port id> and registers it under the name C<chatter>.
145 root 1.4
146 root 1.9 BTW, these "registered port names" should follow similar rules as Perl
147 root 1.4 identifiers, so you should prefix them with your package/module name to
148     make them unique, unless you use them in the main program.
149 elmex 1.1
150 root 1.9 As I<messages> can only be sent to a I<port id> and not just to some name
151     we have to ask the server node for the I<port id> of the port registered
152     as C<chatter>.
153 elmex 1.1
154 root 1.9 =head1 Finding The Chatter Port
155 elmex 1.1
156 root 1.9 Ok, lots of talk, now some code. Now we will actually get the
157     C<$server_port> from the backend:
158 elmex 1.1
159     ...
160    
161     use AnyEvent::MP;
162    
163 root 1.9 my $server_node = "127.0.0.1:1299";
164 elmex 1.1
165 root 1.9 my $client_port = port;
166 elmex 1.1
167 root 1.9 snd $server_node, lookup => "chatter", $client_port, "resolved";
168 elmex 1.1
169 root 1.9 my $resolved_cv = AnyEvent->condvar;
170 elmex 1.1 my $server_port;
171    
172     # setup a receiver callback for the 'resolved' message:
173 root 1.9 rcv $client_port, resolved => sub {
174     my ($tag, $chatter_port_id) = @_;
175 elmex 1.1
176     print "Resolved the server port 'chatter' to $chatter_port_id\n";
177     $server_port = $chatter_port_id;
178    
179     $resolved_cv->send;
180     1
181 root 1.9 };
182 elmex 1.1
183 root 1.9 # lets block the client until we have resolved the server port.
184 elmex 1.1 $resolved_cv->recv;
185    
186     # now setup another receiver callback for the chat messages:
187 root 1.9 rcv $client_port, message => sub {
188     my ($tag, $msg) = @_;
189 elmex 1.1
190     print "chat> $msg\n";
191     0
192 root 1.9 };
193 elmex 1.1
194 root 1.9 # send a 'join' message to the server:
195 elmex 1.1 snd $server_port, join => "$client_port";
196    
197     sub send_message { ...
198    
199 root 1.9 Now that was a lot of new stuff:
200 elmex 1.1
201 root 1.9 First we define the C<$server_node>: In order to refer to another node
202     we need some kind of string to reference it - the node reference. The
203     I<noderef> is basically a comma separated list of C<address:port>
204     pairs. We assume in this tutorial that the server runs on C<127.0.0.1>
205     (localhost) on port 1299, which results in the noderef C<127.0.0.1:1299>.
206    
207     Next, in order to receive a reply from the other node or the server we
208     need to have a I<port> that messages can be sent to. This is what the
209     C<port> function will do for us, it just creates a new local port and
210     returns it's I<port ID> that can then be used to receive messages.
211    
212     When you look carefully, you will see that the first C<snd> uses the
213     C<$server_node> (a noderef) as destination port. Well, what I didn't
214     tell you yet is that each I<node> has a default I<port> to receive
215     messages. The ID of this port is the same as the noderef.
216    
217     This I<default port> provides some special services for us, for example
218     resolving a registered name to a I<port id> (a-ha! finally!).
219    
220     This is exactly what this line does:
221    
222     snd $server_node, lookup => "chatter", $client_port, "resolved";
223    
224     This sends a message with first element being C<lookup>, followed by the
225     (hopefully) registered port name that we want to resolve to a I<port
226     id>: C<chatter>. And in order for the server node to be able to send us
227     back the resolved I<port ID> we have to tell it where to send it: The
228     result message will be sent to C<$client_port> (the I<port id> of the
229     port we just created), and will have the string C<resolved> as the first
230     element.
231    
232     When the node receives this message, it will look up the name, gobble up
233     all the extra arguments we passed, append the resolved name, and send the
234     resulting list as a message.
235 elmex 1.1
236 root 1.9 Next we register a receiver for this C<lookup>-request.
237    
238     rcv $client_port, resolved => sub {
239     my ($tag, $chatter_port_id) = @_;
240     ...
241     1
242     };
243    
244     This sets up a receiver on our own port for messages with the first
245     element being the string C<resolved>. Receivers can match the contents of
246     the messages before actually executing the specified callback.
247    
248     B<Please note> that the every C<rcv> callback has to return either a true
249     or a false value, indicating whether it is B<successful>/B<done> (true) or
250     still wants to B<continue> (false) receiving messages.
251 elmex 1.1
252 root 1.9 In this case we tell the C<$client_port> to look into all the messages
253     it receives and look for the string C<resolved> in the first element of
254     the message. If it is found, the given callback will be called with the
255     message elements as arguments.
256 elmex 1.1
257 root 1.9 Using a string as the first element of the message is called I<tagging>
258     the message. It's common practise to code the 'type' of a message into
259     it's first element, as this allows for simple matching.
260 elmex 1.1
261 root 1.9 The result message will contain the I<port ID> of the well known port
262     C<chatter> as second element, which will be stored in C<$chatter_port_id>.
263 elmex 1.1
264 root 1.9 This port ID will then be stored in C<$server_port>, followed by calling
265     C<send> on $resolved_cv> so the program will continue.
266 elmex 1.1
267 root 1.9 The callback then returns a C<1> (a true value), to indicate that it has
268     done it's job and doesn't want to receive further C<resolved> messages.
269 elmex 1.1
270 root 1.9 After this the chat message receiver callback is registered with the port:
271 elmex 1.1
272 root 1.9 rcv $client_port, message => sub {
273     my ($tag, $msg) = @_;
274 elmex 1.1
275     print "chat> $msg\n";
276 root 1.9
277 elmex 1.1 0
278 root 1.9 };
279 elmex 1.1
280 root 1.9 We assume that all messages that are broadcast to the clients by the
281     server contain the string tag C<message> as first element, and the actual
282     message as second element. The callback returns a false value this time,
283     to indicate that it is not yet done and wants to receive further messages.
284    
285     The last thing to do is to tell the server to send us new chat messages
286     from other clients. We do so by sending the message C<join> followed by
287     our own I<port ID>.
288 elmex 1.1
289     # send the server a 'join' message:
290 root 1.9 snd $server_port, join => $client_port;
291 elmex 1.1
292 root 1.9 This way the server knows where to send all the new messages to.
293 elmex 1.1
294 root 1.8 =head1 The Completed Client
295 elmex 1.1
296     This is the complete client script:
297    
298     #!perl
299 root 1.4
300 elmex 1.1 use AnyEvent;
301     use AnyEvent::Handle;
302     use AnyEvent::MP;
303    
304 root 1.10 my $server_node = "127.0.0.1:1299";
305 elmex 1.1
306 root 1.10 my $client_port = port;
307 elmex 1.1
308 root 1.10 snd $server_node, lookup => "chatter", $client_port, "resolved";
309 elmex 1.1
310 root 1.10 my $resolved_cv = AnyEvent->condvar;
311 elmex 1.1 my $server_port;
312    
313     # setup a receiver callback for the 'resolved' message:
314 root 1.10 rcv $client_port, resolved => sub {
315     my ($tag, $chatter_port_id) = @_;
316 elmex 1.1
317     print "Resolved the server port 'chatter' to $chatter_port_id\n";
318     $server_port = $chatter_port_id;
319    
320     $resolved_cv->send;
321     1
322 root 1.10 };
323 elmex 1.1
324 root 1.10 # lets block the client until we have resolved the server port.
325 elmex 1.1 $resolved_cv->recv;
326    
327     # now setup another receiver callback for the chat messages:
328 root 1.10 rcv $client_port, message => sub {
329     my ($tag, $msg) = @_;
330 elmex 1.1
331     print "chat> $msg\n";
332     0
333 root 1.10 };
334 elmex 1.1
335 root 1.10 # send a 'join' message to the server:
336 elmex 1.1 snd $server_port, join => "$client_port";
337    
338     sub send_message {
339     my ($msg) = @_;
340    
341     snd $server_port, message => $msg;
342     }
343    
344     # make an AnyEvent condition variable for the 'quit' condition
345     # (when we want to exit the client).
346     my $quit_cv = AnyEvent->condvar;
347    
348     my $stdin_hdl = AnyEvent::Handle->new (
349 root 1.10 fh => *STDIN,
350     on_error => sub { $quit_cv->send },
351     on_read => sub {
352 elmex 1.1 my ($hdl) = @_;
353    
354     $hdl->push_read (line => sub {
355     my ($hdl, $line) = @_;
356    
357     if ($line =~ /^\/quit/) { # /quit will end the client
358     $quit_cv->send;
359     } else {
360     send_message ($line);
361     }
362     });
363     }
364     );
365    
366     $quit_cv->recv;
367    
368 root 1.8 =head1 The Server
369 elmex 1.1
370 root 1.10 Ok, we finally come to the server.
371    
372     The server of course also needs to set up a port, and in addition needs to
373     I<register> it, so the clients can find it.
374 elmex 1.1
375 root 1.10 Again, let's jump directly into the code:
376 elmex 1.1
377     #!perl
378 root 1.4
379 elmex 1.1 use AnyEvent;
380     use AnyEvent::MP;
381    
382 root 1.10 become_public "127.0.0.1:1299";
383 elmex 1.1
384 root 1.10 my $chatter_port = port;
385    
386     reg $chatter_port, "chatter";
387 elmex 1.1
388     my %client_ports;
389    
390 root 1.10 rcv $chatter_port,
391     join => sub {
392     my ($tag, $client_port) = @_;
393 elmex 1.1
394 root 1.10 print "got new client port: $client_port\n";
395     $client_ports{$client_port} = 1;
396 elmex 1.5
397 root 1.10 0
398     },
399     message => sub {
400     my ($tag, $msg) = @_;
401 elmex 1.1
402 root 1.10 print "message> $msg\n";
403 elmex 1.1
404 root 1.10 snd $_, message => $msg
405     for keys %client_ports;
406 elmex 1.5
407 root 1.10 0
408 root 1.11 };
409 elmex 1.1
410     AnyEvent->condvar->recv;
411    
412 root 1.10 That is all. Looks much simpler than the client, doesn't it?
413    
414     Let's quickly look over it, as C<rcv> has already been discussed in the
415     client part of this tutorial above.
416 elmex 1.2
417     First this:
418    
419 root 1.10 become_public "127.0.0.1:1299";
420 elmex 1.1
421 root 1.10 This will tell our I<node> to become a I<public> node, which means that it
422     can be contacted via TCP. The first argument should be the I<noderef> the
423     server wants to be reachable at. In this case it's the TCP port 1299 on
424     C<127.0.0.1>.
425    
426     Next we set up two receivers, one for the C<join> messages and another one
427     for the actual messages of type C<messsage>. This is done with a single
428     call to C<rcv>, which allows multiple C<< match => $callback >> pairs.
429    
430     In the C<join> callback we receive the client port, which is simply
431     remembered in the C<%client_ports> hash. In the C<message> callback we
432     just iterate through all known C<%client_ports> and relay the message to
433     them.
434 elmex 1.1
435 root 1.10 That concludes the server.
436 elmex 1.2
437 root 1.8 =head1 The Remaining Problems
438 elmex 1.1
439 root 1.10 The implementation as shown still has some bugs. For instance: How does
440     the server know that the client isn't there anymore, so it can clean up
441     the C<%client_ports> hash? Also, the chat messages have no originator, so
442     we don't know who actually sent the message (which would be quite useful
443 elmex 1.1 for human-to-human interaction: to know who the other one is :).
444    
445 root 1.10 But aside from these issues I hope this tutorial showed you the basics of
446 elmex 1.1 L<AnyEvent::MP> and explained some common idioms.
447    
448 elmex 1.7 How to solve the reliability and C<%client_ports> cleanup problem will
449     be explained later in this tutorial (TODO).
450    
451 root 1.8 =head1 Inside The Protocol
452 elmex 1.7
453     Now, for the interested parties, let me explain some details about the protocol
454     that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
455     interested you can skip this section.
456    
457     Usually TCP is used for communication. Each I<node>, if configured to be a
458     I<public> node with the C<become_public> function will listen on the configured
459 root 1.10 TCP port (default is 4040).
460 elmex 1.7
461 root 1.10 If then one I<node> wants to send a message to another I<node> it will
462     connect to the host and port given in the I<port ID>.
463 elmex 1.7
464 root 1.10 Then some handshaking occurs to check whether both I<nodes> know the
465     I<shared secret>. Optionally, TLS can be enabled (about how to do this
466     exactly please consult the L<AnyEvent::MP> man page, just a hint: It
467     should be enough to put the private key and (self signed) certificate in
468     the C<~/.aemp-secret> file of all nodes).
469    
470     After the handshake, messages will be exchanged using a serialiser
471     (usually L<JSON> is used for this, but it is also possible to use other
472     serialization formats such as L<Storable>).
473 elmex 1.7
474 elmex 1.1 =head1 SEE ALSO
475    
476     L<AnyEvent>
477    
478     L<AnyEvent::Handle>
479    
480     L<AnyEvent::MP>
481    
482     =head1 AUTHOR
483    
484     Robin Redeker <elmex@ta-sa.org>
485 root 1.4