ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.12
Committed: Mon Aug 10 19:23:43 2009 UTC (14 years, 9 months ago) by root
Branch: MAIN
CVS Tags: rel-0_6
Changes since 1.11: +17 -15 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.4 =head1 Message Passing for the Non-Blocked Mind
2 elmex 1.1
3 root 1.8 =head1 Introduction and Terminology
4 elmex 1.1
5 root 1.4 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 root 1.8 module. Which allows us to transparently pass messages to our own process
7     and to other processes on another or the same host.
8 elmex 1.1
9 root 1.4 What kind of messages? Well, basically a message here means a list of
10     Perl strings, numbers, hashes and arrays, mostly everything that can be
11 root 1.8 expressed as a L<JSON> text (as JSON is used by default in the protocol).
12 elmex 1.1
13 root 1.8 And next you might ask: between which entities are those messages
14 root 1.9 being "passed"? Effectively between I<nodes>: a nodes is basically a
15 root 1.8 process/program that use L<AnyEvent::MP> and can run either on the same or
16     different hosts.
17 elmex 1.1
18 root 1.9 To make this more managable, every node can contain any number of
19     I<ports>: Ports are ultimately the receivers of your messages.
20    
21 root 1.4 In this tutorial I'll show you how to write a simple chat server based on
22 elmex 1.1 L<AnyEvent::MP>.
23    
24 root 1.8 =head1 System Requirements
25 elmex 1.7
26     Before we can start we have to make sure some things work on your
27 root 1.8 system.
28 elmex 1.7
29     You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
30     are installed. But how to do that is out of scope of this tutorial.
31    
32 root 1.8 Then we have to setup a I<shared secret>. For two L<AnyEvent::MP> nodes to
33     be able to communicate with each other and authenticate each other it is
34 root 1.12 necessary to setup the same I<shared secret> for both of them.
35    
36     The easiest way is to use the F<aemp> utility:
37    
38     aemp gensecret
39    
40     This creates the F<$HOME/.perl-anyevent-mp> config file and generates a
41     random shared secret. You can copy this file to any other system and then
42     communicate with it. You cna also select your own shared secret (F<aemp
43     setsecret>) and for increased security requirements you can even create a
44     TLS certificate (F<aemp gencert>).
45    
46     Connections will only be successful when the nodes that want to connect to
47     each other have the same I<shared secret> (or successfully verify the TLS
48     certificate).
49 elmex 1.7
50     B<If something does not work as expected, and for example tcpdump shows
51 root 1.8 that the connections are closed almost immediatly, you should make sure
52 root 1.12 that F<~/.perl-anyevent-mp> is the same on all hosts/user accounts that
53     you try to connect with each other!>
54 root 1.8
55     =head1 The Chat Client
56    
57     OK, lets start by implementing the "frontend" of the client. We will
58     develop the client first and postpone the server for later, as the most
59     complex things actually happen in the client.
60    
61     We will use L<AnyEvent::Handle> to do non-blocking IO read on standard
62     input (all of this code deals with actually handling user input, no
63     message passing yet):
64 elmex 1.7
65 root 1.8 #!perl
66 elmex 1.1
67     use AnyEvent;
68     use AnyEvent::Handle;
69    
70     sub send_message {
71     die "This is where we will send the messages to the server"
72     . "in the next step of this tutorial.\n"
73     }
74    
75     # make an AnyEvent condition variable for the 'quit' condition
76     # (when we want to exit the client).
77     my $quit_cv = AnyEvent->condvar;
78    
79     my $stdin_hdl = AnyEvent::Handle->new (
80 root 1.8 fh => *STDIN,
81     on_error => sub { $quit_cv->send },
82     on_read => sub {
83 elmex 1.1 my ($hdl) = @_;
84    
85     $hdl->push_read (line => sub {
86     my ($hdl, $line) = @_;
87    
88     if ($line =~ /^\/quit/) { # /quit will end the client
89     $quit_cv->send;
90     } else {
91     send_message ($line);
92     }
93     });
94     }
95     );
96    
97     $quit_cv->recv;
98    
99 root 1.4 This is now a very basic client. Explaining explicitly what
100     L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
101     of this document, please consult L<AnyEvent::Intro> or the manual pages
102     for L<AnyEvent> and L<AnyEvent::Handle>.
103 elmex 1.1
104 root 1.9 =head1 First Steps Into Messaging
105 elmex 1.1
106 root 1.8 To supply the C<send_message> function we now take a look at
107     L<AnyEvent::MP>. This is an example of how it might look like:
108 elmex 1.1
109     ... # the use lines from the above snippet
110    
111     use AnyEvent::MP;
112    
113     sub send_message {
114     my ($msg) = @_;
115    
116     snd $server_port, message => $msg;
117     }
118    
119     ... # the rest of the above script
120    
121 root 1.8 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send
122     a message'. The first argument is the I<port> (a I<port> is something
123     that can receive messages, represented by a printable string) of the
124     server which will receive the message. How we get this port will be
125     explained in the next step.
126    
127     The remaining arguments of C<snd> are C<message> and C<$msg>, the first
128     two elements of the I<message> (a I<message> in L<AnyEvent::MP> is a
129     simple list of values, which can be sent to a I<port>).
130    
131     So all the function does is send the two values C<message> (a constant
132     string to tell the server what to expect) and the actual message string.
133    
134     Thats all fine and simple so far, but where do we get the
135     C<$server_port>? Well, we need to get the unique I<port id> of the
136     server's port where it wants to receive all the incoming chat messages. A
137 root 1.9 I<port id> is unfortunately a very unique string, which we are unable to
138     know in advance. But L<AnyEvent::MP> supports the concept of 'registered
139     ports', which is basically a port on the server side registered under
140     a well known name.
141    
142     For example, the server has a port for receiving chat messages with a
143     unique I<port id> and registers it under the name C<chatter>.
144 root 1.4
145 root 1.9 BTW, these "registered port names" should follow similar rules as Perl
146 root 1.4 identifiers, so you should prefix them with your package/module name to
147     make them unique, unless you use them in the main program.
148 elmex 1.1
149 root 1.9 As I<messages> can only be sent to a I<port id> and not just to some name
150     we have to ask the server node for the I<port id> of the port registered
151     as C<chatter>.
152 elmex 1.1
153 root 1.9 =head1 Finding The Chatter Port
154 elmex 1.1
155 root 1.9 Ok, lots of talk, now some code. Now we will actually get the
156     C<$server_port> from the backend:
157 elmex 1.1
158     ...
159    
160     use AnyEvent::MP;
161    
162 root 1.9 my $server_node = "127.0.0.1:1299";
163 elmex 1.1
164 root 1.9 my $client_port = port;
165 elmex 1.1
166 root 1.9 snd $server_node, lookup => "chatter", $client_port, "resolved";
167 elmex 1.1
168 root 1.9 my $resolved_cv = AnyEvent->condvar;
169 elmex 1.1 my $server_port;
170    
171     # setup a receiver callback for the 'resolved' message:
172 root 1.9 rcv $client_port, resolved => sub {
173     my ($tag, $chatter_port_id) = @_;
174 elmex 1.1
175     print "Resolved the server port 'chatter' to $chatter_port_id\n";
176     $server_port = $chatter_port_id;
177    
178     $resolved_cv->send;
179     1
180 root 1.9 };
181 elmex 1.1
182 root 1.9 # lets block the client until we have resolved the server port.
183 elmex 1.1 $resolved_cv->recv;
184    
185     # now setup another receiver callback for the chat messages:
186 root 1.9 rcv $client_port, message => sub {
187     my ($tag, $msg) = @_;
188 elmex 1.1
189     print "chat> $msg\n";
190     0
191 root 1.9 };
192 elmex 1.1
193 root 1.9 # send a 'join' message to the server:
194 elmex 1.1 snd $server_port, join => "$client_port";
195    
196     sub send_message { ...
197    
198 root 1.9 Now that was a lot of new stuff:
199 elmex 1.1
200 root 1.9 First we define the C<$server_node>: In order to refer to another node
201     we need some kind of string to reference it - the node reference. The
202     I<noderef> is basically a comma separated list of C<address:port>
203     pairs. We assume in this tutorial that the server runs on C<127.0.0.1>
204     (localhost) on port 1299, which results in the noderef C<127.0.0.1:1299>.
205    
206     Next, in order to receive a reply from the other node or the server we
207     need to have a I<port> that messages can be sent to. This is what the
208     C<port> function will do for us, it just creates a new local port and
209     returns it's I<port ID> that can then be used to receive messages.
210    
211     When you look carefully, you will see that the first C<snd> uses the
212     C<$server_node> (a noderef) as destination port. Well, what I didn't
213     tell you yet is that each I<node> has a default I<port> to receive
214     messages. The ID of this port is the same as the noderef.
215    
216     This I<default port> provides some special services for us, for example
217     resolving a registered name to a I<port id> (a-ha! finally!).
218    
219     This is exactly what this line does:
220    
221     snd $server_node, lookup => "chatter", $client_port, "resolved";
222    
223     This sends a message with first element being C<lookup>, followed by the
224     (hopefully) registered port name that we want to resolve to a I<port
225     id>: C<chatter>. And in order for the server node to be able to send us
226     back the resolved I<port ID> we have to tell it where to send it: The
227     result message will be sent to C<$client_port> (the I<port id> of the
228     port we just created), and will have the string C<resolved> as the first
229     element.
230    
231     When the node receives this message, it will look up the name, gobble up
232     all the extra arguments we passed, append the resolved name, and send the
233     resulting list as a message.
234 elmex 1.1
235 root 1.9 Next we register a receiver for this C<lookup>-request.
236    
237     rcv $client_port, resolved => sub {
238     my ($tag, $chatter_port_id) = @_;
239     ...
240     1
241     };
242    
243     This sets up a receiver on our own port for messages with the first
244     element being the string C<resolved>. Receivers can match the contents of
245     the messages before actually executing the specified callback.
246    
247     B<Please note> that the every C<rcv> callback has to return either a true
248     or a false value, indicating whether it is B<successful>/B<done> (true) or
249     still wants to B<continue> (false) receiving messages.
250 elmex 1.1
251 root 1.9 In this case we tell the C<$client_port> to look into all the messages
252     it receives and look for the string C<resolved> in the first element of
253     the message. If it is found, the given callback will be called with the
254     message elements as arguments.
255 elmex 1.1
256 root 1.9 Using a string as the first element of the message is called I<tagging>
257     the message. It's common practise to code the 'type' of a message into
258     it's first element, as this allows for simple matching.
259 elmex 1.1
260 root 1.9 The result message will contain the I<port ID> of the well known port
261     C<chatter> as second element, which will be stored in C<$chatter_port_id>.
262 elmex 1.1
263 root 1.9 This port ID will then be stored in C<$server_port>, followed by calling
264     C<send> on $resolved_cv> so the program will continue.
265 elmex 1.1
266 root 1.9 The callback then returns a C<1> (a true value), to indicate that it has
267     done it's job and doesn't want to receive further C<resolved> messages.
268 elmex 1.1
269 root 1.9 After this the chat message receiver callback is registered with the port:
270 elmex 1.1
271 root 1.9 rcv $client_port, message => sub {
272     my ($tag, $msg) = @_;
273 elmex 1.1
274     print "chat> $msg\n";
275 root 1.9
276 elmex 1.1 0
277 root 1.9 };
278 elmex 1.1
279 root 1.9 We assume that all messages that are broadcast to the clients by the
280     server contain the string tag C<message> as first element, and the actual
281     message as second element. The callback returns a false value this time,
282     to indicate that it is not yet done and wants to receive further messages.
283    
284     The last thing to do is to tell the server to send us new chat messages
285     from other clients. We do so by sending the message C<join> followed by
286     our own I<port ID>.
287 elmex 1.1
288     # send the server a 'join' message:
289 root 1.9 snd $server_port, join => $client_port;
290 elmex 1.1
291 root 1.9 This way the server knows where to send all the new messages to.
292 elmex 1.1
293 root 1.8 =head1 The Completed Client
294 elmex 1.1
295     This is the complete client script:
296    
297     #!perl
298 root 1.4
299 elmex 1.1 use AnyEvent;
300     use AnyEvent::Handle;
301     use AnyEvent::MP;
302    
303 root 1.10 my $server_node = "127.0.0.1:1299";
304 elmex 1.1
305 root 1.10 my $client_port = port;
306 elmex 1.1
307 root 1.10 snd $server_node, lookup => "chatter", $client_port, "resolved";
308 elmex 1.1
309 root 1.10 my $resolved_cv = AnyEvent->condvar;
310 elmex 1.1 my $server_port;
311    
312     # setup a receiver callback for the 'resolved' message:
313 root 1.10 rcv $client_port, resolved => sub {
314     my ($tag, $chatter_port_id) = @_;
315 elmex 1.1
316     print "Resolved the server port 'chatter' to $chatter_port_id\n";
317     $server_port = $chatter_port_id;
318    
319     $resolved_cv->send;
320     1
321 root 1.10 };
322 elmex 1.1
323 root 1.10 # lets block the client until we have resolved the server port.
324 elmex 1.1 $resolved_cv->recv;
325    
326     # now setup another receiver callback for the chat messages:
327 root 1.10 rcv $client_port, message => sub {
328     my ($tag, $msg) = @_;
329 elmex 1.1
330     print "chat> $msg\n";
331     0
332 root 1.10 };
333 elmex 1.1
334 root 1.10 # send a 'join' message to the server:
335 elmex 1.1 snd $server_port, join => "$client_port";
336    
337     sub send_message {
338     my ($msg) = @_;
339    
340     snd $server_port, message => $msg;
341     }
342    
343     # make an AnyEvent condition variable for the 'quit' condition
344     # (when we want to exit the client).
345     my $quit_cv = AnyEvent->condvar;
346    
347     my $stdin_hdl = AnyEvent::Handle->new (
348 root 1.10 fh => *STDIN,
349     on_error => sub { $quit_cv->send },
350     on_read => sub {
351 elmex 1.1 my ($hdl) = @_;
352    
353     $hdl->push_read (line => sub {
354     my ($hdl, $line) = @_;
355    
356     if ($line =~ /^\/quit/) { # /quit will end the client
357     $quit_cv->send;
358     } else {
359     send_message ($line);
360     }
361     });
362     }
363     );
364    
365     $quit_cv->recv;
366    
367 root 1.8 =head1 The Server
368 elmex 1.1
369 root 1.10 Ok, we finally come to the server.
370    
371     The server of course also needs to set up a port, and in addition needs to
372     I<register> it, so the clients can find it.
373 elmex 1.1
374 root 1.10 Again, let's jump directly into the code:
375 elmex 1.1
376     #!perl
377 root 1.4
378 elmex 1.1 use AnyEvent;
379     use AnyEvent::MP;
380    
381 root 1.10 become_public "127.0.0.1:1299";
382 elmex 1.1
383 root 1.10 my $chatter_port = port;
384    
385     reg $chatter_port, "chatter";
386 elmex 1.1
387     my %client_ports;
388    
389 root 1.10 rcv $chatter_port,
390     join => sub {
391     my ($tag, $client_port) = @_;
392 elmex 1.1
393 root 1.10 print "got new client port: $client_port\n";
394     $client_ports{$client_port} = 1;
395 elmex 1.5
396 root 1.10 0
397     },
398     message => sub {
399     my ($tag, $msg) = @_;
400 elmex 1.1
401 root 1.10 print "message> $msg\n";
402 elmex 1.1
403 root 1.10 snd $_, message => $msg
404     for keys %client_ports;
405 elmex 1.5
406 root 1.10 0
407 root 1.11 };
408 elmex 1.1
409     AnyEvent->condvar->recv;
410    
411 root 1.10 That is all. Looks much simpler than the client, doesn't it?
412    
413     Let's quickly look over it, as C<rcv> has already been discussed in the
414     client part of this tutorial above.
415 elmex 1.2
416     First this:
417    
418 root 1.10 become_public "127.0.0.1:1299";
419 elmex 1.1
420 root 1.10 This will tell our I<node> to become a I<public> node, which means that it
421     can be contacted via TCP. The first argument should be the I<noderef> the
422     server wants to be reachable at. In this case it's the TCP port 1299 on
423     C<127.0.0.1>.
424    
425     Next we set up two receivers, one for the C<join> messages and another one
426     for the actual messages of type C<messsage>. This is done with a single
427     call to C<rcv>, which allows multiple C<< match => $callback >> pairs.
428    
429     In the C<join> callback we receive the client port, which is simply
430     remembered in the C<%client_ports> hash. In the C<message> callback we
431     just iterate through all known C<%client_ports> and relay the message to
432     them.
433 elmex 1.1
434 root 1.10 That concludes the server.
435 elmex 1.2
436 root 1.8 =head1 The Remaining Problems
437 elmex 1.1
438 root 1.10 The implementation as shown still has some bugs. For instance: How does
439     the server know that the client isn't there anymore, so it can clean up
440     the C<%client_ports> hash? Also, the chat messages have no originator, so
441     we don't know who actually sent the message (which would be quite useful
442 elmex 1.1 for human-to-human interaction: to know who the other one is :).
443    
444 root 1.10 But aside from these issues I hope this tutorial showed you the basics of
445 elmex 1.1 L<AnyEvent::MP> and explained some common idioms.
446    
447 elmex 1.7 How to solve the reliability and C<%client_ports> cleanup problem will
448     be explained later in this tutorial (TODO).
449    
450 root 1.8 =head1 Inside The Protocol
451 elmex 1.7
452     Now, for the interested parties, let me explain some details about the protocol
453     that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
454     interested you can skip this section.
455    
456     Usually TCP is used for communication. Each I<node>, if configured to be a
457     I<public> node with the C<become_public> function will listen on the configured
458 root 1.10 TCP port (default is 4040).
459 elmex 1.7
460 root 1.10 If then one I<node> wants to send a message to another I<node> it will
461     connect to the host and port given in the I<port ID>.
462 elmex 1.7
463 root 1.10 Then some handshaking occurs to check whether both I<nodes> know the
464     I<shared secret>. Optionally, TLS can be enabled (about how to do this
465     exactly please consult the L<AnyEvent::MP> man page, just a hint: It
466     should be enough to put the private key and (self signed) certificate in
467     the C<~/.aemp-secret> file of all nodes).
468    
469     After the handshake, messages will be exchanged using a serialiser
470     (usually L<JSON> is used for this, but it is also possible to use other
471     serialization formats such as L<Storable>).
472 elmex 1.7
473 elmex 1.1 =head1 SEE ALSO
474    
475     L<AnyEvent>
476    
477     L<AnyEvent::Handle>
478    
479     L<AnyEvent::MP>
480    
481     =head1 AUTHOR
482    
483     Robin Redeker <elmex@ta-sa.org>
484 root 1.4