ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.9
Committed: Tue Aug 4 20:47:39 2009 UTC (14 years, 9 months ago) by root
Branch: MAIN
Changes since 1.8: +109 -104 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.4 =head1 Message Passing for the Non-Blocked Mind
2 elmex 1.1
3 root 1.8 =head1 Introduction and Terminology
4 elmex 1.1
5 root 1.4 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 root 1.8 module. Which allows us to transparently pass messages to our own process
7     and to other processes on another or the same host.
8 elmex 1.1
9 root 1.4 What kind of messages? Well, basically a message here means a list of
10     Perl strings, numbers, hashes and arrays, mostly everything that can be
11 root 1.8 expressed as a L<JSON> text (as JSON is used by default in the protocol).
12 elmex 1.1
13 root 1.8 And next you might ask: between which entities are those messages
14 root 1.9 being "passed"? Effectively between I<nodes>: a nodes is basically a
15 root 1.8 process/program that use L<AnyEvent::MP> and can run either on the same or
16     different hosts.
17 elmex 1.1
18 root 1.9 To make this more managable, every node can contain any number of
19     I<ports>: Ports are ultimately the receivers of your messages.
20    
21 root 1.4 In this tutorial I'll show you how to write a simple chat server based on
22 elmex 1.1 L<AnyEvent::MP>.
23    
24 root 1.8 =head1 System Requirements
25 elmex 1.7
26     Before we can start we have to make sure some things work on your
27 root 1.8 system.
28 elmex 1.7
29     You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
30     are installed. But how to do that is out of scope of this tutorial.
31    
32 root 1.8 Then we have to setup a I<shared secret>. For two L<AnyEvent::MP> nodes to
33     be able to communicate with each other and authenticate each other it is
34     necessary to setup the same I<shared secret> for both of them. For testing
35     you can write a random string into the file C<.aemp-secret> in your home
36     directory:
37 elmex 1.7
38 root 1.8 mcookie > ~/.aemp-secret
39    
40     # or something more predictable
41 elmex 1.7 echo "secret123#4blabla_please_pick_your_own" > ~/.aemp-secret
42    
43 root 1.8 Connections will only be successful when the nodes that want to connect
44     to each other have the same I<shared secret>. For more security, you can
45     put a self-signed SSL/TLS key/certificate pair into the file (or a normal
46     key/certificate and it's CA certificate).
47 elmex 1.7
48     B<If something does not work as expected, and for example tcpdump shows
49 root 1.8 that the connections are closed almost immediatly, you should make sure
50     that F<~/.aemp-secret> is the same on all hosts/user accounts that you try
51     to connect with each other!>
52    
53     =head1 The Chat Client
54    
55     OK, lets start by implementing the "frontend" of the client. We will
56     develop the client first and postpone the server for later, as the most
57     complex things actually happen in the client.
58    
59     We will use L<AnyEvent::Handle> to do non-blocking IO read on standard
60     input (all of this code deals with actually handling user input, no
61     message passing yet):
62 elmex 1.7
63 root 1.8 #!perl
64 elmex 1.1
65     use AnyEvent;
66     use AnyEvent::Handle;
67    
68     sub send_message {
69     die "This is where we will send the messages to the server"
70     . "in the next step of this tutorial.\n"
71     }
72    
73     # make an AnyEvent condition variable for the 'quit' condition
74     # (when we want to exit the client).
75     my $quit_cv = AnyEvent->condvar;
76    
77     my $stdin_hdl = AnyEvent::Handle->new (
78 root 1.8 fh => *STDIN,
79     on_error => sub { $quit_cv->send },
80     on_read => sub {
81 elmex 1.1 my ($hdl) = @_;
82    
83     $hdl->push_read (line => sub {
84     my ($hdl, $line) = @_;
85    
86     if ($line =~ /^\/quit/) { # /quit will end the client
87     $quit_cv->send;
88     } else {
89     send_message ($line);
90     }
91     });
92     }
93     );
94    
95     $quit_cv->recv;
96    
97 root 1.4 This is now a very basic client. Explaining explicitly what
98     L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
99     of this document, please consult L<AnyEvent::Intro> or the manual pages
100     for L<AnyEvent> and L<AnyEvent::Handle>.
101 elmex 1.1
102 root 1.9 =head1 First Steps Into Messaging
103 elmex 1.1
104 root 1.8 To supply the C<send_message> function we now take a look at
105     L<AnyEvent::MP>. This is an example of how it might look like:
106 elmex 1.1
107     ... # the use lines from the above snippet
108    
109     use AnyEvent::MP;
110    
111     sub send_message {
112     my ($msg) = @_;
113    
114     snd $server_port, message => $msg;
115     }
116    
117     ... # the rest of the above script
118    
119 root 1.8 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send
120     a message'. The first argument is the I<port> (a I<port> is something
121     that can receive messages, represented by a printable string) of the
122     server which will receive the message. How we get this port will be
123     explained in the next step.
124    
125     The remaining arguments of C<snd> are C<message> and C<$msg>, the first
126     two elements of the I<message> (a I<message> in L<AnyEvent::MP> is a
127     simple list of values, which can be sent to a I<port>).
128    
129     So all the function does is send the two values C<message> (a constant
130     string to tell the server what to expect) and the actual message string.
131    
132     Thats all fine and simple so far, but where do we get the
133     C<$server_port>? Well, we need to get the unique I<port id> of the
134     server's port where it wants to receive all the incoming chat messages. A
135 root 1.9 I<port id> is unfortunately a very unique string, which we are unable to
136     know in advance. But L<AnyEvent::MP> supports the concept of 'registered
137     ports', which is basically a port on the server side registered under
138     a well known name.
139    
140     For example, the server has a port for receiving chat messages with a
141     unique I<port id> and registers it under the name C<chatter>.
142 root 1.4
143 root 1.9 BTW, these "registered port names" should follow similar rules as Perl
144 root 1.4 identifiers, so you should prefix them with your package/module name to
145     make them unique, unless you use them in the main program.
146 elmex 1.1
147 root 1.9 As I<messages> can only be sent to a I<port id> and not just to some name
148     we have to ask the server node for the I<port id> of the port registered
149     as C<chatter>.
150 elmex 1.1
151 root 1.9 =head1 Finding The Chatter Port
152 elmex 1.1
153 root 1.9 Ok, lots of talk, now some code. Now we will actually get the
154     C<$server_port> from the backend:
155 elmex 1.1
156     ...
157    
158     use AnyEvent::MP;
159    
160 root 1.9 my $server_node = "127.0.0.1:1299";
161 elmex 1.1
162 root 1.9 my $client_port = port;
163 elmex 1.1
164 root 1.9 snd $server_node, lookup => "chatter", $client_port, "resolved";
165 elmex 1.1
166 root 1.9 my $resolved_cv = AnyEvent->condvar;
167 elmex 1.1 my $server_port;
168    
169     # setup a receiver callback for the 'resolved' message:
170 root 1.9 rcv $client_port, resolved => sub {
171     my ($tag, $chatter_port_id) = @_;
172 elmex 1.1
173     print "Resolved the server port 'chatter' to $chatter_port_id\n";
174     $server_port = $chatter_port_id;
175    
176     $resolved_cv->send;
177     1
178 root 1.9 };
179 elmex 1.1
180 root 1.9 # lets block the client until we have resolved the server port.
181 elmex 1.1 $resolved_cv->recv;
182    
183     # now setup another receiver callback for the chat messages:
184 root 1.9 rcv $client_port, message => sub {
185     my ($tag, $msg) = @_;
186 elmex 1.1
187     print "chat> $msg\n";
188     0
189 root 1.9 };
190 elmex 1.1
191 root 1.9 # send a 'join' message to the server:
192 elmex 1.1 snd $server_port, join => "$client_port";
193    
194     sub send_message { ...
195    
196 root 1.9 Now that was a lot of new stuff:
197 elmex 1.1
198 root 1.9 First we define the C<$server_node>: In order to refer to another node
199     we need some kind of string to reference it - the node reference. The
200     I<noderef> is basically a comma separated list of C<address:port>
201     pairs. We assume in this tutorial that the server runs on C<127.0.0.1>
202     (localhost) on port 1299, which results in the noderef C<127.0.0.1:1299>.
203    
204     Next, in order to receive a reply from the other node or the server we
205     need to have a I<port> that messages can be sent to. This is what the
206     C<port> function will do for us, it just creates a new local port and
207     returns it's I<port ID> that can then be used to receive messages.
208    
209     When you look carefully, you will see that the first C<snd> uses the
210     C<$server_node> (a noderef) as destination port. Well, what I didn't
211     tell you yet is that each I<node> has a default I<port> to receive
212     messages. The ID of this port is the same as the noderef.
213    
214     This I<default port> provides some special services for us, for example
215     resolving a registered name to a I<port id> (a-ha! finally!).
216    
217     This is exactly what this line does:
218    
219     snd $server_node, lookup => "chatter", $client_port, "resolved";
220    
221     This sends a message with first element being C<lookup>, followed by the
222     (hopefully) registered port name that we want to resolve to a I<port
223     id>: C<chatter>. And in order for the server node to be able to send us
224     back the resolved I<port ID> we have to tell it where to send it: The
225     result message will be sent to C<$client_port> (the I<port id> of the
226     port we just created), and will have the string C<resolved> as the first
227     element.
228    
229     When the node receives this message, it will look up the name, gobble up
230     all the extra arguments we passed, append the resolved name, and send the
231     resulting list as a message.
232 elmex 1.1
233 root 1.9 Next we register a receiver for this C<lookup>-request.
234    
235     rcv $client_port, resolved => sub {
236     my ($tag, $chatter_port_id) = @_;
237     ...
238     1
239     };
240    
241     This sets up a receiver on our own port for messages with the first
242     element being the string C<resolved>. Receivers can match the contents of
243     the messages before actually executing the specified callback.
244    
245     B<Please note> that the every C<rcv> callback has to return either a true
246     or a false value, indicating whether it is B<successful>/B<done> (true) or
247     still wants to B<continue> (false) receiving messages.
248 elmex 1.1
249 root 1.9 In this case we tell the C<$client_port> to look into all the messages
250     it receives and look for the string C<resolved> in the first element of
251     the message. If it is found, the given callback will be called with the
252     message elements as arguments.
253 elmex 1.1
254 root 1.9 Using a string as the first element of the message is called I<tagging>
255     the message. It's common practise to code the 'type' of a message into
256     it's first element, as this allows for simple matching.
257 elmex 1.1
258 root 1.9 The result message will contain the I<port ID> of the well known port
259     C<chatter> as second element, which will be stored in C<$chatter_port_id>.
260 elmex 1.1
261 root 1.9 This port ID will then be stored in C<$server_port>, followed by calling
262     C<send> on $resolved_cv> so the program will continue.
263 elmex 1.1
264 root 1.9 The callback then returns a C<1> (a true value), to indicate that it has
265     done it's job and doesn't want to receive further C<resolved> messages.
266 elmex 1.1
267 root 1.9 After this the chat message receiver callback is registered with the port:
268 elmex 1.1
269 root 1.9 rcv $client_port, message => sub {
270     my ($tag, $msg) = @_;
271 elmex 1.1
272     print "chat> $msg\n";
273 root 1.9
274 elmex 1.1 0
275 root 1.9 };
276 elmex 1.1
277 root 1.9 We assume that all messages that are broadcast to the clients by the
278     server contain the string tag C<message> as first element, and the actual
279     message as second element. The callback returns a false value this time,
280     to indicate that it is not yet done and wants to receive further messages.
281    
282     The last thing to do is to tell the server to send us new chat messages
283     from other clients. We do so by sending the message C<join> followed by
284     our own I<port ID>.
285 elmex 1.1
286     # send the server a 'join' message:
287 root 1.9 snd $server_port, join => $client_port;
288 elmex 1.1
289 root 1.9 This way the server knows where to send all the new messages to.
290 elmex 1.1
291 root 1.8 =head1 The Completed Client
292 elmex 1.1
293     This is the complete client script:
294    
295     #!perl
296 root 1.4
297 elmex 1.1 use AnyEvent;
298     use AnyEvent::Handle;
299     use AnyEvent::MP;
300    
301     my $resolved_cv = AnyEvent->condvar;
302    
303     my $client_port = create_port;
304    
305     my $server_node = "localhost:1299#";
306    
307     snd $server_node, wkp => "chatter", "$client_port", "resolved";
308    
309     my $server_port;
310    
311     # setup a receiver callback for the 'resolved' message:
312     $client_port->rcv (resolved => sub {
313     my ($client_port, $type, $chatter_port_id) = @_;
314    
315     print "Resolved the server port 'chatter' to $chatter_port_id\n";
316     $server_port = $chatter_port_id;
317    
318     $resolved_cv->send;
319     1
320     });
321    
322     # lets block the client until we resolved the server port.
323     $resolved_cv->recv;
324    
325     # now setup another receiver callback for the chat messages:
326     $client_port->rcv (message => sub {
327     my ($client_port, $type, $msg) = @_;
328    
329     print "chat> $msg\n";
330     0
331     });
332    
333     # send the server a 'join' message:
334     snd $server_port, join => "$client_port";
335    
336     sub send_message {
337     my ($msg) = @_;
338    
339     snd $server_port, message => $msg;
340     }
341    
342     # make an AnyEvent condition variable for the 'quit' condition
343     # (when we want to exit the client).
344     my $quit_cv = AnyEvent->condvar;
345    
346     my $stdin_hdl = AnyEvent::Handle->new (
347     fh => \*STDIN,
348     on_read => sub {
349     my ($hdl) = @_;
350    
351     $hdl->push_read (line => sub {
352     my ($hdl, $line) = @_;
353    
354     if ($line =~ /^\/quit/) { # /quit will end the client
355     $quit_cv->send;
356    
357     } else {
358     send_message ($line);
359     }
360     });
361     }
362     );
363    
364     $quit_cv->recv;
365    
366 root 1.8 =head1 The Server
367 elmex 1.1
368     Ok, now finally to the server. What do we need? Well, we need to setup
369 root 1.4 the well known port C<chatter> where all clients send their messages to.
370 elmex 1.1
371     Up and into code right now:
372    
373     #!perl
374 root 1.4
375 elmex 1.1 use AnyEvent;
376     use AnyEvent::MP;
377    
378     become_public "localhost:1299";
379    
380     my $chatter_port = create_port;
381     $chatter_port->register ("chatter");
382    
383     my %client_ports;
384    
385     $chatter_port->rcv (join => sub {
386     my ($chatter_port, $type, $client_port) = @_;
387    
388 elmex 1.5 print "got new client port: $client_port\n";
389    
390 elmex 1.1 $client_ports{$client_port} = 1;
391     0
392     });
393    
394     $chatter_port->rcv (message => sub {
395     my ($chatter_port, $type, $msg) = @_;
396    
397 elmex 1.5 print "message> $msg\n";
398    
399 elmex 1.1 snd $_, message => $msg for keys %client_ports;
400     0
401     });
402    
403     AnyEvent->condvar->recv;
404    
405     This is all. Looks much easier, doesn't it? I'll explain it only shortly, as
406     we had the discussion of the C<rcv> method in the client part of this tutorial
407 elmex 1.2 above.
408    
409     First this:
410    
411     become_public "localhost:1299";
412    
413     This will tell our I<node> to become a I<public> node, which means that it can
414     be contacted via TCP. The first argument should be the I<noderef> the server
415     wants to be reachable at. In this case it's the TCP port 1299 on localhost.
416    
417 elmex 1.7 Next we basically setup two receivers, one for the C<join> messages and
418 elmex 1.1 another one for the actual messages of type C<messsage>.
419    
420     In the C<join> message we get the client's port, which we just remember in the
421     C<%client_ports> hash. In the receiver for the message type C<message> we will
422     just iterate through all known C<%client_ports> and relay the message to them.
423    
424 elmex 1.2 And thats it.
425    
426 root 1.8 =head1 The Remaining Problems
427 elmex 1.1
428     The shown implementation still has some bugs. For instance: How does the
429     server know that the client isn't there anymore, and can cleanup the
430     C<%client_ports> hash? And also the chat messages have no originator,
431     so we don't know who actually sent the message (which would be quite useful
432     for human-to-human interaction: to know who the other one is :).
433    
434     But aside from these issues I hope this tutorial got you the swing of
435     L<AnyEvent::MP> and explained some common idioms.
436    
437 elmex 1.7 How to solve the reliability and C<%client_ports> cleanup problem will
438     be explained later in this tutorial (TODO).
439    
440 root 1.8 =head1 Inside The Protocol
441 elmex 1.7
442     Now, for the interested parties, let me explain some details about the protocol
443     that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
444     interested you can skip this section.
445    
446     Usually TCP is used for communication. Each I<node>, if configured to be a
447     I<public> node with the C<become_public> function will listen on the configured
448     TCP port (default is usually 4040).
449    
450     If now one I<node> wants to send a message to another I<node> it will connect
451     to the host and port given in the I<port id>.
452    
453     Then some handshaking occurs to check whether both I<nodes> have the same
454     I<shared secret> configured. Optionally even TLS can be enabled (about how to
455     do this exactly please consult the L<AnyEvent::MP> man pages, just a hint: It
456     should be enough to put the private key and (self signed) certificate in the
457     C<~/.aemp-secret> file of all nodes).
458    
459     Now the serialized (usually L<JSON> is used for this, but it is also possible
460     to use other serialization formats, like L<Storable>) messages are sent over
461     the wire using a simple framing protocol.
462    
463 elmex 1.1 =head1 SEE ALSO
464    
465     L<AnyEvent>
466    
467     L<AnyEvent::Handle>
468    
469     L<AnyEvent::MP>
470    
471     =head1 AUTHOR
472    
473     Robin Redeker <elmex@ta-sa.org>
474 root 1.4