ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.8
Committed: Tue Aug 4 20:00:00 2009 UTC (14 years, 10 months ago) by root
Branch: MAIN
Changes since 1.7: +69 -55 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.4 =head1 Message Passing for the Non-Blocked Mind
2 elmex 1.1
3 root 1.8 =head1 Introduction and Terminology
4 elmex 1.1
5 root 1.4 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 root 1.8 module. Which allows us to transparently pass messages to our own process
7     and to other processes on another or the same host.
8 elmex 1.1
9 root 1.4 What kind of messages? Well, basically a message here means a list of
10     Perl strings, numbers, hashes and arrays, mostly everything that can be
11 root 1.8 expressed as a L<JSON> text (as JSON is used by default in the protocol).
12 elmex 1.1
13 root 1.8 And next you might ask: between which entities are those messages
14     being "passed"? Basically between C<nodes>: a nodes is basically a
15     process/program that use L<AnyEvent::MP> and can run either on the same or
16     different hosts.
17 elmex 1.1
18 root 1.4 In this tutorial I'll show you how to write a simple chat server based on
19 elmex 1.1 L<AnyEvent::MP>.
20    
21 root 1.8 =head1 System Requirements
22 elmex 1.7
23     Before we can start we have to make sure some things work on your
24 root 1.8 system.
25 elmex 1.7
26     You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
27     are installed. But how to do that is out of scope of this tutorial.
28    
29 root 1.8 Then we have to setup a I<shared secret>. For two L<AnyEvent::MP> nodes to
30     be able to communicate with each other and authenticate each other it is
31     necessary to setup the same I<shared secret> for both of them. For testing
32     you can write a random string into the file C<.aemp-secret> in your home
33     directory:
34 elmex 1.7
35 root 1.8 mcookie > ~/.aemp-secret
36    
37     # or something more predictable
38 elmex 1.7 echo "secret123#4blabla_please_pick_your_own" > ~/.aemp-secret
39    
40 root 1.8 Connections will only be successful when the nodes that want to connect
41     to each other have the same I<shared secret>. For more security, you can
42     put a self-signed SSL/TLS key/certificate pair into the file (or a normal
43     key/certificate and it's CA certificate).
44 elmex 1.7
45     B<If something does not work as expected, and for example tcpdump shows
46 root 1.8 that the connections are closed almost immediatly, you should make sure
47     that F<~/.aemp-secret> is the same on all hosts/user accounts that you try
48     to connect with each other!>
49    
50     =head1 The Chat Client
51    
52     OK, lets start by implementing the "frontend" of the client. We will
53     develop the client first and postpone the server for later, as the most
54     complex things actually happen in the client.
55    
56     We will use L<AnyEvent::Handle> to do non-blocking IO read on standard
57     input (all of this code deals with actually handling user input, no
58     message passing yet):
59 elmex 1.7
60 root 1.8 #!perl
61 elmex 1.1
62     use AnyEvent;
63     use AnyEvent::Handle;
64    
65     sub send_message {
66     die "This is where we will send the messages to the server"
67     . "in the next step of this tutorial.\n"
68     }
69    
70     # make an AnyEvent condition variable for the 'quit' condition
71     # (when we want to exit the client).
72     my $quit_cv = AnyEvent->condvar;
73    
74     my $stdin_hdl = AnyEvent::Handle->new (
75 root 1.8 fh => *STDIN,
76     on_error => sub { $quit_cv->send },
77     on_read => sub {
78 elmex 1.1 my ($hdl) = @_;
79    
80     $hdl->push_read (line => sub {
81     my ($hdl, $line) = @_;
82    
83     if ($line =~ /^\/quit/) { # /quit will end the client
84     $quit_cv->send;
85     } else {
86     send_message ($line);
87     }
88     });
89     }
90     );
91    
92     $quit_cv->recv;
93    
94 root 1.4 This is now a very basic client. Explaining explicitly what
95     L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
96     of this document, please consult L<AnyEvent::Intro> or the manual pages
97     for L<AnyEvent> and L<AnyEvent::Handle>.
98 elmex 1.1
99 root 1.8 =head1 First Step Into Messaging
100 elmex 1.1
101 root 1.8 To supply the C<send_message> function we now take a look at
102     L<AnyEvent::MP>. This is an example of how it might look like:
103 elmex 1.1
104     ... # the use lines from the above snippet
105    
106     use AnyEvent::MP;
107    
108     sub send_message {
109     my ($msg) = @_;
110    
111     snd $server_port, message => $msg;
112     }
113    
114     ... # the rest of the above script
115    
116 root 1.8 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send
117     a message'. The first argument is the I<port> (a I<port> is something
118     that can receive messages, represented by a printable string) of the
119     server which will receive the message. How we get this port will be
120     explained in the next step.
121    
122     The remaining arguments of C<snd> are C<message> and C<$msg>, the first
123     two elements of the I<message> (a I<message> in L<AnyEvent::MP> is a
124     simple list of values, which can be sent to a I<port>).
125    
126     So all the function does is send the two values C<message> (a constant
127     string to tell the server what to expect) and the actual message string.
128    
129     Thats all fine and simple so far, but where do we get the
130     C<$server_port>? Well, we need to get the unique I<port id> of the
131     server's port where it wants to receive all the incoming chat messages. A
132     I<port id> is unfortunately a very unique string, which we are unable
133     to know in advance. But L<AnyEvent::MP> supports the concept of 'well
134     known ports', which is basically a port on the server side registered
135     under a well known name. For example, the server has a port for receiving
136     chat messages with a unique I<port id> and registered it under the name
137     C<chatter>.
138 root 1.4
139     BTW, these "well known port names" should follow similar rules as Perl
140     identifiers, so you should prefix them with your package/module name to
141     make them unique, unless you use them in the main program.
142 elmex 1.1
143     As I<messages> can only be sent to a I<port id> and not just to a name we have
144     to ask the server I<node> what I<port id> has the well known port with the
145 root 1.4 name C<chatter>.
146 elmex 1.1
147     Another new term, what is a I<node>: The messaging network that can be created with
148     L<AnyEvent::MP> consists of I<nodes>. A I<node> handles all the connection and
149 root 1.4 low level message sending logic for its application. The application in this
150 elmex 1.1 case is the server. Also every client has/is a I<node>.
151    
152 root 1.8 =head1 Getting The Chatter Port
153 elmex 1.1
154     Ok, lots of talk, now some code. Now we will actually get the C<$server_port>
155     from the backend:
156    
157     ...
158    
159     use AnyEvent::MP;
160    
161     my $resolved_cv = AnyEvent->condvar;
162    
163     my $client_port = create_port;
164    
165     my $server_node = "localhost:1299#";
166    
167     snd $server_node, wkp => "chatter", "$client_port", "resolved";
168    
169     my $server_port;
170    
171     # setup a receiver callback for the 'resolved' message:
172     $client_port->rcv (resolved => sub {
173     my ($client_port, $type, $chatter_port_id) = @_;
174    
175     print "Resolved the server port 'chatter' to $chatter_port_id\n";
176     $server_port = $chatter_port_id;
177    
178     $resolved_cv->send;
179     1
180     });
181    
182     # lets block the client until we resolved the server port.
183     $resolved_cv->recv;
184    
185     # now setup another receiver callback for the chat messages:
186     $client_port->rcv (message => sub {
187     my ($client_port, $type, $msg) = @_;
188    
189     print "chat> $msg\n";
190     0
191     });
192    
193     # send the server a 'join' message:
194     snd $server_port, join => "$client_port";
195    
196     sub send_message { ...
197    
198     Now that was a lot of new stuff. In order to ask the server and receive an
199     answer we need to have a I<port> where we can receive the answer.
200     This is what C<create_port> will do for us, it just creates a new local
201     port and returns us an object (that will btw. stringify to the I<port id>),
202     that we can use to receive messages.
203    
204     Next thing is the C<$server_node>. In order to refer to another node we need
205     some kind of string to reference it. The I<noderef> is basically a comma
206 root 1.6 separated list of C<host:port> pairs. We assume in this tutorial that the
207 elmex 1.1 server runs on your localhost at port 1299, this gives us the noderef
208 root 1.4 C<localhost:1299>.
209 elmex 1.1
210 root 1.4 Now you might ask what the C<#> at the end in C<$server_node> the above
211 elmex 1.1 example is about. Well, what I didn't tell you yet is that each I<node> has a
212     default I<port> to receive messages. The default port is the empty string
213     C<"">. The I<default port> of a I<node> also provides some special services for
214     us, for example resolving a well known port to a I<port id>.
215    
216     Now to this line:
217    
218     snd $server_node, wkp => "chatter", "$client_port", "resolved";
219    
220 root 1.4 We send a message with first element being C<wkp> (standing for 'well known
221 elmex 1.1 port'). Then the well known port name that we want to resolve to a I<port id>:
222 root 1.4 C<chatter>. And in order for the server node to be able to send us back the
223 elmex 1.1 resolved I<port id> we have to tell it where to send the result message: The
224 root 1.4 result message will have as it's first argument the string C<resolved> and
225     will be sent to C<$client_port> (the I<port id> of our own just created
226 elmex 1.1 port).
227    
228 root 1.4 Next comes the receiver for this C<wkp> request.
229 elmex 1.1
230     $client_port->rcv (resolved => sub {
231     my ($client_port, $type, $chatter_port_id) = @_;
232     ...
233     1
234     });
235    
236     This sets up a receiver on our own port for the result message with the first
237 root 1.4 argument being the string C<resolved>. Receivers can match the contents of
238 elmex 1.1 the messages before actually 'sending' it to the given callback.
239    
240     B<Please note> that the given callback has to return either a true or a false
241     value for indicating whether it is B<done> (true value) or still wants to
242     B<continue> (false value) receiving messages.
243    
244     In this case we tell the C<$client_port> to look into the received messages and
245 root 1.4 look for the string C<resolved> in the first element of the message. If it is
246 elmex 1.1 found, the given callback will be called with the C<$client_port> as first
247     argument, and the message as the remaining arguments.
248    
249     We name the first element of the message C<$type> in this case. It's a common
250     idiom to code the 'type' of a message into it's first element, this allows for
251     simple matching.
252    
253 root 1.4 The result message will contain the I<port id> of the well known port C<chatter>
254 elmex 1.1 as next element, and will be put in C<$chatter_port_id>.
255    
256     Next we just assign C<$server_port> and return a 1 (a true value)
257     from the callback. It indicates that we are done and don't want to receive
258 root 1.4 further C<resolved> messages with this callback.
259 elmex 1.1
260     Now we continue to the rest of the client by calling C<send> on
261     C<$resolved_cv>.
262    
263     First new step after this is setting up the chat message receiver callback.
264    
265     $client_port->rcv (message => sub {
266     my ($client_port, $type, $msg) = @_;
267    
268     print "chat> $msg\n";
269     0
270     });
271    
272 root 1.4 We assume that all messages that are broadcast to all clients by the server
273     will contain the string C<message> as first element, and the actual message as
274 elmex 1.1 second element. The callback returns a false value this time, to indicate that
275     it wants to continue receiving messages.
276    
277     Last but not least we actually tell the server to send us
278     the new chat messages from other clients. We do so by sending the
279 root 1.4 message type C<join> followed by our own I<port id>.
280 elmex 1.1
281     # send the server a 'join' message:
282     snd $server_port, join => "$client_port";
283    
284     Then the server knows where to send all the new messages to.
285    
286 root 1.8 =head1 The Completed Client
287 elmex 1.1
288     This is the complete client script:
289    
290     #!perl
291 root 1.4
292 elmex 1.1 use AnyEvent;
293     use AnyEvent::Handle;
294     use AnyEvent::MP;
295    
296     my $resolved_cv = AnyEvent->condvar;
297    
298     my $client_port = create_port;
299    
300     my $server_node = "localhost:1299#";
301    
302     snd $server_node, wkp => "chatter", "$client_port", "resolved";
303    
304     my $server_port;
305    
306     # setup a receiver callback for the 'resolved' message:
307     $client_port->rcv (resolved => sub {
308     my ($client_port, $type, $chatter_port_id) = @_;
309    
310     print "Resolved the server port 'chatter' to $chatter_port_id\n";
311     $server_port = $chatter_port_id;
312    
313     $resolved_cv->send;
314     1
315     });
316    
317     # lets block the client until we resolved the server port.
318     $resolved_cv->recv;
319    
320     # now setup another receiver callback for the chat messages:
321     $client_port->rcv (message => sub {
322     my ($client_port, $type, $msg) = @_;
323    
324     print "chat> $msg\n";
325     0
326     });
327    
328     # send the server a 'join' message:
329     snd $server_port, join => "$client_port";
330    
331     sub send_message {
332     my ($msg) = @_;
333    
334     snd $server_port, message => $msg;
335     }
336    
337     # make an AnyEvent condition variable for the 'quit' condition
338     # (when we want to exit the client).
339     my $quit_cv = AnyEvent->condvar;
340    
341     my $stdin_hdl = AnyEvent::Handle->new (
342     fh => \*STDIN,
343     on_read => sub {
344     my ($hdl) = @_;
345    
346     $hdl->push_read (line => sub {
347     my ($hdl, $line) = @_;
348    
349     if ($line =~ /^\/quit/) { # /quit will end the client
350     $quit_cv->send;
351    
352     } else {
353     send_message ($line);
354     }
355     });
356     }
357     );
358    
359     $quit_cv->recv;
360    
361 root 1.8 =head1 The Server
362 elmex 1.1
363     Ok, now finally to the server. What do we need? Well, we need to setup
364 root 1.4 the well known port C<chatter> where all clients send their messages to.
365 elmex 1.1
366     Up and into code right now:
367    
368     #!perl
369 root 1.4
370 elmex 1.1 use AnyEvent;
371     use AnyEvent::MP;
372    
373     become_public "localhost:1299";
374    
375     my $chatter_port = create_port;
376     $chatter_port->register ("chatter");
377    
378     my %client_ports;
379    
380     $chatter_port->rcv (join => sub {
381     my ($chatter_port, $type, $client_port) = @_;
382    
383 elmex 1.5 print "got new client port: $client_port\n";
384    
385 elmex 1.1 $client_ports{$client_port} = 1;
386     0
387     });
388    
389     $chatter_port->rcv (message => sub {
390     my ($chatter_port, $type, $msg) = @_;
391    
392 elmex 1.5 print "message> $msg\n";
393    
394 elmex 1.1 snd $_, message => $msg for keys %client_ports;
395     0
396     });
397    
398     AnyEvent->condvar->recv;
399    
400     This is all. Looks much easier, doesn't it? I'll explain it only shortly, as
401     we had the discussion of the C<rcv> method in the client part of this tutorial
402 elmex 1.2 above.
403    
404     First this:
405    
406     become_public "localhost:1299";
407    
408     This will tell our I<node> to become a I<public> node, which means that it can
409     be contacted via TCP. The first argument should be the I<noderef> the server
410     wants to be reachable at. In this case it's the TCP port 1299 on localhost.
411    
412 elmex 1.7 Next we basically setup two receivers, one for the C<join> messages and
413 elmex 1.1 another one for the actual messages of type C<messsage>.
414    
415     In the C<join> message we get the client's port, which we just remember in the
416     C<%client_ports> hash. In the receiver for the message type C<message> we will
417     just iterate through all known C<%client_ports> and relay the message to them.
418    
419 elmex 1.2 And thats it.
420    
421 root 1.8 =head1 The Remaining Problems
422 elmex 1.1
423     The shown implementation still has some bugs. For instance: How does the
424     server know that the client isn't there anymore, and can cleanup the
425     C<%client_ports> hash? And also the chat messages have no originator,
426     so we don't know who actually sent the message (which would be quite useful
427     for human-to-human interaction: to know who the other one is :).
428    
429     But aside from these issues I hope this tutorial got you the swing of
430     L<AnyEvent::MP> and explained some common idioms.
431    
432 elmex 1.7 How to solve the reliability and C<%client_ports> cleanup problem will
433     be explained later in this tutorial (TODO).
434    
435 root 1.8 =head1 Inside The Protocol
436 elmex 1.7
437     Now, for the interested parties, let me explain some details about the protocol
438     that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
439     interested you can skip this section.
440    
441     Usually TCP is used for communication. Each I<node>, if configured to be a
442     I<public> node with the C<become_public> function will listen on the configured
443     TCP port (default is usually 4040).
444    
445     If now one I<node> wants to send a message to another I<node> it will connect
446     to the host and port given in the I<port id>.
447    
448     Then some handshaking occurs to check whether both I<nodes> have the same
449     I<shared secret> configured. Optionally even TLS can be enabled (about how to
450     do this exactly please consult the L<AnyEvent::MP> man pages, just a hint: It
451     should be enough to put the private key and (self signed) certificate in the
452     C<~/.aemp-secret> file of all nodes).
453    
454     Now the serialized (usually L<JSON> is used for this, but it is also possible
455     to use other serialization formats, like L<Storable>) messages are sent over
456     the wire using a simple framing protocol.
457    
458 elmex 1.1 =head1 SEE ALSO
459    
460     L<AnyEvent>
461    
462     L<AnyEvent::Handle>
463    
464     L<AnyEvent::MP>
465    
466     =head1 AUTHOR
467    
468     Robin Redeker <elmex@ta-sa.org>
469 root 1.4