ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.10
Committed: Tue Aug 4 20:59:37 2009 UTC (14 years, 10 months ago) by root
Branch: MAIN
Changes since 1.9: +74 -66 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 root 1.4 =head1 Message Passing for the Non-Blocked Mind
2 elmex 1.1
3 root 1.8 =head1 Introduction and Terminology
4 elmex 1.1
5 root 1.4 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 root 1.8 module. Which allows us to transparently pass messages to our own process
7     and to other processes on another or the same host.
8 elmex 1.1
9 root 1.4 What kind of messages? Well, basically a message here means a list of
10     Perl strings, numbers, hashes and arrays, mostly everything that can be
11 root 1.8 expressed as a L<JSON> text (as JSON is used by default in the protocol).
12 elmex 1.1
13 root 1.8 And next you might ask: between which entities are those messages
14 root 1.9 being "passed"? Effectively between I<nodes>: a nodes is basically a
15 root 1.8 process/program that use L<AnyEvent::MP> and can run either on the same or
16     different hosts.
17 elmex 1.1
18 root 1.9 To make this more managable, every node can contain any number of
19     I<ports>: Ports are ultimately the receivers of your messages.
20    
21 root 1.4 In this tutorial I'll show you how to write a simple chat server based on
22 elmex 1.1 L<AnyEvent::MP>.
23    
24 root 1.8 =head1 System Requirements
25 elmex 1.7
26     Before we can start we have to make sure some things work on your
27 root 1.8 system.
28 elmex 1.7
29     You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
30     are installed. But how to do that is out of scope of this tutorial.
31    
32 root 1.8 Then we have to setup a I<shared secret>. For two L<AnyEvent::MP> nodes to
33     be able to communicate with each other and authenticate each other it is
34     necessary to setup the same I<shared secret> for both of them. For testing
35     you can write a random string into the file C<.aemp-secret> in your home
36     directory:
37 elmex 1.7
38 root 1.8 mcookie > ~/.aemp-secret
39    
40     # or something more predictable
41 elmex 1.7 echo "secret123#4blabla_please_pick_your_own" > ~/.aemp-secret
42    
43 root 1.8 Connections will only be successful when the nodes that want to connect
44     to each other have the same I<shared secret>. For more security, you can
45     put a self-signed SSL/TLS key/certificate pair into the file (or a normal
46     key/certificate and it's CA certificate).
47 elmex 1.7
48     B<If something does not work as expected, and for example tcpdump shows
49 root 1.8 that the connections are closed almost immediatly, you should make sure
50     that F<~/.aemp-secret> is the same on all hosts/user accounts that you try
51     to connect with each other!>
52    
53     =head1 The Chat Client
54    
55     OK, lets start by implementing the "frontend" of the client. We will
56     develop the client first and postpone the server for later, as the most
57     complex things actually happen in the client.
58    
59     We will use L<AnyEvent::Handle> to do non-blocking IO read on standard
60     input (all of this code deals with actually handling user input, no
61     message passing yet):
62 elmex 1.7
63 root 1.8 #!perl
64 elmex 1.1
65     use AnyEvent;
66     use AnyEvent::Handle;
67    
68     sub send_message {
69     die "This is where we will send the messages to the server"
70     . "in the next step of this tutorial.\n"
71     }
72    
73     # make an AnyEvent condition variable for the 'quit' condition
74     # (when we want to exit the client).
75     my $quit_cv = AnyEvent->condvar;
76    
77     my $stdin_hdl = AnyEvent::Handle->new (
78 root 1.8 fh => *STDIN,
79     on_error => sub { $quit_cv->send },
80     on_read => sub {
81 elmex 1.1 my ($hdl) = @_;
82    
83     $hdl->push_read (line => sub {
84     my ($hdl, $line) = @_;
85    
86     if ($line =~ /^\/quit/) { # /quit will end the client
87     $quit_cv->send;
88     } else {
89     send_message ($line);
90     }
91     });
92     }
93     );
94    
95     $quit_cv->recv;
96    
97 root 1.4 This is now a very basic client. Explaining explicitly what
98     L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
99     of this document, please consult L<AnyEvent::Intro> or the manual pages
100     for L<AnyEvent> and L<AnyEvent::Handle>.
101 elmex 1.1
102 root 1.9 =head1 First Steps Into Messaging
103 elmex 1.1
104 root 1.8 To supply the C<send_message> function we now take a look at
105     L<AnyEvent::MP>. This is an example of how it might look like:
106 elmex 1.1
107     ... # the use lines from the above snippet
108    
109     use AnyEvent::MP;
110    
111     sub send_message {
112     my ($msg) = @_;
113    
114     snd $server_port, message => $msg;
115     }
116    
117     ... # the rest of the above script
118    
119 root 1.8 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send
120     a message'. The first argument is the I<port> (a I<port> is something
121     that can receive messages, represented by a printable string) of the
122     server which will receive the message. How we get this port will be
123     explained in the next step.
124    
125     The remaining arguments of C<snd> are C<message> and C<$msg>, the first
126     two elements of the I<message> (a I<message> in L<AnyEvent::MP> is a
127     simple list of values, which can be sent to a I<port>).
128    
129     So all the function does is send the two values C<message> (a constant
130     string to tell the server what to expect) and the actual message string.
131    
132     Thats all fine and simple so far, but where do we get the
133     C<$server_port>? Well, we need to get the unique I<port id> of the
134     server's port where it wants to receive all the incoming chat messages. A
135 root 1.9 I<port id> is unfortunately a very unique string, which we are unable to
136     know in advance. But L<AnyEvent::MP> supports the concept of 'registered
137     ports', which is basically a port on the server side registered under
138     a well known name.
139    
140     For example, the server has a port for receiving chat messages with a
141     unique I<port id> and registers it under the name C<chatter>.
142 root 1.4
143 root 1.9 BTW, these "registered port names" should follow similar rules as Perl
144 root 1.4 identifiers, so you should prefix them with your package/module name to
145     make them unique, unless you use them in the main program.
146 elmex 1.1
147 root 1.9 As I<messages> can only be sent to a I<port id> and not just to some name
148     we have to ask the server node for the I<port id> of the port registered
149     as C<chatter>.
150 elmex 1.1
151 root 1.9 =head1 Finding The Chatter Port
152 elmex 1.1
153 root 1.9 Ok, lots of talk, now some code. Now we will actually get the
154     C<$server_port> from the backend:
155 elmex 1.1
156     ...
157    
158     use AnyEvent::MP;
159    
160 root 1.9 my $server_node = "127.0.0.1:1299";
161 elmex 1.1
162 root 1.9 my $client_port = port;
163 elmex 1.1
164 root 1.9 snd $server_node, lookup => "chatter", $client_port, "resolved";
165 elmex 1.1
166 root 1.9 my $resolved_cv = AnyEvent->condvar;
167 elmex 1.1 my $server_port;
168    
169     # setup a receiver callback for the 'resolved' message:
170 root 1.9 rcv $client_port, resolved => sub {
171     my ($tag, $chatter_port_id) = @_;
172 elmex 1.1
173     print "Resolved the server port 'chatter' to $chatter_port_id\n";
174     $server_port = $chatter_port_id;
175    
176     $resolved_cv->send;
177     1
178 root 1.9 };
179 elmex 1.1
180 root 1.9 # lets block the client until we have resolved the server port.
181 elmex 1.1 $resolved_cv->recv;
182    
183     # now setup another receiver callback for the chat messages:
184 root 1.9 rcv $client_port, message => sub {
185     my ($tag, $msg) = @_;
186 elmex 1.1
187     print "chat> $msg\n";
188     0
189 root 1.9 };
190 elmex 1.1
191 root 1.9 # send a 'join' message to the server:
192 elmex 1.1 snd $server_port, join => "$client_port";
193    
194     sub send_message { ...
195    
196 root 1.9 Now that was a lot of new stuff:
197 elmex 1.1
198 root 1.9 First we define the C<$server_node>: In order to refer to another node
199     we need some kind of string to reference it - the node reference. The
200     I<noderef> is basically a comma separated list of C<address:port>
201     pairs. We assume in this tutorial that the server runs on C<127.0.0.1>
202     (localhost) on port 1299, which results in the noderef C<127.0.0.1:1299>.
203    
204     Next, in order to receive a reply from the other node or the server we
205     need to have a I<port> that messages can be sent to. This is what the
206     C<port> function will do for us, it just creates a new local port and
207     returns it's I<port ID> that can then be used to receive messages.
208    
209     When you look carefully, you will see that the first C<snd> uses the
210     C<$server_node> (a noderef) as destination port. Well, what I didn't
211     tell you yet is that each I<node> has a default I<port> to receive
212     messages. The ID of this port is the same as the noderef.
213    
214     This I<default port> provides some special services for us, for example
215     resolving a registered name to a I<port id> (a-ha! finally!).
216    
217     This is exactly what this line does:
218    
219     snd $server_node, lookup => "chatter", $client_port, "resolved";
220    
221     This sends a message with first element being C<lookup>, followed by the
222     (hopefully) registered port name that we want to resolve to a I<port
223     id>: C<chatter>. And in order for the server node to be able to send us
224     back the resolved I<port ID> we have to tell it where to send it: The
225     result message will be sent to C<$client_port> (the I<port id> of the
226     port we just created), and will have the string C<resolved> as the first
227     element.
228    
229     When the node receives this message, it will look up the name, gobble up
230     all the extra arguments we passed, append the resolved name, and send the
231     resulting list as a message.
232 elmex 1.1
233 root 1.9 Next we register a receiver for this C<lookup>-request.
234    
235     rcv $client_port, resolved => sub {
236     my ($tag, $chatter_port_id) = @_;
237     ...
238     1
239     };
240    
241     This sets up a receiver on our own port for messages with the first
242     element being the string C<resolved>. Receivers can match the contents of
243     the messages before actually executing the specified callback.
244    
245     B<Please note> that the every C<rcv> callback has to return either a true
246     or a false value, indicating whether it is B<successful>/B<done> (true) or
247     still wants to B<continue> (false) receiving messages.
248 elmex 1.1
249 root 1.9 In this case we tell the C<$client_port> to look into all the messages
250     it receives and look for the string C<resolved> in the first element of
251     the message. If it is found, the given callback will be called with the
252     message elements as arguments.
253 elmex 1.1
254 root 1.9 Using a string as the first element of the message is called I<tagging>
255     the message. It's common practise to code the 'type' of a message into
256     it's first element, as this allows for simple matching.
257 elmex 1.1
258 root 1.9 The result message will contain the I<port ID> of the well known port
259     C<chatter> as second element, which will be stored in C<$chatter_port_id>.
260 elmex 1.1
261 root 1.9 This port ID will then be stored in C<$server_port>, followed by calling
262     C<send> on $resolved_cv> so the program will continue.
263 elmex 1.1
264 root 1.9 The callback then returns a C<1> (a true value), to indicate that it has
265     done it's job and doesn't want to receive further C<resolved> messages.
266 elmex 1.1
267 root 1.9 After this the chat message receiver callback is registered with the port:
268 elmex 1.1
269 root 1.9 rcv $client_port, message => sub {
270     my ($tag, $msg) = @_;
271 elmex 1.1
272     print "chat> $msg\n";
273 root 1.9
274 elmex 1.1 0
275 root 1.9 };
276 elmex 1.1
277 root 1.9 We assume that all messages that are broadcast to the clients by the
278     server contain the string tag C<message> as first element, and the actual
279     message as second element. The callback returns a false value this time,
280     to indicate that it is not yet done and wants to receive further messages.
281    
282     The last thing to do is to tell the server to send us new chat messages
283     from other clients. We do so by sending the message C<join> followed by
284     our own I<port ID>.
285 elmex 1.1
286     # send the server a 'join' message:
287 root 1.9 snd $server_port, join => $client_port;
288 elmex 1.1
289 root 1.9 This way the server knows where to send all the new messages to.
290 elmex 1.1
291 root 1.8 =head1 The Completed Client
292 elmex 1.1
293     This is the complete client script:
294    
295     #!perl
296 root 1.4
297 elmex 1.1 use AnyEvent;
298     use AnyEvent::Handle;
299     use AnyEvent::MP;
300    
301 root 1.10 my $server_node = "127.0.0.1:1299";
302 elmex 1.1
303 root 1.10 my $client_port = port;
304 elmex 1.1
305 root 1.10 snd $server_node, lookup => "chatter", $client_port, "resolved";
306 elmex 1.1
307 root 1.10 my $resolved_cv = AnyEvent->condvar;
308 elmex 1.1 my $server_port;
309    
310     # setup a receiver callback for the 'resolved' message:
311 root 1.10 rcv $client_port, resolved => sub {
312     my ($tag, $chatter_port_id) = @_;
313 elmex 1.1
314     print "Resolved the server port 'chatter' to $chatter_port_id\n";
315     $server_port = $chatter_port_id;
316    
317     $resolved_cv->send;
318     1
319 root 1.10 };
320 elmex 1.1
321 root 1.10 # lets block the client until we have resolved the server port.
322 elmex 1.1 $resolved_cv->recv;
323    
324     # now setup another receiver callback for the chat messages:
325 root 1.10 rcv $client_port, message => sub {
326     my ($tag, $msg) = @_;
327 elmex 1.1
328     print "chat> $msg\n";
329     0
330 root 1.10 };
331 elmex 1.1
332 root 1.10 # send a 'join' message to the server:
333 elmex 1.1 snd $server_port, join => "$client_port";
334    
335     sub send_message {
336     my ($msg) = @_;
337    
338     snd $server_port, message => $msg;
339     }
340    
341     # make an AnyEvent condition variable for the 'quit' condition
342     # (when we want to exit the client).
343     my $quit_cv = AnyEvent->condvar;
344    
345     my $stdin_hdl = AnyEvent::Handle->new (
346 root 1.10 fh => *STDIN,
347     on_error => sub { $quit_cv->send },
348     on_read => sub {
349 elmex 1.1 my ($hdl) = @_;
350    
351     $hdl->push_read (line => sub {
352     my ($hdl, $line) = @_;
353    
354     if ($line =~ /^\/quit/) { # /quit will end the client
355     $quit_cv->send;
356     } else {
357     send_message ($line);
358     }
359     });
360     }
361     );
362    
363     $quit_cv->recv;
364    
365 root 1.8 =head1 The Server
366 elmex 1.1
367 root 1.10 Ok, we finally come to the server.
368    
369     The server of course also needs to set up a port, and in addition needs to
370     I<register> it, so the clients can find it.
371 elmex 1.1
372 root 1.10 Again, let's jump directly into the code:
373 elmex 1.1
374     #!perl
375 root 1.4
376 elmex 1.1 use AnyEvent;
377     use AnyEvent::MP;
378    
379 root 1.10 become_public "127.0.0.1:1299";
380 elmex 1.1
381 root 1.10 my $chatter_port = port;
382    
383     reg $chatter_port, "chatter";
384 elmex 1.1
385     my %client_ports;
386    
387 root 1.10 rcv $chatter_port,
388     join => sub {
389     my ($tag, $client_port) = @_;
390 elmex 1.1
391 root 1.10 print "got new client port: $client_port\n";
392     $client_ports{$client_port} = 1;
393 elmex 1.5
394 root 1.10 0
395     },
396     message => sub {
397     my ($tag, $msg) = @_;
398 elmex 1.1
399 root 1.10 print "message> $msg\n";
400 elmex 1.1
401 root 1.10 snd $_, message => $msg
402     for keys %client_ports;
403 elmex 1.5
404 root 1.10 0
405     });
406 elmex 1.1
407     AnyEvent->condvar->recv;
408    
409 root 1.10 That is all. Looks much simpler than the client, doesn't it?
410    
411     Let's quickly look over it, as C<rcv> has already been discussed in the
412     client part of this tutorial above.
413 elmex 1.2
414     First this:
415    
416 root 1.10 become_public "127.0.0.1:1299";
417 elmex 1.1
418 root 1.10 This will tell our I<node> to become a I<public> node, which means that it
419     can be contacted via TCP. The first argument should be the I<noderef> the
420     server wants to be reachable at. In this case it's the TCP port 1299 on
421     C<127.0.0.1>.
422    
423     Next we set up two receivers, one for the C<join> messages and another one
424     for the actual messages of type C<messsage>. This is done with a single
425     call to C<rcv>, which allows multiple C<< match => $callback >> pairs.
426    
427     In the C<join> callback we receive the client port, which is simply
428     remembered in the C<%client_ports> hash. In the C<message> callback we
429     just iterate through all known C<%client_ports> and relay the message to
430     them.
431 elmex 1.1
432 root 1.10 That concludes the server.
433 elmex 1.2
434 root 1.8 =head1 The Remaining Problems
435 elmex 1.1
436 root 1.10 The implementation as shown still has some bugs. For instance: How does
437     the server know that the client isn't there anymore, so it can clean up
438     the C<%client_ports> hash? Also, the chat messages have no originator, so
439     we don't know who actually sent the message (which would be quite useful
440 elmex 1.1 for human-to-human interaction: to know who the other one is :).
441    
442 root 1.10 But aside from these issues I hope this tutorial showed you the basics of
443 elmex 1.1 L<AnyEvent::MP> and explained some common idioms.
444    
445 elmex 1.7 How to solve the reliability and C<%client_ports> cleanup problem will
446     be explained later in this tutorial (TODO).
447    
448 root 1.8 =head1 Inside The Protocol
449 elmex 1.7
450     Now, for the interested parties, let me explain some details about the protocol
451     that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
452     interested you can skip this section.
453    
454     Usually TCP is used for communication. Each I<node>, if configured to be a
455     I<public> node with the C<become_public> function will listen on the configured
456 root 1.10 TCP port (default is 4040).
457 elmex 1.7
458 root 1.10 If then one I<node> wants to send a message to another I<node> it will
459     connect to the host and port given in the I<port ID>.
460 elmex 1.7
461 root 1.10 Then some handshaking occurs to check whether both I<nodes> know the
462     I<shared secret>. Optionally, TLS can be enabled (about how to do this
463     exactly please consult the L<AnyEvent::MP> man page, just a hint: It
464     should be enough to put the private key and (self signed) certificate in
465     the C<~/.aemp-secret> file of all nodes).
466    
467     After the handshake, messages will be exchanged using a serialiser
468     (usually L<JSON> is used for this, but it is also possible to use other
469     serialization formats such as L<Storable>).
470 elmex 1.7
471 elmex 1.1 =head1 SEE ALSO
472    
473     L<AnyEvent>
474    
475     L<AnyEvent::Handle>
476    
477     L<AnyEvent::MP>
478    
479     =head1 AUTHOR
480    
481     Robin Redeker <elmex@ta-sa.org>
482 root 1.4