ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.7
Committed: Tue Aug 4 07:58:53 2009 UTC (14 years, 10 months ago) by elmex
Branch: MAIN
Changes since 1.6: +51 -1 lines
Log Message:
enhanced the Intro a bit.

File Contents

# User Rev Content
1 root 1.4 =head1 Message Passing for the Non-Blocked Mind
2 elmex 1.1
3     =head2 Introduction and Terminology
4    
5 root 1.4 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 elmex 1.1 module. Which allows us to transparently pass messages to our own process and
7     to other process on other or the same host.
8    
9 root 1.4 What kind of messages? Well, basically a message here means a list of
10     Perl strings, numbers, hashes and arrays, mostly everything that can be
11     expressed in a L<JSON> text (as JSON is used by default in the protocol).
12 elmex 1.1
13 root 1.4 And next you might ask: between which entities are those messages being "passed"?
14 elmex 1.1 Basically between C<nodes>, which are basically your applications (as in
15     processes) that use L<AnyEvent::MP> that run either on the same or different
16     hosts.
17    
18 root 1.4 In this tutorial I'll show you how to write a simple chat server based on
19 elmex 1.1 L<AnyEvent::MP>.
20    
21 elmex 1.7 =head2 System Requirements
22    
23     Before we can start we have to make sure some things work on your
24     system. First of all the host C<localhost> should resolve to a local
25     IP address. Next you should be able to do TCP over that address.
26    
27     You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
28     are installed. But how to do that is out of scope of this tutorial.
29    
30     Then we have to setup a I<shared secret>. For two L<AnyEvent::MP> nodes
31     to be able to communicate with each other and authenticate each other
32     it is necessary to setup a I<shared secret>. For testing you can write a
33     random string followed by a newline into the file C<.aemp-secret> in your
34     home directory:
35    
36     echo "secret123#4blabla_please_pick_your_own" > ~/.aemp-secret
37    
38     Only if the nodes that want to connect to each other have the same I<shared
39     secret> connections will be successful.
40    
41     B<If something does not work as expected, and for example tcpdump shows
42     that the connections are broken up early, you should make sure that ~/.aemp-secret
43     is the same on both hosts/user accounts you are connecting!>
44    
45 elmex 1.1 =head2 The Chat Client
46    
47 root 1.4 OK, lets start by implementing the "frontend" of the client. We will delay the
48 elmex 1.1 explanation and the code of the server until we finished the client, as the
49     most complex things actually happen in the client.
50    
51     We will use L<AnyEvent::Handle> to do non-blocking IO read on standard input:
52    
53     #!perl
54     use AnyEvent;
55     use AnyEvent::Handle;
56    
57     sub send_message {
58     die "This is where we will send the messages to the server"
59     . "in the next step of this tutorial.\n"
60     }
61    
62     # make an AnyEvent condition variable for the 'quit' condition
63     # (when we want to exit the client).
64     my $quit_cv = AnyEvent->condvar;
65    
66     my $stdin_hdl = AnyEvent::Handle->new (
67     fh => \*STDIN,
68     on_read => sub {
69     my ($hdl) = @_;
70    
71     $hdl->push_read (line => sub {
72     my ($hdl, $line) = @_;
73    
74     if ($line =~ /^\/quit/) { # /quit will end the client
75     $quit_cv->send;
76    
77     } else {
78     send_message ($line);
79     }
80     });
81     }
82     );
83    
84     $quit_cv->recv;
85    
86 root 1.4 This is now a very basic client. Explaining explicitly what
87     L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
88     of this document, please consult L<AnyEvent::Intro> or the manual pages
89     for L<AnyEvent> and L<AnyEvent::Handle>.
90 elmex 1.1
91     =head2 First Step Into Messaging
92    
93     Now we take a look at L<AnyEvent::MP>. We need to know what to do in
94 root 1.4 C<send_message>. This is an example of how it might look like:
95 elmex 1.1
96     ... # the use lines from the above snippet
97    
98     use AnyEvent::MP;
99    
100     sub send_message {
101     my ($msg) = @_;
102    
103     snd $server_port, message => $msg;
104     }
105    
106     ... # the rest of the above script
107    
108     The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send a
109     message'. The first argument is the I<port> (a I<port> is something that can
110     receive messages) of the server which will receive the message . How we get it
111     will be explained in the next step. The next arguments of C<snd> are
112 root 1.4 C<message> and C<$msg> are the first two elements of the I<message> (a
113 elmex 1.1 I<message> in L<AnyEvent::MP> is a be a simple list of values, which can be
114     sent to I<ports>).
115    
116     Thats all fine so far, but how do we get the C<$server_port>? Well, we will
117     need to get the unique I<port id> of the server's port where he wants to
118     receive all the incoming chat messages. A I<port id> is unfortunately a very
119     unique string, which we are unable to know in advance. But L<AnyEvent::MP>
120     supports the concept of 'well known ports', which is basically a port on the
121     server side registered under a well known name. For example, the server has a
122     port for receiving chat messages with a unique I<port id> and registered it
123 root 1.4 under the name C<chatter>.
124    
125     BTW, these "well known port names" should follow similar rules as Perl
126     identifiers, so you should prefix them with your package/module name to
127     make them unique, unless you use them in the main program.
128 elmex 1.1
129     As I<messages> can only be sent to a I<port id> and not just to a name we have
130     to ask the server I<node> what I<port id> has the well known port with the
131 root 1.4 name C<chatter>.
132 elmex 1.1
133     Another new term, what is a I<node>: The messaging network that can be created with
134     L<AnyEvent::MP> consists of I<nodes>. A I<node> handles all the connection and
135 root 1.4 low level message sending logic for its application. The application in this
136 elmex 1.1 case is the server. Also every client has/is a I<node>.
137    
138     =head2 Getting The Chatter Port
139    
140     Ok, lots of talk, now some code. Now we will actually get the C<$server_port>
141     from the backend:
142    
143     ...
144    
145     use AnyEvent::MP;
146    
147     my $resolved_cv = AnyEvent->condvar;
148    
149     my $client_port = create_port;
150    
151     my $server_node = "localhost:1299#";
152    
153     snd $server_node, wkp => "chatter", "$client_port", "resolved";
154    
155     my $server_port;
156    
157     # setup a receiver callback for the 'resolved' message:
158     $client_port->rcv (resolved => sub {
159     my ($client_port, $type, $chatter_port_id) = @_;
160    
161     print "Resolved the server port 'chatter' to $chatter_port_id\n";
162     $server_port = $chatter_port_id;
163    
164     $resolved_cv->send;
165     1
166     });
167    
168     # lets block the client until we resolved the server port.
169     $resolved_cv->recv;
170    
171     # now setup another receiver callback for the chat messages:
172     $client_port->rcv (message => sub {
173     my ($client_port, $type, $msg) = @_;
174    
175     print "chat> $msg\n";
176     0
177     });
178    
179     # send the server a 'join' message:
180     snd $server_port, join => "$client_port";
181    
182     sub send_message { ...
183    
184     Now that was a lot of new stuff. In order to ask the server and receive an
185     answer we need to have a I<port> where we can receive the answer.
186     This is what C<create_port> will do for us, it just creates a new local
187     port and returns us an object (that will btw. stringify to the I<port id>),
188     that we can use to receive messages.
189    
190     Next thing is the C<$server_node>. In order to refer to another node we need
191     some kind of string to reference it. The I<noderef> is basically a comma
192 root 1.6 separated list of C<host:port> pairs. We assume in this tutorial that the
193 elmex 1.1 server runs on your localhost at port 1299, this gives us the noderef
194 root 1.4 C<localhost:1299>.
195 elmex 1.1
196 root 1.4 Now you might ask what the C<#> at the end in C<$server_node> the above
197 elmex 1.1 example is about. Well, what I didn't tell you yet is that each I<node> has a
198     default I<port> to receive messages. The default port is the empty string
199     C<"">. The I<default port> of a I<node> also provides some special services for
200     us, for example resolving a well known port to a I<port id>.
201    
202     Now to this line:
203    
204     snd $server_node, wkp => "chatter", "$client_port", "resolved";
205    
206 root 1.4 We send a message with first element being C<wkp> (standing for 'well known
207 elmex 1.1 port'). Then the well known port name that we want to resolve to a I<port id>:
208 root 1.4 C<chatter>. And in order for the server node to be able to send us back the
209 elmex 1.1 resolved I<port id> we have to tell it where to send the result message: The
210 root 1.4 result message will have as it's first argument the string C<resolved> and
211     will be sent to C<$client_port> (the I<port id> of our own just created
212 elmex 1.1 port).
213    
214 root 1.4 Next comes the receiver for this C<wkp> request.
215 elmex 1.1
216     $client_port->rcv (resolved => sub {
217     my ($client_port, $type, $chatter_port_id) = @_;
218     ...
219     1
220     });
221    
222     This sets up a receiver on our own port for the result message with the first
223 root 1.4 argument being the string C<resolved>. Receivers can match the contents of
224 elmex 1.1 the messages before actually 'sending' it to the given callback.
225    
226     B<Please note> that the given callback has to return either a true or a false
227     value for indicating whether it is B<done> (true value) or still wants to
228     B<continue> (false value) receiving messages.
229    
230     In this case we tell the C<$client_port> to look into the received messages and
231 root 1.4 look for the string C<resolved> in the first element of the message. If it is
232 elmex 1.1 found, the given callback will be called with the C<$client_port> as first
233     argument, and the message as the remaining arguments.
234    
235     We name the first element of the message C<$type> in this case. It's a common
236     idiom to code the 'type' of a message into it's first element, this allows for
237     simple matching.
238    
239 root 1.4 The result message will contain the I<port id> of the well known port C<chatter>
240 elmex 1.1 as next element, and will be put in C<$chatter_port_id>.
241    
242     Next we just assign C<$server_port> and return a 1 (a true value)
243     from the callback. It indicates that we are done and don't want to receive
244 root 1.4 further C<resolved> messages with this callback.
245 elmex 1.1
246     Now we continue to the rest of the client by calling C<send> on
247     C<$resolved_cv>.
248    
249     First new step after this is setting up the chat message receiver callback.
250    
251     $client_port->rcv (message => sub {
252     my ($client_port, $type, $msg) = @_;
253    
254     print "chat> $msg\n";
255     0
256     });
257    
258 root 1.4 We assume that all messages that are broadcast to all clients by the server
259     will contain the string C<message> as first element, and the actual message as
260 elmex 1.1 second element. The callback returns a false value this time, to indicate that
261     it wants to continue receiving messages.
262    
263     Last but not least we actually tell the server to send us
264     the new chat messages from other clients. We do so by sending the
265 root 1.4 message type C<join> followed by our own I<port id>.
266 elmex 1.1
267     # send the server a 'join' message:
268     snd $server_port, join => "$client_port";
269    
270     Then the server knows where to send all the new messages to.
271    
272 root 1.6 =head2 The Completed Client
273 elmex 1.1
274     This is the complete client script:
275    
276     #!perl
277 root 1.4
278 elmex 1.1 use AnyEvent;
279     use AnyEvent::Handle;
280     use AnyEvent::MP;
281    
282     my $resolved_cv = AnyEvent->condvar;
283    
284     my $client_port = create_port;
285    
286     my $server_node = "localhost:1299#";
287    
288     snd $server_node, wkp => "chatter", "$client_port", "resolved";
289    
290     my $server_port;
291    
292     # setup a receiver callback for the 'resolved' message:
293     $client_port->rcv (resolved => sub {
294     my ($client_port, $type, $chatter_port_id) = @_;
295    
296     print "Resolved the server port 'chatter' to $chatter_port_id\n";
297     $server_port = $chatter_port_id;
298    
299     $resolved_cv->send;
300     1
301     });
302    
303     # lets block the client until we resolved the server port.
304     $resolved_cv->recv;
305    
306     # now setup another receiver callback for the chat messages:
307     $client_port->rcv (message => sub {
308     my ($client_port, $type, $msg) = @_;
309    
310     print "chat> $msg\n";
311     0
312     });
313    
314     # send the server a 'join' message:
315     snd $server_port, join => "$client_port";
316    
317     sub send_message {
318     my ($msg) = @_;
319    
320     snd $server_port, message => $msg;
321     }
322    
323     # make an AnyEvent condition variable for the 'quit' condition
324     # (when we want to exit the client).
325     my $quit_cv = AnyEvent->condvar;
326    
327     my $stdin_hdl = AnyEvent::Handle->new (
328     fh => \*STDIN,
329     on_read => sub {
330     my ($hdl) = @_;
331    
332     $hdl->push_read (line => sub {
333     my ($hdl, $line) = @_;
334    
335     if ($line =~ /^\/quit/) { # /quit will end the client
336     $quit_cv->send;
337    
338     } else {
339     send_message ($line);
340     }
341     });
342     }
343     );
344    
345     $quit_cv->recv;
346    
347     =head2 The Server
348    
349     Ok, now finally to the server. What do we need? Well, we need to setup
350 root 1.4 the well known port C<chatter> where all clients send their messages to.
351 elmex 1.1
352     Up and into code right now:
353    
354     #!perl
355 root 1.4
356 elmex 1.1 use AnyEvent;
357     use AnyEvent::MP;
358    
359     become_public "localhost:1299";
360    
361     my $chatter_port = create_port;
362     $chatter_port->register ("chatter");
363    
364     my %client_ports;
365    
366     $chatter_port->rcv (join => sub {
367     my ($chatter_port, $type, $client_port) = @_;
368    
369 elmex 1.5 print "got new client port: $client_port\n";
370    
371 elmex 1.1 $client_ports{$client_port} = 1;
372     0
373     });
374    
375     $chatter_port->rcv (message => sub {
376     my ($chatter_port, $type, $msg) = @_;
377    
378 elmex 1.5 print "message> $msg\n";
379    
380 elmex 1.1 snd $_, message => $msg for keys %client_ports;
381     0
382     });
383    
384     AnyEvent->condvar->recv;
385    
386     This is all. Looks much easier, doesn't it? I'll explain it only shortly, as
387     we had the discussion of the C<rcv> method in the client part of this tutorial
388 elmex 1.2 above.
389    
390     First this:
391    
392     become_public "localhost:1299";
393    
394     This will tell our I<node> to become a I<public> node, which means that it can
395     be contacted via TCP. The first argument should be the I<noderef> the server
396     wants to be reachable at. In this case it's the TCP port 1299 on localhost.
397    
398 elmex 1.7 Next we basically setup two receivers, one for the C<join> messages and
399 elmex 1.1 another one for the actual messages of type C<messsage>.
400    
401     In the C<join> message we get the client's port, which we just remember in the
402     C<%client_ports> hash. In the receiver for the message type C<message> we will
403     just iterate through all known C<%client_ports> and relay the message to them.
404    
405 elmex 1.2 And thats it.
406    
407 elmex 1.1 =head2 The Remaining Problems
408    
409     The shown implementation still has some bugs. For instance: How does the
410     server know that the client isn't there anymore, and can cleanup the
411     C<%client_ports> hash? And also the chat messages have no originator,
412     so we don't know who actually sent the message (which would be quite useful
413     for human-to-human interaction: to know who the other one is :).
414    
415     But aside from these issues I hope this tutorial got you the swing of
416     L<AnyEvent::MP> and explained some common idioms.
417    
418 elmex 1.7 How to solve the reliability and C<%client_ports> cleanup problem will
419     be explained later in this tutorial (TODO).
420    
421     =head2 Inside The Protocol
422    
423     Now, for the interested parties, let me explain some details about the protocol
424     that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
425     interested you can skip this section.
426    
427     Usually TCP is used for communication. Each I<node>, if configured to be a
428     I<public> node with the C<become_public> function will listen on the configured
429     TCP port (default is usually 4040).
430    
431     If now one I<node> wants to send a message to another I<node> it will connect
432     to the host and port given in the I<port id>.
433    
434     Then some handshaking occurs to check whether both I<nodes> have the same
435     I<shared secret> configured. Optionally even TLS can be enabled (about how to
436     do this exactly please consult the L<AnyEvent::MP> man pages, just a hint: It
437     should be enough to put the private key and (self signed) certificate in the
438     C<~/.aemp-secret> file of all nodes).
439    
440     Now the serialized (usually L<JSON> is used for this, but it is also possible
441     to use other serialization formats, like L<Storable>) messages are sent over
442     the wire using a simple framing protocol.
443    
444 elmex 1.1 =head1 SEE ALSO
445    
446     L<AnyEvent>
447    
448     L<AnyEvent::Handle>
449    
450     L<AnyEvent::MP>
451    
452     =head1 AUTHOR
453    
454     Robin Redeker <elmex@ta-sa.org>
455 root 1.4