ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.7
Committed: Tue Aug 4 07:58:53 2009 UTC (14 years, 10 months ago) by elmex
Branch: MAIN
Changes since 1.6: +51 -1 lines
Log Message:
enhanced the Intro a bit.

File Contents

# Content
1 =head1 Message Passing for the Non-Blocked Mind
2
3 =head2 Introduction and Terminology
4
5 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 module. Which allows us to transparently pass messages to our own process and
7 to other process on other or the same host.
8
9 What kind of messages? Well, basically a message here means a list of
10 Perl strings, numbers, hashes and arrays, mostly everything that can be
11 expressed in a L<JSON> text (as JSON is used by default in the protocol).
12
13 And next you might ask: between which entities are those messages being "passed"?
14 Basically between C<nodes>, which are basically your applications (as in
15 processes) that use L<AnyEvent::MP> that run either on the same or different
16 hosts.
17
18 In this tutorial I'll show you how to write a simple chat server based on
19 L<AnyEvent::MP>.
20
21 =head2 System Requirements
22
23 Before we can start we have to make sure some things work on your
24 system. First of all the host C<localhost> should resolve to a local
25 IP address. Next you should be able to do TCP over that address.
26
27 You should of course also make sure that L<AnyEvent> and L<AnyEvent::MP>
28 are installed. But how to do that is out of scope of this tutorial.
29
30 Then we have to setup a I<shared secret>. For two L<AnyEvent::MP> nodes
31 to be able to communicate with each other and authenticate each other
32 it is necessary to setup a I<shared secret>. For testing you can write a
33 random string followed by a newline into the file C<.aemp-secret> in your
34 home directory:
35
36 echo "secret123#4blabla_please_pick_your_own" > ~/.aemp-secret
37
38 Only if the nodes that want to connect to each other have the same I<shared
39 secret> connections will be successful.
40
41 B<If something does not work as expected, and for example tcpdump shows
42 that the connections are broken up early, you should make sure that ~/.aemp-secret
43 is the same on both hosts/user accounts you are connecting!>
44
45 =head2 The Chat Client
46
47 OK, lets start by implementing the "frontend" of the client. We will delay the
48 explanation and the code of the server until we finished the client, as the
49 most complex things actually happen in the client.
50
51 We will use L<AnyEvent::Handle> to do non-blocking IO read on standard input:
52
53 #!perl
54 use AnyEvent;
55 use AnyEvent::Handle;
56
57 sub send_message {
58 die "This is where we will send the messages to the server"
59 . "in the next step of this tutorial.\n"
60 }
61
62 # make an AnyEvent condition variable for the 'quit' condition
63 # (when we want to exit the client).
64 my $quit_cv = AnyEvent->condvar;
65
66 my $stdin_hdl = AnyEvent::Handle->new (
67 fh => \*STDIN,
68 on_read => sub {
69 my ($hdl) = @_;
70
71 $hdl->push_read (line => sub {
72 my ($hdl, $line) = @_;
73
74 if ($line =~ /^\/quit/) { # /quit will end the client
75 $quit_cv->send;
76
77 } else {
78 send_message ($line);
79 }
80 });
81 }
82 );
83
84 $quit_cv->recv;
85
86 This is now a very basic client. Explaining explicitly what
87 L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
88 of this document, please consult L<AnyEvent::Intro> or the manual pages
89 for L<AnyEvent> and L<AnyEvent::Handle>.
90
91 =head2 First Step Into Messaging
92
93 Now we take a look at L<AnyEvent::MP>. We need to know what to do in
94 C<send_message>. This is an example of how it might look like:
95
96 ... # the use lines from the above snippet
97
98 use AnyEvent::MP;
99
100 sub send_message {
101 my ($msg) = @_;
102
103 snd $server_port, message => $msg;
104 }
105
106 ... # the rest of the above script
107
108 The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send a
109 message'. The first argument is the I<port> (a I<port> is something that can
110 receive messages) of the server which will receive the message . How we get it
111 will be explained in the next step. The next arguments of C<snd> are
112 C<message> and C<$msg> are the first two elements of the I<message> (a
113 I<message> in L<AnyEvent::MP> is a be a simple list of values, which can be
114 sent to I<ports>).
115
116 Thats all fine so far, but how do we get the C<$server_port>? Well, we will
117 need to get the unique I<port id> of the server's port where he wants to
118 receive all the incoming chat messages. A I<port id> is unfortunately a very
119 unique string, which we are unable to know in advance. But L<AnyEvent::MP>
120 supports the concept of 'well known ports', which is basically a port on the
121 server side registered under a well known name. For example, the server has a
122 port for receiving chat messages with a unique I<port id> and registered it
123 under the name C<chatter>.
124
125 BTW, these "well known port names" should follow similar rules as Perl
126 identifiers, so you should prefix them with your package/module name to
127 make them unique, unless you use them in the main program.
128
129 As I<messages> can only be sent to a I<port id> and not just to a name we have
130 to ask the server I<node> what I<port id> has the well known port with the
131 name C<chatter>.
132
133 Another new term, what is a I<node>: The messaging network that can be created with
134 L<AnyEvent::MP> consists of I<nodes>. A I<node> handles all the connection and
135 low level message sending logic for its application. The application in this
136 case is the server. Also every client has/is a I<node>.
137
138 =head2 Getting The Chatter Port
139
140 Ok, lots of talk, now some code. Now we will actually get the C<$server_port>
141 from the backend:
142
143 ...
144
145 use AnyEvent::MP;
146
147 my $resolved_cv = AnyEvent->condvar;
148
149 my $client_port = create_port;
150
151 my $server_node = "localhost:1299#";
152
153 snd $server_node, wkp => "chatter", "$client_port", "resolved";
154
155 my $server_port;
156
157 # setup a receiver callback for the 'resolved' message:
158 $client_port->rcv (resolved => sub {
159 my ($client_port, $type, $chatter_port_id) = @_;
160
161 print "Resolved the server port 'chatter' to $chatter_port_id\n";
162 $server_port = $chatter_port_id;
163
164 $resolved_cv->send;
165 1
166 });
167
168 # lets block the client until we resolved the server port.
169 $resolved_cv->recv;
170
171 # now setup another receiver callback for the chat messages:
172 $client_port->rcv (message => sub {
173 my ($client_port, $type, $msg) = @_;
174
175 print "chat> $msg\n";
176 0
177 });
178
179 # send the server a 'join' message:
180 snd $server_port, join => "$client_port";
181
182 sub send_message { ...
183
184 Now that was a lot of new stuff. In order to ask the server and receive an
185 answer we need to have a I<port> where we can receive the answer.
186 This is what C<create_port> will do for us, it just creates a new local
187 port and returns us an object (that will btw. stringify to the I<port id>),
188 that we can use to receive messages.
189
190 Next thing is the C<$server_node>. In order to refer to another node we need
191 some kind of string to reference it. The I<noderef> is basically a comma
192 separated list of C<host:port> pairs. We assume in this tutorial that the
193 server runs on your localhost at port 1299, this gives us the noderef
194 C<localhost:1299>.
195
196 Now you might ask what the C<#> at the end in C<$server_node> the above
197 example is about. Well, what I didn't tell you yet is that each I<node> has a
198 default I<port> to receive messages. The default port is the empty string
199 C<"">. The I<default port> of a I<node> also provides some special services for
200 us, for example resolving a well known port to a I<port id>.
201
202 Now to this line:
203
204 snd $server_node, wkp => "chatter", "$client_port", "resolved";
205
206 We send a message with first element being C<wkp> (standing for 'well known
207 port'). Then the well known port name that we want to resolve to a I<port id>:
208 C<chatter>. And in order for the server node to be able to send us back the
209 resolved I<port id> we have to tell it where to send the result message: The
210 result message will have as it's first argument the string C<resolved> and
211 will be sent to C<$client_port> (the I<port id> of our own just created
212 port).
213
214 Next comes the receiver for this C<wkp> request.
215
216 $client_port->rcv (resolved => sub {
217 my ($client_port, $type, $chatter_port_id) = @_;
218 ...
219 1
220 });
221
222 This sets up a receiver on our own port for the result message with the first
223 argument being the string C<resolved>. Receivers can match the contents of
224 the messages before actually 'sending' it to the given callback.
225
226 B<Please note> that the given callback has to return either a true or a false
227 value for indicating whether it is B<done> (true value) or still wants to
228 B<continue> (false value) receiving messages.
229
230 In this case we tell the C<$client_port> to look into the received messages and
231 look for the string C<resolved> in the first element of the message. If it is
232 found, the given callback will be called with the C<$client_port> as first
233 argument, and the message as the remaining arguments.
234
235 We name the first element of the message C<$type> in this case. It's a common
236 idiom to code the 'type' of a message into it's first element, this allows for
237 simple matching.
238
239 The result message will contain the I<port id> of the well known port C<chatter>
240 as next element, and will be put in C<$chatter_port_id>.
241
242 Next we just assign C<$server_port> and return a 1 (a true value)
243 from the callback. It indicates that we are done and don't want to receive
244 further C<resolved> messages with this callback.
245
246 Now we continue to the rest of the client by calling C<send> on
247 C<$resolved_cv>.
248
249 First new step after this is setting up the chat message receiver callback.
250
251 $client_port->rcv (message => sub {
252 my ($client_port, $type, $msg) = @_;
253
254 print "chat> $msg\n";
255 0
256 });
257
258 We assume that all messages that are broadcast to all clients by the server
259 will contain the string C<message> as first element, and the actual message as
260 second element. The callback returns a false value this time, to indicate that
261 it wants to continue receiving messages.
262
263 Last but not least we actually tell the server to send us
264 the new chat messages from other clients. We do so by sending the
265 message type C<join> followed by our own I<port id>.
266
267 # send the server a 'join' message:
268 snd $server_port, join => "$client_port";
269
270 Then the server knows where to send all the new messages to.
271
272 =head2 The Completed Client
273
274 This is the complete client script:
275
276 #!perl
277
278 use AnyEvent;
279 use AnyEvent::Handle;
280 use AnyEvent::MP;
281
282 my $resolved_cv = AnyEvent->condvar;
283
284 my $client_port = create_port;
285
286 my $server_node = "localhost:1299#";
287
288 snd $server_node, wkp => "chatter", "$client_port", "resolved";
289
290 my $server_port;
291
292 # setup a receiver callback for the 'resolved' message:
293 $client_port->rcv (resolved => sub {
294 my ($client_port, $type, $chatter_port_id) = @_;
295
296 print "Resolved the server port 'chatter' to $chatter_port_id\n";
297 $server_port = $chatter_port_id;
298
299 $resolved_cv->send;
300 1
301 });
302
303 # lets block the client until we resolved the server port.
304 $resolved_cv->recv;
305
306 # now setup another receiver callback for the chat messages:
307 $client_port->rcv (message => sub {
308 my ($client_port, $type, $msg) = @_;
309
310 print "chat> $msg\n";
311 0
312 });
313
314 # send the server a 'join' message:
315 snd $server_port, join => "$client_port";
316
317 sub send_message {
318 my ($msg) = @_;
319
320 snd $server_port, message => $msg;
321 }
322
323 # make an AnyEvent condition variable for the 'quit' condition
324 # (when we want to exit the client).
325 my $quit_cv = AnyEvent->condvar;
326
327 my $stdin_hdl = AnyEvent::Handle->new (
328 fh => \*STDIN,
329 on_read => sub {
330 my ($hdl) = @_;
331
332 $hdl->push_read (line => sub {
333 my ($hdl, $line) = @_;
334
335 if ($line =~ /^\/quit/) { # /quit will end the client
336 $quit_cv->send;
337
338 } else {
339 send_message ($line);
340 }
341 });
342 }
343 );
344
345 $quit_cv->recv;
346
347 =head2 The Server
348
349 Ok, now finally to the server. What do we need? Well, we need to setup
350 the well known port C<chatter> where all clients send their messages to.
351
352 Up and into code right now:
353
354 #!perl
355
356 use AnyEvent;
357 use AnyEvent::MP;
358
359 become_public "localhost:1299";
360
361 my $chatter_port = create_port;
362 $chatter_port->register ("chatter");
363
364 my %client_ports;
365
366 $chatter_port->rcv (join => sub {
367 my ($chatter_port, $type, $client_port) = @_;
368
369 print "got new client port: $client_port\n";
370
371 $client_ports{$client_port} = 1;
372 0
373 });
374
375 $chatter_port->rcv (message => sub {
376 my ($chatter_port, $type, $msg) = @_;
377
378 print "message> $msg\n";
379
380 snd $_, message => $msg for keys %client_ports;
381 0
382 });
383
384 AnyEvent->condvar->recv;
385
386 This is all. Looks much easier, doesn't it? I'll explain it only shortly, as
387 we had the discussion of the C<rcv> method in the client part of this tutorial
388 above.
389
390 First this:
391
392 become_public "localhost:1299";
393
394 This will tell our I<node> to become a I<public> node, which means that it can
395 be contacted via TCP. The first argument should be the I<noderef> the server
396 wants to be reachable at. In this case it's the TCP port 1299 on localhost.
397
398 Next we basically setup two receivers, one for the C<join> messages and
399 another one for the actual messages of type C<messsage>.
400
401 In the C<join> message we get the client's port, which we just remember in the
402 C<%client_ports> hash. In the receiver for the message type C<message> we will
403 just iterate through all known C<%client_ports> and relay the message to them.
404
405 And thats it.
406
407 =head2 The Remaining Problems
408
409 The shown implementation still has some bugs. For instance: How does the
410 server know that the client isn't there anymore, and can cleanup the
411 C<%client_ports> hash? And also the chat messages have no originator,
412 so we don't know who actually sent the message (which would be quite useful
413 for human-to-human interaction: to know who the other one is :).
414
415 But aside from these issues I hope this tutorial got you the swing of
416 L<AnyEvent::MP> and explained some common idioms.
417
418 How to solve the reliability and C<%client_ports> cleanup problem will
419 be explained later in this tutorial (TODO).
420
421 =head2 Inside The Protocol
422
423 Now, for the interested parties, let me explain some details about the protocol
424 that L<AnyEvent::MP> nodes use to communicate to each other. If you are not
425 interested you can skip this section.
426
427 Usually TCP is used for communication. Each I<node>, if configured to be a
428 I<public> node with the C<become_public> function will listen on the configured
429 TCP port (default is usually 4040).
430
431 If now one I<node> wants to send a message to another I<node> it will connect
432 to the host and port given in the I<port id>.
433
434 Then some handshaking occurs to check whether both I<nodes> have the same
435 I<shared secret> configured. Optionally even TLS can be enabled (about how to
436 do this exactly please consult the L<AnyEvent::MP> man pages, just a hint: It
437 should be enough to put the private key and (self signed) certificate in the
438 C<~/.aemp-secret> file of all nodes).
439
440 Now the serialized (usually L<JSON> is used for this, but it is also possible
441 to use other serialization formats, like L<Storable>) messages are sent over
442 the wire using a simple framing protocol.
443
444 =head1 SEE ALSO
445
446 L<AnyEvent>
447
448 L<AnyEvent::Handle>
449
450 L<AnyEvent::MP>
451
452 =head1 AUTHOR
453
454 Robin Redeker <elmex@ta-sa.org>
455