AnyEvent-MP/MP/Intro.pod

=head1 Message Passing for the Non-Blocked Mind

=head2 Introduction and Terminology

This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
module. Which allows us to transparently pass messages to our own process and
to other process on other or the same host.

What kind of messages? Well, basically a message here means a list of
Perl strings, numbers, hashes and arrays, mostly everything that can be
expressed in a L<JSON> text (as JSON is used by default in the protocol).

And next you might ask: between which entities are those messages being "passed"?
Basically between C<nodes>, which are basically your applications (as in
processes) that use L<AnyEvent::MP> that run either on the same or different
hosts.

In this tutorial I'll show you how to write a simple chat server based on
L<AnyEvent::MP>.

=head2 The Chat Client

OK, lets start by implementing the "frontend" of the client. We will delay the
explanation and the code of the server until we finished the client, as the
most complex things actually happen in the client.

We will use L<AnyEvent::Handle> to do non-blocking IO read on standard input:

   #!perl
   use AnyEvent;
   use AnyEvent::Handle;

   sub send_message {
      die "This is where we will send the messages to the server"
         . "in the next step of this tutorial.\n"
   }

   # make an AnyEvent condition variable for the 'quit' condition
   # (when we want to exit the client).
   my $quit_cv = AnyEvent->condvar;

   my $stdin_hdl = AnyEvent::Handle->new (
      fh      => \*STDIN,
      on_read => sub {
         my ($hdl) = @_;

         $hdl->push_read (line => sub {
            my ($hdl, $line) = @_;

            if ($line =~ /^\/quit/) { # /quit will end the client
               $quit_cv->send;

            } else {
               send_message ($line);
            }
         });
      }
   );

   $quit_cv->recv;

This is now a very basic client. Explaining explicitly what
L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
of this document, please consult L<AnyEvent::Intro> or the manual pages
for L<AnyEvent> and L<AnyEvent::Handle>.

=head2 First Step Into Messaging

Now we take a look at L<AnyEvent::MP>. We need to know what to do in
C<send_message>. This is an example of how it might look like:

   ... # the use lines from the above snippet

   use AnyEvent::MP;

   sub send_message {
      my ($msg) = @_;

      snd $server_port, message => $msg;
   }

   ... # the rest of the above script

The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send a
message'. The first argument is the I<port> (a I<port> is something that can
receive messages) of the server which will receive the message . How we get it
will be explained in the next step.  The next arguments of C<snd> are
C<message> and C<$msg> are the first two elements of the I<message> (a
I<message> in L<AnyEvent::MP> is a be a simple list of values, which can be
sent to I<ports>).

Thats all fine so far, but how do we get the C<$server_port>? Well, we will
need to get the unique I<port id> of the server's port where he wants to
receive all the incoming chat messages. A I<port id> is unfortunately a very
unique string, which we are unable to know in advance. But L<AnyEvent::MP>
supports the concept of 'well known ports', which is basically a port on the
server side registered under a well known name. For example, the server has a
port for receiving chat messages with a unique I<port id> and registered it
under the name C<chatter>.

BTW, these "well known port names" should follow similar rules as Perl
identifiers, so you should prefix them with your package/module name to
make them unique, unless you use them in the main program.

As I<messages> can only be sent to a I<port id> and not just to a name we have
to ask the server I<node> what I<port id> has the well known port with the
name C<chatter>.

Another new term, what is a I<node>: The messaging network that can be created with
L<AnyEvent::MP> consists of I<nodes>. A I<node> handles all the connection and
low level message sending logic for its application. The application in this
case is the server. Also every client has/is a I<node>.

=head2 Getting The Chatter Port

Ok, lots of talk, now some code. Now we will actually get the C<$server_port>
from the backend:

   ...

   use AnyEvent::MP;

   my $resolved_cv = AnyEvent->condvar;

   my $client_port = create_port;

   my $server_node = "localhost:1299#";

   snd $server_node, wkp => "chatter", "$client_port", "resolved";

   my $server_port;

   # setup a receiver callback for the 'resolved' message:
   $client_port->rcv (resolved => sub {
      my ($client_port, $type, $chatter_port_id) = @_;

      print "Resolved the server port 'chatter' to $chatter_port_id\n";
      $server_port = $chatter_port_id;

      $resolved_cv->send;
      1
   });

   # lets block the client until we resolved the server port.
   $resolved_cv->recv;

   # now setup another receiver callback for the chat messages:
   $client_port->rcv (message => sub {
      my ($client_port, $type, $msg) = @_;

      print "chat> $msg\n";
      0
   });

   # send the server a 'join' message:
   snd $server_port, join => "$client_port";

   sub send_message { ...

Now that was a lot of new stuff. In order to ask the server and receive an
answer we need to have a I<port> where we can receive the answer.
This is what C<create_port> will do for us, it just creates a new local
port and returns us an object (that will btw. stringify to the I<port id>),
that we can use to receive messages.

Next thing is the C<$server_node>. In order to refer to another node we need
some kind of string to reference it. The I<noderef> is basically a comma
seperated list of C<host:port> pairs. We assume in this tutorial that the
server runs on your localhost at port 1299, this gives us the noderef
C<localhost:1299>.

Now you might ask what the C<#> at the end in C<$server_node> the above
example is about. Well, what I didn't tell you yet is that each I<node> has a
default I<port> to receive messages. The default port is the empty string
C<"">. The I<default port> of a I<node> also provides some special services for
us, for example resolving a well known port to a I<port id>.

Now to this line:

   snd $server_node, wkp => "chatter", "$client_port", "resolved";

We send a message with first element being C<wkp> (standing for 'well known
port'). Then the well known port name that we want to resolve to a I<port id>:
C<chatter>. And in order for the server node to be able to send us back the
resolved I<port id> we have to tell it where to send the result message: The
result message will have as it's first argument the string C<resolved> and
will be sent to C<$client_port> (the I<port id> of our own just created
port).

Next comes the receiver for this C<wkp> request.

   $client_port->rcv (resolved => sub {
      my ($client_port, $type, $chatter_port_id) = @_;
      ...
      1
   });

This sets up a receiver on our own port for the result message with the first
argument being the string C<resolved>. Receivers can match the contents of
the messages before actually 'sending' it to the given callback.

B<Please note> that the given callback has to return either a true or a false
value for indicating whether it is B<done> (true value) or still wants to
B<continue> (false value) receiving messages.

In this case we tell the C<$client_port> to look into the received messages and
look for the string C<resolved> in the first element of the message. If it is
found, the given callback will be called with the C<$client_port> as first
argument, and the message as the remaining arguments.

We name the first element of the message C<$type> in this case. It's a common
idiom to code the 'type' of a message into it's first element, this allows for
simple matching.

The result message will contain the I<port id> of the well known port C<chatter>
as next element, and will be put in C<$chatter_port_id>.

Next we just assign C<$server_port> and return a 1 (a true value)
from the callback. It indicates that we are done and don't want to receive
further C<resolved> messages with this callback.

Now we continue to the rest of the client by calling C<send> on
C<$resolved_cv>.

First new step after this is setting up the chat message receiver callback.

   $client_port->rcv (message => sub {
      my ($client_port, $type, $msg) = @_;

      print "chat> $msg\n";
      0
   });

We assume that all messages that are broadcast to all clients by the server
will contain the string C<message> as first element, and the actual message as
second element. The callback returns a false value this time, to indicate that
it wants to continue receiving messages.

Last but not least we actually tell the server to send us
the new chat messages from other clients. We do so by sending the
message type C<join> followed by our own I<port id>.

   # send the server a 'join' message:
   snd $server_port, join => "$client_port";

Then the server knows where to send all the new messages to.

=head2 The Finished Client

This is the complete client script:

   #!perl

   use AnyEvent;
   use AnyEvent::Handle;
   use AnyEvent::MP;

   my $resolved_cv = AnyEvent->condvar;

   my $client_port = create_port;

   my $server_node = "localhost:1299#";

   snd $server_node, wkp => "chatter", "$client_port", "resolved";

   my $server_port;

   # setup a receiver callback for the 'resolved' message:
   $client_port->rcv (resolved => sub {
      my ($client_port, $type, $chatter_port_id) = @_;

      print "Resolved the server port 'chatter' to $chatter_port_id\n";
      $server_port = $chatter_port_id;

      $resolved_cv->send;
      1
   });

   # lets block the client until we resolved the server port.
   $resolved_cv->recv;

   # now setup another receiver callback for the chat messages:
   $client_port->rcv (message => sub {
      my ($client_port, $type, $msg) = @_;

      print "chat> $msg\n";
      0
   });

   # send the server a 'join' message:
   snd $server_port, join => "$client_port";

   sub send_message {
      my ($msg) = @_;

      snd $server_port, message => $msg;
   }

   # make an AnyEvent condition variable for the 'quit' condition
   # (when we want to exit the client).
   my $quit_cv = AnyEvent->condvar;

   my $stdin_hdl = AnyEvent::Handle->new (
      fh      => \*STDIN,
      on_read => sub {
         my ($hdl) = @_;

         $hdl->push_read (line => sub {
            my ($hdl, $line) = @_;

            if ($line =~ /^\/quit/) { # /quit will end the client
               $quit_cv->send;

            } else {
               send_message ($line);
            }
         });
      }
   );

   $quit_cv->recv;

=head2 The Server

Ok, now finally to the server. What do we need? Well, we need to setup
the well known port C<chatter> where all clients send their messages to.

Up and into code right now:

   #!perl

   use AnyEvent;
   use AnyEvent::MP;

   become_public "localhost:1299";

   my $chatter_port = create_port;
   $chatter_port->register ("chatter");

   my %client_ports;

   $chatter_port->rcv (join => sub {
      my ($chatter_port, $type, $client_port) = @_;

      print "got new client port: $client_port\n";

      $client_ports{$client_port} = 1;
      0
   });

   $chatter_port->rcv (message => sub {
      my ($chatter_port, $type, $msg) = @_;

      print "message> $msg\n";

      snd $_, message => $msg for keys %client_ports;
      0
   });

   AnyEvent->condvar->recv;

This is all. Looks much easier, doesn't it? I'll explain it only shortly, as
we had the discussion of the C<rcv> method in the client part of this tutorial
above.

First this:

   become_public "localhost:1299";

This will tell our I<node> to become a I<public> node, which means that it can
be contacted via TCP. The first argument should be the I<noderef> the server
wants to be reachable at. In this case it's the TCP port 1299 on localhost.

Next we bascially setup two receivers, one for the C<join> messages and
another one for the actual messages of type C<messsage>.

In the C<join> message we get the client's port, which we just remember in the
C<%client_ports> hash. In the receiver for the message type C<message> we will
just iterate through all known C<%client_ports> and relay the message to them.

And thats it.

=head2 The Remaining Problems

The shown implementation still has some bugs. For instance: How does the
server know that the client isn't there anymore, and can cleanup the
C<%client_ports> hash? And also the chat messages have no originator,
so we don't know who actually sent the message (which would be quite useful
for human-to-human interaction: to know who the other one is :).

But aside from these issues I hope this tutorial got you the swing of
L<AnyEvent::MP> and explained some common idioms.

=head1 SEE ALSO

L<AnyEvent>

L<AnyEvent::Handle>

L<AnyEvent::MP>

=head1 AUTHOR

  Robin Redeker <elmex@ta-sa.org>

Revision:	1.5
Committed:	Mon Aug 3 09:43:15 2009 UTC (14 years, 9 months ago) by elmex
Branch:	MAIN
Changes since 1.4:	+4 -0 lines
Log Message:	added examples from intro.
#	Content
1	=head1 Message Passing for the Non-Blocked Mind
2
3	=head2 Introduction and Terminology
4
5	This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6	module. Which allows us to transparently pass messages to our own process and
7	to other process on other or the same host.
8
9	What kind of messages? Well, basically a message here means a list of
10	Perl strings, numbers, hashes and arrays, mostly everything that can be
11	expressed in a L<JSON> text (as JSON is used by default in the protocol).
12
13	And next you might ask: between which entities are those messages being "passed"?
14	Basically between C<nodes>, which are basically your applications (as in
15	processes) that use L<AnyEvent::MP> that run either on the same or different
16	hosts.
17
18	In this tutorial I'll show you how to write a simple chat server based on
19	L<AnyEvent::MP>.
20
21	=head2 The Chat Client
22
23	OK, lets start by implementing the "frontend" of the client. We will delay the
24	explanation and the code of the server until we finished the client, as the
25	most complex things actually happen in the client.
26
27	We will use L<AnyEvent::Handle> to do non-blocking IO read on standard input:
28
29	#!perl
30	use AnyEvent;
31	use AnyEvent::Handle;
32
33	sub send_message {
34	die "This is where we will send the messages to the server"
35	. "in the next step of this tutorial.\n"
36	}
37
38	# make an AnyEvent condition variable for the 'quit' condition
39	# (when we want to exit the client).
40	my $quit_cv = AnyEvent->condvar;
41
42	my $stdin_hdl = AnyEvent::Handle->new (
43	fh => \*STDIN,
44	on_read => sub {
45	my ($hdl) = @_;
46
47	$hdl->push_read (line => sub {
48	my ($hdl, $line) = @_;
49
50	if ($line =~ /^\/quit/) { # /quit will end the client
51	$quit_cv->send;
52
53	} else {
54	send_message ($line);
55	}
56	});
57	}
58	);
59
60	$quit_cv->recv;
61
62	This is now a very basic client. Explaining explicitly what
63	L<AnyEvent::Handle> does or what a I<condvar> is all about is out of scope
64	of this document, please consult L<AnyEvent::Intro> or the manual pages
65	for L<AnyEvent> and L<AnyEvent::Handle>.
66
67	=head2 First Step Into Messaging
68
69	Now we take a look at L<AnyEvent::MP>. We need to know what to do in
70	C<send_message>. This is an example of how it might look like:
71
72	... # the use lines from the above snippet
73
74	use AnyEvent::MP;
75
76	sub send_message {
77	my ($msg) = @_;
78
79	snd $server_port, message => $msg;
80	}
81
82	... # the rest of the above script
83
84	The C<snd> function is exported by L<AnyEvent::MP>, it stands for 'send a
85	message'. The first argument is the I<port> (a I<port> is something that can
86	receive messages) of the server which will receive the message . How we get it
87	will be explained in the next step. The next arguments of C<snd> are
88	C<message> and C<$msg> are the first two elements of the I<message> (a
89	I<message> in L<AnyEvent::MP> is a be a simple list of values, which can be
90	sent to I<ports>).
91
92	Thats all fine so far, but how do we get the C<$server_port>? Well, we will
93	need to get the unique I<port id> of the server's port where he wants to
94	receive all the incoming chat messages. A I<port id> is unfortunately a very
95	unique string, which we are unable to know in advance. But L<AnyEvent::MP>
96	supports the concept of 'well known ports', which is basically a port on the
97	server side registered under a well known name. For example, the server has a
98	port for receiving chat messages with a unique I<port id> and registered it
99	under the name C<chatter>.
100
101	BTW, these "well known port names" should follow similar rules as Perl
102	identifiers, so you should prefix them with your package/module name to
103	make them unique, unless you use them in the main program.
104
105	As I<messages> can only be sent to a I<port id> and not just to a name we have
106	to ask the server I<node> what I<port id> has the well known port with the
107	name C<chatter>.
108
109	Another new term, what is a I<node>: The messaging network that can be created with
110	L<AnyEvent::MP> consists of I<nodes>. A I<node> handles all the connection and
111	low level message sending logic for its application. The application in this
112	case is the server. Also every client has/is a I<node>.
113
114	=head2 Getting The Chatter Port
115
116	Ok, lots of talk, now some code. Now we will actually get the C<$server_port>
117	from the backend:
118
119	...
120
121	use AnyEvent::MP;
122
123	my $resolved_cv = AnyEvent->condvar;
124
125	my $client_port = create_port;
126
127	my $server_node = "localhost:1299#";
128
129	snd $server_node, wkp => "chatter", "$client_port", "resolved";
130
131	my $server_port;
132
133	# setup a receiver callback for the 'resolved' message:
134	$client_port->rcv (resolved => sub {
135	my ($client_port, $type, $chatter_port_id) = @_;
136
137	print "Resolved the server port 'chatter' to $chatter_port_id\n";
138	$server_port = $chatter_port_id;
139
140	$resolved_cv->send;
141	1
142	});
143
144	# lets block the client until we resolved the server port.
145	$resolved_cv->recv;
146
147	# now setup another receiver callback for the chat messages:
148	$client_port->rcv (message => sub {
149	my ($client_port, $type, $msg) = @_;
150
151	print "chat> $msg\n";
152	0
153	});
154
155	# send the server a 'join' message:
156	snd $server_port, join => "$client_port";
157
158	sub send_message { ...
159
160	Now that was a lot of new stuff. In order to ask the server and receive an
161	answer we need to have a I<port> where we can receive the answer.
162	This is what C<create_port> will do for us, it just creates a new local
163	port and returns us an object (that will btw. stringify to the I<port id>),
164	that we can use to receive messages.
165
166	Next thing is the C<$server_node>. In order to refer to another node we need
167	some kind of string to reference it. The I<noderef> is basically a comma
168	seperated list of C<host:port> pairs. We assume in this tutorial that the
169	server runs on your localhost at port 1299, this gives us the noderef
170	C<localhost:1299>.
171
172	Now you might ask what the C<#> at the end in C<$server_node> the above
173	example is about. Well, what I didn't tell you yet is that each I<node> has a
174	default I<port> to receive messages. The default port is the empty string
175	C<"">. The I<default port> of a I<node> also provides some special services for
176	us, for example resolving a well known port to a I<port id>.
177
178	Now to this line:
179
180	snd $server_node, wkp => "chatter", "$client_port", "resolved";
181
182	We send a message with first element being C<wkp> (standing for 'well known
183	port'). Then the well known port name that we want to resolve to a I<port id>:
184	C<chatter>. And in order for the server node to be able to send us back the
185	resolved I<port id> we have to tell it where to send the result message: The
186	result message will have as it's first argument the string C<resolved> and
187	will be sent to C<$client_port> (the I<port id> of our own just created
188	port).
189
190	Next comes the receiver for this C<wkp> request.
191
192	$client_port->rcv (resolved => sub {
193	my ($client_port, $type, $chatter_port_id) = @_;
194	...
195	1
196	});
197
198	This sets up a receiver on our own port for the result message with the first
199	argument being the string C<resolved>. Receivers can match the contents of
200	the messages before actually 'sending' it to the given callback.
201
202	B<Please note> that the given callback has to return either a true or a false
203	value for indicating whether it is B<done> (true value) or still wants to
204	B<continue> (false value) receiving messages.
205
206	In this case we tell the C<$client_port> to look into the received messages and
207	look for the string C<resolved> in the first element of the message. If it is
208	found, the given callback will be called with the C<$client_port> as first
209	argument, and the message as the remaining arguments.
210
211	We name the first element of the message C<$type> in this case. It's a common
212	idiom to code the 'type' of a message into it's first element, this allows for
213	simple matching.
214
215	The result message will contain the I<port id> of the well known port C<chatter>
216	as next element, and will be put in C<$chatter_port_id>.
217
218	Next we just assign C<$server_port> and return a 1 (a true value)
219	from the callback. It indicates that we are done and don't want to receive
220	further C<resolved> messages with this callback.
221
222	Now we continue to the rest of the client by calling C<send> on
223	C<$resolved_cv>.
224
225	First new step after this is setting up the chat message receiver callback.
226
227	$client_port->rcv (message => sub {
228	my ($client_port, $type, $msg) = @_;
229
230	print "chat> $msg\n";
231	0
232	});
233
234	We assume that all messages that are broadcast to all clients by the server
235	will contain the string C<message> as first element, and the actual message as
236	second element. The callback returns a false value this time, to indicate that
237	it wants to continue receiving messages.
238
239	Last but not least we actually tell the server to send us
240	the new chat messages from other clients. We do so by sending the
241	message type C<join> followed by our own I<port id>.
242
243	# send the server a 'join' message:
244	snd $server_port, join => "$client_port";
245
246	Then the server knows where to send all the new messages to.
247
248	=head2 The Finished Client
249
250	This is the complete client script:
251
252	#!perl
253
254	use AnyEvent;
255	use AnyEvent::Handle;
256	use AnyEvent::MP;
257
258	my $resolved_cv = AnyEvent->condvar;
259
260	my $client_port = create_port;
261
262	my $server_node = "localhost:1299#";
263
264	snd $server_node, wkp => "chatter", "$client_port", "resolved";
265
266	my $server_port;
267
268	# setup a receiver callback for the 'resolved' message:
269	$client_port->rcv (resolved => sub {
270	my ($client_port, $type, $chatter_port_id) = @_;
271
272	print "Resolved the server port 'chatter' to $chatter_port_id\n";
273	$server_port = $chatter_port_id;
274
275	$resolved_cv->send;
276	1
277	});
278
279	# lets block the client until we resolved the server port.
280	$resolved_cv->recv;
281
282	# now setup another receiver callback for the chat messages:
283	$client_port->rcv (message => sub {
284	my ($client_port, $type, $msg) = @_;
285
286	print "chat> $msg\n";
287	0
288	});
289
290	# send the server a 'join' message:
291	snd $server_port, join => "$client_port";
292
293	sub send_message {
294	my ($msg) = @_;
295
296	snd $server_port, message => $msg;
297	}
298
299	# make an AnyEvent condition variable for the 'quit' condition
300	# (when we want to exit the client).
301	my $quit_cv = AnyEvent->condvar;
302
303	my $stdin_hdl = AnyEvent::Handle->new (
304	fh => \*STDIN,
305	on_read => sub {
306	my ($hdl) = @_;
307
308	$hdl->push_read (line => sub {
309	my ($hdl, $line) = @_;
310
311	if ($line =~ /^\/quit/) { # /quit will end the client
312	$quit_cv->send;
313
314	} else {
315	send_message ($line);
316	}
317	});
318	}
319	);
320
321	$quit_cv->recv;
322
323	=head2 The Server
324
325	Ok, now finally to the server. What do we need? Well, we need to setup
326	the well known port C<chatter> where all clients send their messages to.
327
328	Up and into code right now:
329
330	#!perl
331
332	use AnyEvent;
333	use AnyEvent::MP;
334
335	become_public "localhost:1299";
336
337	my $chatter_port = create_port;
338	$chatter_port->register ("chatter");
339
340	my %client_ports;
341
342	$chatter_port->rcv (join => sub {
343	my ($chatter_port, $type, $client_port) = @_;
344
345	print "got new client port: $client_port\n";
346
347	$client_ports{$client_port} = 1;
348	0
349	});
350
351	$chatter_port->rcv (message => sub {
352	my ($chatter_port, $type, $msg) = @_;
353
354	print "message> $msg\n";
355
356	snd $_, message => $msg for keys %client_ports;
357	0
358	});
359
360	AnyEvent->condvar->recv;
361
362	This is all. Looks much easier, doesn't it? I'll explain it only shortly, as
363	we had the discussion of the C<rcv> method in the client part of this tutorial
364	above.
365
366	First this:
367
368	become_public "localhost:1299";
369
370	This will tell our I<node> to become a I<public> node, which means that it can
371	be contacted via TCP. The first argument should be the I<noderef> the server
372	wants to be reachable at. In this case it's the TCP port 1299 on localhost.
373
374	Next we bascially setup two receivers, one for the C<join> messages and
375	another one for the actual messages of type C<messsage>.
376
377	In the C<join> message we get the client's port, which we just remember in the
378	C<%client_ports> hash. In the receiver for the message type C<message> we will
379	just iterate through all known C<%client_ports> and relay the message to them.
380
381	And thats it.
382
383	=head2 The Remaining Problems
384
385	The shown implementation still has some bugs. For instance: How does the
386	server know that the client isn't there anymore, and can cleanup the
387	C<%client_ports> hash? And also the chat messages have no originator,
388	so we don't know who actually sent the message (which would be quite useful
389	for human-to-human interaction: to know who the other one is :).
390
391	But aside from these issues I hope this tutorial got you the swing of
392	L<AnyEvent::MP> and explained some common idioms.
393
394	=head1 SEE ALSO
395
396	L<AnyEvent>
397
398	L<AnyEvent::Handle>
399
400	L<AnyEvent::MP>
401
402	=head1 AUTHOR
403
404	Robin Redeker <elmex@ta-sa.org>
405