ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-MP/MP/Intro.pod
Revision: 1.20
Committed: Fri Aug 28 15:44:39 2009 UTC (14 years, 9 months ago) by elmex
Branch: MAIN
Changes since 1.19: +38 -407 lines
Log Message:
finished intro of second example.

File Contents

# User Rev Content
1 root 1.4 =head1 Message Passing for the Non-Blocked Mind
2 elmex 1.1
3 root 1.8 =head1 Introduction and Terminology
4 elmex 1.1
5 root 1.4 This is a tutorial about how to get the swing of the new L<AnyEvent::MP>
6 root 1.14 module, which allows us to transparently pass messages to our own process
7 root 1.8 and to other processes on another or the same host.
8 elmex 1.1
9 root 1.15 What kind of messages? Well, basically a message here means a list of Perl
10     strings, numbers, hashes and arrays, anything that can be expressed as a
11     L<JSON> text (as JSON is used by default in the protocol).
12 elmex 1.1
13 elmex 1.17 It's custom in L<AnyEvent::MP> to have a string which describes the type of the
14     message as first element (this is called a I<tag> in L<AnyEvent::MP>), as some
15     API functions (C<rcv>) support matching it directly. So supposedly you want to
16     send a ping message with your current time to something, this is how such a
17     message might look like (in Perl syntax):
18 elmex 1.1
19 elmex 1.17 ['ping', 1251381636]
20 elmex 1.7
21 elmex 1.17 And next you might ask: between which entities are those messages being
22     I<passed>? They are I<passed> between I<ports>. I<ports> are just sources and
23     destinations for messages. How do these ports relate to things you know? Well,
24     each I<port> belongs to a I<node>, and a I<node> is just the UNIX process that
25     runs your L<AnyEvent::MP> application.
26    
27     Each I<node> is distinguished from other I<nodes> running on the same host or
28     multiple hosts in a network by it's I<node ID>. A I<node ID> can be manually
29     assigned or L<AnyEvent::MP> will assign one it self for you.
30    
31 elmex 1.18 So, you might want to visualize it like this (setup is two nodes (more are of
32     course possible): Node C<A> (in UNIX process 7066) with ports C<ABC> and C<DEF>
33     and C<B> (in UNIX process 8321) with ports C<FOO> and C<BAR>).
34 elmex 1.17
35    
36     |- PID: 7066 -| |- PID: 8321 -|
37     | | | |
38     | Node ID: A | | Node ID: B |
39     | | | |
40     | Port ABC =|= <----\ /-----> =|= Port FOO |
41     | | X | |
42     | Port DEF =|= <----/ \-----> =|= Port BAR |
43     | | | |
44     |-------------| |-------------|
45    
46     The strings for the ports here are just for illustrative purposes. Even if in
47     reality I<ports> in L<AnyEvent::MP> are also identified by strings they can't
48     be choosen manually and are assigned randomly. These I<port ids> should also
49 elmex 1.18 not be used directly for other purposes than referring to an endpoint for
50 elmex 1.17 messages.
51    
52     The next sections will explain the API of L<AnyEvent::MP>. First the API is
53 elmex 1.18 laid out by simple examples. Later some more complex idioms are introduced,
54     which are maybe useful to solve some real world purposes.
55 elmex 1.17
56     # In this tutorial I'll show you how to write a simple chat server based on
57     # L<AnyEvent::MP>. This example is used because it nicely shows how to organise a
58     # simple application, but keep in mind that every node trusts any other, so this
59     # chat cannot be used to implement a real public chat server and client system,
60     # but it can be used to implement a distributed chat server for example.
61 root 1.8
62 elmex 1.16 =head1 Passing Your First Message
63    
64     As start lets have a look at the messaging API. The next example is just a
65     demo to show the basic elements of message passing with L<AnyEvent::MP>.
66     It shout just print: "Ending with: 123". So here the code:
67    
68     use AnyEvent;
69     use AnyEvent::MP;
70    
71     my $end_cv = AnyEvent->condvar;
72    
73     my $port = port;
74    
75     rcv $port, test => sub {
76     my ($data) = @_;
77     $end_cv->send ($data);
78     };
79    
80     snd $port, test => 123;
81    
82     print "Ending with: " . $end_cv->recv . "\n";
83    
84 elmex 1.17 It already contains most functions of the essential L<AnyEvent::MP> API.
85    
86     First there is the C<port> function which will create a I<port> and will return
87     it's I<port id>.
88    
89     That I<port id> can be used to send and receive messages. That I<port id> is a
90     simple string and can be safely passed to other I<nodes> in the network to
91     refer to that specific port (usually used for RPC, where you need to
92     tell the other end which I<port> to send the reply to).
93    
94     Next function is C<rcv>:
95 elmex 1.16
96 elmex 1.17 rcv $port, test => sub { ... };
97 elmex 1.16
98 elmex 1.17 It sets up a receiver callback on a specific I<port> which needs to be
99     specified as the first argument. The next argument, in this example C<test>, is
100     a I<tag> match. This means that whenever a message, with the first element
101     being the string C<tag>, is received the callback is called with the remaining
102     parts of that message.
103    
104     Messages can be send with the C<snd> function, which looks like this in the
105     example above:
106    
107     snd $port, test => 123;
108    
109     This will send the message C<['test', 123]> to the I<port> with the I<port id>
110     in C<$port>. The receiver got a I<tag> match on C<test> and will call the
111     callback with the first argument being the number C<123>.
112    
113     That callback then just passes that number on to the I<condition variable>
114     C<$end_cv> which will then pass the value to the print. But I<condition
115     variables> are out of the scope of this tutorial. So please consult the
116     L<AnyEvent::Intro> about them.
117    
118     But passing messages inside one process is boring, but before we can continue
119     and take the next step to interprocess message passing we first have to make
120     sure some things have been setup.
121    
122     =head1 System Requirements and System Setup
123    
124     Before we can start with real IPC we have to make sure some things work on your
125     system.
126    
127     First we have to setup a I<shared secret>: for two L<AnyEvent::MP> I<nodes> to
128     be able to communicate with each other and authenticate each other it is
129     necessary to setup the same I<shared secret> for both of them (or use TLS
130     certificates).
131    
132     The easiest way is to set this up is to use the F<aemp> utility:
133    
134     aemp gensecret
135    
136     This creates a F<$HOME/.perl-anyevent-mp> config file and generates a random
137     shared secret. You can copy this file to any other system and then communicate
138     over the network (via TCP) with it. You can also select your own shared secret
139     (F<aemp setsecret>) and for increased security requirements you can even create
140     a TLS certificate (F<aemp gencert>), causing connections to not just be
141     authenticated, but also to be encrypted.
142    
143     Connections will only be successful when the I<nodes> that want to connect to
144     each other have the same I<shared secret> (or successfully verify the TLS
145     certificate of the other side).
146    
147     B<If something does not work as expected, and for example tcpdump shows
148     that the connections are closed almost immediately, you should make sure
149     that F<~/.perl-anyevent-mp> is the same on all hosts/user accounts that
150     you try to connect with each other!>
151 elmex 1.16
152 elmex 1.18 Thats all for now, there is more fiddling around with the C<aemp> utility
153     later.
154    
155     =head1 Passing Messages Between Processes
156    
157     =head2 The Receiver
158    
159     Lets split the previous example up into two small programs. First the
160     receiver application:
161    
162     #!/opt/perl/bin/perl
163     use AnyEvent;
164     use AnyEvent::MP;
165     use AnyEvent::MP::Global;
166    
167     initialise_node "eg_simple_receiver";
168    
169     my $port = port;
170    
171     AnyEvent::MP::Global::register $port, "eg_receivers";
172    
173     rcv $port, test => sub {
174     my ($data, $reply_port) = @_;
175    
176     print "Received data: " . $data . "\n";
177     };
178    
179     AnyEvent->condvar->recv;
180    
181     =head3 AnyEvent::MP::Global
182    
183     Now, that wasn't too bad, was it? Ok, lets step through the new functions
184     and modules that have been used. For starters there is now an additional
185     module loaded: L<AnyEvent::MP::Global>.
186    
187     That module provides us with a I<global registry>, which lets us share data
188     among all I<nodes> in a network. Why do we need it you might ask?
189    
190 elmex 1.20 The thing is, that the I<port ids> are just random strings, assigned by
191     L<AnyEvent::MP>. We can't know those I<port ids> in advance, so we don't know
192     which I<port id> to send messages to if the message is to be passed between
193     I<nodes> (or UNIX processes). To find the right I<port> of another I<node> in
194     the network we will need to communicate that somehow to the sender. And
195     exactly that is what L<AnyEvent::MP::Global> provides.
196 elmex 1.18
197     =head3 initialise_node And The Network
198    
199     Now, lets have a look at the next new thing, the C<initialise_node>:
200    
201     initialise_node "eg_simple_receiver";
202    
203     Before we are able to send messages to other nodes we have to initialise
204     ourself. The first argument, the string C<"eg_simple_receiver">, is called the
205     I<profile> of this node. A profile holds some information about the application
206     that is going to be a node in an L<AnyEvent::MP> network.
207    
208     Most importantly the profile allows you to set the I<node id> that your
209     application will use. You can also set I<binds> in the profile, meaning that
210     you can define TCP ports that the application will listen on for incoming
211     connections from other nodes of the network.
212    
213     Next you can configure I<seeds> in profile. A I<seed> is just a TCP endpoint
214     which tells the application where to find other nodes of it's network. To
215     explain this a bit more detailed we have to look at the topology of an
216     L<AnyEvent::MP> network. The topology is called a I<fully connected mesh>, here
217     an example with 4 nodes:
218    
219     N1--N2
220     | \/ |
221     | /\ |
222     N3--N4
223    
224     Now imagine another I<node> C<N5>. wants to connect itself to that network:
225    
226     N1--N2
227     | \/ | N5
228     | /\ |
229     N3--N4
230    
231     The new node needs to know the I<binds> of all of those 4 already connected
232     nodes. And exactly this is what the I<seeds> are for. Now lets assume that
233     the new node C<N5> has as I<seed> the TCP endpoint of the node C<N2>.
234     It then connects to C<N2>:
235    
236     N1--N2____
237     | \/ | N5
238     | /\ |
239     N3--N4
240    
241     C<N2> then tells C<N5> the I<binds> of the other nodes it is connected to,
242     and C<N5> builds up the rest of the connections:
243    
244     /--------\
245     N1--N2____|
246     | \/ | N5
247     | /\ | /|
248     N3--N4--- |
249     \________/
250    
251     Finished. C<N5> is now happily connected to the rest of the network.
252    
253 elmex 1.19 =head3 Setting Up The Profiles
254    
255     Ok, so much to the profile. Now lets setup the C<eg_simple_receiver> I<profile>
256     for later. For the receiver we just give the receiver a I<bind>:
257    
258     aemp profile eg_simple_receiver setbinds localhost:12266
259    
260     And while we are at it, just setup the I<profile> for the sender in the second
261     part of this example too. We will call the sender I<profile>
262     C<eg_simple_sender>. For the sender we will just setup a I<seed> to the
263     receiver:
264    
265     aemp profile eg_simple_sender setseeds localhost:12266
266    
267     You might wonder why we don't setup a I<bind> here. Well, there can be
268     exceptions to the I<fully> in the I<fully connected mesh> in L<AnyEvent::MP>.
269     If you don't configure a I<bind> for a node's profile it won't bind itself
270     somewhere. These kinds of I<nodes> will not be able to send messages to other
271     I<nodes> that also didn't I<bind> them self to some TCP address. For this
272     example, as well as some cases in the real world, we can live with this
273     limitation.
274    
275     =head3 Registering The Receiver
276    
277     Ok, where were we. We now discussed the basic purpose of L<AnyEvent::MP::Global>
278     and initialise_node with it's relations to profiles. We also setup our profiles
279     for later use and now have to continue talking about the receiver example.
280    
281     Lets look at the next undiscussed line(s) of code:
282    
283     my $port = port;
284     AnyEvent::MP::Global::register $port, "eg_receivers";
285    
286     The C<port> function already has been discussed. It just creates a new I<port>
287     and gives us the I<port id>. Now to the C<register> function of
288     L<AnyEvent::MP::Global>: The first argument is a I<port id> that we want to add
289     to a I<global group>, and it's second argument is the name of that I<global
290     group>.
291    
292     You can choose that name of such a I<global group> freely, and it's purpose is
293     to store a set of I<port ids>. That set is made available throughout the whole
294     L<AnyEvent::MP> network, so that each node can see which ports belong to that
295     group.
296    
297     The sender will later look for the ports in that I<global group> and send
298     messages to them.
299    
300     Last step in the example is to setup a receiver callback for those messages
301     like we have discussed in the first example. We again match for the I<tag>
302     C<test>. The difference is just that we don't end the application after
303     receiving the first message. We just infinitely continue to look out for new
304     messages.
305    
306 elmex 1.20 =head2 The Sender
307 root 1.8
308 elmex 1.20 Ok, now lets take a look at the sender:
309 root 1.4
310 elmex 1.20 #!/opt/perl/bin/perl
311 elmex 1.1 use AnyEvent;
312     use AnyEvent::MP;
313 elmex 1.20 use AnyEvent::MP::Global;
314 elmex 1.1
315 elmex 1.20 initialise_node "eg_simple_sender";
316 elmex 1.1
317 elmex 1.20 my $find_timer =
318     AnyEvent->timer (after => 0, interval => 1, cb => sub {
319     my $ports = AnyEvent::MP::Global::find "eg_receivers"
320     or return;
321    
322     snd $_, test => time
323     for @$ports;
324     });
325 elmex 1.1
326     AnyEvent->condvar->recv;
327    
328 elmex 1.20 It's even less code. The C<initialise_node> is known now from the receiver
329     above. As discussed in the section where we setup the profiles we configure
330     this application to use the I<profile> C<eg_simple_sender>.
331 root 1.10
332 elmex 1.20 Next we setup a timer that repeatedly calls this chunk of code:
333 elmex 1.1
334 elmex 1.20 my $ports = AnyEvent::MP::Global::find "eg_receivers"
335     or return;
336 elmex 1.2
337 elmex 1.20 snd $_, test => time
338     for @$ports;
339 elmex 1.1
340 elmex 1.20 The new function here is the C<find> function of L<AnyEvent::MP::Global>. It
341     searches in the I<global group> named C<eg_receivers> for ports. If none are
342     found C<undef> is returned and we wait for the next time the timer fires.
343 elmex 1.1
344 elmex 1.20 In case the receiver application has been connected and the newly added port by
345     the receiver has propagated to the sender C<find> returns an array reference
346     that contains the I<port id> of the receiver I<port(s)>.
347 elmex 1.1
348 elmex 1.20 We then just send to every I<port> in the I<global group> a message consisting
349     of the I<tag> C<test> and the current time in form of a UNIX timestamp.
350 elmex 1.7
351 elmex 1.20 And thats all.
352 elmex 1.7
353 elmex 1.1 =head1 SEE ALSO
354    
355     L<AnyEvent>
356    
357     L<AnyEvent::Handle>
358    
359     L<AnyEvent::MP>
360    
361 elmex 1.20 L<AnyEvent::MP::Global>
362    
363 elmex 1.1 =head1 AUTHOR
364    
365     Robin Redeker <elmex@ta-sa.org>
366 root 1.4