[ViewVC] Diff of: cvs/AnyEvent/lib/AnyEvent/Intro.pod

Comparing AnyEvent/lib/AnyEvent/Intro.pod (file contents):
Revision 1.10 by root, Mon Jun 2 06:04:08 2008 UTC vs.
Revision 1.11 by root, Mon Jun 2 09:10:38 2008 UTC

…		…
52		52
53	L<AnyEvent> solves this dilemma, by B<not> forcing module authors to either	53	L<AnyEvent> solves this dilemma, by B<not> forcing module authors to either
54		54
55	=over 4	55	=over 4
56		56
57	=item write their own event loop (because guarantees to offer one	57	=item - write their own event loop (because guarantees to offer one
58	everywhere - even on windows).	58	everywhere - even on windows).
59		59
60	=item choose one fixed event loop (because AnyEvent works with all	60	=item - choose one fixed event loop (because AnyEvent works with all
61	important event loops available for Perl, and adding others is trivial).	61	important event loops available for Perl, and adding others is trivial).
62		62
63	=back	63	=back
64		64
65	If the module author uses L<AnyEvent> for all his event needs (IO events,	65	If the module author uses L<AnyEvent> for all his event needs (IO events,
…		…
442	successful? That unsuccessful TCP connects might never be reported back	442	successful? That unsuccessful TCP connects might never be reported back
443	to your program? That C<WSAEINPROGRESS> means your C<connect> call was	443	to your program? That C<WSAEINPROGRESS> means your C<connect> call was
444	ignored instead of being in progress? AnyEvent::Socket works around all of	444	ignored instead of being in progress? AnyEvent::Socket works around all of
445	these Windows/Perl bugs for you).	445	these Windows/Perl bugs for you).
446		446
447	=head2 First experiments with non-blocking connects: a parallel finger	447	=head2 Implementing a parallel finger client with non-blocking connects
448	client.
449		448
450	The finger protocol is one of the simplest protocols in use on the	449	The finger protocol is one of the simplest protocols in use on the
451	internet. Or in use in the past, as almost nobody uses it anymore.	450	internet. Or in use in the past, as almost nobody uses it anymore.
452		451
453	It works by connecting to the finger port on another host, writing a	452	It works by connecting to the finger port on another host, writing a
…		…
466	Now on the web:	465	Now on the web:
467	[...]	466	[...]
468		467
469	Connection closed by foreign host.	468	Connection closed by foreign host.
470		469
471	Yeah, I<was> used indeed, but at least the finger daemon still works, so	470	"Now on the web..." yeah, I<was> used indeed, but at least the finger
472	let's write a little AnyEvent function that makes a finger request:	471	daemon still works, so let's write a little AnyEvent function that makes a
		472	finger request:
473		473
474	use AnyEvent;	474	use AnyEvent;
475	use AnyEvent::Socket;	475	use AnyEvent::Socket;
476		476
477	sub finger($$) {	477	sub finger($$) {
…		…
509		509
510	# pass $cv to the caller	510	# pass $cv to the caller
511	$cv	511	$cv
512	}	512	}
513		513
514	That's a mouthful! Let's dissect this function a bit, first the overall function:	514	That's a mouthful! Let's dissect this function a bit, first the overall
		515	function and execution flow:
515		516
516	sub finger($$) {	517	sub finger($$) {
517	my ($user, $host) = @_;	518	my ($user, $host) = @_;
518		519
519	# use a condvar to return results	520	# use a condvar to return results
…		…
525	};	526	};
526		527
527	$cv	528	$cv
528	}	529	}
529		530
530	This isn't too complicated, just a function with two parameters, which	531	This isn't too complicated, just a function with two parameters, that
531	creates a condition variable, returns it, and while it does that,	532	creates a condition variable, returns it, and while it does that,
532	initiates a TCP connect to C<$host>. The condition variable	533	initiates a TCP connect to C<$host>. The condition variable will be used
533	will be used by the caller to receive the finger response.	534	by the caller to receive the finger response, but one could equally well
		535	pass a third argument, a callback, to the function.
534		536
535	Since we are event-based programmers, we do not wait for the connect to	537	Since we are programming event'ish, we do not wait for the connect to
536	finish - it could block your program for a minute or longer! Instead,	538	finish - it could block the program for a minute or longer!
		539
537	we pass the callback it should invoke when the connect is done to	540	Instead, we pass the callback it should invoke when the connect is done to
538	C<tcp_connect>. If it is successful, our callback gets called with the	541	C<tcp_connect>. If it is successful, that callback gets called with the
539	socket handle as first argument, otherwise, nothing will be passed to our	542	socket handle as first argument, otherwise, nothing will be passed to our
540	callback.	543	callback. The important point is that it will always be called as soon as
		544	the outcome of the TCP connect is known.
541		545
		546	This style of programming is also called "continuation style": the
		547	"continuation" is simply the way the program continues - normally, a
		548	program continues at the next line after some statement (the exception
		549	is loops or things like C<return>). When we are interested in events,
		550	however, we instead specify the "continuation" of our program by passing a
		551	closure, which makes that closure the "continuation" of the program. The
		552	C<tcp_connect> call is like saying "return now, and when the connection is
		553	established or it failed, continue there".
		554
542	Let's look at our callback in more detail:	555	Now let's look at the callback/closure in more detail:
543		556
544	# the callback gets the socket handle - or nothing	557	# the callback receives the socket handle - or nothing
545	my ($fh) = @_	558	my ($fh) = @_
546	or return $cv->send;	559	or return $cv->send;
547		560
548	The first thing the callback does is indeed save the socket handle in	561	The first thing the callback does is indeed save the socket handle in
549	C<$fh>. When there was an error (no arguments), then our instinct as	562	C<$fh>. When there was an error (no arguments), then our instinct as
550	expert Perl programmers would tell us to die:	563	expert Perl programmers would tell us to C<die>:
551		564
552	my ($fh) = @_	565	my ($fh) = @_
553	or die "$host: $!";	566	or die "$host: $!";
554		567
555	While this would give good feedback to the user, our program would	568	While this would give good feedback to the user (if he happens to watch
556	probably freeze here, as we never report the results to anybody, certainly	569	standard error), our program would probably stop working here, as we never
557	not the caller of our C<finger> function!	570	report the results to anybody, certainly not the caller of our C<finger>
		571	function, and most event loops continue even after a C<die>!
558		572
559	This is why we instead return, but also call C<< $cv->send >> without any	573	This is why we instead C<return>, but also call C<< $cv->send >> without
560	arguments to signal to our consumer that something bad has happened. The	574	any arguments to signal to the condvar consumer that something bad has
561	return value of C<< $cv->send >> is irrelevant, as is the return value of	575	happened. The return value of C<< $cv->send >> is irrelevant, as is the
562	our callback. The return statement is simply used for the side effect of,	576	return value of our callback. The return statement is simply used for the
563	well, returning immediately from the callback.	577	side effect of, well, returning immediately from the callback. Checking
		578	for errors and handling them this way is very common, which is why this
		579	compact idiom is so handy.
564		580
565	As the next step in the finger protocol, we send the username to the	581	As the next step in the finger protocol, we send the username to the
566	finger daemon on the other side of our connection:	582	finger daemon on the other side of our connection:
567		583
568	syswrite $fh, "$user\015\012";	584	syswrite $fh, "$user\015\012";
569		585
570	Note that this isn't 100% clean - the socket could, for whatever reasons,	586	Note that this isn't 100% clean socket programming - the socket could,
571	not accept our data. When writing a small amount of data like in this	587	for whatever reasons, not accept our data. When writing a small amount
572	example it doesn't matter, but for real-world cases you might need to	588	of data like in this example it doesn't matter, as a socket buffer is
573	implement some kind of write buffering - or use L<AnyEvent::Handle>, which	589	almost always big enough for a mere "username", but for real-world
574	handles these matters for you.	590	cases you might need to implement some kind of write buffering - or use
		591	L<AnyEvent::Handle>, which handles these matters for you, as shown in the
		592	next section.
575		593
576	What we do have to do is to implement our own read buffer - the response	594	What we I<do> have to do is to implement our own read buffer - the response
577	data could arrive late or in multiple chunks, and we cannot just wait for	595	data could arrive late or in multiple chunks, and we cannot just wait for
578	it (event-based programming, you know?).	596	it (event-based programming, you know?).
579		597
580	To do that, we register a read watcher on the socket which waits for data:	598	To do that, we register a read watcher on the socket which waits for data:
581		599
…		…
591	To avoid that, we C<undef>ine the variable in the watcher callback. This	609	To avoid that, we C<undef>ine the variable in the watcher callback. This
592	means that, when the C<tcp_connect> callback returns, that perl thinks	610	means that, when the C<tcp_connect> callback returns, that perl thinks
593	(quite correctly) that the read watcher is still in use - namely in the	611	(quite correctly) that the read watcher is still in use - namely in the
594	callback.	612	callback.
595		613
		614	The trick, however, is that instead of:
		615
		616	my $read_watcher = AnyEvent->io (...
		617
		618	The program does:
		619
		620	my $read_watcher; $read_watcher = AnyEvent->io (...
		621
		622	The reason for this is a quirk in the way Perl works: variable names
		623	declared with C<my> are only visible in the I<next> statement. If the
		624	whole C<< AnyEvent->io >> call, including the callback, would be done in
		625	a single statement, the callback could not refer to the C<$read_watcher>
		626	variable to undefine it, so it is done in two statements.
		627
		628	Whether you'd want to format it like this is of course a matter of style,
		629	this way emphasizes that the declaration and assignment really are one
		630	logical statement.
		631
596	The callback itself calls C<sysread> for as many times as necessary, until	632	The callback itself calls C<sysread> for as many times as necessary, until
597	C<sysread> returns an error or end-of-file:	633	C<sysread> returns either an error or end-of-file:
598		634
599	cb => sub {	635	cb => sub {
600	my $len = sysread $fh, $response, 1024, length $response;	636	my $len = sysread $fh, $response, 1024, length $response;
601		637
602	if ($len <= 0) {	638	if ($len <= 0) {
603		639
604	Note that C<sysread> has the ability to append data it reads to a scalar,	640	Note that C<sysread> has the ability to append data it reads to a scalar,
605	which is what we make good use of in this example.	641	by specifying an offset, which is what we make good use of in this
		642	example.
606		643
607	When C<sysread> indicates we are done, the callback C<undef>ines	644	When C<sysread> indicates we are done, the callback C<undef>ines
608	the watcher and then C<send>'s the response data to the condition	645	the watcher and then C<send>'s the response data to the condition
609	variable. All this has the following effects:	646	variable. All this has the following effects:
610		647
…		…
631	print "trouble ticket #1736:\n", $f2->recv, "\n";	668	print "trouble ticket #1736:\n", $f2->recv, "\n";
632	print "john carmacks finger file: ", $f3->recv, "\n";	669	print "john carmacks finger file: ", $f3->recv, "\n";
633		670
634	It doesn't look like it, but in fact all three requests run in	671	It doesn't look like it, but in fact all three requests run in
635	parallel. The code waits for the first finger request to finish first, but	672	parallel. The code waits for the first finger request to finish first, but
636	that doesn't keep it from executing in parallel, because when the first	673	that doesn't keep it from executing them parallel: when the first C<recv>
637	C<recv> call sees that the data isn't ready yet, it serves events for all	674	call sees that the data isn't ready yet, it serves events for all three
638	three requests automatically.	675	requests automatically, until the first request has finished.
		676
		677	The second C<recv> call might either find the data is already there, or it
		678	will continue handling events until that is the case, and so on.
639		679
640	By taking advantage of network latencies, which allows us to serve other	680	By taking advantage of network latencies, which allows us to serve other
641	requests and events while we wait for an event on one socket, the overall	681	requests and events while we wait for an event on one socket, the overall
642	time to do these three requests will be greatly reduces, typically all	682	time to do these three requests will be greatly reduced, typically all
643	three are done in the same time as the slowest of them would use.	683	three are done in the same time as the slowest of them would need to finish.
644		684
645	By the way, you do not actually have to wait in the C<recv> method on an	685	By the way, you do not actually have to wait in the C<recv> method on an
646	AnyEvent condition variable, you can also register a callback:	686	AnyEvent condition variable - after all, waiting is evil - you can also
		687	register a callback:
647		688
648	$cv->cb (sub {	689	$cv->cb (sub {
649	my $response = shift->recv;	690	my $response = shift->recv;
650	# ...	691	# ...
651	});	692	});
…		…
656	response:	697	response:
657		698
658	sub finger($$$) {	699	sub finger($$$) {
659	my ($user, $host, $cb) = @_;	700	my ($user, $host, $cb) = @_;
660		701
661	What you use is a matter of taste - if you expect your function to be	702	How you implement it is a matter of taste - if you expect your function to
662	used mainly in an event-based program you would normally prefer to pass a	703	be used mainly in an event-based program you would normally prefer to pass
663	callback directly.	704	a callback directly. If you write a module and expect your users to use
		705	it "synchronously" often (for example, a simple http-get script would not
		706	really care much for events), then you would use a condition variable and
		707	tell them "simply ->recv the data".
664		708
665	=head3 Criticism and fix	709	=head3 Problems with the implementation and how to fix them
666		710
667	To make this example more real-world-ready, we would not only implement	711	To make this example more real-world-ready, we would not only implement
668	some write buffering (for the paranoid), but we would also have to handle	712	some write buffering (for the paranoid), but we would also have to handle
669	timeouts and maybe protocol errors.	713	timeouts and maybe protocol errors.
670		714
671	This quickly gets unwieldy, which is why we introduce L<AnyEvent::Handle>	715	Doing this quickly gets unwieldy, which is why we introduce
672	in the next section, which takes care of all these details for us.	716	L<AnyEvent::Handle> in the next section, which takes care of all these
		717	details for you and let's you concentrate on the actual protocol.
673		718
674		719
675	=head2 First experiments with AnyEvent::Handle	720	=head2 Implementing simple HTTP and HTTPS GET requests with AnyEvent::Handle
		721
		722	The L<AnyEvent::Handle> module has been hyped quite a bit so far, so let's
		723	see what it really offers.
		724
		725	As finger is such a simple protocol, let's try something slightly more
		726	complicated: HTTP/1.0.
		727
		728	An HTTP GET request works by sending a single request line that indicates
		729	what you want the server to do and the URI you want to act it on, followed
		730	by as many "header" lines (C<Header: data>, same as e-mail headers) as
		731	required for the request, ended by an empty line.
		732
		733	The response is formatted very similarly, first a line with the response
		734	status, then again as many header lines as required, then an empty line,
		735	followed by any data that the server might send.
		736
		737	Again, let's try it out with C<telnet> (I condensed the output a bit - if
		738	you want to see the full response, do it yourself).
		739
		740	# telnet www.google.com 80
		741	Trying 209.85.135.99...
		742	Connected to www.google.com (209.85.135.99).
		743	Escape character is '^]'.
		744	GET /test HTTP/1.0
		745
		746	HTTP/1.0 404 Not Found
		747	Date: Mon, 02 Jun 2008 07:05:54 GMT
		748	Content-Type: text/html; charset=UTF-8
		749
		750	<html><head>
		751	[...]
		752	Connection closed by foreign host.
		753
		754	The C<GET ...> and the empty line were entered manually, the rest of the
		755	telnet output is google's response, in which case a C<404 not found> one.
		756
		757	So, here is how you would do it with C<AnyEvent::Handle>:
		758
		759	###TODO
		760
		761	And now let's go through it step by step. First, as usual, the overall
		762	C<http_get> function structure:
		763
		764	sub http_get {
		765	my ($host, $uri, $cb) = @_;
		766
		767	tcp_connect $host, "http", sub {
		768	...
		769	};
		770	}
		771
		772	Unlike in the finger example, this time the caller has to pass a callback
		773	to C<http_get>. Also, instead of passing a URL as one would expect, the
		774	caller has to provide the hostname and URI - normally you would use the
		775	C<URI> module to parse a URL and separate it into those parts, but that is
		776	left to the inspired reader :)
		777
		778	Since everything else is left to the caller, all C<http_get> does it to
		779	initiate the connection with C<tcp_connect> and leave everything else to
		780	it's callback.
		781
		782	The first thing the callback does is check for connection errors and
		783	declare some variables:
		784
		785	my ($fh) = @_
		786	or $cb->("HTTP/1.0 500 $!");
		787
		788	my ($response, $header, $body);
		789
		790	Instead of having an extra mechanism to signal errors, connection errors
		791	are signalled by crafting a special "response status line", like this:
		792
		793	HTTP/1.0 500 Connection refused
		794
		795	This means the caller cannot distinguish (easily) between
		796	locally-generated errors and server errors, but it simplifies error
		797	handling for the caller a lot.
		798
		799	The next step finally involves L<AnyEvent::Handle>, namely it creates the
		800	handle object:
		801
		802	my $handle; $handle = new AnyEvent::Handle
		803	fh => $fh,
		804	on_error => sub {
		805	undef $handle;
		806	$cb->("HTTP/1.0 500 $!");
		807	},
		808	on_eof => sub {
		809	undef $handle; # keep it alive till eof
		810	$cb->($response, $header, $body);
		811	};
		812
		813	The constructor expects a file handle, which gets passed via the C<fh>
		814	argument.
		815
		816	The remaining two argument pairs specify two callbacks to be called on
		817	any errors (C<on_error>) and in the case of a normal connection close
		818	(C<on_eof>).
		819
		820	In the first case, we C<undef>ine the handle object and pass the error to
		821	the callback provided by the callback - done.
		822
		823	In the second case we assume everything went fine and pass the results
		824	gobbled up so far to the caller-provided callback. This is not quite
		825	perfect, as when the server "cleanly" closes the connection in the middle
		826	of sending headers we might wrongly report this as an "OK" to the caller,
		827	but then, HTTP doesn't support a perfect mechanism that would detect such
		828	problems in all cases, so we don't bother either.
		829
		830	=head3 The write queue
		831
		832	The next line sends the actual request:
		833
		834	$handle->push_write ("GET $uri HTTP/1.0\015\012\015\012");
		835
		836	No headers will be sent (this is fine for simple requests), so the whole
		837	request is just a single line followed by an empty line to signal the end
		838	of the headers to the server.
		839
		840	The more interesting question is why the method is called C<push_write>
		841	and not just write. The reason is that you can I<always> add some write
		842	data without blocking, and to do this, AnyEvent::Handle needs some write
		843	queue internally - and C<push_write> simply pushes some data at the end of
		844	that queue, just like Perl's C<push> pushes data at the end of an array.
		845
		846	The deeper reason is that at some point in the future, there might
		847	be C<unshift_write> as well, and in any case, we will shortly meet
		848	C<push_read> and C<unshift_read>, and it's usually easiest if all those
		849	functions have some symmetry in their name.
		850
		851	If C<push_write> is called with more than one argument, then you can even
		852	do I<formatted> I/O, which simply means your data will be transformed in
		853	some ways. For example, this would JSON-encode your data before pushing it
		854	to the write queue:
		855
		856	$handle->push_write (json => [1, 2, 3]);
		857
		858	Apart from that, this pretty much summarises the write queue, there is
		859	little else to it.
		860
		861	Reading the response if far more interesting:
		862
		863	=head3 The read queue
		864
		865	the response consists of three parts: a single line of response status, a
		866	single paragraph of headers ended by an empty line, and the request body,
		867	which is simply the remaining data on that connection.
		868
		869	For the first two, we push two read requests onto the read queue:
		870
		871	# now fetch response status line
		872	$handle->push_read (line => sub {
		873	my ($handle, $line) = @_;
		874	$response = $line;
		875	});
		876
		877	# then the headers
		878	$handle->push_read (line => "\015\012\015\012", sub {
		879	my ($handle, $line) = @_;
		880	$header = $line;
		881	});
		882
		883	While one can simply push a single callback to the queue, I<formatted> I/O
		884	really comes to out advantage here, as there is a ready-made "read line"
		885	read type. The first read expects a single line, ended by C<\015\012> (the
		886	standard end-of-line marker in internet protocols).
		887
		888	The second "line" is actually a single paragraph - instead of reading it
		889	line by line we tell C<push_read> that the end-of-line marker is really
		890	C<\015\012\015\012>, which is an empty line. The result is that the whole
		891	header paragraph will be treated as a single line and read. The word
		892	"line" is interpreted very freely, much like Perl itself does it.
		893
		894	Note that push read requests are pushed immediately after creating the
		895	handle object - since AnyEvent::Handle provides a queue we can push as
		896	many requests as we want, and AnyEvent::Handle will handle them in order.
		897
		898	There is, however, no read type for "the remaining data". For that, we
		899	install our own C<on_read> callback:
		900
		901	# and finally handle any remaining data as body
		902	$handle->on_read (sub {
		903	$body .= $_[0]->rbuf;
		904	$_[0]->rbuf = "";
		905	});
		906
		907	This callback is invoked every time data arrives and the read queue is
		908	empty - which in this example will only be the case when both response and
		909	header have been read.
		910
		911
		912	#############################################################################
676		913
677	Now let's start with something simple: a program that reads from standard	914	Now let's start with something simple: a program that reads from standard
678	input in a non-blocking way, that is, in a way that lets your program do	915	input in a non-blocking way, that is, in a way that lets your program do
679	other things while it is waiting for input.	916	other things while it is waiting for input.
680		917

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing AnyEvent/lib/AnyEvent/Intro.pod (file contents): Revision 1.10 by root, Mon Jun 2 06:04:08 2008 UTC vs. Revision 1.11 by root, Mon Jun 2 09:10:38 2008 UTC

Diff Legend

Comparing AnyEvent/lib/AnyEvent/Intro.pod (file contents):
Revision 1.10 by root, Mon Jun 2 06:04:08 2008 UTC vs.
Revision 1.11 by root, Mon Jun 2 09:10:38 2008 UTC