Coro/Coro/LWP.pm

=head1 NAME

Coro::LWP - make LWP non-blocking - as much as possible

=head1 SYNOPSIS

 use Coro::LWP; # afterwards LWP should not block

=head1 ALTERNATIVES

Over the years, a number of less-invasive alternatives have popped up,
which you might find more acceptable than this rather invasive and fragile
module. All of them only support HTTP (and sometimes HTTPS).

=over 4

=item L<AnyEvent::HTTP>

Works fine without Coro. Requires using a very different API than
LWP. Probably the best choice I<iff> you can do with a completely
different event-based API.

=item L<LWP::Protocol::AnyEvent::http>

Makes LWP use L<AnyEvent::HTTP>. Does not make LWP event-based, but allows
Coro threads to schedule unimpeded through its AnyEvent integration.

Let's you use the LWP API normally.

=item L<LWP::Protocol::Coro::http>

Basically the same as above, distinction unclear. :)

=item L<AnyEvent::HTTP::LWP::UserAgent>

A different user agent implementation, not completely transparent to
users, requires Coro.

=back

=head1 DESCRIPTION

This module is an L<AnyEvent> user, you need to make sure that you use and
run a supported event loop.

This module tries to make L<LWP|LWP> non-blocking with respect to other
coroutines as much as possible, and with whatever means it takes.

LWP really tries very hard to be blocking (and relies on a lot of
undocumented functionality in IO::Socket), so this module had to be very
invasive and must be loaded very early to take the proper effect.

Note that the module L<AnyEvent::HTTP> might offer an alternative to the
full L<LWP> that is designed to be non-blocking.

Here is what it currently does (future versions of LWP might require
different tricks):

=over 4

=item It loads Coro::Select, overwriting the perl C<select> builtin I<globally>.

This is necessary because LWP calls select quite often for timeouts and
who-knows-what.

Impact: everybody else uses this (slower) version of select, too. It should be quite
compatible to perls builtin select, though.

=item It overwrites Socket::inet_aton with Coro::Util::inet_aton.

This is necessary because LWP might (and does) try to resolve hostnames
this way.

Impact: some code might not expect coroutine semantics, for example, when
you fork you might prefer the blocking variant because other coroutines
shouldn't actually run.

=item It replaces the base class of Net::HTTP, Net::FTP, Net::NNTP.

This is necessary because LWP does not always use select to see whether
a filehandle can be read/written without blocking, so the base class
C<IO::Socket::INET> needs to be replaced by C<Coro::Socket>.

Impact: Coro::Socket is not at all compatible to IO::Socket::INET. While
it duplicates some undocumented functionality required by LWP, it does not
have all the methods of IO::Socket::INET and might act quite differently
in practise. Also, protocols other than the above mentioned will still block,
at least some of the time.

=back

All this likely makes other libraries than just LWP not block, but thats
just a side effect you cannot rely on.

Increases parallelism is not supported by all libraries, some might cache
data globally.

=cut

package Coro::LWP;

use common::sense;

BEGIN {
   # suppress warnings
   local $^W = 0;
   require Net::Config;
}

# do it as early as possible
use Coro::Select;

# import these so they can grab Socket::inet_aton
use AnyEvent::Util ();
use AnyEvent::DNS ();

use Coro::Util ();
use Coro::Socket ();
use Coro::AnyEvent ();

use Socket ();

use IO::Socket::INET ();

use Net::HTTP ();
use Net::FTP ();
use Net::NNTP ();

our $VERSION = 6.42;

*Socket::inet_aton = \&Coro::Util::inet_aton;

for (@Net::HTTP::ISA, @Net::FTP::ISA, @Net::NTTP::ISA) {
   $_ = Coro::LWP::Socket:: if $_ eq IO::Socket::INET::;
}

package Coro::LWP::Socket;

no warnings;

use base Coro::Socket::;

sub new {
   my $self = shift;

   $self->SUPER::new (@_, partial => 1)
}

1;

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut


Revision:	1.80
Committed:	Wed Feb 11 19:30:15 2015 UTC (9 years, 3 months ago) by root
Branch:	MAIN
CVS Tags:	rel-6_42
Changes since 1.79:	+1 -1 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			Coro::LWP - make LWP non-blocking - as much as possible
4
5			=head1 SYNOPSIS
6
7			use Coro::LWP; # afterwards LWP should not block
8
9	root	1.57	=head1 ALTERNATIVES
10
11			Over the years, a number of less-invasive alternatives have popped up,
12	root	1.58	which you might find more acceptable than this rather invasive and fragile
13			module. All of them only support HTTP (and sometimes HTTPS).
14	root	1.57
15			=over 4
16
17			=item L<AnyEvent::HTTP>
18
19	root	1.58	Works fine without Coro. Requires using a very different API than
20			LWP. Probably the best choice I<iff> you can do with a completely
21			different event-based API.
22	root	1.57
23			=item L<LWP::Protocol::AnyEvent::http>
24
25			Makes LWP use L<AnyEvent::HTTP>. Does not make LWP event-based, but allows
26			Coro threads to schedule unimpeded through its AnyEvent integration.
27
28			Let's you use the LWP API normally.
29
30			=item L<LWP::Protocol::Coro::http>
31
32			Basically the same as above, distinction unclear. :)
33
34			=item L<AnyEvent::HTTP::LWP::UserAgent>
35
36			A different user agent implementation, not completely transparent to
37			users, requires Coro.
38
39			=back
40
41	root	1.1	=head1 DESCRIPTION
42
43	root	1.6	This module is an L<AnyEvent> user, you need to make sure that you use and
44			run a supported event loop.
45
46	root	1.1	This module tries to make L<LWP\|LWP> non-blocking with respect to other
47			coroutines as much as possible, and with whatever means it takes.
48
49	root	1.7	LWP really tries very hard to be blocking (and relies on a lot of
50			undocumented functionality in IO::Socket), so this module had to be very
51	root	1.1	invasive and must be loaded very early to take the proper effect.
52
53	root	1.32	Note that the module L<AnyEvent::HTTP> might offer an alternative to the
54			full L<LWP> that is designed to be non-blocking.
55
56	root	1.1	Here is what it currently does (future versions of LWP might require
57			different tricks):
58
59			=over 4
60
61			=item It loads Coro::Select, overwriting the perl C<select> builtin I<globally>.
62
63			This is necessary because LWP calls select quite often for timeouts and
64			who-knows-what.
65
66			Impact: everybody else uses this (slower) version of select, too. It should be quite
67			compatible to perls builtin select, though.
68
69			=item It overwrites Socket::inet_aton with Coro::Util::inet_aton.
70
71	root	1.2	This is necessary because LWP might (and does) try to resolve hostnames
72			this way.
73	root	1.1
74	root	1.8	Impact: some code might not expect coroutine semantics, for example, when
75			you fork you might prefer the blocking variant because other coroutines
76			shouldn't actually run.
77	root	1.1
78	root	1.2	=item It replaces the base class of Net::HTTP, Net::FTP, Net::NNTP.
79	root	1.1
80	root	1.18	This is necessary because LWP does not always use select to see whether
81	root	1.2	a filehandle can be read/written without blocking, so the base class
82			C<IO::Socket::INET> needs to be replaced by C<Coro::Socket>.
83	root	1.1
84			Impact: Coro::Socket is not at all compatible to IO::Socket::INET. While
85			it duplicates some undocumented functionality required by LWP, it does not
86			have all the methods of IO::Socket::INET and might act quite differently
87	root	1.2	in practise. Also, protocols other than the above mentioned will still block,
88			at least some of the time.
89	root	1.1
90			=back
91
92			All this likely makes other libraries than just LWP not block, but thats
93			just a side effect you cannot rely on.
94
95			Increases parallelism is not supported by all libraries, some might cache
96			data globally.
97
98			=cut
99
100			package Coro::LWP;
101
102	root	1.46	use common::sense;
103	root	1.11
104			BEGIN {
105			# suppress warnings
106			local $^W = 0;
107			require Net::Config;
108			}
109	root	1.1
110	root	1.33	# do it as early as possible
111			use Coro::Select;
112
113			# import these so they can grab Socket::inet_aton
114	root	1.9	use AnyEvent::Util ();
115			use AnyEvent::DNS ();
116
117	root	1.8	use Coro::Util ();
118			use Coro::Socket ();
119	root	1.39	use Coro::AnyEvent ();
120	root	1.8
121			use Socket ();
122	root	1.11
123	root	1.8	use IO::Socket::INET ();
124
125			use Net::HTTP ();
126			use Net::FTP ();
127			use Net::NNTP ();
128	root	1.2
129	root	1.80	our $VERSION = 6.42;
130	root	1.3
131	root	1.1	*Socket::inet_aton = \&Coro::Util::inet_aton;
132
133	root	1.2	for (@Net::HTTP::ISA, @Net::FTP::ISA, @Net::NTTP::ISA) {
134			$_ = Coro::LWP::Socket:: if $_ eq IO::Socket::INET::;
135			}
136
137			package Coro::LWP::Socket;
138
139	root	1.11	no warnings;
140
141	root	1.2	use base Coro::Socket::;
142
143			sub new {
144			my $self = shift;
145
146			$self->SUPER::new (@_, partial => 1)
147			}
148
149			1;
150	root	1.1
151			=head1 AUTHOR
152
153			Marc Lehmann <schmorp@schmorp.de>
154			http://home.schmorp.de/
155
156			=cut
157
158