ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-HTTP/README
(Generate patch)

Comparing AnyEvent-HTTP/README (file contents):
Revision 1.2 by root, Thu Jun 5 18:42:16 2008 UTC vs.
Revision 1.13 by root, Wed Jun 16 19:17:30 2010 UTC

1NAME 1NAME
2 AnyEvent::HTTP - simple but non-blocking HTTP/HTTPS client 2 AnyEvent::HTTP - simple but non-blocking HTTP/HTTPS client
3 3
4SYNOPSIS 4SYNOPSIS
5 use AnyEvent::HTTP; 5 use AnyEvent::HTTP;
6
7 http_get "http://www.nethype.de/", sub { print $_[1] };
8
9 # ... do something else here
6 10
7DESCRIPTION 11DESCRIPTION
8 This module is an AnyEvent user, you need to make sure that you use and 12 This module is an AnyEvent user, you need to make sure that you use and
9 run a supported event loop. 13 run a supported event loop.
10 14
… …
24 limited support. 28 limited support.
25 29
26 METHODS 30 METHODS
27 http_get $url, key => value..., $cb->($data, $headers) 31 http_get $url, key => value..., $cb->($data, $headers)
28 Executes an HTTP-GET request. See the http_request function for 32 Executes an HTTP-GET request. See the http_request function for
29 details on additional parameters. 33 details on additional parameters and the return value.
30 34
31 http_head $url, key => value..., $cb->($data, $headers) 35 http_head $url, key => value..., $cb->($data, $headers)
32 Executes an HTTP-HEAD request. See the http_request function for 36 Executes an HTTP-HEAD request. See the http_request function for
33 details on additional parameters. 37 details on additional parameters and the return value.
34 38
35 http_post $url, $body, key => value..., $cb->($data, $headers) 39 http_post $url, $body, key => value..., $cb->($data, $headers)
36 Executes an HTTP-POST request with a request body of $bod. See the 40 Executes an HTTP-POST request with a request body of $body. See the
37 http_request function for details on additional parameters. 41 http_request function for details on additional parameters and the
42 return value.
38 43
39 http_request $method => $url, key => value..., $cb->($data, $headers) 44 http_request $method => $url, key => value..., $cb->($data, $headers)
40 Executes a HTTP request of type $method (e.g. "GET", "POST"). The 45 Executes a HTTP request of type $method (e.g. "GET", "POST"). The
41 URL must be an absolute http or https URL. 46 URL must be an absolute http or https URL.
42 47
48 When called in void context, nothing is returned. In other contexts,
49 "http_request" returns a "cancellation guard" - you have to keep the
50 object at least alive until the callback get called. If the object
51 gets destroyed before the callbakc is called, the request will be
52 cancelled.
53
43 The callback will be called with the response data as first argument 54 The callback will be called with the response body data as first
44 (or "undef" if it wasn't available due to errors), and a hash-ref 55 argument (or "undef" if an error occured), and a hash-ref with
45 with response headers as second argument. 56 response headers as second argument.
46 57
47 All the headers in that hash are lowercased. In addition to the 58 All the headers in that hash are lowercased. In addition to the
48 response headers, the three "pseudo-headers" "HTTPVersion", "Status" 59 response headers, the "pseudo-headers" (uppercase to avoid clashing
60 with possible response headers) "HTTPVersion", "Status" and "Reason"
49 and "Reason" contain the three parts of the HTTP Status-Line of the 61 contain the three parts of the HTTP Status-Line of the same name.
62
63 The pseudo-header "URL" contains the actual URL (which can differ
64 from the requested URL when following redirects - for example, you
65 might get an error that your URL scheme is not supported even though
66 your URL is a valid http URL because it redirected to an ftp URL, in
67 which case you can look at the URL pseudo header).
68
69 The pseudo-header "Redirect" only exists when the request was a
70 result of an internal redirect. In that case it is an array
71 reference with the "($data, $headers)" from the redirect response.
72 Note that this response could in turn be the result of a redirect
73 itself, and "$headers->{Redirect}[1]{Redirect}" will then contain
74 the original response, and so on.
75
50 same name. If the server sends a header multiple lines, then their 76 If the server sends a header multiple times, then their contents
51 contents will be joined together with "\x00". 77 will be joined together with a comma (","), as per the HTTP spec.
52 78
53 If an internal error occurs, such as not being able to resolve a 79 If an internal error occurs, such as not being able to resolve a
54 hostname, then $data will be "undef", "$headers->{Status}" will be 80 hostname, then $data will be "undef", "$headers->{Status}" will be
55 599 and the "Reason" pseudo-header will contain an error message. 81 "59x" (usually 599) and the "Reason" pseudo-header will contain an
82 error message.
56 83
57 A typical callback might look like this: 84 A typical callback might look like this:
58 85
59 sub { 86 sub {
60 my ($body, $hdr) = @_; 87 my ($body, $hdr) = @_;
… …
75 102
76 headers => hashref 103 headers => hashref
77 The request headers to use. Currently, "http_request" may 104 The request headers to use. Currently, "http_request" may
78 provide its own "Host:", "Content-Length:", "Connection:" and 105 provide its own "Host:", "Content-Length:", "Connection:" and
79 "Cookie:" headers and will provide defaults for "User-Agent:" 106 "Cookie:" headers and will provide defaults for "User-Agent:"
80 and "Referer:". 107 and "Referer:" (this can be suppressed by using "undef" for
108 these headers in which case they won't be sent at all).
81 109
82 timeout => $seconds 110 timeout => $seconds
83 The time-out to use for various stages - each connect attempt 111 The time-out to use for various stages - each connect attempt
84 will reset the timeout, as will read or write activity. Default 112 will reset the timeout, as will read or write activity, i.e.
113 this is not an overall timeout.
114
85 timeout is 5 minutes. 115 Default timeout is 5 minutes.
86 116
87 proxy => [$host, $port[, $scheme]] or undef 117 proxy => [$host, $port[, $scheme]] or undef
88 Use the given http proxy for all requests. If not specified, 118 Use the given http proxy for all requests. If not specified,
89 then the default proxy (as specified by $ENV{http_proxy}) is 119 then the default proxy (as specified by $ENV{http_proxy}) is
90 used. 120 used.
91 121
92 $scheme must be either missing or "http" for HTTP, or "https" 122 $scheme must be either missing, "http" for HTTP or "https" for
93 for HTTPS. 123 HTTPS.
94 124
95 body => $string 125 body => $string
96 The request body, usually empty. Will be-sent as-is (future 126 The request body, usually empty. Will be-sent as-is (future
97 versions of this module might offer more options). 127 versions of this module might offer more options).
98 128
… …
101 loosely based on the original netscape specification. 131 loosely based on the original netscape specification.
102 132
103 The $hash_ref must be an (initially empty) hash reference which 133 The $hash_ref must be an (initially empty) hash reference which
104 will get updated automatically. It is possible to save the 134 will get updated automatically. It is possible to save the
105 cookie_jar to persistent storage with something like JSON or 135 cookie_jar to persistent storage with something like JSON or
106 Storable, but this is not recommended, as expire times are 136 Storable, but this is not recommended, as expiry times are
107 currently being ignored. 137 currently being ignored.
108 138
109 Note that this cookie implementation is not of very high 139 Note that this cookie implementation is not of very high
110 quality, nor meant to be complete. If you want complete cookie 140 quality, nor meant to be complete. If you want complete cookie
111 management you have to do that on your own. "cookie_jar" is 141 management you have to do that on your own. "cookie_jar" is
112 meant as a quick fix to get some cookie-using sites working. 142 meant as a quick fix to get some cookie-using sites working.
113 Cookies are a privacy disaster, do not use them unless required 143 Cookies are a privacy disaster, do not use them unless required
114 to. 144 to.
145
146 tls_ctx => $scheme | $tls_ctx
147 Specifies the AnyEvent::TLS context to be used for https
148 connections. This parameter follows the same rules as the
149 "tls_ctx" parameter to AnyEvent::Handle, but additionally, the
150 two strings "low" or "high" can be specified, which give you a
151 predefined low-security (no verification, highest compatibility)
152 and high-security (CA and common-name verification) TLS context.
153
154 The default for this option is "low", which could be interpreted
155 as "give me the page, no matter what".
156
157 on_prepare => $callback->($fh)
158 In rare cases you need to "tune" the socket before it is used to
159 connect (for exmaple, to bind it on a given IP address). This
160 parameter overrides the prepare callback passed to
161 "AnyEvent::Socket::tcp_connect" and behaves exactly the same way
162 (e.g. it has to provide a timeout). See the description for the
163 $prepare_cb argument of "AnyEvent::Socket::tcp_connect" for
164 details.
165
166 on_header => $callback->($headers)
167 When specified, this callback will be called with the header
168 hash as soon as headers have been successfully received from the
169 remote server (not on locally-generated errors).
170
171 It has to return either true (in which case AnyEvent::HTTP will
172 continue), or false, in which case AnyEvent::HTTP will cancel
173 the download (and call the finish callback with an error code of
174 598).
175
176 This callback is useful, among other things, to quickly reject
177 unwanted content, which, if it is supposed to be rare, can be
178 faster than first doing a "HEAD" request.
179
180 Example: cancel the request unless the content-type is
181 "text/html".
182
183 on_header => sub {
184 $_[0]{"content-type"} =~ /^text\/html\s*(?:;|$)/
185 },
186
187 on_body => $callback->($partial_body, $headers)
188 When specified, all body data will be passed to this callback
189 instead of to the completion callback. The completion callback
190 will get the empty string instead of the body data.
191
192 It has to return either true (in which case AnyEvent::HTTP will
193 continue), or false, in which case AnyEvent::HTTP will cancel
194 the download (and call the completion callback with an error
195 code of 598).
196
197 This callback is useful when the data is too large to be held in
198 memory (so the callback writes it to a file) or when only some
199 information should be extracted, or when the body should be
200 processed incrementally.
201
202 It is usually preferred over doing your own body handling via
203 "want_body_handle", but in case of streaming APIs, where HTTP is
204 only used to create a connection, "want_body_handle" is the
205 better alternative, as it allows you to install your own event
206 handler, reducing resource usage.
207
208 want_body_handle => $enable
209 When enabled (default is disabled), the behaviour of
210 AnyEvent::HTTP changes considerably: after parsing the headers,
211 and instead of downloading the body (if any), the completion
212 callback will be called. Instead of the $body argument
213 containing the body data, the callback will receive the
214 AnyEvent::Handle object associated with the connection. In error
215 cases, "undef" will be passed. When there is no body (e.g.
216 status 304), the empty string will be passed.
217
218 The handle object might or might not be in TLS mode, might be
219 connected to a proxy, be a persistent connection etc., and
220 configured in unspecified ways. The user is responsible for this
221 handle (it will not be used by this module anymore).
222
223 This is useful with some push-type services, where, after the
224 initial headers, an interactive protocol is used (typical
225 example would be the push-style twitter API which starts a
226 JSON/XML stream).
227
228 If you think you need this, first have a look at "on_body", to
229 see if that doesn't solve your problem in a better way.
115 230
116 Example: make a simple HTTP GET request for http://www.nethype.de/ 231 Example: make a simple HTTP GET request for http://www.nethype.de/
117 232
118 http_request GET => "http://www.nethype.de/", sub { 233 http_request GET => "http://www.nethype.de/", sub {
119 my ($body, $hdr) = @_; 234 my ($body, $hdr) = @_;
… …
131 use Data::Dumper; 246 use Data::Dumper;
132 print Dumper $hdr; 247 print Dumper $hdr;
133 } 248 }
134 ; 249 ;
135 250
251 Example: make another simple HTTP GET request, but immediately try
252 to cancel it.
253
254 my $request = http_request GET => "http://www.nethype.de/", sub {
255 my ($body, $hdr) = @_;
256 print "$body\n";
257 };
258
259 undef $request;
260
261 DNS CACHING
262 AnyEvent::HTTP uses the AnyEvent::Socket::tcp_connect function for the
263 actual connection, which in turn uses AnyEvent::DNS to resolve
264 hostnames. The latter is a simple stub resolver and does no caching on
265 its own. If you want DNS caching, you currently have to provide your own
266 default resolver (by storing a suitable resolver object in
267 $AnyEvent::DNS::RESOLVER).
268
136 GLOBAL FUNCTIONS AND VARIABLES 269 GLOBAL FUNCTIONS AND VARIABLES
137 AnyEvent::HTTP::set_proxy "proxy-url" 270 AnyEvent::HTTP::set_proxy "proxy-url"
138 Sets the default proxy server to use. The proxy-url must begin with 271 Sets the default proxy server to use. The proxy-url must begin with
139 a string of the form "http://host:port" (optionally "https:..."). 272 a string of the form "http://host:port" (optionally "https:..."),
273 croaks otherwise.
274
275 To clear an already-set proxy, use "undef".
140 276
141 $AnyEvent::HTTP::MAX_RECURSE 277 $AnyEvent::HTTP::MAX_RECURSE
142 The default value for the "recurse" request parameter (default: 10). 278 The default value for the "recurse" request parameter (default: 10).
143 279
144 $AnyEvent::HTTP::USERAGENT 280 $AnyEvent::HTTP::USERAGENT
145 The default value for the "User-Agent" header (the default is 281 The default value for the "User-Agent" header (the default is
146 "Mozilla/5.0 (compatible; AnyEvent::HTTP/$VERSION; 282 "Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION;
147 +http://software.schmorp.de/pkg/AnyEvent)"). 283 +http://software.schmorp.de/pkg/AnyEvent)").
148 284
149 $AnyEvent::HTTP::MAX_PERSISTENT 285 $AnyEvent::HTTP::MAX_PER_HOST
150 The maximum number of persistent connections to keep open (default: 286 The maximum number of concurrent connections to the same host
151 8). 287 (identified by the hostname). If the limit is exceeded, then the
288 additional requests are queued until previous connections are
289 closed.
152 290
153 Not implemented currently. 291 The default value for this is 4, and it is highly advisable to not
154 292 increase it.
155 $AnyEvent::HTTP::PERSISTENT_TIMEOUT
156 The maximum time to cache a persistent connection, in seconds
157 (default: 2).
158
159 Not implemented currently.
160 293
161 $AnyEvent::HTTP::ACTIVE 294 $AnyEvent::HTTP::ACTIVE
162 The number of active connections. This is not the number of 295 The number of active connections. This is not the number of
163 currently running requests, but the number of currently open and 296 currently running requests, but the number of currently open and
164 non-idle TCP connections. This number of can be useful for 297 non-idle TCP connections. This number of can be useful for
… …
166 299
167SEE ALSO 300SEE ALSO
168 AnyEvent. 301 AnyEvent.
169 302
170AUTHOR 303AUTHOR
171 Marc Lehmann <schmorp@schmorp.de> 304 Marc Lehmann <schmorp@schmorp.de>
172 http://home.schmorp.de/ 305 http://home.schmorp.de/
173 306
307 With many thanks to Дмитрий Шалашов, who provided
308 countless testcases and bugreports.
309

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines