ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-HTTP/README
(Generate patch)

Comparing AnyEvent-HTTP/README (file contents):
Revision 1.21 by root, Tue Jun 14 05:23:12 2011 UTC vs.
Revision 1.27 by root, Sun Aug 28 09:31:29 2016 UTC

50 object at least alive until the callback get called. If the object 50 object at least alive until the callback get called. If the object
51 gets destroyed before the callback is called, the request will be 51 gets destroyed before the callback is called, the request will be
52 cancelled. 52 cancelled.
53 53
54 The callback will be called with the response body data as first 54 The callback will be called with the response body data as first
55 argument (or "undef" if an error occured), and a hash-ref with 55 argument (or "undef" if an error occurred), and a hash-ref with
56 response headers (and trailers) as second argument. 56 response headers (and trailers) as second argument.
57 57
58 All the headers in that hash are lowercased. In addition to the 58 All the headers in that hash are lowercased. In addition to the
59 response headers, the "pseudo-headers" (uppercase to avoid clashing 59 response headers, the "pseudo-headers" (uppercase to avoid clashing
60 with possible response headers) "HTTPVersion", "Status" and "Reason" 60 with possible response headers) "HTTPVersion", "Status" and "Reason"
82 If an internal error occurs, such as not being able to resolve a 82 If an internal error occurs, such as not being able to resolve a
83 hostname, then $data will be "undef", "$headers->{Status}" will be 83 hostname, then $data will be "undef", "$headers->{Status}" will be
84 590-599 and the "Reason" pseudo-header will contain an error 84 590-599 and the "Reason" pseudo-header will contain an error
85 message. Currently the following status codes are used: 85 message. Currently the following status codes are used:
86 86
87 595 - errors during connection etsbalishment, proxy handshake. 87 595 - errors during connection establishment, proxy handshake.
88 596 - errors during TLS negotiation, request sending and header 88 596 - errors during TLS negotiation, request sending and header
89 processing. 89 processing.
90 597 - errors during body receiving or processing. 90 597 - errors during body receiving or processing.
91 598 - user aborted request via "on_header" or "on_body". 91 598 - user aborted request via "on_header" or "on_body".
92 599 - other, usually nonretryable, errors (garbled URL etc.). 92 599 - other, usually nonretryable, errors (garbled URL etc.).
106 Additional parameters are key-value pairs, and are fully optional. 106 Additional parameters are key-value pairs, and are fully optional.
107 They include: 107 They include:
108 108
109 recurse => $count (default: $MAX_RECURSE) 109 recurse => $count (default: $MAX_RECURSE)
110 Whether to recurse requests or not, e.g. on redirects, 110 Whether to recurse requests or not, e.g. on redirects,
111 authentication retries and so on, and how often to do so. 111 authentication and other retries and so on, and how often to do
112 so.
113
114 Only redirects to http and https URLs are supported. While most
115 common redirection forms are handled entirely within this
116 module, some require the use of the optional URI module. If it
117 is required but missing, then the request will fail with an
118 error.
112 119
113 headers => hashref 120 headers => hashref
114 The request headers to use. Currently, "http_request" may 121 The request headers to use. Currently, "http_request" may
115 provide its own "Host:", "Content-Length:", "Connection:" and 122 provide its own "Host:", "Content-Length:", "Connection:" and
116 "Cookie:" headers and will provide defaults at least for "TE:", 123 "Cookie:" headers and will provide defaults at least for "TE:",
140 $scheme must be either missing or must be "http" for HTTP. 147 $scheme must be either missing or must be "http" for HTTP.
141 148
142 If not specified, then the default proxy is used (see 149 If not specified, then the default proxy is used (see
143 "AnyEvent::HTTP::set_proxy"). 150 "AnyEvent::HTTP::set_proxy").
144 151
152 Currently, if your proxy requires authorization, you have to
153 specify an appropriate "Proxy-Authorization" header in every
154 request.
155
145 body => $string 156 body => $string
146 The request body, usually empty. Will be sent as-is (future 157 The request body, usually empty. Will be sent as-is (future
147 versions of this module might offer more options). 158 versions of this module might offer more options).
148 159
149 cookie_jar => $hash_ref 160 cookie_jar => $hash_ref
188 object storing your state data, or the TLS context) - only 199 object storing your state data, or the TLS context) - only
189 connections using the same unique ID will be reused. 200 connections using the same unique ID will be reused.
190 201
191 on_prepare => $callback->($fh) 202 on_prepare => $callback->($fh)
192 In rare cases you need to "tune" the socket before it is used to 203 In rare cases you need to "tune" the socket before it is used to
193 connect (for exmaple, to bind it on a given IP address). This 204 connect (for example, to bind it on a given IP address). This
194 parameter overrides the prepare callback passed to 205 parameter overrides the prepare callback passed to
195 "AnyEvent::Socket::tcp_connect" and behaves exactly the same way 206 "AnyEvent::Socket::tcp_connect" and behaves exactly the same way
196 (e.g. it has to provide a timeout). See the description for the 207 (e.g. it has to provide a timeout). See the description for the
197 $prepare_cb argument of "AnyEvent::Socket::tcp_connect" for 208 $prepare_cb argument of "AnyEvent::Socket::tcp_connect" for
198 details. 209 details.
336 347
337 Example: do a HTTP HEAD request on https://www.google.com/, use a 348 Example: do a HTTP HEAD request on https://www.google.com/, use a
338 timeout of 30 seconds. 349 timeout of 30 seconds.
339 350
340 http_request 351 http_request
341 GET => "https://www.google.com", 352 HEAD => "https://www.google.com",
342 headers => { "user-agent" => "MySearchClient 1.0" }, 353 headers => { "user-agent" => "MySearchClient 1.0" },
343 timeout => 30, 354 timeout => 30,
344 sub { 355 sub {
345 my ($body, $hdr) = @_; 356 my ($body, $hdr) = @_;
346 use Data::Dumper; 357 use Data::Dumper;
371 Sets the default proxy server to use. The proxy-url must begin with 382 Sets the default proxy server to use. The proxy-url must begin with
372 a string of the form "http://host:port", croaks otherwise. 383 a string of the form "http://host:port", croaks otherwise.
373 384
374 To clear an already-set proxy, use "undef". 385 To clear an already-set proxy, use "undef".
375 386
376 When AnyEvent::HTTP is laoded for the first time it will query the 387 When AnyEvent::HTTP is loaded for the first time it will query the
377 default proxy from the operating system, currently by looking at 388 default proxy from the operating system, currently by looking at
378 "$ENV{http_proxy"}. 389 "$ENV{http_proxy"}.
379 390
380 AnyEvent::HTTP::cookie_jar_expire $jar[, $session_end] 391 AnyEvent::HTTP::cookie_jar_expire $jar[, $session_end]
381 Remove all cookies from the cookie jar that have been expired. If 392 Remove all cookies from the cookie jar that have been expired. If
383 cookies. 394 cookies.
384 395
385 You should call this function (with a true $session_end) before you 396 You should call this function (with a true $session_end) before you
386 save cookies to disk, and you should call this function after 397 save cookies to disk, and you should call this function after
387 loading them again. If you have a long-running program you can 398 loading them again. If you have a long-running program you can
388 additonally call this function from time to time. 399 additionally call this function from time to time.
389 400
390 A cookie jar is initially an empty hash-reference that is managed by 401 A cookie jar is initially an empty hash-reference that is managed by
391 this module. It's format is subject to change, but currently it is 402 this module. Its format is subject to change, but currently it is as
392 like this: 403 follows:
393 404
394 The key "version" has to contain 1, otherwise the hash gets emptied. 405 The key "version" has to contain 1, otherwise the hash gets emptied.
395 All other keys are hostnames or IP addresses pointing to 406 All other keys are hostnames or IP addresses pointing to
396 hash-references. The key for these inner hash references is the 407 hash-references. The key for these inner hash references is the
397 server path for which this cookie is meant, and the values are again 408 server path for which this cookie is meant, and the values are again
398 hash-references. The keys of those hash-references is the cookie 409 hash-references. Each key of those hash-references is a cookie name,
399 name, and the value, you guessed it, is another hash-reference, this 410 and the value, you guessed it, is another hash-reference, this time
400 time with the key-value pairs from the cookie, except for "expires" 411 with the key-value pairs from the cookie, except for "expires" and
401 and "max-age", which have been replaced by a "_expires" key that 412 "max-age", which have been replaced by a "_expires" key that
402 contains the cookie expiry timestamp. 413 contains the cookie expiry timestamp. Session cookies are indicated
414 by not having an "_expires" key.
403 415
404 Here is an example of a cookie jar with a single cookie, so you have 416 Here is an example of a cookie jar with a single cookie, so you have
405 a chance of understanding the above paragraph: 417 a chance of understanding the above paragraph:
406 418
407 { 419 {
428 440
429 $AnyEvent::HTTP::MAX_RECURSE 441 $AnyEvent::HTTP::MAX_RECURSE
430 The default value for the "recurse" request parameter (default: 10). 442 The default value for the "recurse" request parameter (default: 10).
431 443
432 $AnyEvent::HTTP::TIMEOUT 444 $AnyEvent::HTTP::TIMEOUT
433 The default timeout for conenction operations (default: 300). 445 The default timeout for connection operations (default: 300).
434 446
435 $AnyEvent::HTTP::USERAGENT 447 $AnyEvent::HTTP::USERAGENT
436 The default value for the "User-Agent" header (the default is 448 The default value for the "User-Agent" header (the default is
437 "Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION; 449 "Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION;
438 +http://software.schmorp.de/pkg/AnyEvent)"). 450 +http://software.schmorp.de/pkg/AnyEvent)").
439 451
440 $AnyEvent::HTTP::MAX_PER_HOST 452 $AnyEvent::HTTP::MAX_PER_HOST
441 The maximum number of concurrent connections to the same host 453 The maximum number of concurrent connections to the same host
442 (identified by the hostname). If the limit is exceeded, then the 454 (identified by the hostname). If the limit is exceeded, then
443 additional requests are queued until previous connections are 455 additional requests are queued until previous connections are
444 closed. Both persistent and non-persistent connections are counted 456 closed. Both persistent and non-persistent connections are counted
445 in this limit. 457 in this limit.
446 458
447 The default value for this is 4, and it is highly advisable to not 459 The default value for this is 4, and it is highly advisable to not
448 increase it much. 460 increase it much.
449 461
450 For comparison: the RFC's recommend 4 non-persistent or 2 persistent 462 For comparison: the RFC's recommend 4 non-persistent or 2 persistent
451 connections, older browsers used 2, newers (such as firefox 3) 463 connections, older browsers used 2, newer ones (such as firefox 3)
452 typically use 6, and Opera uses 8 because like, they have the 464 typically use 6, and Opera uses 8 because like, they have the
453 fastest browser and give a shit for everybody else on the planet. 465 fastest browser and give a shit for everybody else on the planet.
454 466
455 $AnyEvent::HTTP::PERSISTENT_TIMEOUT 467 $AnyEvent::HTTP::PERSISTENT_TIMEOUT
456 The time after which idle persistent conenctions get closed by 468 The time after which idle persistent connections get closed by
457 AnyEvent::HTTP (default: 3). 469 AnyEvent::HTTP (default: 3).
458 470
459 $AnyEvent::HTTP::ACTIVE 471 $AnyEvent::HTTP::ACTIVE
460 The number of active connections. This is not the number of 472 The number of active connections. This is not the number of
461 currently running requests, but the number of currently open and 473 currently running requests, but the number of currently open and
462 non-idle TCP connections. This number can be useful for 474 non-idle TCP connections. This number can be useful for
463 load-leveling. 475 load-leveling.
464 476
465 SHOWCASE 477 SHOWCASE
466 This section contaisn some more elaborate "real-world" examples or code 478 This section contains some more elaborate "real-world" examples or code
467 snippets. 479 snippets.
468 480
469 HTTP/1.1 FILE DOWNLOAD 481 HTTP/1.1 FILE DOWNLOAD
470 Downloading files with HTTP can be quite tricky, especially when 482 Downloading files with HTTP can be quite tricky, especially when
471 something goes wrong and you want to resume. 483 something goes wrong and you want to resume.
474 last modified time to check for file content changes, and works with 486 last modified time to check for file content changes, and works with
475 many HTTP/1.0 servers as well, and usually falls back to a complete 487 many HTTP/1.0 servers as well, and usually falls back to a complete
476 re-download on older servers. 488 re-download on older servers.
477 489
478 It calls the completion callback with either "undef", which means a 490 It calls the completion callback with either "undef", which means a
479 nonretryable error occured, 0 when the download was partial and should 491 nonretryable error occurred, 0 when the download was partial and should
480 be retried, and 1 if it was successful. 492 be retried, and 1 if it was successful.
481 493
482 use AnyEvent::HTTP; 494 use AnyEvent::HTTP;
483 495
484 sub download($$$) { 496 sub download($$$) {
488 or die "$file: $!"; 500 or die "$file: $!";
489 501
490 my %hdr; 502 my %hdr;
491 my $ofs = 0; 503 my $ofs = 0;
492 504
493 warn stat $fh;
494 warn -s _;
495 if (stat $fh and -s _) { 505 if (stat $fh and -s _) {
496 $ofs = -s _; 506 $ofs = -s _;
497 warn "-s is ", $ofs; 507 warn "-s is ", $ofs;
498 $hdr{"if-unmodified-since"} = AnyEvent::HTTP::format_date +(stat _)[9]; 508 $hdr{"if-unmodified-since"} = AnyEvent::HTTP::format_date +(stat _)[9];
499 $hdr{"range"} = "bytes=$ofs-"; 509 $hdr{"range"} = "bytes=$ofs-";
527 my (undef, $hdr) = @_; 537 my (undef, $hdr) = @_;
528 538
529 my $status = $hdr->{Status}; 539 my $status = $hdr->{Status};
530 540
531 if (my $time = AnyEvent::HTTP::parse_date $hdr->{"last-modified"}) { 541 if (my $time = AnyEvent::HTTP::parse_date $hdr->{"last-modified"}) {
532 utime $fh, $time, $time; 542 utime $time, $time, $fh;
533 } 543 }
534 544
535 if ($status == 200 || $status == 206 || $status == 416) { 545 if ($status == 200 || $status == 206 || $status == 416) {
536 # download ok || resume ok || file already fully downloaded 546 # download ok || resume ok || file already fully downloaded
537 $cb->(1, $hdr); 547 $cb->(1, $hdr);
622 632
623AUTHOR 633AUTHOR
624 Marc Lehmann <schmorp@schmorp.de> 634 Marc Lehmann <schmorp@schmorp.de>
625 http://home.schmorp.de/ 635 http://home.schmorp.de/
626 636
627 With many thanks to Дмитрий Шалашов, who provided 637 With many thanks to Дмитрий Шалашов, who provided countless testcases
628 countless testcases and bugreports. 638 and bugreports.
629 639

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines