ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-HTTP/HTTP.pm
(Generate patch)

Comparing AnyEvent-HTTP/HTTP.pm (file contents):
Revision 1.94 by root, Tue Jan 11 23:49:37 2011 UTC vs.
Revision 1.119 by root, Sun Jun 8 23:33:28 2014 UTC

… …
46use AnyEvent::Util (); 46use AnyEvent::Util ();
47use AnyEvent::Handle (); 47use AnyEvent::Handle ();
48 48
49use base Exporter::; 49use base Exporter::;
50 50
51our $VERSION = '2.01'; 51our $VERSION = 2.2;
52 52
53our @EXPORT = qw(http_get http_post http_head http_request); 53our @EXPORT = qw(http_get http_post http_head http_request);
54 54
55our $USERAGENT = "Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION; +http://software.schmorp.de/pkg/AnyEvent)"; 55our $USERAGENT = "Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION; +http://software.schmorp.de/pkg/AnyEvent)";
56our $MAX_RECURSE = 10; 56our $MAX_RECURSE = 10;
… …
89C<http_request> returns a "cancellation guard" - you have to keep the 89C<http_request> returns a "cancellation guard" - you have to keep the
90object at least alive until the callback get called. If the object gets 90object at least alive until the callback get called. If the object gets
91destroyed before the callback is called, the request will be cancelled. 91destroyed before the callback is called, the request will be cancelled.
92 92
93The callback will be called with the response body data as first argument 93The callback will be called with the response body data as first argument
94(or C<undef> if an error occured), and a hash-ref with response headers 94(or C<undef> if an error occurred), and a hash-ref with response headers
95(and trailers) as second argument. 95(and trailers) as second argument.
96 96
97All the headers in that hash are lowercased. In addition to the response 97All the headers in that hash are lowercased. In addition to the response
98headers, the "pseudo-headers" (uppercase to avoid clashing with possible 98headers, the "pseudo-headers" (uppercase to avoid clashing with possible
99response headers) C<HTTPVersion>, C<Status> and C<Reason> contain the 99response headers) C<HTTPVersion>, C<Status> and C<Reason> contain the
… …
123C<590>-C<599> and the C<Reason> pseudo-header will contain an error 123C<590>-C<599> and the C<Reason> pseudo-header will contain an error
124message. Currently the following status codes are used: 124message. Currently the following status codes are used:
125 125
126=over 4 126=over 4
127 127
128=item 595 - errors during connection etsbalishment, proxy handshake. 128=item 595 - errors during connection establishment, proxy handshake.
129 129
130=item 596 - errors during TLS negotiation, request sending and header processing. 130=item 596 - errors during TLS negotiation, request sending and header processing.
131 131
132=item 597 - errors during body receiving or processing. 132=item 597 - errors during body receiving or processing.
133 133
… …
154 154
155=over 4 155=over 4
156 156
157=item recurse => $count (default: $MAX_RECURSE) 157=item recurse => $count (default: $MAX_RECURSE)
158 158
159Whether to recurse requests or not, e.g. on redirects, authentication 159Whether to recurse requests or not, e.g. on redirects, authentication and
160retries and so on, and how often to do so. 160other retries and so on, and how often to do so.
161
162Only redirects to http and https URLs are supported. While most common
163redirection forms are handled entirely within this module, some require
164the use of the optional L<URI> module. If it is required but missing, then
165the request will fail with an error.
161 166
162=item headers => hashref 167=item headers => hashref
163 168
164The request headers to use. Currently, C<http_request> may provide its own 169The request headers to use. Currently, C<http_request> may provide its own
165C<Host:>, C<Content-Length:>, C<Connection:> and C<Cookie:> headers and 170C<Host:>, C<Content-Length:>, C<Connection:> and C<Cookie:> headers and
… …
169 174
170You really should provide your own C<User-Agent:> header value that is 175You really should provide your own C<User-Agent:> header value that is
171appropriate for your program - I wouldn't be surprised if the default 176appropriate for your program - I wouldn't be surprised if the default
172AnyEvent string gets blocked by webservers sooner or later. 177AnyEvent string gets blocked by webservers sooner or later.
173 178
179Also, make sure that your headers names and values do not contain any
180embedded newlines.
181
174=item timeout => $seconds 182=item timeout => $seconds
175 183
176The time-out to use for various stages - each connect attempt will reset 184The time-out to use for various stages - each connect attempt will reset
177the timeout, as will read or write activity, i.e. this is not an overall 185the timeout, as will read or write activity, i.e. this is not an overall
178timeout. 186timeout.
179 187
180Default timeout is 5 minutes. 188Default timeout is 5 minutes.
181 189
182=item proxy => [$host, $port[, $scheme]] or undef 190=item proxy => [$host, $port[, $scheme]] or undef
183 191
184Use the given http proxy for all requests. If not specified, then the 192Use the given http proxy for all requests, or no proxy if C<undef> is
185default proxy (as specified by C<$ENV{http_proxy}>) is used. 193used.
186 194
187C<$scheme> must be either missing or must be C<http> for HTTP. 195C<$scheme> must be either missing or must be C<http> for HTTP.
196
197If not specified, then the default proxy is used (see
198C<AnyEvent::HTTP::set_proxy>).
188 199
189=item body => $string 200=item body => $string
190 201
191The request body, usually empty. Will be sent as-is (future versions of 202The request body, usually empty. Will be sent as-is (future versions of
192this module might offer more options). 203this module might offer more options).
… …
236context) - only connections using the same unique ID will be reused. 247context) - only connections using the same unique ID will be reused.
237 248
238=item on_prepare => $callback->($fh) 249=item on_prepare => $callback->($fh)
239 250
240In rare cases you need to "tune" the socket before it is used to 251In rare cases you need to "tune" the socket before it is used to
241connect (for exmaple, to bind it on a given IP address). This parameter 252connect (for example, to bind it on a given IP address). This parameter
242overrides the prepare callback passed to C<AnyEvent::Socket::tcp_connect> 253overrides the prepare callback passed to C<AnyEvent::Socket::tcp_connect>
243and behaves exactly the same way (e.g. it has to provide a 254and behaves exactly the same way (e.g. it has to provide a
244timeout). See the description for the C<$prepare_cb> argument of 255timeout). See the description for the C<$prepare_cb> argument of
245C<AnyEvent::Socket::tcp_connect> for details. 256C<AnyEvent::Socket::tcp_connect> for details.
246 257
… …
378 389
379Example: do a HTTP HEAD request on https://www.google.com/, use a 390Example: do a HTTP HEAD request on https://www.google.com/, use a
380timeout of 30 seconds. 391timeout of 30 seconds.
381 392
382 http_request 393 http_request
383 GET => "https://www.google.com", 394 HEAD => "https://www.google.com",
384 headers => { "user-agent" => "MySearchClient 1.0" }, 395 headers => { "user-agent" => "MySearchClient 1.0" },
385 timeout => 30, 396 timeout => 30,
386 sub { 397 sub {
387 my ($body, $hdr) = @_; 398 my ($body, $hdr) = @_;
388 use Data::Dumper; 399 use Data::Dumper;
… …
529 while ( 540 while (
530 m{ 541 m{
531 \G\s* 542 \G\s*
532 (?: 543 (?:
533 expires \s*=\s* ([A-Z][a-z][a-z]+,\ [^,;]+) 544 expires \s*=\s* ([A-Z][a-z][a-z]+,\ [^,;]+)
534 | ([^=;,[:space:]]+) (?: \s*=\s* (?: "((?:[^\\"]+|\\.)*)" | ([^=;,[:space:]]*) ) )? 545 | ([^=;,[:space:]]+) (?: \s*=\s* (?: "((?:[^\\"]+|\\.)*)" | ([^;,[:space:]]*) ) )?
535 ) 546 )
536 }gcxsi 547 }gcxsi
537 ) { 548 ) {
538 my $name = $2; 549 my $name = $2;
539 my $value = $4; 550 my $value = $4;
… …
546 # quoted 557 # quoted
547 $value = $3; 558 $value = $3;
548 $value =~ s/\\(.)/$1/gs; 559 $value =~ s/\\(.)/$1/gs;
549 } 560 }
550 561
551 push @kv, lc $name, $value; 562 push @kv, @kv ? lc $name : $name, $value;
552 563
553 last unless /\G\s*;/gc; 564 last unless /\G\s*;/gc;
554 } 565 }
555 566
556 last unless @kv; 567 last unless @kv;
… …
683 694
684 $cb->(undef, $hdr); 695 $cb->(undef, $hdr);
685 () 696 ()
686} 697}
687 698
699our %IDEMPOTENT = (
700 DELETE => 1,
701 GET => 1,
702 HEAD => 1,
703 OPTIONS => 1,
704 PUT => 1,
705 TRACE => 1,
706
707 ACL => 1,
708 "BASELINE-CONTROL" => 1,
709 BIND => 1,
710 CHECKIN => 1,
711 CHECKOUT => 1,
712 COPY => 1,
713 LABEL => 1,
714 LINK => 1,
715 MERGE => 1,
716 MKACTIVITY => 1,
717 MKCALENDAR => 1,
718 MKCOL => 1,
719 MKREDIRECTREF => 1,
720 MKWORKSPACE => 1,
721 MOVE => 1,
722 ORDERPATCH => 1,
723 PROPFIND => 1,
724 PROPPATCH => 1,
725 REBIND => 1,
726 REPORT => 1,
727 SEARCH => 1,
728 UNBIND => 1,
729 UNCHECKOUT => 1,
730 UNLINK => 1,
731 UNLOCK => 1,
732 UPDATE => 1,
733 UPDATEREDIRECTREF => 1,
734 "VERSION-CONTROL" => 1,
735);
736
688sub http_request($$@) { 737sub http_request($$@) {
689 my $cb = pop; 738 my $cb = pop;
690 my ($method, $url, %arg) = @_; 739 my ($method, $url, %arg) = @_;
691 740
692 my %hdr; 741 my %hdr;
… …
709 my $recurse = exists $arg{recurse} ? delete $arg{recurse} : $MAX_RECURSE; 758 my $recurse = exists $arg{recurse} ? delete $arg{recurse} : $MAX_RECURSE;
710 759
711 return $cb->(undef, { @pseudo, Status => 599, Reason => "Too many redirections" }) 760 return $cb->(undef, { @pseudo, Status => 599, Reason => "Too many redirections" })
712 if $recurse < 0; 761 if $recurse < 0;
713 762
714 my $proxy = $arg{proxy} || $PROXY; 763 my $proxy = exists $arg{proxy} ? $arg{proxy} : $PROXY;
715 my $timeout = $arg{timeout} || $TIMEOUT; 764 my $timeout = $arg{timeout} || $TIMEOUT;
716 765
717 my ($uscheme, $uauthority, $upath, $query, undef) = # ignore fragment 766 my ($uscheme, $uauthority, $upath, $query, undef) = # ignore fragment
718 $url =~ m|(?:([^:/?#]+):)?(?://([^/?#]*))?([^?#]*)(?:(\?[^#]*))?(?:#(.*))?|; 767 $url =~ m|^([^:]+):(?://([^/?#]*))?([^?#]*)(?:(\?[^#]*))?(?:#(.*))?$|;
719 768
720 $uscheme = lc $uscheme; 769 $uscheme = lc $uscheme;
721 770
722 my $uport = $uscheme eq "http" ? 80 771 my $uport = $uscheme eq "http" ? 80
723 : $uscheme eq "https" ? 443 772 : $uscheme eq "https" ? 443
… …
767 $hdr{"user-agent"} = $USERAGENT unless exists $hdr{"user-agent"}; 816 $hdr{"user-agent"} = $USERAGENT unless exists $hdr{"user-agent"};
768 817
769 $hdr{"content-length"} = length $arg{body} 818 $hdr{"content-length"} = length $arg{body}
770 if length $arg{body} || $method ne "GET"; 819 if length $arg{body} || $method ne "GET";
771 820
772 my $idempotent = $method =~ /^(?:GET|HEAD|PUT|DELETE|OPTIONS|TRACE)$/; 821 my $idempotent = $IDEMPOTENT{$method};
773 822
774 # default value for keepalive is true iff the request is for an idempotent method 823 # default value for keepalive is true iff the request is for an idempotent method
775 my $keepalive = exists $arg{keepalive} ? !!$arg{keepalive} : $idempotent; 824 my $persistent = exists $arg{persistent} ? !!$arg{persistent} : $idempotent;
776 my $keepalive10 = exists $arg{keepalive10} ? $arg{keepalive10} : !$proxy; 825 my $keepalive = exists $arg{keepalive} ? !!$arg{keepalive} : !$proxy;
777 my $keptalive; # true if this is actually a recycled connection 826 my $was_persistent; # true if this is actually a recycled connection
778 827
779 # the key to use in the keepalive cache 828 # the key to use in the keepalive cache
780 my $ka_key = "$uhost\x00$arg{sessionid}"; 829 my $ka_key = "$uscheme\x00$uhost\x00$uport\x00$arg{sessionid}";
781 830
782 $hdr{connection} = ($keepalive ? $keepalive10 ? "keep-alive " : "" : "close ") . "Te"; #1.1 831 $hdr{connection} = ($persistent ? $keepalive ? "keep-alive, " : "" : "close, ") . "Te"; #1.1
783 $hdr{te} = "trailers" unless exists $hdr{te}; #1.1 832 $hdr{te} = "trailers" unless exists $hdr{te}; #1.1
784 833
785 my %state = (connect_guard => 1); 834 my %state = (connect_guard => 1);
786 835
787 my $ae_error = 595; # connecting 836 my $ae_error = 595; # connecting
… …
800 . (join "", map "\u$_: $hdr{$_}\015\012", grep defined $hdr{$_}, keys %hdr) 849 . (join "", map "\u$_: $hdr{$_}\015\012", grep defined $hdr{$_}, keys %hdr)
801 . "\015\012" 850 . "\015\012"
802 . (delete $arg{body}) 851 . (delete $arg{body})
803 ); 852 );
804 853
805 # return if error occured during push_write() 854 # return if error occurred during push_write()
806 return unless %state; 855 return unless %state;
807 856
808 # reduce memory usage, save a kitten, also re-use it for the response headers. 857 # reduce memory usage, save a kitten, also re-use it for the response headers.
809 %hdr = (); 858 %hdr = ();
810 859
… …
837 886
838 %hdr = (%$hdr, @pseudo); 887 %hdr = (%$hdr, @pseudo);
839 } 888 }
840 889
841 # redirect handling 890 # redirect handling
842 # microsoft and other shitheads don't give a shit for following standards, 891 # relative uri handling forced by microsoft and other shitheads.
843 # try to support some common forms of broken Location headers. 892 # we give our best and fall back to URI if available.
844 if ($hdr{location} !~ /^(?: $ | [^:\/?\#]+ : )/x) { 893 if (exists $hdr{location}) {
894 my $loc = $hdr{location};
895
896 if ($loc =~ m%^//%) { # //
897 $loc = "$rscheme:$loc";
898
899 } elsif ($loc eq "") {
900 $loc = $url;
901
902 } elsif ($loc !~ /^(?: $ | [^:\/?\#]+ : )/x) { # anything "simple"
845 $hdr{location} =~ s/^\.\/+//; 903 $loc =~ s/^\.\/+//;
846 904
905 if ($loc !~ m%^[.?#]%) {
847 my $url = "$rscheme://$uhost:$uport"; 906 my $prefix = "$rscheme://$uhost:$uport";
848 907
849 unless ($hdr{location} =~ s/^\///) { 908 unless ($loc =~ s/^\///) {
850 $url .= $upath; 909 $prefix .= $upath;
851 $url =~ s/\/[^\/]*$//; 910 $prefix =~ s/\/[^\/]*$//;
911 }
912
913 $loc = "$prefix/$loc";
914
915 } elsif (eval { require URI }) { # uri
916 $loc = URI->new_abs ($loc, $url)->as_string;
917
918 } else {
919 return _error %state, $cb, { @pseudo, Status => 599, Reason => "Cannot parse Location (URI module missing)" };
920 #$hdr{Status} = 599;
921 #$hdr{Reason} = "Unparsable Redirect (URI module missing)";
922 #$recurse = 0;
923 }
852 } 924 }
853 925
854 $hdr{location} = "$url/$hdr{location}"; 926 $hdr{location} = $loc;
855 } 927 }
856 928
857 my $redirect; 929 my $redirect;
858 930
859 if ($recurse) { 931 if ($recurse) {
… …
861 933
862 # industry standard is to redirect POST as GET for 934 # industry standard is to redirect POST as GET for
863 # 301, 302 and 303, in contrast to HTTP/1.0 and 1.1. 935 # 301, 302 and 303, in contrast to HTTP/1.0 and 1.1.
864 # also, the UA should ask the user for 301 and 307 and POST, 936 # also, the UA should ask the user for 301 and 307 and POST,
865 # industry standard seems to be to simply follow. 937 # industry standard seems to be to simply follow.
866 # we go with the industry standard. 938 # we go with the industry standard. 308 is defined
939 # by rfc7238
867 if ($status == 301 or $status == 302 or $status == 303) { 940 if ($status == 301 or $status == 302 or $status == 303) {
868 # HTTP/1.1 is unclear on how to mutate the method 941 # HTTP/1.1 is unclear on how to mutate the method
869 $method = "GET" unless $method eq "HEAD"; 942 $method = "GET" unless $method eq "HEAD";
870 $redirect = 1; 943 $redirect = 1;
871 } elsif ($status == 307) { 944 } elsif ($status == 307 or $status == 308) {
872 $redirect = 1; 945 $redirect = 1;
873 } 946 }
874 } 947 }
875 948
876 my $finish = sub { # ($data, $err_status, $err_reason[, $keepalive]) 949 my $finish = sub { # ($data, $err_status, $err_reason[, $persistent])
877 if ($state{handle}) { 950 if ($state{handle}) {
878 # handle keepalive 951 # handle keepalive
879 if ( 952 if (
880 $keepalive 953 $persistent
881 && $_[3] 954 && $_[3]
882 && ($hdr{HTTPVersion} < 1.1 955 && ($hdr{HTTPVersion} < 1.1
883 ? $hdr{connection} =~ /\bkeep-?alive\b/i 956 ? $hdr{connection} =~ /\bkeep-?alive\b/i
884 : $hdr{connection} !~ /\bclose\b/i) 957 : $hdr{connection} !~ /\bclose\b/i)
885 ) { 958 ) {
… …
904 977
905 if ($redirect && exists $hdr{location}) { 978 if ($redirect && exists $hdr{location}) {
906 # we ignore any errors, as it is very common to receive 979 # we ignore any errors, as it is very common to receive
907 # Content-Length != 0 but no actual body 980 # Content-Length != 0 but no actual body
908 # we also access %hdr, as $_[1] might be an erro 981 # we also access %hdr, as $_[1] might be an erro
982 $state{recurse} =
909 http_request ( 983 http_request (
910 $method => $hdr{location}, 984 $method => $hdr{location},
911 %arg, 985 %arg,
912 recurse => $recurse - 1, 986 recurse => $recurse - 1,
913 Redirect => [$_[0], \%hdr], 987 Redirect => [$_[0], \%hdr],
988 sub {
989 %state = ();
914 $cb 990 &$cb
991 },
915 ); 992 );
916 } else { 993 } else {
917 $cb->($_[0], \%hdr); 994 $cb->($_[0], \%hdr);
918 } 995 }
919 }; 996 };
920 997
… …
952 my $body = ""; 1029 my $body = "";
953 my $on_body = $arg{on_body} || sub { $body .= shift; 1 }; 1030 my $on_body = $arg{on_body} || sub { $body .= shift; 1 };
954 1031
955 $state{read_chunk} = sub { 1032 $state{read_chunk} = sub {
956 $_[1] =~ /^([0-9a-fA-F]+)/ 1033 $_[1] =~ /^([0-9a-fA-F]+)/
957 or $finish->(undef, $ae_error => "Garbled chunked transfer encoding"); 1034 or return $finish->(undef, $ae_error => "Garbled chunked transfer encoding");
958 1035
959 my $len = hex $1; 1036 my $len = hex $1;
960 1037
961 if ($len) { 1038 if ($len) {
962 $cl += $len; 1039 $cl += $len;
… …
1032 } 1109 }
1033 }; 1110 };
1034 1111
1035 # if keepalive is enabled, then the server closing the connection 1112 # if keepalive is enabled, then the server closing the connection
1036 # before a response can happen legally - we retry on idempotent methods. 1113 # before a response can happen legally - we retry on idempotent methods.
1037 if ($keptalive && $idempotent) { 1114 if ($was_persistent && $idempotent) {
1038 my $old_eof = $hdl->{on_eof}; 1115 my $old_eof = $hdl->{on_eof};
1039 $hdl->{on_eof} = sub { 1116 $hdl->{on_eof} = sub {
1040 _destroy_state %state; 1117 _destroy_state %state;
1041 1118
1119 %state = ();
1120 $state{recurse} =
1042 http_request ( 1121 http_request (
1043 $method => $url, 1122 $method => $url,
1044 %arg, 1123 %arg,
1124 recurse => $recurse - 1,
1045 keepalive => 0, 1125 keepalive => 0,
1126 sub {
1127 %state = ();
1046 $cb 1128 &$cb
1129 }
1047 ); 1130 );
1048 }; 1131 };
1049 $hdl->on_read (sub { 1132 $hdl->on_read (sub {
1050 return unless %state; 1133 return unless %state;
1051 1134
1052 # as soon as we receive something, a connection close 1135 # as soon as we receive something, a connection close
… …
1060 }; 1143 };
1061 1144
1062 my $prepare_handle = sub { 1145 my $prepare_handle = sub {
1063 my ($hdl) = $state{handle}; 1146 my ($hdl) = $state{handle};
1064 1147
1065 $hdl->timeout ($timeout);
1066 $hdl->on_error (sub { 1148 $hdl->on_error (sub {
1067 _error %state, $cb, { @pseudo, Status => $ae_error, Reason => $_[2] }; 1149 _error %state, $cb, { @pseudo, Status => $ae_error, Reason => $_[2] };
1068 }); 1150 });
1069 $hdl->on_eof (sub { 1151 $hdl->on_eof (sub {
1070 _error %state, $cb, { @pseudo, Status => $ae_error, Reason => "Unexpected end-of-file" }; 1152 _error %state, $cb, { @pseudo, Status => $ae_error, Reason => "Unexpected end-of-file" };
1071 }); 1153 });
1154 $hdl->timeout_reset;
1155 $hdl->timeout ($timeout);
1072 }; 1156 };
1073 1157
1074 # connected to proxy (or origin server) 1158 # connected to proxy (or origin server)
1075 my $connect_cb = sub { 1159 my $connect_cb = sub {
1076 my $fh = shift 1160 my $fh = shift
… …
1117 1201
1118 return unless $state{connect_guard}; 1202 return unless $state{connect_guard};
1119 1203
1120 # try to use an existing keepalive connection, but only if we, ourselves, plan 1204 # try to use an existing keepalive connection, but only if we, ourselves, plan
1121 # on a keepalive request (in theory, this should be a separate config option). 1205 # on a keepalive request (in theory, this should be a separate config option).
1122 if ($keepalive && $KA_CACHE{$ka_key}) { 1206 if ($persistent && $KA_CACHE{$ka_key}) {
1123 $keptalive = 1; 1207 $was_persistent = 1;
1208
1124 $state{handle} = ka_fetch $ka_key; 1209 $state{handle} = ka_fetch $ka_key;
1210 $state{handle}->destroyed
1211 and die "AnyEvent::HTTP: unexpectedly got a destructed handle (1), please report.";#d#
1125 $prepare_handle->(); 1212 $prepare_handle->();
1213 $state{handle}->destroyed
1214 and die "AnyEvent::HTTP: unexpectedly got a destructed handle (2), please report.";#d#
1126 $handle_actual_request->(); 1215 $handle_actual_request->();
1127 1216
1128 } else { 1217 } else {
1129 my $tcp_connect = $arg{tcp_connect} 1218 my $tcp_connect = $arg{tcp_connect}
1130 || do { require AnyEvent::Socket; \&AnyEvent::Socket::tcp_connect }; 1219 || do { require AnyEvent::Socket; \&AnyEvent::Socket::tcp_connect };
… …
1172Sets the default proxy server to use. The proxy-url must begin with a 1261Sets the default proxy server to use. The proxy-url must begin with a
1173string of the form C<http://host:port>, croaks otherwise. 1262string of the form C<http://host:port>, croaks otherwise.
1174 1263
1175To clear an already-set proxy, use C<undef>. 1264To clear an already-set proxy, use C<undef>.
1176 1265
1266When AnyEvent::HTTP is loaded for the first time it will query the
1267default proxy from the operating system, currently by looking at
1268C<$ENV{http_proxy>}.
1269
1177=item AnyEvent::HTTP::cookie_jar_expire $jar[, $session_end] 1270=item AnyEvent::HTTP::cookie_jar_expire $jar[, $session_end]
1178 1271
1179Remove all cookies from the cookie jar that have been expired. If 1272Remove all cookies from the cookie jar that have been expired. If
1180C<$session_end> is given and true, then additionally remove all session 1273C<$session_end> is given and true, then additionally remove all session
1181cookies. 1274cookies.
1182 1275
1183You should call this function (with a true C<$session_end>) before you 1276You should call this function (with a true C<$session_end>) before you
1184save cookies to disk, and you should call this function after loading them 1277save cookies to disk, and you should call this function after loading them
1185again. If you have a long-running program you can additonally call this 1278again. If you have a long-running program you can additionally call this
1186function from time to time. 1279function from time to time.
1187 1280
1188A cookie jar is initially an empty hash-reference that is managed by this 1281A cookie jar is initially an empty hash-reference that is managed by this
1189module. It's format is subject to change, but currently it is like this: 1282module. It's format is subject to change, but currently it is like this:
1190 1283
1191The key C<version> has to contain C<1>, otherwise the hash gets 1284The key C<version> has to contain C<1>, otherwise the hash gets
1192emptied. All other keys are hostnames or IP addresses pointing to 1285emptied. All other keys are hostnames or IP addresses pointing to
1193hash-references. The key for these inner hash references is the 1286hash-references. The key for these inner hash references is the
1194server path for which this cookie is meant, and the values are again 1287server path for which this cookie is meant, and the values are again
1195hash-references. The keys of those hash-references is the cookie name, and 1288hash-references. Each key of those hash-references is a cookie name, and
1196the value, you guessed it, is another hash-reference, this time with the 1289the value, you guessed it, is another hash-reference, this time with the
1197key-value pairs from the cookie, except for C<expires> and C<max-age>, 1290key-value pairs from the cookie, except for C<expires> and C<max-age>,
1198which have been replaced by a C<_expires> key that contains the cookie 1291which have been replaced by a C<_expires> key that contains the cookie
1199expiry timestamp. 1292expiry timestamp. Session cookies are indicated by not having an
1293C<_expires> key.
1200 1294
1201Here is an example of a cookie jar with a single cookie, so you have a 1295Here is an example of a cookie jar with a single cookie, so you have a
1202chance of understanding the above paragraph: 1296chance of understanding the above paragraph:
1203 1297
1204 { 1298 {
… …
1228 1322
1229The default value for the C<recurse> request parameter (default: C<10>). 1323The default value for the C<recurse> request parameter (default: C<10>).
1230 1324
1231=item $AnyEvent::HTTP::TIMEOUT 1325=item $AnyEvent::HTTP::TIMEOUT
1232 1326
1233The default timeout for conenction operations (default: C<300>). 1327The default timeout for connection operations (default: C<300>).
1234 1328
1235=item $AnyEvent::HTTP::USERAGENT 1329=item $AnyEvent::HTTP::USERAGENT
1236 1330
1237The default value for the C<User-Agent> header (the default is 1331The default value for the C<User-Agent> header (the default is
1238C<Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION; +http://software.schmorp.de/pkg/AnyEvent)>). 1332C<Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION; +http://software.schmorp.de/pkg/AnyEvent)>).
… …
1246 1340
1247The default value for this is C<4>, and it is highly advisable to not 1341The default value for this is C<4>, and it is highly advisable to not
1248increase it much. 1342increase it much.
1249 1343
1250For comparison: the RFC's recommend 4 non-persistent or 2 persistent 1344For comparison: the RFC's recommend 4 non-persistent or 2 persistent
1251connections, older browsers used 2, newers (such as firefox 3) typically 1345connections, older browsers used 2, newer ones (such as firefox 3)
1252use 6, and Opera uses 8 because like, they have the fastest browser and 1346typically use 6, and Opera uses 8 because like, they have the fastest
1253give a shit for everybody else on the planet. 1347browser and give a shit for everybody else on the planet.
1254 1348
1255=item $AnyEvent::HTTP::PERSISTENT_TIMEOUT 1349=item $AnyEvent::HTTP::PERSISTENT_TIMEOUT
1256 1350
1257The time after which idle persistent conenctions get closed by 1351The time after which idle persistent connections get closed by
1258AnyEvent::HTTP (default: C<3>). 1352AnyEvent::HTTP (default: C<3>).
1259 1353
1260=item $AnyEvent::HTTP::ACTIVE 1354=item $AnyEvent::HTTP::ACTIVE
1261 1355
1262The number of active connections. This is not the number of currently 1356The number of active connections. This is not the number of currently
… …
1303 # other formats fail in the loop below 1397 # other formats fail in the loop below
1304 1398
1305 for (0..11) { 1399 for (0..11) {
1306 if ($m eq $month[$_]) { 1400 if ($m eq $month[$_]) {
1307 require Time::Local; 1401 require Time::Local;
1308 return Time::Local::timegm ($S, $M, $H, $d, $_, $y); 1402 return eval { Time::Local::timegm ($S, $M, $H, $d, $_, $y) };
1309 } 1403 }
1310 } 1404 }
1311 1405
1312 undef 1406 undef
1313} 1407}
… …
1327 set_proxy $ENV{http_proxy}; 1421 set_proxy $ENV{http_proxy};
1328}; 1422};
1329 1423
1330=head2 SHOWCASE 1424=head2 SHOWCASE
1331 1425
1332This section contaisn some more elaborate "real-world" examples or code 1426This section contains some more elaborate "real-world" examples or code
1333snippets. 1427snippets.
1334 1428
1335=head2 HTTP/1.1 FILE DOWNLOAD 1429=head2 HTTP/1.1 FILE DOWNLOAD
1336 1430
1337Downloading files with HTTP cna be quite tricky, especially when something 1431Downloading files with HTTP can be quite tricky, especially when something
1338goes wrong and you want tor esume. 1432goes wrong and you want to resume.
1339 1433
1340Here is a function that initiates and resumes a download. It uses the 1434Here is a function that initiates and resumes a download. It uses the
1341last modified time to check for file content changes, and works with many 1435last modified time to check for file content changes, and works with many
1342HTTP/1.0 servers as well, and usually falls back to a complete re-download 1436HTTP/1.0 servers as well, and usually falls back to a complete re-download
1343on older servers. 1437on older servers.
1344 1438
1345It calls the completion callback with either C<undef>, which means a 1439It calls the completion callback with either C<undef>, which means a
1346nonretryable error occured, C<0> when the download was partial and should 1440nonretryable error occurred, C<0> when the download was partial and should
1347be retried, and C<1> if it was successful. 1441be retried, and C<1> if it was successful.
1348 1442
1349 use AnyEvent::HTTP; 1443 use AnyEvent::HTTP;
1350 1444
1351 sub download($$$) { 1445 sub download($$$) {
… …
1359 1453
1360 warn stat $fh; 1454 warn stat $fh;
1361 warn -s _; 1455 warn -s _;
1362 if (stat $fh and -s _) { 1456 if (stat $fh and -s _) {
1363 $ofs = -s _; 1457 $ofs = -s _;
1364 warn "-s is ", $ofs;#d# 1458 warn "-s is ", $ofs;
1365 $hdr{"if-unmodified-since"} = AnyEvent::HTTP::format_date +(stat _)[9]; 1459 $hdr{"if-unmodified-since"} = AnyEvent::HTTP::format_date +(stat _)[9];
1366 $hdr{"range"} = "bytes=$ofs-"; 1460 $hdr{"range"} = "bytes=$ofs-";
1367 } 1461 }
1368 1462
1369 http_get $url, 1463 http_get $url,

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines