ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/AnyEvent-HTTP/HTTP.pm
(Generate patch)

Comparing AnyEvent-HTTP/HTTP.pm (file contents):
Revision 1.129 by root, Thu Aug 30 00:08:16 2018 UTC vs.
Revision 1.134 by root, Fri Sep 7 22:11:31 2018 UTC

46use AnyEvent::Util (); 46use AnyEvent::Util ();
47use AnyEvent::Handle (); 47use AnyEvent::Handle ();
48 48
49use base Exporter::; 49use base Exporter::;
50 50
51our $VERSION = 2.25; 51our $VERSION = 2.24;
52 52
53our @EXPORT = qw(http_get http_post http_head http_request); 53our @EXPORT = qw(http_get http_post http_head http_request);
54 54
55our $USERAGENT = "Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION; +http://software.schmorp.de/pkg/AnyEvent)"; 55our $USERAGENT = "Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION; +http://software.schmorp.de/pkg/AnyEvent)";
56our $MAX_RECURSE = 10; 56our $MAX_RECURSE = 10;
264establishes connections. Normally it uses L<AnyEvent::Socket::tcp_connect> 264establishes connections. Normally it uses L<AnyEvent::Socket::tcp_connect>
265to do this, but you can provide your own C<tcp_connect> function - 265to do this, but you can provide your own C<tcp_connect> function -
266obviously, it has to follow the same calling conventions, except that it 266obviously, it has to follow the same calling conventions, except that it
267may always return a connection guard object. 267may always return a connection guard object.
268 268
269The connections made by this hook will be treated as equivalent to
270connecitons made the built-in way, specifically, they will be put into
271and taken from the persistent conneciton cache. If your C<$tcp_connect>
272function is incompatible with this kind of re-use, consider switching off
273C<persistent> connections and/or providing a C<session> identifier.
274
269There are probably lots of weird uses for this function, starting from 275There are probably lots of weird uses for this function, starting from
270tracing the hosts C<http_request> actually tries to connect, to (inexact 276tracing the hosts C<http_request> actually tries to connect, to (inexact
271but fast) host => IP address caching or even socks protocol support. 277but fast) host => IP address caching or even socks protocol support.
272 278
273=item on_header => $callback->($headers) 279=item on_header => $callback->($headers)
342=item persistent => $boolean 348=item persistent => $boolean
343 349
344Try to create/reuse a persistent connection. When this flag is set 350Try to create/reuse a persistent connection. When this flag is set
345(default: true for idempotent requests, false for all others), then 351(default: true for idempotent requests, false for all others), then
346C<http_request> tries to re-use an existing (previously-created) 352C<http_request> tries to re-use an existing (previously-created)
347persistent connection to the host and, failing that, tries to create a new 353persistent connection to same host (i.e. identical URL scheme, hostname,
348one. 354port and session) and, failing that, tries to create a new one.
349 355
350Requests failing in certain ways will be automatically retried once, which 356Requests failing in certain ways will be automatically retried once, which
351is dangerous for non-idempotent requests, which is why it defaults to off 357is dangerous for non-idempotent requests, which is why it defaults to off
352for them. The reason for this is because the bozos who designed HTTP/1.1 358for them. The reason for this is because the bozos who designed HTTP/1.1
353made it impossible to distinguish between a fatal error and a normal 359made it impossible to distinguish between a fatal error and a normal
454 460
455# expire cookies 461# expire cookies
456sub cookie_jar_expire($;$) { 462sub cookie_jar_expire($;$) {
457 my ($jar, $session_end) = @_; 463 my ($jar, $session_end) = @_;
458 464
459 %$jar = () if $jar->{version} != 1; 465 %$jar = () if $jar->{version} != 2;
460 466
461 my $anow = AE::now; 467 my $anow = AE::now;
462 468
463 while (my ($chost, $paths) = each %$jar) { 469 while (my ($chost, $paths) = each %$jar) {
464 next unless ref $paths; 470 next unless ref $paths;
484 490
485# extract cookies from jar 491# extract cookies from jar
486sub cookie_jar_extract($$$$) { 492sub cookie_jar_extract($$$$) {
487 my ($jar, $scheme, $host, $path) = @_; 493 my ($jar, $scheme, $host, $path) = @_;
488 494
489 %$jar = () if $jar->{version} != 1; 495 %$jar = () if $jar->{version} != 2;
496
497 $host = AnyEvent::Util::idn_to_ascii $host
498 if $host =~ /[^\x00-\x7f]/;
490 499
491 my @cookies; 500 my @cookies;
492 501
493 while (my ($chost, $paths) = each %$jar) { 502 while (my ($chost, $paths) = each %$jar) {
494 next unless ref $paths; 503 next unless ref $paths;
495 504
496 if ($chost =~ /^\./) { 505 # exact match or suffix including . match
497 next unless $chost eq substr $host, -length $chost; 506 $chost eq $host or ".$chost" eq substr $host, -1 - length $chost
498 } elsif ($chost =~ /\./) {
499 next unless $chost eq $host;
500 } else {
501 next; 507 or next;
502 }
503 508
504 while (my ($cpath, $cookies) = each %$paths) { 509 while (my ($cpath, $cookies) = each %$paths) {
505 next unless $cpath eq substr $path, 0, length $cpath; 510 next unless $cpath eq substr $path, 0, length $cpath;
506 511
507 while (my ($cookie, $kv) = each %$cookies) { 512 while (my ($cookie, $kv) = each %$cookies) {
528} 533}
529 534
530# parse set_cookie header into jar 535# parse set_cookie header into jar
531sub cookie_jar_set_cookie($$$$) { 536sub cookie_jar_set_cookie($$$$) {
532 my ($jar, $set_cookie, $host, $date) = @_; 537 my ($jar, $set_cookie, $host, $date) = @_;
538
539 %$jar = () if $jar->{version} != 2;
533 540
534 my $anow = int AE::now; 541 my $anow = int AE::now;
535 my $snow; # server-now 542 my $snow; # server-now
536 543
537 for ($set_cookie) { 544 for ($set_cookie) {
583 590
584 my $cdom; 591 my $cdom;
585 my $cpath = (delete $kv{path}) || "/"; 592 my $cpath = (delete $kv{path}) || "/";
586 593
587 if (exists $kv{domain}) { 594 if (exists $kv{domain}) {
588 $cdom = delete $kv{domain}; 595 $cdom = $kv{domain};
589 596
590 $cdom =~ s/^\.?/./; # make sure it starts with a "." 597 $cdom =~ s/^\.?/./; # make sure it starts with a "."
591 598
592 next if $cdom =~ /\.$/; 599 next if $cdom =~ /\.$/;
593 600
594 # this is not rfc-like and not netscape-like. go figure. 601 # this is not rfc-like and not netscape-like. go figure.
595 my $ndots = $cdom =~ y/.//; 602 my $ndots = $cdom =~ y/.//;
596 next if $ndots < ($cdom =~ /\.[^.][^.]\.[^.][^.]$/ ? 3 : 2); 603 next if $ndots < ($cdom =~ /\.[^.][^.]\.[^.][^.]$/ ? 3 : 2);
604
605 $cdom = substr $cdom, 1; # remove initial .
597 } else { 606 } else {
598 $cdom = $host; 607 $cdom = $host;
599 } 608 }
600 609
601 # store it 610 # store it
602 $jar->{version} = 1; 611 $jar->{version} = 2;
603 $jar->{lc $cdom}{$cpath}{$name} = \%kv; 612 $jar->{lc $cdom}{$cpath}{$name} = \%kv;
604 613
605 redo if /\G\s*,/gc; 614 redo if /\G\s*,/gc;
606 } 615 }
607} 616}
1291function from time to time. 1300function from time to time.
1292 1301
1293A cookie jar is initially an empty hash-reference that is managed by this 1302A cookie jar is initially an empty hash-reference that is managed by this
1294module. Its format is subject to change, but currently it is as follows: 1303module. Its format is subject to change, but currently it is as follows:
1295 1304
1296The key C<version> has to contain C<1>, otherwise the hash gets 1305The key C<version> has to contain C<2>, otherwise the hash gets
1297emptied. All other keys are hostnames or IP addresses pointing to 1306cleared. All other keys are hostnames or IP addresses pointing to
1298hash-references. The key for these inner hash references is the 1307hash-references. The key for these inner hash references is the
1299server path for which this cookie is meant, and the values are again 1308server path for which this cookie is meant, and the values are again
1300hash-references. Each key of those hash-references is a cookie name, and 1309hash-references. Each key of those hash-references is a cookie name, and
1301the value, you guessed it, is another hash-reference, this time with the 1310the value, you guessed it, is another hash-reference, this time with the
1302key-value pairs from the cookie, except for C<expires> and C<max-age>, 1311key-value pairs from the cookie, except for C<expires> and C<max-age>,
1306 1315
1307Here is an example of a cookie jar with a single cookie, so you have a 1316Here is an example of a cookie jar with a single cookie, so you have a
1308chance of understanding the above paragraph: 1317chance of understanding the above paragraph:
1309 1318
1310 { 1319 {
1311 version => 1, 1320 version => 2,
1312 "10.0.0.1" => { 1321 "10.0.0.1" => {
1313 "/" => { 1322 "/" => {
1314 "mythweb_id" => { 1323 "mythweb_id" => {
1315 _expires => 1293917923, 1324 _expires => 1293917923,
1316 value => "ooRung9dThee3ooyXooM1Ohm", 1325 value => "ooRung9dThee3ooyXooM1Ohm",

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines