--- AnyEvent/lib/AnyEvent/Handle.pm 2008/05/31 13:38:01 1.51 +++ AnyEvent/lib/AnyEvent/Handle.pm 2009/02/10 13:58:49 1.115 @@ -1,7 +1,7 @@ package AnyEvent::Handle; no warnings; -use strict; +use strict qw(subs vars); use AnyEvent (); use AnyEvent::Util qw(WSAEWOULDBLOCK); @@ -16,7 +16,7 @@ =cut -our $VERSION = 4.1; +our $VERSION = 4.331; =head1 SYNOPSIS @@ -29,7 +29,7 @@ AnyEvent::Handle->new ( fh => \*STDIN, on_eof => sub { - $cv->broadcast; + $cv->send; }, ); @@ -51,6 +51,9 @@ filehandles. For utility functions for doing non-blocking connects and accepts on sockets see L. +The L tutorial contains some well-documented +AnyEvent::Handle examples. + In the following, when the documentation refers to of "bytes" then this means characters. As sysread and syswrite are used for all I/O, their treatment of characters applies to this module as well. @@ -72,40 +75,58 @@ The filehandle this L object will operate on. -NOTE: The filehandle will be set to non-blocking (using -AnyEvent::Util::fh_nonblocking). +NOTE: The filehandle will be set to non-blocking mode (using +C) by the constructor and needs to stay in +that mode. =item on_eof => $cb->($handle) -Set the callback to be called on EOF. +Set the callback to be called when an end-of-file condition is detected, +i.e. in the case of a socket, when the other side has closed the +connection cleanly. + +For sockets, this just means that the other side has stopped sending data, +you can still try to write data, and, in fact, one can return from the EOF +callback and continue writing data, as only the read part has been shut +down. -While not mandatory, it is highly recommended to set an eof callback, +While not mandatory, it is I recommended to set an EOF callback, otherwise you might end up with a closed socket while you are still waiting for data. -=item on_error => $cb->($handle) +If an EOF condition has been detected but no C callback has been +set, then a fatal error will be raised with C<$!> set to <0>. -This is the fatal error callback, that is called when, well, a fatal error -occurs, such as not being able to resolve the hostname, failure to connect -or a read error. +=item on_error => $cb->($handle, $fatal) -The object will not be in a usable state when this callback has been -called. +This is the error callback, which is called when, well, some error +occured, such as not being able to resolve the hostname, failure to +connect or a read error. + +Some errors are fatal (which is indicated by C<$fatal> being true). On +fatal errors the handle object will be shut down and will not be usable +(but you are free to look at the current C<< ->rbuf >>). Examples of fatal +errors are an EOF condition with active (but unsatisifable) read watchers +(C) or I/O errors. + +Non-fatal errors can be retried by simply returning, but it is recommended +to simply ignore this parameter and instead abondon the handle object +when this callback is invoked. Examples of non-fatal errors are timeouts +C) or badly-formatted data (C). On callback entrance, the value of C<$!> contains the operating system error (or C, C, C or C). -The callback should throw an exception. If it returns, then -AnyEvent::Handle will C for you. - While not mandatory, it is I recommended to set this callback, as you will not be notified of errors otherwise. The default simply calls -die. +C. =item on_read => $cb->($handle) This sets the default read callback, which is called when data arrives -and no read request is in the queue. +and no read request is in the queue (unlike read queue callbacks, this +callback will only be called when at least one octet of data is in the +read buffer). To access (and remove data from) the read buffer, use the C<< ->rbuf >> method or access the C<$handle->{rbuf}> member directly. @@ -122,17 +143,24 @@ To append to the write buffer, use the C<< ->push_write >> method. +This callback is useful when you don't want to put all of your write data +into the queue at once, for example, when you want to write the contents +of some file to the socket you might not want to read the whole file into +memory and push it into the queue, but instead only read more data from +the file when the write queue becomes empty. + =item timeout => $fractional_seconds If non-zero, then this enables an "inactivity" timeout: whenever this many seconds pass without a successful read or write on the underlying file handle, the C callback will be invoked (and if that one is -missing, an C error will be raised). +missing, a non-fatal C error will be raised). Note that timeout processing is also active when you currently do not have any outstanding read or write requests: If you plan to keep the connection idle then you should disable the timout temporarily or ignore the timeout -in the C callback. +in the C callback, in which case AnyEvent::Handle will simply +restart the timeout. Zero (the default) disables this timeout. @@ -146,7 +174,7 @@ If defined, then a fatal error will be raised (with C<$!> set to C) when the read buffer ever (strictly) exceeds this size. This is useful to -avoid denial-of-service attacks. +avoid some forms of denial-of-service attacks. For example, a server accepting connections from untrusted sources should be configured to accept only so-and-so much data that it cannot act on @@ -154,10 +182,37 @@ amount of data without a callback ever being called as long as the line isn't finished). +=item autocork => + +When disabled (the default), then C will try to immediately +write the data to the handle, if possible. This avoids having to register +a write watcher and wait for the next event loop iteration, but can +be inefficient if you write multiple small chunks (on the wire, this +disadvantage is usually avoided by your kernel's nagle algorithm, see +C, but this option can save costly syscalls). + +When enabled, then writes will always be queued till the next event loop +iteration. This is efficient when you do many small writes per iteration, +but less efficient when you do a single write only per iteration (or when +the write buffer often is full). It also increases write latency. + +=item no_delay => + +When doing small writes on sockets, your operating system kernel might +wait a bit for more data before actually sending it out. This is called +the Nagle algorithm, and usually it is beneficial. + +In some situations you want as low a delay as possible, which can be +accomplishd by setting this option to a true value. + +The default is your opertaing system's default behaviour (most likely +enabled), this option explicitly enables or disables it, if possible. + =item read_size => -The default read block size (the amount of bytes this module will try to read -during each (loop iteration). Default: C<8192>. +The default read block size (the amount of bytes this module will +try to read during each loop iteration, which affects memory +requirements). Default: C<8192>. =item low_water_mark => @@ -165,28 +220,53 @@ buffer: If the write reaches this size or gets even samller it is considered empty. +Sometimes it can be beneficial (for performance reasons) to add data to +the write buffer before it is fully drained, but this is a rare case, as +the operating system kernel usually buffers data as well, so the default +is good in almost all cases. + +=item linger => + +If non-zero (default: C<3600>), then the destructor of the +AnyEvent::Handle object will check whether there is still outstanding +write data and will install a watcher that will write this data to the +socket. No errors will be reported (this mostly matches how the operating +system treats outstanding data at socket close time). + +This will not work for partial TLS data that could not be encoded +yet. This data will be lost. Calling the C method in time might +help. + =item tls => "accept" | "connect" | Net::SSLeay::SSL object -When this parameter is given, it enables TLS (SSL) mode, that means it -will start making tls handshake and will transparently encrypt/decrypt -data. +When this parameter is given, it enables TLS (SSL) mode, that means +AnyEvent will start a TLS handshake as soon as the conenction has been +established and will transparently encrypt/decrypt data afterwards. TLS mode requires Net::SSLeay to be installed (it will be loaded -automatically when you try to create a TLS handle). - -For the TLS server side, use C, and for the TLS client side of a -connection, use C mode. +automatically when you try to create a TLS handle): this module doesn't +have a dependency on that module, so if your module requires it, you have +to add the dependency yourself. + +Unlike TCP, TLS has a server and client side: for the TLS server side, use +C, and for the TLS client side of a connection, use C +mode. You can also provide your own TLS connection object, but you have to make sure that you call either C or C on it before you pass it to AnyEvent::Handle. -See the C method if you need to start TLs negotiation later. +B since Net::SSLeay "objects" are really only integers, +passing in the wrong integer will lead to certain crash. This most often +happens when one uses a stylish C<< tls => 1 >> and is surprised about the +segmentation fault. + +See the C<< ->starttls >> method for when need to start TLS negotiation later. =item tls_ctx => $ssl_ctx -Use the given Net::SSLeay::CTX object to create the new TLS connection +Use the given C object to create the new TLS connection (unless a connection object was specified directly). If this parameter is missing, then AnyEvent::Handle will use C. @@ -195,17 +275,12 @@ This is the json coder object used by the C read and write types. If you don't supply it, then AnyEvent::Handle will create and use a -suitable one, which will write and expect UTF-8 encoded JSON texts. +suitable one (on demand), which will write and expect UTF-8 encoded JSON +texts. Note that you are responsible to depend on the JSON module if you want to use this functionality, as AnyEvent does not have a dependency itself. -=item filter_r => $cb - -=item filter_w => $cb - -These exist, but are undocumented at this time. - =back =cut @@ -219,20 +294,17 @@ AnyEvent::Util::fh_nonblocking $self->{fh}, 1; - if ($self->{tls}) { - require Net::SSLeay; - $self->starttls (delete $self->{tls}, delete $self->{tls_ctx}); - } - -# $self->on_eof (delete $self->{on_eof} ) if $self->{on_eof}; # nop -# $self->on_error (delete $self->{on_error}) if $self->{on_error}; # nop -# $self->on_read (delete $self->{on_read} ) if $self->{on_read}; # nop - $self->on_drain (delete $self->{on_drain}) if $self->{on_drain}; + $self->starttls (delete $self->{tls}, delete $self->{tls_ctx}) + if $self->{tls}; $self->{_activity} = AnyEvent->now; $self->_timeout; - $self->start_read; + $self->on_drain (delete $self->{on_drain}) if exists $self->{on_drain}; + $self->no_delay (delete $self->{no_delay}) if exists $self->{no_delay}; + + $self->start_read + if $self->{on_read}; $self } @@ -244,25 +316,31 @@ delete $self->{_rw}; delete $self->{_ww}; delete $self->{fh}; + + &_freetls; + + delete $self->{on_read}; + delete $self->{_queue}; } -sub error { - my ($self) = @_; +sub _error { + my ($self, $errno, $fatal) = @_; - { - local $!; - $self->_shutdown; - } + $self->_shutdown + if $fatal; - $self->{on_error}($self) - if $self->{on_error}; + $! = $errno; - Carp::croak "AnyEvent::Handle uncaught fatal error: $!"; + if ($self->{on_error}) { + $self->{on_error}($self, $fatal); + } elsif ($self->{fh}) { + Carp::croak "AnyEvent::Handle uncaught error: $!"; + } } =item $fh = $handle->fh -This method returns the file handle of the L object. +This method returns the file handle used to create the L object. =cut @@ -290,9 +368,9 @@ =item $handle->on_timeout ($cb) -Replace the current C callback, or disables the callback -(but not the timeout) if C<$cb> = C. See C constructor -argument. +Replace the current C callback, or disables the callback (but +not the timeout) if C<$cb> = C. See the C constructor +argument and method. =cut @@ -300,6 +378,33 @@ $_[0]{on_timeout} = $_[1]; } +=item $handle->autocork ($boolean) + +Enables or disables the current autocork behaviour (see C +constructor argument). Changes will only take effect on the next write. + +=cut + +sub autocork { + $_[0]{autocork} = $_[1]; +} + +=item $handle->no_delay ($boolean) + +Enables or disables the C setting (see constructor argument of +the same name for details). + +=cut + +sub no_delay { + $_[0]{no_delay} = $_[1]; + + eval { + local $SIG{__DIE__}; + setsockopt $_[0]{fh}, &Socket::IPPROTO_TCP, &Socket::TCP_NODELAY, int $_[1]; + }; +} + ############################################################################# =item $handle->timeout ($seconds) @@ -333,11 +438,10 @@ if ($self->{on_timeout}) { $self->{on_timeout}($self); } else { - $! = Errno::ETIMEDOUT; - $self->error; + $self->_error (&Errno::ETIMEDOUT); } - # callbakx could have changed timeout value, optimise + # callback could have changed timeout value, optimise return unless $self->{timeout}; # calculate new after @@ -345,6 +449,7 @@ } Scalar::Util::weaken $self; + return unless $self; # ->error could have destroyed $self $self->{_tw} ||= AnyEvent->timer (after => $after, cb => sub { delete $self->{_tw}; @@ -385,7 +490,7 @@ $self->{on_drain} = $cb; $cb->($self) - if $cb && $self->{low_water_mark} >= length $self->{wbuf}; + if $cb && $self->{low_water_mark} >= (length $self->{wbuf}) + (length $self->{_tls_wbuf}); } =item $handle->push_write ($data) @@ -412,17 +517,17 @@ $self->{_activity} = AnyEvent->now; $self->{on_drain}($self) - if $self->{low_water_mark} >= length $self->{wbuf} + if $self->{low_water_mark} >= (length $self->{wbuf}) + (length $self->{_tls_wbuf}) && $self->{on_drain}; delete $self->{_ww} unless length $self->{wbuf}; } elsif ($! != EAGAIN && $! != EINTR && $! != WSAEWOULDBLOCK) { - $self->error; + $self->_error ($!, 1); } }; # try to write data immediately - $cb->(); + $cb->() unless $self->{autocork}; # if still data left in wbuf, we need to poll $self->{_ww} = AnyEvent->io (fh => $self->{fh}, poll => "w", cb => $cb) @@ -446,8 +551,10 @@ ->($self, @_); } - if ($self->{filter_w}) { - $self->{filter_w}($self, \$_[0]); + if ($self->{tls}) { + $self->{_tls_wbuf} .= $_[0]; + + &_dotls ($self); } else { $self->{wbuf} .= $_[0]; $self->_drain_wbuf; @@ -456,8 +563,6 @@ =item $handle->push_write (type => @args) -=item $handle->unshift_write (type => @args) - Instead of formatting your data yourself, you can also let this module do the job by specifying a type and type-specific arguments. @@ -471,14 +576,27 @@ Formats the given value as netstring (http://cr.yp.to/proto/netstrings.txt, this is not a recommendation to use them). -=back - =cut register_write_type netstring => sub { my ($self, $string) = @_; - sprintf "%d:%s,", (length $string), $string + (length $string) . ":$string," +}; + +=item packstring => $format, $data + +An octet string prefixed with an encoded length. The encoding C<$format> +uses the same format as a Perl C format, but must specify a single +integer only (only one of C is allowed, plus an +optional C, C<< < >> or C<< > >> modifier). + +=cut + +register_write_type packstring => sub { + my ($self, $format, $string) = @_; + + pack "$format/a*", $string }; =item json => $array_or_hashref @@ -522,6 +640,23 @@ : JSON::encode_json ($ref) }; +=item storable => $reference + +Freezes the given reference using L and writes it to the +handle. Uses the C format. + +=cut + +register_write_type storable => sub { + my ($self, $ref) = @_; + + require Storable; + + pack "w/a*", Storable::nfreeze ($ref) +}; + +=back + =item AnyEvent::Handle::register_write_type type => $coderef->($handle, @args) This function (not method) lets you add your own types to C. @@ -551,13 +686,14 @@ In the simple case, you just install an C callback and whenever new data arrives, it will be called. You can then remove some data (if -enough is there) from the read buffer (C<< $handle->rbuf >>) if you want -or not. +enough is there) from the read buffer (C<< $handle->rbuf >>). Or you cna +leave the data there if you want to accumulate more (e.g. when only a +partial message has been received so far). In the more complex case, you want to queue multiple callbacks. In this case, AnyEvent::Handle will call the first queued callback each time new -data arrives and removes it when it has done its job (see C, -below). +data arrives (also the first time it is queued) and removes it when it has +done its job (see C, below). This way you can, for example, push three line-reads, followed by reading a chunk of data, and AnyEvent::Handle will execute them in order. @@ -568,46 +704,50 @@ # in the default state, expect some header bytes $handle->on_read (sub { # some data is here, now queue the length-header-read (4 octets) - shift->unshift_read_chunk (4, sub { + shift->unshift_read (chunk => 4, sub { # header arrived, decode my $len = unpack "N", $_[1]; # now read the payload - shift->unshift_read_chunk ($len, sub { + shift->unshift_read (chunk => $len, sub { my $xml = $_[1]; # handle xml }); }); }); -Example 2: Implement a client for a protocol that replies either with -"OK" and another line or "ERROR" for one request, and 64 bytes for the -second request. Due tot he availability of a full queue, we can just -pipeline sending both requests and manipulate the queue as necessary in -the callbacks: +Example 2: Implement a client for a protocol that replies either with "OK" +and another line or "ERROR" for the first request that is sent, and 64 +bytes for the second request. Due to the availability of a queue, we can +just pipeline sending both requests and manipulate the queue as necessary +in the callbacks. + +When the first callback is called and sees an "OK" response, it will +C another line-read. This line-read will be queued I the +64-byte chunk callback. - # request one + # request one, returns either "OK + extra line" or "ERROR" $handle->push_write ("request 1\015\012"); # we expect "ERROR" or "OK" as response, so push a line read - $handle->push_read_line (sub { + $handle->push_read (line => sub { # if we got an "OK", we have to _prepend_ another line, # so it will be read before the second request reads its 64 bytes # which are already in the queue when this callback is called # we don't do this in case we got an error if ($_[1] eq "OK") { - $_[0]->unshift_read_line (sub { + $_[0]->unshift_read (line => sub { my $response = $_[1]; ... }); } }); - # request two + # request two, simply returns 64 octets $handle->push_write ("request 2\015\012"); # simply read 64 bytes, always - $handle->push_read_chunk (64, sub { + $handle->push_read (chunk => 64, sub { my $response = $_[1]; ... }); @@ -619,52 +759,67 @@ sub _drain_rbuf { my ($self) = @_; + local $self->{_in_drain} = 1; + if ( defined $self->{rbuf_max} && $self->{rbuf_max} < length $self->{rbuf} ) { - $! = &Errno::ENOSPC; - $self->error; + $self->_error (&Errno::ENOSPC, 1), return; } - return if $self->{in_drain}; - local $self->{in_drain} = 1; + while () { + $self->{rbuf} .= delete $self->{tls_rbuf} if exists $self->{tls_rbuf};#d# + + my $len = length $self->{rbuf}; - while (my $len = length $self->{rbuf}) { - no strict 'refs'; if (my $cb = shift @{ $self->{_queue} }) { unless ($cb->($self)) { if ($self->{_eof}) { # no progress can be made (not enough data and no data forthcoming) - $! = &Errno::EPIPE; - $self->error; + $self->_error (&Errno::EPIPE, 1), return; } unshift @{ $self->{_queue} }, $cb; - return; + last; } } elsif ($self->{on_read}) { + last unless $len; + $self->{on_read}($self); if ( - $self->{_eof} # if no further data will arrive - && $len == length $self->{rbuf} # and no data has been consumed - && !@{ $self->{_queue} } # and the queue is still empty - && $self->{on_read} # and we still want to read data + $len == length $self->{rbuf} # if no data has been consumed + && !@{ $self->{_queue} } # and the queue is still empty + && $self->{on_read} # but we still have on_read ) { - # then no progress can be made - $! = &Errno::EPIPE; - $self->error; + # no further data will arrive + # so no progress can be made + $self->_error (&Errno::EPIPE, 1), return + if $self->{_eof}; + + last; # more data might arrive } } else { # read side becomes idle - delete $self->{_rw}; - return; + delete $self->{_rw} unless $self->{tls}; + last; } } - $self->{on_eof}($self) - if $self->{_eof} && $self->{on_eof}; + if ($self->{_eof}) { + if ($self->{on_eof}) { + $self->{on_eof}($self) + } else { + $self->_error (0, 1); + } + } + + # may need to restart read watcher + unless ($self->{_rw}) { + $self->start_read + if $self->{on_read} || @{ $self->{_queue} }; + } } =item $handle->on_read ($cb) @@ -679,6 +834,7 @@ my ($self, $cb) = @_; $self->{on_read} = $cb; + $self->_drain_rbuf if $cb && !$self->{_in_drain}; } =item $handle->rbuf @@ -737,7 +893,7 @@ } push @{ $self->{_queue} }, $cb; - $self->_drain_rbuf; + $self->_drain_rbuf unless $self->{_in_drain}; } sub unshift_read { @@ -753,7 +909,7 @@ unshift @{ $self->{_queue} }, $cb; - $self->_drain_rbuf; + $self->_drain_rbuf unless $self->{_in_drain}; } =item $handle->push_read (type => @args, $cb) @@ -793,15 +949,6 @@ } }; -# compatibility with older API -sub push_read_chunk { - $_[0]->push_read (chunk => $_[1], $_[2]); -} - -sub unshift_read_chunk { - $_[0]->unshift_read (chunk => $_[1], $_[2]); -} - =item line => [$eol, ]$cb->($handle, $line, $eol) The callback will be called only once a full line (including the end of @@ -826,64 +973,24 @@ register_read_type line => sub { my ($self, $cb, $eol) = @_; - $eol = qr|(\015?\012)| if @_ < 3; - $eol = quotemeta $eol unless ref $eol; - $eol = qr|^(.*?)($eol)|s; - - sub { - $_[0]{rbuf} =~ s/$eol// or return; - - $cb->($_[0], $1, $2); - 1 - } -}; - -# compatibility with older API -sub push_read_line { - my $self = shift; - $self->push_read (line => @_); -} - -sub unshift_read_line { - my $self = shift; - $self->unshift_read (line => @_); -} - -=item netstring => $cb->($handle, $string) - -A netstring (http://cr.yp.to/proto/netstrings.txt, this is not an endorsement). - -Throws an error with C<$!> set to EBADMSG on format violations. - -=cut - -register_read_type netstring => sub { - my ($self, $cb) = @_; + if (@_ < 3) { + # this is more than twice as fast as the generic code below + sub { + $_[0]{rbuf} =~ s/^([^\015\012]*)(\015?\012)// or return; - sub { - unless ($_[0]{rbuf} =~ s/^(0|[1-9][0-9]*)://) { - if ($_[0]{rbuf} =~ /[^0-9]/) { - $! = &Errno::EBADMSG; - $self->error; - } - return; + $cb->($_[0], $1, $2); + 1 } + } else { + $eol = quotemeta $eol unless ref $eol; + $eol = qr|^(.*?)($eol)|s; - my $len = $1; + sub { + $_[0]{rbuf} =~ s/$eol// or return; - $self->unshift_read (chunk => $len, sub { - my $string = $_[1]; - $_[0]->unshift_read (chunk => 1, sub { - if ($_[1] eq ",") { - $cb->($_[0], $string); - } else { - $! = &Errno::EBADMSG; - $self->error; - } - }); - }); - - 1 + $cb->($_[0], $1, $2); + 1 + } } }; @@ -945,8 +1052,7 @@ # reject if ($reject && $$rbuf =~ $reject) { - $! = &Errno::EBADMSG; - $self->error; + $self->_error (&Errno::EBADMSG); } # skip @@ -958,9 +1064,92 @@ } }; +=item netstring => $cb->($handle, $string) + +A netstring (http://cr.yp.to/proto/netstrings.txt, this is not an endorsement). + +Throws an error with C<$!> set to EBADMSG on format violations. + +=cut + +register_read_type netstring => sub { + my ($self, $cb) = @_; + + sub { + unless ($_[0]{rbuf} =~ s/^(0|[1-9][0-9]*)://) { + if ($_[0]{rbuf} =~ /[^0-9]/) { + $self->_error (&Errno::EBADMSG); + } + return; + } + + my $len = $1; + + $self->unshift_read (chunk => $len, sub { + my $string = $_[1]; + $_[0]->unshift_read (chunk => 1, sub { + if ($_[1] eq ",") { + $cb->($_[0], $string); + } else { + $self->_error (&Errno::EBADMSG); + } + }); + }); + + 1 + } +}; + +=item packstring => $format, $cb->($handle, $string) + +An octet string prefixed with an encoded length. The encoding C<$format> +uses the same format as a Perl C format, but must specify a single +integer only (only one of C is allowed, plus an +optional C, C<< < >> or C<< > >> modifier). + +For example, DNS over TCP uses a prefix of C (2 octet network order), +EPP uses a prefix of C (4 octtes). + +Example: read a block of data prefixed by its length in BER-encoded +format (very efficient). + + $handle->push_read (packstring => "w", sub { + my ($handle, $data) = @_; + }); + +=cut + +register_read_type packstring => sub { + my ($self, $cb, $format) = @_; + + sub { + # when we can use 5.10 we can use ".", but for 5.8 we use the re-pack method + defined (my $len = eval { unpack $format, $_[0]{rbuf} }) + or return; + + $format = length pack $format, $len; + + # bypass unshift if we already have the remaining chunk + if ($format + $len <= length $_[0]{rbuf}) { + my $data = substr $_[0]{rbuf}, $format, $len; + substr $_[0]{rbuf}, 0, $format + $len, ""; + $cb->($_[0], $data); + } else { + # remove prefix + substr $_[0]{rbuf}, 0, $format, ""; + + # read remaining chunk + $_[0]->unshift_read (chunk => $len, $cb); + } + + 1 + } +}; + =item json => $cb->($handle, $hash_or_arrayref) -Reads a JSON object or array, decodes it and passes it to the callback. +Reads a JSON object or array, decodes it and passes it to the +callback. When a parse error occurs, an C error will be raised. If a C object was passed to the constructor, then that will be used for the final decode, otherwise it will create a JSON coder expecting UTF-8. @@ -977,7 +1166,7 @@ =cut register_read_type json => sub { - my ($self, $cb, $accept, $reject, $skip) = @_; + my ($self, $cb) = @_; require JSON; @@ -987,7 +1176,7 @@ my $json = $self->{json} ||= JSON->new->utf8; sub { - my $ref = $json->incr_parse ($self->{rbuf}); + my $ref = eval { $json->incr_parse ($self->{rbuf}) }; if ($ref) { $self->{rbuf} = $json->incr_text; @@ -995,13 +1184,69 @@ $cb->($self, $ref); 1 + } elsif ($@) { + # error case + $json->incr_skip; + + $self->{rbuf} = $json->incr_text; + $json->incr_text = ""; + + $self->_error (&Errno::EBADMSG); + + () } else { $self->{rbuf} = ""; + () } } }; +=item storable => $cb->($handle, $ref) + +Deserialises a L frozen representation as written by the +C write type (BER-encoded length prefix followed by nfreeze'd +data). + +Raises C error if the data could not be decoded. + +=cut + +register_read_type storable => sub { + my ($self, $cb) = @_; + + require Storable; + + sub { + # when we can use 5.10 we can use ".", but for 5.8 we use the re-pack method + defined (my $len = eval { unpack "w", $_[0]{rbuf} }) + or return; + + my $format = length pack "w", $len; + + # bypass unshift if we already have the remaining chunk + if ($format + $len <= length $_[0]{rbuf}) { + my $data = substr $_[0]{rbuf}, $format, $len; + substr $_[0]{rbuf}, 0, $format + $len, ""; + $cb->($_[0], Storable::thaw ($data)); + } else { + # remove prefix + substr $_[0]{rbuf}, 0, $format, ""; + + # read remaining chunk + $_[0]->unshift_read (chunk => $len, sub { + if (my $ref = eval { Storable::thaw ($_[1]) }) { + $cb->($_[0], $ref); + } else { + $self->_error (&Errno::EBADMSG); + } + }); + } + + 1 + } +}; + =back =item AnyEvent::Handle::register_read_type type => $coderef->($handle, $cb, @args) @@ -1029,16 +1274,24 @@ =item $handle->start_read In rare cases you actually do not want to read anything from the -socket. In this case you can call C. Neither C no +socket. In this case you can call C. Neither C nor any queued callbacks will be executed then. To start reading again, call C. +Note that AnyEvent::Handle will automatically C for you when +you change the C callback or push/unshift a read callback, and it +will automatically C for you when neither C is set nor +there are any read requests in the queue. + +These methods will have no effect when in TLS mode (as TLS doesn't support +half-duplex connections). + =cut sub stop_read { my ($self) = @_; - delete $self->{_rw}; + delete $self->{_rw} unless $self->{tls}; } sub start_read { @@ -1048,58 +1301,72 @@ Scalar::Util::weaken $self; $self->{_rw} = AnyEvent->io (fh => $self->{fh}, poll => "r", cb => sub { - my $rbuf = $self->{filter_r} ? \my $buf : \$self->{rbuf}; + my $rbuf = \($self->{tls} ? my $buf : $self->{rbuf}); my $len = sysread $self->{fh}, $$rbuf, $self->{read_size} || 8192, length $$rbuf; if ($len > 0) { $self->{_activity} = AnyEvent->now; - $self->{filter_r} - ? $self->{filter_r}($self, $rbuf) - : $self->_drain_rbuf; + if ($self->{tls}) { + Net::SSLeay::BIO_write ($self->{_rbio}, $$rbuf); + + &_dotls ($self); + } else { + $self->_drain_rbuf unless $self->{_in_drain}; + } } elsif (defined $len) { delete $self->{_rw}; $self->{_eof} = 1; - $self->_drain_rbuf; + $self->_drain_rbuf unless $self->{_in_drain}; } elsif ($! != EAGAIN && $! != EINTR && $! != WSAEWOULDBLOCK) { - return $self->error; + return $self->_error ($!, 1); } }); } } +# poll the write BIO and send the data if applicable sub _dotls { my ($self) = @_; + my $tmp; + if (length $self->{_tls_wbuf}) { - while ((my $len = Net::SSLeay::write ($self->{tls}, $self->{_tls_wbuf})) > 0) { - substr $self->{_tls_wbuf}, 0, $len, ""; + while (($tmp = Net::SSLeay::write ($self->{tls}, $self->{_tls_wbuf})) > 0) { + substr $self->{_tls_wbuf}, 0, $tmp, ""; } } - if (defined (my $buf = Net::SSLeay::BIO_read ($self->{_wbio}))) { - $self->{wbuf} .= $buf; - $self->_drain_wbuf; - } + while (defined ($tmp = Net::SSLeay::read ($self->{tls}))) { + unless (length $tmp) { + # let's treat SSL-eof as we treat normal EOF + delete $self->{_rw}; + $self->{_eof} = 1; + &_freetls; + } - while (defined (my $buf = Net::SSLeay::read ($self->{tls}))) { - $self->{rbuf} .= $buf; - $self->_drain_rbuf; + $self->{tls_rbuf} .= $tmp;#d# + $self->_drain_rbuf unless $self->{_in_drain}; + $self->{tls} or return; # tls session might have gone away in callback } - my $err = Net::SSLeay::get_error ($self->{tls}, -1); - - if ($err!= Net::SSLeay::ERROR_WANT_READ ()) { - if ($err == Net::SSLeay::ERROR_SYSCALL ()) { - $self->error; - } elsif ($err == Net::SSLeay::ERROR_SSL ()) { - $! = &Errno::EIO; - $self->error; + $tmp = Net::SSLeay::get_error ($self->{tls}, -1); + + if ($tmp != Net::SSLeay::ERROR_WANT_READ ()) { + if ($tmp == Net::SSLeay::ERROR_SYSCALL ()) { + return $self->_error ($!, 1); + } elsif ($tmp == Net::SSLeay::ERROR_SSL ()) { + return $self->_error (&Errno::EIO, 1); } - # all others are fine for our purposes + # all other errors are fine for our purposes + } + + while (length ($tmp = Net::SSLeay::BIO_read ($self->{_wbio}))) { + $self->{wbuf} .= $tmp; + $self->_drain_wbuf; } } @@ -1119,13 +1386,19 @@ call and can be used or changed to your liking. Note that the handshake might have already started when this function returns. +If it an error to start a TLS handshake more than once per +AnyEvent::Handle object (this is due to bugs in OpenSSL). + =cut sub starttls { my ($self, $ssl, $ctx) = @_; - $self->stoptls; + require Net::SSLeay; + Carp::croak "it is an error to call starttls more than once on an AnyEvent::Handle object" + if $self->{tls}; + if ($ssl eq "accept") { $ssl = Net::SSLeay::new ($ctx || TLS_CTX ()); Net::SSLeay::set_accept_state ($ssl); @@ -1141,6 +1414,13 @@ # (unfortunately, we have to hardcode constants because the abysmally misdesigned # and mismaintained ssleay-module doesn't even offer them). # http://www.mail-archive.com/openssl-dev@openssl.org/msg22420.html + # + # in short: this is a mess. + # + # note that we do not try to keep the length constant between writes as we are required to do. + # we assume that most (but not all) of this insanity only applies to non-blocking cases, + # and we drive openssl fully in blocking mode here. Or maybe we don't - openssl seems to + # have identity issues in that area. Net::SSLeay::CTX_set_mode ($self->{tls}, (eval { local $SIG{__DIE__}; Net::SSLeay::MODE_ENABLE_PARTIAL_WRITE () } || 1) | (eval { local $SIG{__DIE__}; Net::SSLeay::MODE_ACCEPT_MOVING_WRITE_BUFFER () } || 2)); @@ -1150,39 +1430,94 @@ Net::SSLeay::set_bio ($ssl, $self->{_rbio}, $self->{_wbio}); - $self->{filter_w} = sub { - $_[0]{_tls_wbuf} .= ${$_[1]}; - &_dotls; - }; - $self->{filter_r} = sub { - Net::SSLeay::BIO_write ($_[0]{_rbio}, ${$_[1]}); - &_dotls; - }; + &_dotls; # need to trigger the initial handshake + $self->start_read; # make sure we actually do read } =item $handle->stoptls -Destroys the SSL connection, if any. Partial read or write data will be -lost. +Shuts down the SSL connection - this makes a proper EOF handshake by +sending a close notify to the other side, but since OpenSSL doesn't +support non-blocking shut downs, it is not possible to re-use the stream +afterwards. =cut sub stoptls { my ($self) = @_; - Net::SSLeay::free (delete $self->{tls}) if $self->{tls}; + if ($self->{tls}) { + Net::SSLeay::shutdown ($self->{tls}); + + &_dotls; + + # we don't give a shit. no, we do, but we can't. no... + # we, we... have to use openssl :/ + &_freetls; + } +} + +sub _freetls { + my ($self) = @_; + + return unless $self->{tls}; - delete $self->{_rbio}; - delete $self->{_wbio}; - delete $self->{_tls_wbuf}; - delete $self->{filter_r}; - delete $self->{filter_w}; + Net::SSLeay::free (delete $self->{tls}); + + delete @$self{qw(_rbio _wbio _tls_wbuf)}; } sub DESTROY { my $self = shift; - $self->stoptls; + &_freetls; + + my $linger = exists $self->{linger} ? $self->{linger} : 3600; + + if ($linger && length $self->{wbuf}) { + my $fh = delete $self->{fh}; + my $wbuf = delete $self->{wbuf}; + + my @linger; + + push @linger, AnyEvent->io (fh => $fh, poll => "w", cb => sub { + my $len = syswrite $fh, $wbuf, length $wbuf; + + if ($len > 0) { + substr $wbuf, 0, $len, ""; + } else { + @linger = (); # end + } + }); + push @linger, AnyEvent->timer (after => $linger, cb => sub { + @linger = (); + }); + } +} + +=item $handle->destroy + +Shuts down the handle object as much as possible - this call ensures that +no further callbacks will be invoked and resources will be freed as much +as possible. You must not call any methods on the object afterwards. + +Normally, you can just "forget" any references to an AnyEvent::Handle +object and it will simply shut down. This works in fatal error and EOF +callbacks, as well as code outside. It does I work in a read or write +callback, so when you want to destroy the AnyEvent::Handle object from +within such an callback. You I call C<< ->destroy >> explicitly in +that case. + +The handle might still linger in the background and write out remaining +data, as specified by the C option, however. + +=cut + +sub destroy { + my ($self) = @_; + + $self->DESTROY; + %$self = (); } =item AnyEvent::Handle::TLS_CTX @@ -1222,6 +1557,78 @@ =back + +=head1 NONFREQUENTLY ASKED QUESTIONS + +=over 4 + +=item I C the AnyEvent::Handle reference inside my callback and +still get further invocations! + +That's because AnyEvent::Handle keeps a reference to itself when handling +read or write callbacks. + +It is only safe to "forget" the reference inside EOF or error callbacks, +from within all other callbacks, you need to explicitly call the C<< +->destroy >> method. + +=item I get different callback invocations in TLS mode/Why can't I pause +reading? + +Unlike, say, TCP, TLS connections do not consist of two independent +communication channels, one for each direction. Or put differently. The +read and write directions are not independent of each other: you cannot +write data unless you are also prepared to read, and vice versa. + +This can mean than, in TLS mode, you might get C or C +callback invocations when you are not expecting any read data - the reason +is that AnyEvent::Handle always reads in TLS mode. + +During the connection, you have to make sure that you always have a +non-empty read-queue, or an C watcher. At the end of the +connection (or when you no longer want to use it) you can call the +C method. + +=item How do I read data until the other side closes the connection? + +If you just want to read your data into a perl scalar, the easiest way +to achieve this is by setting an C callback that does nothing, +clearing the C callback and in the C callback, the data +will be in C<$_[0]{rbuf}>: + + $handle->on_read (sub { }); + $handle->on_eof (undef); + $handle->on_error (sub { + my $data = delete $_[0]{rbuf}; + undef $handle; + }); + +The reason to use C is that TCP connections, due to latencies +and packets loss, might get closed quite violently with an error, when in +fact, all data has been received. + +It is usually better to use acknowledgements when transferring data, +to make sure the other side hasn't just died and you got the data +intact. This is also one reason why so many internet protocols have an +explicit QUIT command. + +=item I don't want to destroy the handle too early - how do I wait until +all data has been written? + +After writing your last bits of data, set the C callback +and destroy the handle in there - with the default setting of +C this will be called precisely when all data has been +written to the socket: + + $handle->push_write (...); + $handle->on_drain (sub { + warn "all data submitted to the kernel\n"; + undef $handle; + }); + +=back + + =head1 SUBCLASSING AnyEvent::Handle In many cases, you might want to subclass AnyEvent::Handle. @@ -1234,7 +1641,7 @@ =item * all constructor arguments become object members. At least initially, when you pass a C-argument to the constructor it -will end up in C<< $handle->{tls} >>. Those members might be changes or +will end up in C<< $handle->{tls} >>. Those members might be changed or mutated later on (for example C will hold the TLS connection object). =item * other object member names are prefixed with an C<_>.