--- AnyEvent/lib/AnyEvent/Handle.pm 2008/10/01 08:52:06 1.92 +++ AnyEvent/lib/AnyEvent/Handle.pm 2009/07/26 00:17:25 1.162 @@ -1,22 +1,19 @@ package AnyEvent::Handle; -no warnings; -use strict qw(subs vars); - -use AnyEvent (); -use AnyEvent::Util qw(WSAEWOULDBLOCK); use Scalar::Util (); use Carp (); -use Fcntl (); use Errno qw(EAGAIN EINTR); +use AnyEvent (); BEGIN { AnyEvent::common_sense } +use AnyEvent::Util qw(WSAEWOULDBLOCK); + =head1 NAME AnyEvent::Handle - non-blocking I/O on file handles via AnyEvent =cut -our $VERSION = 4.3; +our $VERSION = 4.87; =head1 SYNOPSIS @@ -25,21 +22,22 @@ my $cv = AnyEvent->condvar; - my $handle = - AnyEvent::Handle->new ( - fh => \*STDIN, - on_eof => sub { - $cv->broadcast; - }, + my $hdl; $hdl = new AnyEvent::Handle + fh => \*STDIN, + on_error => sub { + my ($hdl, $fatal, $msg) = @_; + warn "got error $msg\n"; + $hdl->destroy; + $cv->send; ); # send some request line - $handle->push_write ("getinfo\015\012"); + $hdl->push_write ("getinfo\015\012"); # read the response line - $handle->push_read (line => sub { - my ($handle, $line) = @_; - warn "read line <$line>\n"; + $hdl->push_read (line => sub { + my ($hdl, $line) = @_; + warn "got line <$line>\n"; $cv->send; }); @@ -48,8 +46,7 @@ =head1 DESCRIPTION This module is a helper module to make it easier to do event-based I/O on -filehandles. For utility functions for doing non-blocking connects and accepts -on sockets see L. +filehandles. The L tutorial contains some well-documented AnyEvent::Handle examples. @@ -58,64 +55,102 @@ means characters. As sysread and syswrite are used for all I/O, their treatment of characters applies to this module as well. +At the very minimum, you should specify C or C, and the +C callback. + All callbacks will be invoked with the handle object as their first argument. -=head2 SIGPIPE is not handled by this module - -SIGPIPE is not handled by this module, so one of the practical -requirements of using it is to ignore SIGPIPE (C<$SIG{PIPE} = -'IGNORE'>). At least, this is highly recommend in a networked program: If -you use AnyEvent::Handle in a filter program (like sort), exiting on -SIGPIPE is probably the right thing to do. - =head1 METHODS =over 4 -=item B +=item $handle = B AnyEvent::TLS fh => $filehandle, key => value... -The constructor supports these arguments (all as key => value pairs). +The constructor supports these arguments (all as C<< key => value >> pairs). =over 4 -=item fh => $filehandle [MANDATORY] +=item fh => $filehandle [C or C MANDATORY] The filehandle this L object will operate on. - NOTE: The filehandle will be set to non-blocking mode (using C) by the constructor and needs to stay in that mode. -=item on_eof => $cb->($handle) +=item connect => [$host, $service] [C or C MANDATORY] -Set the callback to be called when an end-of-file condition is detected, -i.e. in the case of a socket, when the other side has closed the -connection cleanly. +Try to connect to the specified host and service (port), using +C. The C<$host> additionally becomes the +default C. -For sockets, this just means that the other side has stopped sending data, -you can still try to write data, and, in fact, one can return from the eof -callback and continue writing data, as only the read part has been shut -down. +You have to specify either this parameter, or C, above. -While not mandatory, it is I recommended to set an eof callback, -otherwise you might end up with a closed socket while you are still -waiting for data. +It is possible to push requests on the read and write queues, and modify +properties of the stream, even while AnyEvent::Handle is connecting. -If an EOF condition has been detected but no C callback has been -set, then a fatal error will be raised with C<$!> set to <0>. +When this parameter is specified, then the C, +C and C callbacks will be called under the +appropriate circumstances: + +=over 4 + +=item on_prepare => $cb->($handle) + +This (rarely used) callback is called before a new connection is +attempted, but after the file handle has been created. It could be used to +prepare the file handle with parameters required for the actual connect +(as opposed to settings that can be changed when the connection is already +established). + +The return value of this callback should be the connect timeout value in +seconds (or C<0>, or C, or the empty list, to indicate the default +timeout is to be used). + +=item on_connect => $cb->($handle, $host, $port, $retry->()) + +This callback is called when a connection has been successfully established. + +The actual numeric host and port (the socket peername) are passed as +parameters, together with a retry callback. + +When, for some reason, the handle is not acceptable, then calling +C<$retry> will continue with the next conenction target (in case of +multi-homed hosts or SRV records there can be multiple connection +endpoints). When it is called then the read and write queues, eof status, +tls status and similar properties of the handle are being reset. -=item on_error => $cb->($handle, $fatal) +In most cases, ignoring the C<$retry> parameter is the way to go. + +=item on_connect_error => $cb->($handle, $message) + +This callback is called when the conenction could not be +established. C<$!> will contain the relevant error code, and C<$message> a +message describing it (usually the same as C<"$!">). + +If this callback isn't specified, then C will be called with a +fatal error instead. + +=back + +=item on_error => $cb->($handle, $fatal, $message) This is the error callback, which is called when, well, some error occured, such as not being able to resolve the hostname, failure to connect or a read error. Some errors are fatal (which is indicated by C<$fatal> being true). On -fatal errors the handle object will be shut down and will not be usable -(but you are free to look at the current C<< ->rbuf >>). Examples of fatal -errors are an EOF condition with active (but unsatisifable) read watchers -(C) or I/O errors. +fatal errors the handle object will be destroyed (by a call to C<< -> +destroy >>) after invoking the error callback (which means you are free to +examine the handle object). Examples of fatal errors are an EOF condition +with active (but unsatisifable) read watchers (C) or I/O errors. In +cases where the other side can close the connection at their will it is +often easiest to not report C errors in this callback. + +AnyEvent::Handle tries to find an appropriate error code for you to check +against, but in some cases (TLS errors), this does not work well. It is +recommended to always output the C<$message> argument in human-readable +error messages (it's usually the same as C<"$!">). Non-fatal errors can be retried by simply returning, but it is recommended to simply ignore this parameter and instead abondon the handle object @@ -123,7 +158,8 @@ C) or badly-formatted data (C). On callback entrance, the value of C<$!> contains the operating system -error (or C, C, C or C). +error code (or C, C, C, C or +C). While not mandatory, it is I recommended to set this callback, as you will not be notified of errors otherwise. The default simply calls @@ -137,13 +173,36 @@ read buffer). To access (and remove data from) the read buffer, use the C<< ->rbuf >> -method or access the C<$handle->{rbuf}> member directly. +method or access the C<< $handle->{rbuf} >> member directly. Note that you +must not enlarge or modify the read buffer, you can only remove data at +the beginning from it. When an EOF condition is detected then AnyEvent::Handle will first try to feed all the remaining data to the queued callbacks and C before calling the C callback. If no progress can be made, then a fatal error will be raised (with C<$!> set to C). +Note that, unlike requests in the read queue, an C callback +doesn't mean you I some data: if there is an EOF and there +are outstanding read requests then an error will be flagged. With an +C callback, the C callback will be invoked. + +=item on_eof => $cb->($handle) + +Set the callback to be called when an end-of-file condition is detected, +i.e. in the case of a socket, when the other side has closed the +connection cleanly, and there are no outstanding read requests in the +queue (if there are read requests, then an EOF counts as an unexpected +connection close and will be flagged as an error). + +For sockets, this just means that the other side has stopped sending data, +you can still try to write data, and, in fact, one can return from the EOF +callback and continue writing data, as only the read part has been shut +down. + +If an EOF condition has been detected but no C callback has been +set, then a fatal error will be raised with C<$!> set to <0>. + =item on_drain => $cb->($handle) This sets the callback that is called when the write buffer becomes empty @@ -242,7 +301,18 @@ system treats outstanding data at socket close time). This will not work for partial TLS data that could not be encoded -yet. This data will be lost. +yet. This data will be lost. Calling the C method in time might +help. + +=item peername => $string + +A string used to identify the remote site - usually the DNS hostname +(I IDN!) used to create the connection, rarely the IP address. + +Apart from being useful in error messages, this string is also used in TLS +peername verification (see C in L). This +verification will be skipped when C is not specified or +C. =item tls => "accept" | "connect" | Net::SSLeay::SSL object @@ -250,6 +320,9 @@ AnyEvent will start a TLS handshake as soon as the conenction has been established and will transparently encrypt/decrypt data afterwards. +All TLS protocol errors will be signalled as C, with an +appropriate error message. + TLS mode requires Net::SSLeay to be installed (it will be loaded automatically when you try to create a TLS handle): this module doesn't have a dependency on that module, so if your module requires it, you have @@ -262,16 +335,62 @@ You can also provide your own TLS connection object, but you have to make sure that you call either C or C on it before you pass it to -AnyEvent::Handle. +AnyEvent::Handle. Also, this module will take ownership of this connection +object. + +At some future point, AnyEvent::Handle might switch to another TLS +implementation, then the option to use your own session object will go +away. + +B since Net::SSLeay "objects" are really only integers, +passing in the wrong integer will lead to certain crash. This most often +happens when one uses a stylish C<< tls => 1 >> and is surprised about the +segmentation fault. See the C<< ->starttls >> method for when need to start TLS negotiation later. -=item tls_ctx => $ssl_ctx +=item tls_ctx => $anyevent_tls -Use the given C object to create the new TLS connection +Use the given C object to create the new TLS connection (unless a connection object was specified directly). If this parameter is missing, then AnyEvent::Handle will use C. +Instead of an object, you can also specify a hash reference with C<< key +=> value >> pairs. Those will be passed to L to create a +new TLS context object. + +=item on_starttls => $cb->($handle, $success[, $error_message]) + +This callback will be invoked when the TLS/SSL handshake has finished. If +C<$success> is true, then the TLS handshake succeeded, otherwise it failed +(C will not be called in this case). + +The session in C<< $handle->{tls} >> can still be examined in this +callback, even when the handshake was not successful. + +TLS handshake failures will not cause C to be invoked when this +callback is in effect, instead, the error message will be passed to C. + +Without this callback, handshake failures lead to C being +called, as normal. + +Note that you cannot call C right again in this callback. If you +need to do that, start an zero-second timer instead whose callback can +then call C<< ->starttls >> again. + +=item on_stoptls => $cb->($handle) + +When a SSLv3/TLS shutdown/close notify/EOF is detected and this callback is +set, then it will be invoked after freeing the TLS session. If it is not, +then a TLS shutdown condition will be treated like a normal EOF condition +on the handle. + +The session in C<< $handle->{tls} >> can still be examined in this +callback. + +This callback will only be called on TLS shutdowns, not when the +underlying handle signals EOF. + =item json => JSON or JSON::XS object This is the json coder object used by the C read and write types. @@ -283,69 +402,117 @@ Note that you are responsible to depend on the JSON module if you want to use this functionality, as AnyEvent does not have a dependency itself. -=item filter_r => $cb - -=item filter_w => $cb - -These exist, but are undocumented at this time. (They are used internally -by the TLS code). - =back =cut sub new { my $class = shift; - my $self = bless { @_ }, $class; - $self->{fh} or Carp::croak "mandatory argument fh is missing"; - - AnyEvent::Util::fh_nonblocking $self->{fh}, 1; + if ($self->{fh}) { + $self->_start; + return unless $self->{fh}; # could be gone by now + + } elsif ($self->{connect}) { + require AnyEvent::Socket; + + $self->{peername} = $self->{connect}[0] + unless exists $self->{peername}; + + $self->{_skip_drain_rbuf} = 1; + + { + Scalar::Util::weaken (my $self = $self); + + $self->{_connect} = + AnyEvent::Socket::tcp_connect ( + $self->{connect}[0], + $self->{connect}[1], + sub { + my ($fh, $host, $port, $retry) = @_; + + if ($fh) { + $self->{fh} = $fh; + + delete $self->{_skip_drain_rbuf}; + $self->_start; + + $self->{on_connect} + and $self->{on_connect}($self, $host, $port, sub { + delete @$self{qw(fh _tw _ww _rw _eof _queue rbuf _wbuf tls _tls_rbuf _tls_wbuf)}; + $self->{_skip_drain_rbuf} = 1; + &$retry; + }); + + } else { + if ($self->{on_connect_error}) { + $self->{on_connect_error}($self, "$!"); + $self->destroy; + } else { + $self->_error ($!, 1); + } + } + }, + sub { + local $self->{fh} = $_[0]; + + $self->{on_prepare} + ? $self->{on_prepare}->($self) + : () + } + ); + } - if ($self->{tls}) { - require Net::SSLeay; - $self->starttls (delete $self->{tls}, delete $self->{tls_ctx}); + } else { + Carp::croak "AnyEvent::Handle: either an existing fh or the connect parameter must be specified"; } + $self +} + +sub _start { + my ($self) = @_; + + AnyEvent::Util::fh_nonblocking $self->{fh}, 1; + $self->{_activity} = AnyEvent->now; $self->_timeout; - $self->on_drain (delete $self->{on_drain}) if exists $self->{on_drain}; $self->no_delay (delete $self->{no_delay}) if exists $self->{no_delay}; - $self->start_read - if $self->{on_read}; - - $self -} - -sub _shutdown { - my ($self) = @_; + $self->starttls (delete $self->{tls}, delete $self->{tls_ctx}) + if $self->{tls}; - delete $self->{_tw}; - delete $self->{_rw}; - delete $self->{_ww}; - delete $self->{fh}; + $self->on_drain (delete $self->{on_drain}) if $self->{on_drain}; - &_freetls; + $self->start_read + if $self->{on_read} || @{ $self->{_queue} }; - delete $self->{on_read}; - delete $self->{_queue}; + $self->_drain_wbuf; } -sub _error { - my ($self, $errno, $fatal) = @_; +#sub _shutdown { +# my ($self) = @_; +# +# delete @$self{qw(_tw _rw _ww fh wbuf on_read _queue)}; +# $self->{_eof} = 1; # tell starttls et. al to stop trying +# +# &_freetls; +#} - $self->_shutdown - if $fatal; +sub _error { + my ($self, $errno, $fatal, $message) = @_; $! = $errno; + $message ||= "$!"; if ($self->{on_error}) { - $self->{on_error}($self, $fatal); - } else { - Carp::croak "AnyEvent::Handle uncaught error: $!"; + $self->{on_error}($self, $fatal, $message); + $self->destroy if $fatal; + } elsif ($self->{fh}) { + $self->destroy; + Carp::croak "AnyEvent::Handle uncaught error: $message"; } } @@ -392,10 +559,14 @@ =item $handle->autocork ($boolean) Enables or disables the current autocork behaviour (see C -constructor argument). +constructor argument). Changes will only take effect on the next write. =cut +sub autocork { + $_[0]{autocork} = $_[1]; +} + =item $handle->no_delay ($boolean) Enables or disables the C setting (see constructor argument of @@ -408,10 +579,31 @@ eval { local $SIG{__DIE__}; - setsockopt $_[0]{fh}, &Socket::IPPROTO_TCP, &Socket::TCP_NODELAY, int $_[1]; + setsockopt $_[0]{fh}, &Socket::IPPROTO_TCP, &Socket::TCP_NODELAY, int $_[1] + if $_[0]{fh}; }; } +=item $handle->on_starttls ($cb) + +Replace the current C callback (see the C constructor argument). + +=cut + +sub on_starttls { + $_[0]{on_starttls} = $_[1]; +} + +=item $handle->on_stoptls ($cb) + +Replace the current C callback (see the C constructor argument). + +=cut + +sub on_starttls { + $_[0]{on_stoptls} = $_[1]; +} + ############################################################################# =item $handle->timeout ($seconds) @@ -432,7 +624,7 @@ sub _timeout { my ($self) = @_; - if ($self->{timeout}) { + if ($self->{timeout} && $self->{fh}) { my $NOW = AnyEvent->now; # when would the timeout trigger? @@ -445,7 +637,7 @@ if ($self->{on_timeout}) { $self->{on_timeout}($self); } else { - $self->_error (&Errno::ETIMEDOUT); + $self->_error (Errno::ETIMEDOUT); } # callback could have changed timeout value, optimise @@ -497,7 +689,7 @@ $self->{on_drain} = $cb; $cb->($self) - if $cb && $self->{low_water_mark} >= length $self->{wbuf}; + if $cb && $self->{low_water_mark} >= (length $self->{wbuf}) + (length $self->{_tls_wbuf}); } =item $handle->push_write ($data) @@ -518,13 +710,13 @@ my $cb = sub { my $len = syswrite $self->{fh}, $self->{wbuf}; - if ($len >= 0) { + if (defined $len) { substr $self->{wbuf}, 0, $len, ""; $self->{_activity} = AnyEvent->now; $self->{on_drain}($self) - if $self->{low_water_mark} >= length $self->{wbuf} + if $self->{low_water_mark} >= (length $self->{wbuf}) + (length $self->{_tls_wbuf}) && $self->{on_drain}; delete $self->{_ww} unless length $self->{wbuf}; @@ -558,11 +750,12 @@ ->($self, @_); } - if ($self->{filter_w}) { - $self->{filter_w}($self, \$_[0]); + if ($self->{tls}) { + $self->{_tls_wbuf} .= $_[0]; + &_dotls ($self) if $self->{fh}; } else { - $self->{wbuf} .= $_[0]; - $self->_drain_wbuf; + $self->{wbuf} .= $_[0]; + $self->_drain_wbuf if $self->{fh}; } } @@ -586,7 +779,7 @@ register_write_type netstring => sub { my ($self, $string) = @_; - sprintf "%d:%s,", (length $string), $string + (length $string) . ":$string," }; =item packstring => $format, $data @@ -662,6 +855,31 @@ =back +=item $handle->push_shutdown + +Sometimes you know you want to close the socket after writing your data +before it was actually written. One way to do that is to replace your +C handler by a callback that shuts down the socket (and set +C to C<0>). This method is a shorthand for just that, and +replaces the C callback with: + + sub { shutdown $_[0]{fh}, 1 } # for push_shutdown + +This simply shuts down the write side and signals an EOF condition to the +the peer. + +You can rely on the normal read queue and C handling +afterwards. This is the cleanest way to close a connection. + +=cut + +sub push_shutdown { + my ($self) = @_; + + delete $self->{low_water_mark}; + $self->on_drain (sub { shutdown $_[0]{fh}, 1 }); +} + =item AnyEvent::Handle::register_write_type type => $coderef->($handle, @args) This function (not method) lets you add your own types to C. @@ -764,23 +982,29 @@ sub _drain_rbuf { my ($self) = @_; - local $self->{_in_drain} = 1; + # avoid recursion + return if exists $self->{_skip_drain_rbuf}; + local $self->{_skip_drain_rbuf} = 1; if ( defined $self->{rbuf_max} && $self->{rbuf_max} < length $self->{rbuf} ) { - $self->_error (&Errno::ENOSPC, 1), return; + $self->_error (Errno::ENOSPC, 1), return; } while () { + # we need to use a separate tls read buffer, as we must not receive data while + # we are draining the buffer, and this can only happen with TLS. + $self->{rbuf} .= delete $self->{_tls_rbuf} if exists $self->{_tls_rbuf}; + my $len = length $self->{rbuf}; if (my $cb = shift @{ $self->{_queue} }) { unless ($cb->($self)) { if ($self->{_eof}) { # no progress can be made (not enough data and no data forthcoming) - $self->_error (&Errno::EPIPE, 1), return; + $self->_error (Errno::EPIPE, 1), return; } unshift @{ $self->{_queue} }, $cb; @@ -798,14 +1022,14 @@ ) { # no further data will arrive # so no progress can be made - $self->_error (&Errno::EPIPE, 1), return + $self->_error (Errno::EPIPE, 1), return if $self->{_eof}; last; # more data might arrive } } else { # read side becomes idle - delete $self->{_rw}; + delete $self->{_rw} unless $self->{tls}; last; } } @@ -814,7 +1038,7 @@ if ($self->{on_eof}) { $self->{on_eof}($self) } else { - $self->_error (0, 1); + $self->_error (0, 1, "Unexpected end-of-file"); } } @@ -837,15 +1061,18 @@ my ($self, $cb) = @_; $self->{on_read} = $cb; - $self->_drain_rbuf if $cb && !$self->{_in_drain}; + $self->_drain_rbuf if $cb; } =item $handle->rbuf Returns the read buffer (as a modifiable lvalue). -You can access the read buffer directly as the C<< ->{rbuf} >> member, if -you want. +You can access the read buffer directly as the C<< ->{rbuf} >> +member, if you want. However, the only operation allowed on the +read buffer (apart from looking at it) is removing data from its +beginning. Otherwise modifying or appending to it is not allowed and will +lead to hard-to-track-down bugs. NOTE: The read buffer should only be used or modified if the C, C or C methods are used. The other read methods @@ -896,7 +1123,7 @@ } push @{ $self->{_queue} }, $cb; - $self->_drain_rbuf unless $self->{_in_drain}; + $self->_drain_rbuf; } sub unshift_read { @@ -912,7 +1139,7 @@ unshift @{ $self->{_queue} }, $cb; - $self->_drain_rbuf unless $self->{_in_drain}; + $self->_drain_rbuf; } =item $handle->push_read (type => @args, $cb) @@ -1055,7 +1282,7 @@ # reject if ($reject && $$rbuf =~ $reject) { - $self->_error (&Errno::EBADMSG); + $self->_error (Errno::EBADMSG); } # skip @@ -1081,7 +1308,7 @@ sub { unless ($_[0]{rbuf} =~ s/^(0|[1-9][0-9]*)://) { if ($_[0]{rbuf} =~ /[^0-9]/) { - $self->_error (&Errno::EBADMSG); + $self->_error (Errno::EBADMSG); } return; } @@ -1094,7 +1321,7 @@ if ($_[1] eq ",") { $cb->($_[0], $string); } else { - $self->_error (&Errno::EBADMSG); + $self->_error (Errno::EBADMSG); } }); }); @@ -1110,7 +1337,8 @@ integer only (only one of C is allowed, plus an optional C, C<< < >> or C<< > >> modifier). -DNS over TCP uses a prefix of C, EPP uses a prefix of C. +For example, DNS over TCP uses a prefix of C (2 octet network order), +EPP uses a prefix of C (4 octtes). Example: read a block of data prefixed by its length in BER-encoded format (very efficient). @@ -1150,7 +1378,8 @@ =item json => $cb->($handle, $hash_or_arrayref) -Reads a JSON object or array, decodes it and passes it to the callback. +Reads a JSON object or array, decodes it and passes it to the +callback. When a parse error occurs, an C error will be raised. If a C object was passed to the constructor, then that will be used for the final decode, otherwise it will create a JSON coder expecting UTF-8. @@ -1169,15 +1398,15 @@ register_read_type json => sub { my ($self, $cb) = @_; - require JSON; + my $json = $self->{json} ||= + eval { require JSON::XS; JSON::XS->new->utf8 } + || do { require JSON; JSON->new->utf8 }; my $data; my $rbuf = \$self->{rbuf}; - my $json = $self->{json} ||= JSON->new->utf8; - sub { - my $ref = $json->incr_parse ($self->{rbuf}); + my $ref = eval { $json->incr_parse ($self->{rbuf}) }; if ($ref) { $self->{rbuf} = $json->incr_text; @@ -1185,8 +1414,19 @@ $cb->($self, $ref); 1 + } elsif ($@) { + # error case + $json->incr_skip; + + $self->{rbuf} = $json->incr_text; + $json->incr_text = ""; + + $self->_error (Errno::EBADMSG); + + () } else { $self->{rbuf} = ""; + () } } @@ -1228,7 +1468,7 @@ if (my $ref = eval { Storable::thaw ($_[1]) }) { $cb->($_[0], $ref); } else { - $self->_error (&Errno::EBADMSG); + $self->_error (Errno::EBADMSG); } }); } @@ -1273,12 +1513,15 @@ will automatically C for you when neither C is set nor there are any read requests in the queue. +These methods will have no effect when in TLS mode (as TLS doesn't support +half-duplex connections). + =cut sub stop_read { my ($self) = @_; - delete $self->{_rw}; + delete $self->{_rw} unless $self->{tls}; } sub start_read { @@ -1288,20 +1531,24 @@ Scalar::Util::weaken $self; $self->{_rw} = AnyEvent->io (fh => $self->{fh}, poll => "r", cb => sub { - my $rbuf = $self->{filter_r} ? \my $buf : \$self->{rbuf}; + my $rbuf = \($self->{tls} ? my $buf : $self->{rbuf}); my $len = sysread $self->{fh}, $$rbuf, $self->{read_size} || 8192, length $$rbuf; if ($len > 0) { $self->{_activity} = AnyEvent->now; - $self->{filter_r} - ? $self->{filter_r}($self, $rbuf) - : $self->{_in_drain} || $self->_drain_rbuf; + if ($self->{tls}) { + Net::SSLeay::BIO_write ($self->{_rbio}, $$rbuf); + + &_dotls ($self); + } else { + $self->_drain_rbuf; + } } elsif (defined $len) { delete $self->{_rw}; $self->{_eof} = 1; - $self->_drain_rbuf unless $self->{_in_drain}; + $self->_drain_rbuf; } elsif ($! != EAGAIN && $! != EINTR && $! != WSAEWOULDBLOCK) { return $self->_error ($!, 1); @@ -1310,46 +1557,84 @@ } } +our $ERROR_SYSCALL; +our $ERROR_WANT_READ; + +sub _tls_error { + my ($self, $err) = @_; + + return $self->_error ($!, 1) + if $err == Net::SSLeay::ERROR_SYSCALL (); + + my $err =Net::SSLeay::ERR_error_string (Net::SSLeay::ERR_get_error ()); + + # reduce error string to look less scary + $err =~ s/^error:[0-9a-fA-F]{8}:[^:]+:([^:]+):/\L$1: /; + + if ($self->{_on_starttls}) { + (delete $self->{_on_starttls})->($self, undef, $err); + &_freetls; + } else { + &_freetls; + $self->_error (Errno::EPROTO, 1, $err); + } +} + +# poll the write BIO and send the data if applicable +# also decode read data if possible +# this is basiclaly our TLS state machine +# more efficient implementations are possible with openssl, +# but not with the buggy and incomplete Net::SSLeay. sub _dotls { my ($self) = @_; - my $buf; + my $tmp; if (length $self->{_tls_wbuf}) { - while ((my $len = Net::SSLeay::write ($self->{tls}, $self->{_tls_wbuf})) > 0) { - substr $self->{_tls_wbuf}, 0, $len, ""; + while (($tmp = Net::SSLeay::write ($self->{tls}, $self->{_tls_wbuf})) > 0) { + substr $self->{_tls_wbuf}, 0, $tmp, ""; } + + $tmp = Net::SSLeay::get_error ($self->{tls}, $tmp); + return $self->_tls_error ($tmp) + if $tmp != $ERROR_WANT_READ + && ($tmp != $ERROR_SYSCALL || $!); } - while (defined ($buf = Net::SSLeay::read ($self->{tls}))) { - unless (length $buf) { - # let's treat SSL-eof as we treat normal EOF - delete $self->{_rw}; - $self->{_eof} = 1; + while (defined ($tmp = Net::SSLeay::read ($self->{tls}))) { + unless (length $tmp) { + $self->{_on_starttls} + and (delete $self->{_on_starttls})->($self, undef, "EOF during handshake"); # ??? &_freetls; + + if ($self->{on_stoptls}) { + $self->{on_stoptls}($self); + return; + } else { + # let's treat SSL-eof as we treat normal EOF + delete $self->{_rw}; + $self->{_eof} = 1; + } } - $self->{rbuf} .= $buf; - $self->_drain_rbuf unless $self->{_in_drain}; + $self->{_tls_rbuf} .= $tmp; + $self->_drain_rbuf; $self->{tls} or return; # tls session might have gone away in callback } - my $err = Net::SSLeay::get_error ($self->{tls}, -1); + $tmp = Net::SSLeay::get_error ($self->{tls}, -1); + return $self->_tls_error ($tmp) + if $tmp != $ERROR_WANT_READ + && ($tmp != $ERROR_SYSCALL || $!); - if ($err!= Net::SSLeay::ERROR_WANT_READ ()) { - if ($err == Net::SSLeay::ERROR_SYSCALL ()) { - return $self->_error ($!, 1); - } elsif ($err == Net::SSLeay::ERROR_SSL ()) { - return $self->_error (&Errno::EIO, 1); - } - - # all others are fine for our purposes - } - - if (length ($buf = Net::SSLeay::BIO_read ($self->{_wbio}))) { - $self->{wbuf} .= $buf; + while (length ($tmp = Net::SSLeay::BIO_read ($self->{_wbio}))) { + $self->{wbuf} .= $tmp; $self->_drain_wbuf; } + + $self->{_on_starttls} + and Net::SSLeay::state ($self->{tls}) == Net::SSLeay::ST_OK () + and (delete $self->{_on_starttls})->($self, 1, "TLS/SSL connection established"); } =item $handle->starttls ($tls[, $tls_ctx]) @@ -1358,36 +1643,65 @@ object is created, you can also do that at a later time by calling C. +Starting TLS is currently an asynchronous operation - when you push some +write data and then call C<< ->starttls >> then TLS negotiation will start +immediately, after which the queued write data is then sent. + The first argument is the same as the C constructor argument (either C<"connect">, C<"accept"> or an existing Net::SSLeay object). -The second argument is the optional C object that is -used when AnyEvent::Handle has to create its own TLS connection object. +The second argument is the optional C object that is used +when AnyEvent::Handle has to create its own TLS connection object, or +a hash reference with C<< key => value >> pairs that will be used to +construct a new context. -The TLS connection object will end up in C<< $handle->{tls} >> after this -call and can be used or changed to your liking. Note that the handshake -might have already started when this function returns. +The TLS connection object will end up in C<< $handle->{tls} >>, the TLS +context in C<< $handle->{tls_ctx} >> after this call and can be used or +changed to your liking. Note that the handshake might have already started +when this function returns. -If it an error to start a TLS handshake more than once per -AnyEvent::Handle object (this is due to bugs in OpenSSL). +Due to bugs in OpenSSL, it might or might not be possible to do multiple +handshakes on the same stream. Best do not attempt to use the stream after +stopping TLS. =cut +our %TLS_CACHE; #TODO not yet documented, should we? + sub starttls { - my ($self, $ssl, $ctx) = @_; + my ($self, $tls, $ctx) = @_; - Carp::croak "it is an error to call starttls more than once on an Anyevent::Handle object" + Carp::croak "It is an error to call starttls on an AnyEvent::Handle object while TLS is already active, caught" if $self->{tls}; - - if ($ssl eq "accept") { - $ssl = Net::SSLeay::new ($ctx || TLS_CTX ()); - Net::SSLeay::set_accept_state ($ssl); - } elsif ($ssl eq "connect") { - $ssl = Net::SSLeay::new ($ctx || TLS_CTX ()); - Net::SSLeay::set_connect_state ($ssl); - } - $self->{tls} = $ssl; + $self->{tls} = $tls; + $self->{tls_ctx} = $ctx if @_ > 2; + + return unless $self->{fh}; + + require Net::SSLeay; + + $ERROR_SYSCALL = Net::SSLeay::ERROR_SYSCALL (); + $ERROR_WANT_READ = Net::SSLeay::ERROR_WANT_READ (); + + $tls = $self->{tls}; + $ctx = $self->{tls_ctx}; + + local $Carp::CarpLevel = 1; # skip ourselves when creating a new context or session + + if ("HASH" eq ref $ctx) { + require AnyEvent::TLS; + + if ($ctx->{cache}) { + my $key = $ctx+0; + $ctx = $TLS_CACHE{$key} ||= new AnyEvent::TLS %$ctx; + } else { + $ctx = new AnyEvent::TLS %$ctx; + } + } + + $self->{tls_ctx} = $ctx || TLS_CTX (); + $self->{tls} = $tls = $self->{tls_ctx}->_get_session ($tls, $self, $self->{peername}); # basically, this is deep magic (because SSL_read should have the same issues) # but the openssl maintainers basically said: "trust us, it just works". @@ -1397,36 +1711,33 @@ # # in short: this is a mess. # - # note that we do not try to kepe the length constant between writes as we are required to do. + # note that we do not try to keep the length constant between writes as we are required to do. # we assume that most (but not all) of this insanity only applies to non-blocking cases, - # and we drive openssl fully in blocking mode here. - Net::SSLeay::CTX_set_mode ($self->{tls}, - (eval { local $SIG{__DIE__}; Net::SSLeay::MODE_ENABLE_PARTIAL_WRITE () } || 1) - | (eval { local $SIG{__DIE__}; Net::SSLeay::MODE_ACCEPT_MOVING_WRITE_BUFFER () } || 2)); + # and we drive openssl fully in blocking mode here. Or maybe we don't - openssl seems to + # have identity issues in that area. +# Net::SSLeay::CTX_set_mode ($ssl, +# (eval { local $SIG{__DIE__}; Net::SSLeay::MODE_ENABLE_PARTIAL_WRITE () } || 1) +# | (eval { local $SIG{__DIE__}; Net::SSLeay::MODE_ACCEPT_MOVING_WRITE_BUFFER () } || 2)); + Net::SSLeay::CTX_set_mode ($tls, 1|2); $self->{_rbio} = Net::SSLeay::BIO_new (Net::SSLeay::BIO_s_mem ()); $self->{_wbio} = Net::SSLeay::BIO_new (Net::SSLeay::BIO_s_mem ()); - Net::SSLeay::set_bio ($ssl, $self->{_rbio}, $self->{_wbio}); + Net::SSLeay::set_bio ($tls, $self->{_rbio}, $self->{_wbio}); - $self->{filter_w} = sub { - $_[0]{_tls_wbuf} .= ${$_[1]}; - &_dotls; - }; - $self->{filter_r} = sub { - Net::SSLeay::BIO_write ($_[0]{_rbio}, ${$_[1]}); - &_dotls; - }; + $self->{_on_starttls} = sub { $_[0]{on_starttls}(@_) } + if $self->{on_starttls}; - &_dotls; # need to trigger the initial negotiation exchange + &_dotls; # need to trigger the initial handshake + $self->start_read; # make sure we actually do read } =item $handle->stoptls Shuts down the SSL connection - this makes a proper EOF handshake by sending a close notify to the other side, but since OpenSSL doesn't -support non-blocking shut downs, it is not possible to re-use the stream -afterwards. +support non-blocking shut downs, it is not guarenteed that you can re-use +the stream afterwards. =cut @@ -1434,13 +1745,13 @@ my ($self) = @_; if ($self->{tls}) { - Net::SSLeay::shutdown $self->{tls}; + Net::SSLeay::shutdown ($self->{tls}); &_dotls; - # we don't give a shit. no, we do, but we can't. no... - # we, we... have to use openssl :/ - &_freetls; +# # we don't give a shit. no, we do, but we can't. no...#d# +# # we, we... have to use openssl :/#d# +# &_freetls;#d# } } @@ -1449,19 +1760,20 @@ return unless $self->{tls}; - Net::SSLeay::free (delete $self->{tls}); + $self->{tls_ctx}->_put_session (delete $self->{tls}) + if ref $self->{tls}; - delete @$self{qw(_rbio filter_w _wbio filter_r)}; + delete @$self{qw(_rbio _wbio _tls_wbuf _on_starttls)}; } sub DESTROY { - my $self = shift; + my ($self) = @_; &_freetls; my $linger = exists $self->{linger} ? $self->{linger} : 3600; - if ($linger && length $self->{wbuf}) { + if ($linger && length $self->{wbuf} && $self->{fh}) { my $fh = delete $self->{fh}; my $wbuf = delete $self->{wbuf}; @@ -1482,43 +1794,216 @@ } } -=item AnyEvent::Handle::TLS_CTX +=item $handle->destroy + +Shuts down the handle object as much as possible - this call ensures that +no further callbacks will be invoked and as many resources as possible +will be freed. You must not call any methods on the object afterwards. -This function creates and returns the Net::SSLeay::CTX object used by -default for TLS mode. +Normally, you can just "forget" any references to an AnyEvent::Handle +object and it will simply shut down. This works in fatal error and EOF +callbacks, as well as code outside. It does I work in a read or write +callback, so when you want to destroy the AnyEvent::Handle object from +within such an callback. You I call C<< ->destroy >> explicitly in +that case. + +Destroying the handle object in this way has the advantage that callbacks +will be removed as well, so if those are the only reference holders (as +is common), then one doesn't need to do anything special to break any +reference cycles. + +The handle might still linger in the background and write out remaining +data, as specified by the C option, however. + +=cut + +sub destroy { + my ($self) = @_; -The context is created like this: + $self->DESTROY; + %$self = (); +} - Net::SSLeay::load_error_strings; - Net::SSLeay::SSLeay_add_ssl_algorithms; - Net::SSLeay::randomize; +=item AnyEvent::Handle::TLS_CTX - my $CTX = Net::SSLeay::CTX_new; +This function creates and returns the AnyEvent::TLS object used by default +for TLS mode. - Net::SSLeay::CTX_set_options $CTX, Net::SSLeay::OP_ALL +The context is created by calling L without any arguments. =cut our $TLS_CTX; sub TLS_CTX() { - $TLS_CTX || do { - require Net::SSLeay; + $TLS_CTX ||= do { + require AnyEvent::TLS; - Net::SSLeay::load_error_strings (); - Net::SSLeay::SSLeay_add_ssl_algorithms (); - Net::SSLeay::randomize (); + new AnyEvent::TLS + } +} - $TLS_CTX = Net::SSLeay::CTX_new (); +=back - Net::SSLeay::CTX_set_options ($TLS_CTX, Net::SSLeay::OP_ALL ()); - $TLS_CTX - } -} +=head1 NONFREQUENTLY ASKED QUESTIONS + +=over 4 + +=item I C the AnyEvent::Handle reference inside my callback and +still get further invocations! + +That's because AnyEvent::Handle keeps a reference to itself when handling +read or write callbacks. + +It is only safe to "forget" the reference inside EOF or error callbacks, +from within all other callbacks, you need to explicitly call the C<< +->destroy >> method. + +=item I get different callback invocations in TLS mode/Why can't I pause +reading? + +Unlike, say, TCP, TLS connections do not consist of two independent +communication channels, one for each direction. Or put differently. The +read and write directions are not independent of each other: you cannot +write data unless you are also prepared to read, and vice versa. + +This can mean than, in TLS mode, you might get C or C +callback invocations when you are not expecting any read data - the reason +is that AnyEvent::Handle always reads in TLS mode. + +During the connection, you have to make sure that you always have a +non-empty read-queue, or an C watcher. At the end of the +connection (or when you no longer want to use it) you can call the +C method. + +=item How do I read data until the other side closes the connection? + +If you just want to read your data into a perl scalar, the easiest way +to achieve this is by setting an C callback that does nothing, +clearing the C callback and in the C callback, the data +will be in C<$_[0]{rbuf}>: + + $handle->on_read (sub { }); + $handle->on_eof (undef); + $handle->on_error (sub { + my $data = delete $_[0]{rbuf}; + }); + +The reason to use C is that TCP connections, due to latencies +and packets loss, might get closed quite violently with an error, when in +fact, all data has been received. + +It is usually better to use acknowledgements when transferring data, +to make sure the other side hasn't just died and you got the data +intact. This is also one reason why so many internet protocols have an +explicit QUIT command. + +=item I don't want to destroy the handle too early - how do I wait until +all data has been written? + +After writing your last bits of data, set the C callback +and destroy the handle in there - with the default setting of +C this will be called precisely when all data has been +written to the socket: + + $handle->push_write (...); + $handle->on_drain (sub { + warn "all data submitted to the kernel\n"; + undef $handle; + }); + +If you just want to queue some data and then signal EOF to the other side, +consider using C<< ->push_shutdown >> instead. + +=item I want to contact a TLS/SSL server, I don't care about security. + +If your TLS server is a pure TLS server (e.g. HTTPS) that only speaks TLS, +simply connect to it and then create the AnyEvent::Handle with the C +parameter: + + tcp_connect $host, $port, sub { + my ($fh) = @_; + + my $handle = new AnyEvent::Handle + fh => $fh, + tls => "connect", + on_error => sub { ... }; + + $handle->push_write (...); + }; + +=item I want to contact a TLS/SSL server, I do care about security. + +Then you should additionally enable certificate verification, including +peername verification, if the protocol you use supports it (see +L, C). + +E.g. for HTTPS: + + tcp_connect $host, $port, sub { + my ($fh) = @_; + + my $handle = new AnyEvent::Handle + fh => $fh, + peername => $host, + tls => "connect", + tls_ctx => { verify => 1, verify_peername => "https" }, + ... + +Note that you must specify the hostname you connected to (or whatever +"peername" the protocol needs) as the C argument, otherwise no +peername verification will be done. + +The above will use the system-dependent default set of trusted CA +certificates. If you want to check against a specific CA, add the +C (or C) arguments to C: + + tls_ctx => { + verify => 1, + verify_peername => "https", + ca_file => "my-ca-cert.pem", + }, + +=item I want to create a TLS/SSL server, how do I do that? + +Well, you first need to get a server certificate and key. You have +three options: a) ask a CA (buy one, use cacert.org etc.) b) create a +self-signed certificate (cheap. check the search engine of your choice, +there are many tutorials on the net) or c) make your own CA (tinyca2 is a +nice program for that purpose). + +Then create a file with your private key (in PEM format, see +L), followed by the certificate (also in PEM format). The +file should then look like this: + + -----BEGIN RSA PRIVATE KEY----- + ...header data + ... lots of base64'y-stuff + -----END RSA PRIVATE KEY----- + + -----BEGIN CERTIFICATE----- + ... lots of base64'y-stuff + -----END CERTIFICATE----- + +The important bits are the "PRIVATE KEY" and "CERTIFICATE" parts. Then +specify this file as C: + + tcp_server undef, $port, sub { + my ($fh) = @_; + + my $handle = new AnyEvent::Handle + fh => $fh, + tls => "accept", + tls_ctx => { cert_file => "my-server-keycert.pem" }, + ... + +When you have intermediate CA certificates that your clients might not +know about, just append them to the C. =back + =head1 SUBCLASSING AnyEvent::Handle In many cases, you might want to subclass AnyEvent::Handle.