[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.28 by root, Thu Nov 28 16:09:04 2013 UTC vs.
Revision 1.35 by root, Sun Dec 1 16:40:25 2013 UTC

 Regarding compactness, C<CBOR::XS>-encoded data structures are usually
 about 20% smaller than the same data encoded as (compact) JSON or
 L<Storable>.
 In addition to the core CBOR data format, this module implements a
-number of extensions, to support cyclic and shared data structures (see
+number of extensions, to support cyclic and shared data structures
-C<allow_sharing>), string deduplication (see C<pack_strings>) and scalar
+(see C<allow_sharing> and C<allow_cycles>), string deduplication (see
-references (always enabled).
+C<pack_strings>) and scalar references (always enabled).
 The primary goal of this module is to be I<correct> and the secondary goal
 is to be I<fast>. To reach the latter goal it was written in C.
 See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
 package CBOR::XS;
 use common::sense;
-our $VERSION = '1.0';
+our $VERSION = 1.11;
 our @ISA = qw(Exporter);
 our @EXPORT = qw(encode_cbor decode_cbor);
 use Exporter;
 reference to the earlier value.
 This means that such values will only be encoded once, and will not result
 in a deep cloning of the value on decode, in decoders supporting the value
 sharing extension. This also makes it possible to encode cyclic data
-structures.
+structures (which need C<allow_cycles> to ne enabled to be decoded by this
+module).
 It is recommended to leave it off unless you know your
 communication partner supports the value sharing extensions to CBOR
 (L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the
 resulting data structure might be unusable.
 Detecting shared values incurs a runtime overhead when values are encoded
 that have a reference counter large than one, and might unnecessarily
 increase the encoded size, as potentially shared values are encode as
-sharable whether or not they are actually shared.
+shareable whether or not they are actually shared.
 At the moment, only targets of references can be shared (e.g. scalars,
 arrays or hashes pointed to by a reference). Weirder constructs, such as
 an array with multiple "copies" of the I<same> string, which are hard but
 not impossible to create in Perl, are not supported (this is the same as
 structures cannot be encoded in this mode.
 This option does not affect C<decode> in any way - shared values and
 references will always be decoded properly if present.
+=item $cbor = $cbor->allow_cycles ([$enable])
+=item $enabled = $cbor->get_allow_cycles
+If C<$enable> is true (or missing), then C<decode> will happily decode
+self-referential (cyclic) data structures. By default these will not be
+decoded, as they need manual cleanup to avoid memory leaks, so code that
+isn't prepared for this will not leak memory.
+If C<$enable> is false (the default), then C<decode> will throw an error
+when it encounters a self-referential/cyclic data structure.
+This option does not affect C<encode> in any way - shared values and
+references will always be decoded properly if present.
 =item $cbor = $cbor->pack_strings ([$enable])
 =item $enabled = $cbor->get_pack_strings
 If C<$enable> is true (or missing), then C<encode> will try not to encode
 If C<$enable> is false (the default), then C<encode> will encode strings
 the standard CBOR way.
 This option does not affect C<decode> in any way - string references will
 always be decoded properly if present.
+=item $cbor = $cbor->validate_utf8 ([$enable])
+=item $enabled = $cbor->get_validate_utf8
+If C<$enable> is true (or missing), then C<decode> will validate that
+elements (text strings) containing UTF-8 data in fact contain valid UTF-8
+data (instead of blindly accepting it). This validation obviously takes
+extra time during decoding.
+The concept of "valid UTF-8" used is perl's concept, which is a superset
+of the official UTF-8.
+If C<$enable> is false (the default), then C<decode> will blindly accept
+UTF-8 data, marking them as valid UTF-8 in the resulting data structure
+regardless of whether thats true or not.
+Perl isn't too happy about corrupted UTF-8 in strings, but should
+generally not crash or do similarly evil things. Extensions might be not
+so forgiving, so it's recommended to turn on this setting if you receive
+untrusted CBOR.
+This option does not affect C<encode> in any way - strings that are
+supposedly valid UTF-8 will simply be dumped into the resulting CBOR
+string without checking whether that is, in fact, true or not.
 =item $cbor = $cbor->filter ([$cb->($tag, $value)])
 =item $cb_or_undef = $cbor->get_filter
 =back
 =head2 OBJECT SERIALISATION
+This module implements both a CBOR-specific and the generic
+L<Types::Serialier> object serialisation protocol. The following
+subsections explain both methods.
+=head3 ENCODING
 This module knows two way to serialise a Perl object: The CBOR-specific
 way, and the generic way.
-Whenever the encoder encounters a Perl object that it cnanot serialise
+Whenever the encoder encounters a Perl object that it cannot serialise
 directly (most of them), it will first look up the C<TO_CBOR> method on
 it.
 If it has a C<TO_CBOR> method, it will call it with the object as only
 argument, and expects exactly one return value, which it will then
 The C<FREEZE> method can return any number of values (i.e. zero or
 more). These will be encoded as CBOR perl object, together with the
 classname.
+These methods I<MUST NOT> change the data structure that is being
+serialised. Failure to comply to this can result in memory corruption -
+and worse.
 If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
 with an error.
+=head3 DECODING
-Objects encoded via C<TO_CBOR> cannot be automatically decoded, but
+Objects encoded via C<TO_CBOR> cannot (normally) be automatically decoded,
-objects encoded via C<FREEZE> can be decoded using the following protocol:
+but objects encoded via C<FREEZE> can be decoded using the following
+protocol:
 When an encoded CBOR perl object is encountered by the decoder, it will
 look up the C<THAW> method, by using the stored classname, and will fail
 if the method cannot be found.
 After the lookup it will call the C<THAW> method with the stored classname
 as first argument, the constant string C<CBOR> as second argument, and all
 values returned by C<FREEZE> as remaining arguments.
-=head4 EXAMPLES
+=head3 EXAMPLES
 Here is an example C<TO_CBOR> method:
    sub My::Object::TO_CBOR {
       my ($obj) = @_;
 These tags are automatically created (and decoded) for serialisable
 objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
 serialisation protocol). See L<OBJECT SERIALISATION> for details.
-=item 28, 29 (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)
+=item 28, 29 (shareable, sharedref, L <http://cbor.schmorp.de/value-sharing>)
-These tags are automatically decoded when encountered, resulting in
+These tags are automatically decoded when encountered (and they do not
+result in a cyclic data structure, see C<allow_cycles>), resulting in
 shared values in the decoded object. They are only encoded, however, when
-C<allow_sharable> is enabled.
+C<allow_sharing> is enabled.
+Not all shared values can be successfully decoded: values that reference
+themselves will I<currently> decode as C<undef> (this is not the same
+as a reference pointing to itself, which will be represented as a value
+that contains an indirect reference to itself - these will be decoded
+properly).
+Note that considerably more shared value data structures can be decoded
+than will be encoded - currently, only values pointed to by references
+will be shared, others will not. While non-reference shared values can be
+generated in Perl with some effort, they were considered too unimportant
+to be supported in the encoder. The decoder, however, will decode these
+values as shared values.
 =item 256, 25 (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>)
 These tags are automatically decoded when encountered. They are only
 encoded, however, when C<pack_strings> is enabled.
 perl core distribution (e.g. L<URI>), it is (currently) up to the user to
 provide these modules. The decoding usually fails with an exception if the
 required module cannot be loaded.
 =over 4
+=item 0, 1 (date/time string, seconds since the epoch)
+These tags are decoded into L<Time::Piece> objects. The corresponding
+C<Time::Piece::TO_CBOR> method always encodes into tag 1 values currently.
+The L<Time::Piece> API is generally surprisingly bad, and fractional
+seconds are only accidentally kept intact, so watch out. On the plus side,
+the module comes with perl since 5.10, which has to count for something.
 =item 2, 3 (positive/negative bignum)
 These tags are decoded into L<Math::BigInt> objects. The corresponding
 C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR
 properly. Half precision types are accepted, but not encoded.
 Strict mode and canonical mode are not implemented.
+=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT
+On perls that were built without 64 bit integer support (these are rare
+nowadays, even on 32 bit architectures), support for any kind of 64 bit
+integer in CBOR is very limited - most likely, these 64 bit values will
+be truncated, corrupted, or otherwise not decoded correctly. This also
+includes string, array and map sizes that are stored as 64 bit integers.
 =head1 THREADS
 This module is I<not> guaranteed to be thread safe and there are no
 plans to change this until Perl gets thread support (as opposed to the
 horribly slow so-called "threads" which are simply slow and bloated
 service. I put the contact address into my modules for a reason.
 =cut
 our %FILTER = (
-   # 0 # rfc4287 datetime, utf-8
+   0 => sub { # rfc4287 datetime, utf-8
-   # 1 # unix timestamp, any
+      require Time::Piece;
+      # Time::Piece::Strptime uses the "incredibly flexible date parsing routine"
+      # from FreeBSD, which can't parse ISO 8601, RFC3339, RFC4287 or much of anything
+      # else either. Whats incredibe over standard strptime totally escapes me.
+      # doesn't do fractional times, either. sigh.
+      scalar eval {
+         my $s = $_[1];
+         $s =~ s/Z$/+00:00/;
+         $s =~ s/(\.[0-9]+)?([+-][0-9][0-9]):([0-9][0-9])/$2$3/
+            or die;
+         my $f = $1; # fractional part. hopefully
+         my $d = Time::Piece->strptime ($s, "%Y-%m-%dT%H:%M:%S%z");
+         Time::Piece::gmtime ($d->epoch + $f)
+      } || die "corrupted CBOR date/time string ($_[0])";
+   },
+   1 => sub { # seconds since the epoch, possibly fractional
+      require Time::Piece;
+      scalar Time::Piece::gmtime (pop)
+   },
    2 => sub { # pos bigint
       require Math::BigInt;
       Math::BigInt->new ("0x" . unpack "H*", pop)
    },
 }
 sub URI::TO_CBOR {
    my $uri = $_[0]->as_string;
    utf8::upgrade $uri;
-   CBOR::XS::tag 32, $uri
+   tag 32, $uri
 }
 sub Math::BigInt::TO_CBOR {
    if ($_[0] >= -2147483648 && $_[0] <= 2147483647) {
       $_[0]->numify
    } else {
       my $hex = substr $_[0]->as_hex, 2;
       $hex = "0$hex" if 1 & length $hex; # sigh
-      CBOR::XS::tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
+      tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
    }
 }
 sub Math::BigFloat::TO_CBOR {
    my ($m, $e) = $_[0]->parts;
-   CBOR::XS::tag 4, [$e->numify, $m]
+   tag 4, [$e->numify, $m]
+}
+sub Time::Piece::TO_CBOR {
+   tag 1, $_[0]->epoch
 }
 XSLoader::load "CBOR::XS", $VERSION;
 =head1 SEE ALSO

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.28 by root, Thu Nov 28 16:09:04 2013 UTC vs. Revision 1.35 by root, Sun Dec 1 16:40:25 2013 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.28 by root, Thu Nov 28 16:09:04 2013 UTC vs.
Revision 1.35 by root, Sun Dec 1 16:40:25 2013 UTC