[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.49 by root, Thu Apr 21 16:24:03 2016 UTC vs.
Revision 1.56 by root, Mon Apr 25 21:44:13 2016 UTC

 package CBOR::XS;
 use common::sense;
-our $VERSION = 1.41;
+our $VERSION = 1.5;
 our @ISA = qw(Exporter);
 our @EXPORT = qw(encode_cbor decode_cbor);
 use Exporter;
 the standard CBOR way.
 This option does not affect C<decode> in any way - string references will
 always be decoded properly if present.
+=item $cbor = $cbor->text_keys ([$enable])
+=item $enabled = $cbor->get_text_keys
+If C<$enabled> is true (or missing), then C<encode> will encode all
+perl hash keys as CBOR text strings/UTF-8 string, upgrading them as needed.
+If C<$enable> is false (the default), then C<encode> will encode hash keys
+normally - upgraded perl strings (strings internally encoded as UTF-8) as
+CBOR text strings, and downgraded perl strings as CBOR byte strings.
+This option does not affect C<decode> in any way.
+This option is useful for interoperability with CBOR decoders that don't
+treat byte strings as a form of text. It is especially useful as Perl
+gives very little control over hash keys.
+Enabling this option can be slow, as all downgraded hash keys that are
+encoded need to be scanned and converted to UTF-8.
+=item $cbor = $cbor->text_strings ([$enable])
+=item $enabled = $cbor->get_text_strings
+This option works similar to C<text_keys>, above, but works on all strings
+(including hash keys), so C<text_keys> has no further effect after
+enabling C<text_strings>.
+If C<$enabled> is true (or missing), then C<encode> will encode all perl
+strings as CBOR text strings/UTF-8 strings, upgrading them as needed.
+If C<$enable> is false (the default), then C<encode> will encode strings
+normally (but see C<text_keys>) - upgraded perl strings (strings
+internally encoded as UTF-8) as CBOR text strings, and downgraded perl
+strings as CBOR byte strings.
+This option does not affect C<decode> in any way.
+This option has similar advantages and disadvantages as C<text_keys>. In
+addition, this option effectively removes the ability to encode byte
+strings, which might break some C<FREEZE> and C<TO_CBOR> methods that rely
+on this, such as bignum encoding, so this option is mainly useful for very
+simple data.
 =item $cbor = $cbor->validate_utf8 ([$enable])
 =item $enabled = $cbor->get_validate_utf8
 If C<$enable> is true (or missing), then C<decode> will validate that
 The concept of "valid UTF-8" used is perl's concept, which is a superset
 of the official UTF-8.
 If C<$enable> is false (the default), then C<decode> will blindly accept
 UTF-8 data, marking them as valid UTF-8 in the resulting data structure
-regardless of whether thats true or not.
+regardless of whether that's true or not.
 Perl isn't too happy about corrupted UTF-8 in strings, but should
 generally not crash or do similarly evil things. Extensions might be not
 so forgiving, so it's recommended to turn on this setting if you receive
 untrusted CBOR.
    my $x = 3.1; # some variable containing a number
    "$x";        # stringified
    $x .= "";    # another, more awkward way to stringify
    print $x;    # perl does it for you, too, quite often
-You can force whether a string ie encoded as byte or text string by using
+You can force whether a string is encoded as byte or text string by using
-C<utf8::upgrade> and C<utf8::downgrade>):
+C<utf8::upgrade> and C<utf8::downgrade> (if C<text_strings> is disabled):
   utf8::upgrade $x;   # encode $x as text string
   utf8::downgrade $x; # encode $x as byte string
 Perl doesn't define what operations up- and downgrade strings, so if the
 difference between byte and text is important, you should up- or downgrade
-your string as late as possible before encoding.
+your string as late as possible before encoding. You can also force the
+use of CBOR text strings by using C<text_keys> or C<text_strings>.
 You can force the type to be a CBOR number by numifying it:
    my $x = "3"; # some variable containing a string
    $x += 0;     # numify it, ensuring it will be dumped as a number
       "$self" # encode url string
    }
    sub URI::THAW {
       my ($class, $serialiser, $uri) = @_;
       $class->new ($uri)
    }
 Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For
 example, a C<FREEZE> method that returns "type", "id" and "variant" values
 additional tags (such as base64url).
 =head2 ENFORCED TAGS
 These tags are always handled when decoding, and their handling cannot be
-overriden by the user.
+overridden by the user.
 =over 4
 =item 26 (perl-object, L<http://cbor.schmorp.de/perl-object>)
 encoded, however, when C<pack_strings> is enabled.
 =item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
 This tag is automatically generated when a reference are encountered (with
-the exception of hash and array refernces). It is converted to a reference
+the exception of hash and array references). It is converted to a reference
 when decoding.
 =item 55799 (self-describe CBOR, RFC 7049)
 This value is not generated on encoding (unless explicitly requested by
 =back
 =head2 NON-ENFORCED TAGS
 These tags have default filters provided when decoding. Their handling can
-be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
+be overridden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
 providing a custom C<filter> callback when decoding.
 When they result in decoding into a specific Perl class, the module
 usually provides a corresponding C<TO_CBOR> method as well.
 These tags are decoded into L<Math::BigInt> objects. The corresponding
 C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR
 integers, and others into positive/negative CBOR bignums.
-=item 4, 5 (decimal fraction/bigfloat)
+=item 4, 5, 264, 265 (decimal fraction/bigfloat)
 Both decimal fractions and bigfloats are decoded into L<Math::BigFloat>
 objects. The corresponding C<Math::BigFloat::TO_CBOR> method I<always>
-encodes into a decimal fraction.
+encodes into a decimal fraction (either tag 4 or 264).
-CBOR cannot represent bigfloats with I<very> large exponents - conversion
+NaN and infinities are not encoded properly, as they cannot be represented
-of such big float objects is undefined.
+in CBOR.
-Also, NaN and infinities are not encoded properly.
+See L<BIGNUM SECURITY CONSIDERATIONS> for more info.
 =item 21, 22, 23 (expected later JSON conversion)
 CBOR::XS is not a CBOR-to-JSON converter, and will simply ignore these
 tags.
 C<URI::TO_CBOR> method again results in a CBOR URI value.
 =back
 =cut
-our %FILTER = (
-   # 0 # rfc4287 datetime, utf-8
-   # 1 # unix timestamp, any
-   2 => sub { # pos bigint
-      require Math::BigInt;
-      Math::BigInt->new ("0x" . unpack "H*", pop)
-   },
-   3 => sub { # neg bigint
-      require Math::BigInt;
-      -Math::BigInt->new ("0x" . unpack "H*", pop)
-   },
-   4 => sub { # decimal fraction, array
-      require Math::BigFloat;
-      Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
-   },
-   5 => sub { # bigfloat, array
-      require Math::BigFloat;
-      scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
-   },
-   21 => sub { pop }, # expected conversion to base64url encoding
-   22 => sub { pop }, # expected conversion to base64 encoding
-   23 => sub { pop }, # expected conversion to base16 encoding
-   # 24 # embedded cbor, byte string
-   32 => sub {
-      require URI;
-      URI->new (pop)
-   },
-   # 33 # base64url rfc4648, utf-8
-   # 34 # base64 rfc46484, utf-8
-   # 35 # regex pcre/ecma262, utf-8
-   # 36 # mime message rfc2045, utf-8
-);
 =head1 CBOR and JSON
 CBOR is supposed to implement a superset of the JSON data model, and is,
 with some coercion, able to represent all JSON texts (something that other
 Also keep in mind that CBOR::XS might leak contents of your Perl data
 structures in its error messages, so when you serialise sensitive
 information you might want to make sure that exceptions thrown by CBOR::XS
 will not end up in front of untrusted eyes.
+=head1 BIGNUM SECURITY CONSIDERATIONS
+CBOR::XS provides a C<TO_CBOR> method for both L<Math::BigInt> and
+L<Math::BigFloat> that tries to encode the number in the simplest possible
+way, that is, either a CBOR integer, a CBOR bigint/decimal fraction (tag
+4) or an arbitrary-exponent decimal fraction (tag 264).
+It will also understand base-2 bigfloat or arbitrary-exponent bigfloats
+(tags 5 and 265), but it will never generate these on its own.
+Using the built-in L<Math::BigInt::Calc> support, encoding and decoding
+decimal fractions is generally fast. Decoding bigints can be slow for very
+big numbers, and decoding bigfloats or arbitrary-exponent bigfloats can be
+extremely slow (minutes, decades) for large exponents.
+Additionally, L<Math::BigInt> can take advantage of other bignum
+libraries, such as L<Math::GMP>, which cannot handle big
+floats with large exponents, and might simply abort or crash your program,
+due to their code quality.
+This can be a concern if you want to parse untrusted CBOR. If it is, you
+need to disable decoding of tag 2 (bigint) and 3 (negative bigint) types,
+which will also disable bigfloat support (to be sure, you can also disable
+types 4, 5, 264 and 265).
 =head1 CBOR IMPLEMENTATION NOTES
 This section contains some random implementation notes. They do not
 describe guaranteed behaviour, but merely behaviour as-is implemented
       scalar Time::Piece::gmtime (pop)
    },
    2 => sub { # pos bigint
       require Math::BigInt;
-      Math::BigInt->new ("0x" . unpack "H*", pop)
+      Math::BigInt->from_hex ("0x" . unpack "H*", pop)
    },
    3 => sub { # neg bigint
       require Math::BigInt;
-      -Math::BigInt->new ("0x" . unpack "H*", pop)
+      -Math::BigInt->from_hex ("0x" . unpack "H*", pop)
    },
    4 => sub { # decimal fraction, array
       require Math::BigFloat;
       Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
    },
    5 => sub { # bigfloat, array
       require Math::BigFloat;
-      scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
+      scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
    },
    21 => sub { pop }, # expected conversion to base64url encoding
    22 => sub { pop }, # expected conversion to base64 encoding
    23 => sub { pop }, # expected conversion to base16 encoding
    # 33 # base64url rfc4648, utf-8
    # 34 # base64 rfc46484, utf-8
    # 35 # regex pcre/ecma262, utf-8
    # 36 # mime message rfc2045, utf-8
+   264 => sub { # decimal fraction with arbitrary exponent
+      require Math::BigFloat;
+      Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
+   },
+   265 => sub { # bigfloat with arbitrary exponent
+      require Math::BigFloat;
+      scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
+   },
 );
 sub CBOR::XS::default_filter {
    &{ $FILTER{$_[0]} or return }
 }
    utf8::upgrade $uri;
    tag 32, $uri
 }
 sub Math::BigInt::TO_CBOR {
-   if ($_[0] >= -2147483648 && $_[0] <= 2147483647) {
+   if (-2147483648 <= $_[0] && $_[0] <= 2147483647) {
       $_[0]->numify
    } else {
       my $hex = substr $_[0]->as_hex, 2;
       $hex = "0$hex" if 1 & length $hex; # sigh
       tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
    }
 }
 sub Math::BigFloat::TO_CBOR {
    my ($m, $e) = $_[0]->parts;
+   -9223372036854775808 <= $e && $e <= 18446744073709551615
-   tag 4, [$e->numify, $m]
+      ? tag 4, [$e->numify, $m]
+      : tag 264, [$e, $m]
 }
 sub Time::Piece::TO_CBOR {
    tag 1, 0 + $_[0]->epoch
 }

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.49 by root, Thu Apr 21 16:24:03 2016 UTC vs. Revision 1.56 by root, Mon Apr 25 21:44:13 2016 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.49 by root, Thu Apr 21 16:24:03 2016 UTC vs.
Revision 1.56 by root, Mon Apr 25 21:44:13 2016 UTC