ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/XS.pm
(Generate patch)

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.28 by root, Thu Nov 28 16:09:04 2013 UTC vs.
Revision 1.62 by root, Fri Nov 25 06:13:16 2016 UTC

48Regarding compactness, C<CBOR::XS>-encoded data structures are usually 48Regarding compactness, C<CBOR::XS>-encoded data structures are usually
49about 20% smaller than the same data encoded as (compact) JSON or 49about 20% smaller than the same data encoded as (compact) JSON or
50L<Storable>. 50L<Storable>.
51 51
52In addition to the core CBOR data format, this module implements a 52In addition to the core CBOR data format, this module implements a
53number of extensions, to support cyclic and shared data structures (see 53number of extensions, to support cyclic and shared data structures
54C<allow_sharing>), string deduplication (see C<pack_strings>) and scalar 54(see C<allow_sharing> and C<allow_cycles>), string deduplication (see
55references (always enabled). 55C<pack_strings>) and scalar references (always enabled).
56 56
57The primary goal of this module is to be I<correct> and the secondary goal 57The primary goal of this module is to be I<correct> and the secondary goal
58is to be I<fast>. To reach the latter goal it was written in C. 58is to be I<fast>. To reach the latter goal it was written in C.
59 59
60See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and 60See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
64 64
65package CBOR::XS; 65package CBOR::XS;
66 66
67use common::sense; 67use common::sense;
68 68
69our $VERSION = '1.0'; 69our $VERSION = 1.51;
70our @ISA = qw(Exporter); 70our @ISA = qw(Exporter);
71 71
72our @EXPORT = qw(encode_cbor decode_cbor); 72our @EXPORT = qw(encode_cbor decode_cbor);
73 73
74use Exporter; 74use Exporter;
180reference to the earlier value. 180reference to the earlier value.
181 181
182This means that such values will only be encoded once, and will not result 182This means that such values will only be encoded once, and will not result
183in a deep cloning of the value on decode, in decoders supporting the value 183in a deep cloning of the value on decode, in decoders supporting the value
184sharing extension. This also makes it possible to encode cyclic data 184sharing extension. This also makes it possible to encode cyclic data
185structures. 185structures (which need C<allow_cycles> to be enabled to be decoded by this
186module).
186 187
187It is recommended to leave it off unless you know your 188It is recommended to leave it off unless you know your
188communication partner supports the value sharing extensions to CBOR 189communication partner supports the value sharing extensions to CBOR
189(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the 190(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the
190resulting data structure might be unusable. 191resulting data structure might be unusable.
191 192
192Detecting shared values incurs a runtime overhead when values are encoded 193Detecting shared values incurs a runtime overhead when values are encoded
193that have a reference counter large than one, and might unnecessarily 194that have a reference counter large than one, and might unnecessarily
194increase the encoded size, as potentially shared values are encode as 195increase the encoded size, as potentially shared values are encode as
195sharable whether or not they are actually shared. 196shareable whether or not they are actually shared.
196 197
197At the moment, only targets of references can be shared (e.g. scalars, 198At the moment, only targets of references can be shared (e.g. scalars,
198arrays or hashes pointed to by a reference). Weirder constructs, such as 199arrays or hashes pointed to by a reference). Weirder constructs, such as
199an array with multiple "copies" of the I<same> string, which are hard but 200an array with multiple "copies" of the I<same> string, which are hard but
200not impossible to create in Perl, are not supported (this is the same as 201not impossible to create in Perl, are not supported (this is the same as
205structures cannot be encoded in this mode. 206structures cannot be encoded in this mode.
206 207
207This option does not affect C<decode> in any way - shared values and 208This option does not affect C<decode> in any way - shared values and
208references will always be decoded properly if present. 209references will always be decoded properly if present.
209 210
211=item $cbor = $cbor->allow_cycles ([$enable])
212
213=item $enabled = $cbor->get_allow_cycles
214
215If C<$enable> is true (or missing), then C<decode> will happily decode
216self-referential (cyclic) data structures. By default these will not be
217decoded, as they need manual cleanup to avoid memory leaks, so code that
218isn't prepared for this will not leak memory.
219
220If C<$enable> is false (the default), then C<decode> will throw an error
221when it encounters a self-referential/cyclic data structure.
222
223FUTURE DIRECTION: the motivation behind this option is to avoid I<real>
224cycles - future versions of this module might chose to decode cyclic data
225structures using weak references when this option is off, instead of
226throwing an error.
227
228This option does not affect C<encode> in any way - shared values and
229references will always be encoded properly if present.
230
210=item $cbor = $cbor->pack_strings ([$enable]) 231=item $cbor = $cbor->pack_strings ([$enable])
211 232
212=item $enabled = $cbor->get_pack_strings 233=item $enabled = $cbor->get_pack_strings
213 234
214If C<$enable> is true (or missing), then C<encode> will try not to encode 235If C<$enable> is true (or missing), then C<encode> will try not to encode
226the standard CBOR way. 247the standard CBOR way.
227 248
228This option does not affect C<decode> in any way - string references will 249This option does not affect C<decode> in any way - string references will
229always be decoded properly if present. 250always be decoded properly if present.
230 251
252=item $cbor = $cbor->text_keys ([$enable])
253
254=item $enabled = $cbor->get_text_keys
255
256If C<$enabled> is true (or missing), then C<encode> will encode all
257perl hash keys as CBOR text strings/UTF-8 string, upgrading them as needed.
258
259If C<$enable> is false (the default), then C<encode> will encode hash keys
260normally - upgraded perl strings (strings internally encoded as UTF-8) as
261CBOR text strings, and downgraded perl strings as CBOR byte strings.
262
263This option does not affect C<decode> in any way.
264
265This option is useful for interoperability with CBOR decoders that don't
266treat byte strings as a form of text. It is especially useful as Perl
267gives very little control over hash keys.
268
269Enabling this option can be slow, as all downgraded hash keys that are
270encoded need to be scanned and converted to UTF-8.
271
272=item $cbor = $cbor->text_strings ([$enable])
273
274=item $enabled = $cbor->get_text_strings
275
276This option works similar to C<text_keys>, above, but works on all strings
277(including hash keys), so C<text_keys> has no further effect after
278enabling C<text_strings>.
279
280If C<$enabled> is true (or missing), then C<encode> will encode all perl
281strings as CBOR text strings/UTF-8 strings, upgrading them as needed.
282
283If C<$enable> is false (the default), then C<encode> will encode strings
284normally (but see C<text_keys>) - upgraded perl strings (strings
285internally encoded as UTF-8) as CBOR text strings, and downgraded perl
286strings as CBOR byte strings.
287
288This option does not affect C<decode> in any way.
289
290This option has similar advantages and disadvantages as C<text_keys>. In
291addition, this option effectively removes the ability to encode byte
292strings, which might break some C<FREEZE> and C<TO_CBOR> methods that rely
293on this, such as bignum encoding, so this option is mainly useful for very
294simple data.
295
296=item $cbor = $cbor->validate_utf8 ([$enable])
297
298=item $enabled = $cbor->get_validate_utf8
299
300If C<$enable> is true (or missing), then C<decode> will validate that
301elements (text strings) containing UTF-8 data in fact contain valid UTF-8
302data (instead of blindly accepting it). This validation obviously takes
303extra time during decoding.
304
305The concept of "valid UTF-8" used is perl's concept, which is a superset
306of the official UTF-8.
307
308If C<$enable> is false (the default), then C<decode> will blindly accept
309UTF-8 data, marking them as valid UTF-8 in the resulting data structure
310regardless of whether that's true or not.
311
312Perl isn't too happy about corrupted UTF-8 in strings, but should
313generally not crash or do similarly evil things. Extensions might be not
314so forgiving, so it's recommended to turn on this setting if you receive
315untrusted CBOR.
316
317This option does not affect C<encode> in any way - strings that are
318supposedly valid UTF-8 will simply be dumped into the resulting CBOR
319string without checking whether that is, in fact, true or not.
320
231=item $cbor = $cbor->filter ([$cb->($tag, $value)]) 321=item $cbor = $cbor->filter ([$cb->($tag, $value)])
232 322
233=item $cb_or_undef = $cbor->get_filter 323=item $cb_or_undef = $cbor->get_filter
234 324
235Sets or replaces the tagged value decoding filter (when C<$cb> is 325Sets or replaces the tagged value decoding filter (when C<$cb> is
292 CBOR::XS->new->decode_prefix ("......") 382 CBOR::XS->new->decode_prefix ("......")
293 => ("...", 3) 383 => ("...", 3)
294 384
295=back 385=back
296 386
387=head2 INCREMENTAL PARSING
388
389In some cases, there is the need for incremental parsing of JSON
390texts. While this module always has to keep both CBOR text and resulting
391Perl data structure in memory at one time, it does allow you to parse a
392CBOR stream incrementally, using a similar to using "decode_prefix" to see
393if a full CBOR object is available, but is much more efficient.
394
395It basically works by parsing as much of a CBOR string as possible - if
396the CBOR data is not complete yet, the pasrer will remember where it was,
397to be able to restart when more data has been accumulated. Once enough
398data is available to either decode a complete CBOR value or raise an
399error, a real decode will be attempted.
400
401A typical use case would be a network protocol that consists of sending
402and receiving CBOR-encoded messages. The solution that works with CBOR and
403about anything else is by prepending a length to every CBOR value, so the
404receiver knows how many octets to read. More compact (and slightly slower)
405would be to just send CBOR values back-to-back, as C<CBOR::XS> knows where
406a CBOR value ends, and doesn't need an explicit length.
407
408The following methods help with this:
409
410=over 4
411
412=item @decoded = $cbor->incr_parse ($buffer)
413
414This method attempts to decode exactly one CBOR value from the beginning
415of the given C<$buffer>. The value is removed from the C<$buffer> on
416success. When C<$buffer> doesn't contain a complete value yet, it returns
417nothing. Finally, when the C<$buffer> doesn't start with something
418that could ever be a valid CBOR value, it raises an exception, just as
419C<decode> would. In the latter case the decoder state is undefined and
420must be reset before being able to parse further.
421
422This method modifies the C<$buffer> in place. When no CBOR value can be
423decoded, the decoder stores the current string offset. On the next call,
424continues decoding at the place where it stopped before. For this to make
425sense, the C<$buffer> must begin with the same octets as on previous
426unsuccessful calls.
427
428You can call this method in scalar context, in which case it either
429returns a decoded value or C<undef>. This makes it impossible to
430distinguish between CBOR null values (which decode to C<undef>) and an
431unsuccessful decode, which is often acceptable.
432
433=item @decoded = $cbor->incr_parse_multiple ($buffer)
434
435Same as C<incr_parse>, but attempts to decode as many CBOR values as
436possible in one go, instead of at most one. Calls to C<incr_parse> and
437C<incr_parse_multiple> can be interleaved.
438
439=item $cbor->incr_reset
440
441Resets the incremental decoder. This throws away any saved state, so that
442subsequent calls to C<incr_parse> or C<incr_parse_multiple> start to parse
443a new CBOR value from the beginning of the C<$buffer> again.
444
445This method can be called at any time, but it I<must> be called if you want
446to change your C<$buffer> or there was a decoding error and you want to
447reuse the C<$cbor> object for future incremental parsings.
448
449=back
450
297 451
298=head1 MAPPING 452=head1 MAPPING
299 453
300This section describes how CBOR::XS maps Perl values to CBOR values and 454This section describes how CBOR::XS maps Perl values to CBOR values and
301vice versa. These mappings are designed to "do the right thing" in most 455vice versa. These mappings are designed to "do the right thing" in most
371 525
372=item hash references 526=item hash references
373 527
374Perl hash references become CBOR maps. As there is no inherent ordering in 528Perl hash references become CBOR maps. As there is no inherent ordering in
375hash keys (or CBOR maps), they will usually be encoded in a pseudo-random 529hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
376order. This order can be different each time a hahs is encoded. 530order. This order can be different each time a hash is encoded.
377 531
378Currently, tied hashes will use the indefinite-length format, while normal 532Currently, tied hashes will use the indefinite-length format, while normal
379hashes will use the fixed-length format. 533hashes will use the fixed-length format.
380 534
381=item array references 535=item array references
434 my $x = 3.1; # some variable containing a number 588 my $x = 3.1; # some variable containing a number
435 "$x"; # stringified 589 "$x"; # stringified
436 $x .= ""; # another, more awkward way to stringify 590 $x .= ""; # another, more awkward way to stringify
437 print $x; # perl does it for you, too, quite often 591 print $x; # perl does it for you, too, quite often
438 592
439You can force whether a string ie encoded as byte or text string by using 593You can force whether a string is encoded as byte or text string by using
440C<utf8::upgrade> and C<utf8::downgrade>): 594C<utf8::upgrade> and C<utf8::downgrade> (if C<text_strings> is disabled):
441 595
442 utf8::upgrade $x; # encode $x as text string 596 utf8::upgrade $x; # encode $x as text string
443 utf8::downgrade $x; # encode $x as byte string 597 utf8::downgrade $x; # encode $x as byte string
444 598
445Perl doesn't define what operations up- and downgrade strings, so if the 599Perl doesn't define what operations up- and downgrade strings, so if the
446difference between byte and text is important, you should up- or downgrade 600difference between byte and text is important, you should up- or downgrade
447your string as late as possible before encoding. 601your string as late as possible before encoding. You can also force the
602use of CBOR text strings by using C<text_keys> or C<text_strings>.
448 603
449You can force the type to be a CBOR number by numifying it: 604You can force the type to be a CBOR number by numifying it:
450 605
451 my $x = "3"; # some variable containing a string 606 my $x = "3"; # some variable containing a string
452 $x += 0; # numify it, ensuring it will be dumped as a number 607 $x += 0; # numify it, ensuring it will be dumped as a number
465 620
466=back 621=back
467 622
468=head2 OBJECT SERIALISATION 623=head2 OBJECT SERIALISATION
469 624
625This module implements both a CBOR-specific and the generic
626L<Types::Serialier> object serialisation protocol. The following
627subsections explain both methods.
628
629=head3 ENCODING
630
470This module knows two way to serialise a Perl object: The CBOR-specific 631This module knows two way to serialise a Perl object: The CBOR-specific
471way, and the generic way. 632way, and the generic way.
472 633
473Whenever the encoder encounters a Perl object that it cnanot serialise 634Whenever the encoder encounters a Perl object that it cannot serialise
474directly (most of them), it will first look up the C<TO_CBOR> method on 635directly (most of them), it will first look up the C<TO_CBOR> method on
475it. 636it.
476 637
477If it has a C<TO_CBOR> method, it will call it with the object as only 638If it has a C<TO_CBOR> method, it will call it with the object as only
478argument, and expects exactly one return value, which it will then 639argument, and expects exactly one return value, which it will then
484 645
485The C<FREEZE> method can return any number of values (i.e. zero or 646The C<FREEZE> method can return any number of values (i.e. zero or
486more). These will be encoded as CBOR perl object, together with the 647more). These will be encoded as CBOR perl object, together with the
487classname. 648classname.
488 649
650These methods I<MUST NOT> change the data structure that is being
651serialised. Failure to comply to this can result in memory corruption -
652and worse.
653
489If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail 654If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
490with an error. 655with an error.
491 656
657=head3 DECODING
658
492Objects encoded via C<TO_CBOR> cannot be automatically decoded, but 659Objects encoded via C<TO_CBOR> cannot (normally) be automatically decoded,
493objects encoded via C<FREEZE> can be decoded using the following protocol: 660but objects encoded via C<FREEZE> can be decoded using the following
661protocol:
494 662
495When an encoded CBOR perl object is encountered by the decoder, it will 663When an encoded CBOR perl object is encountered by the decoder, it will
496look up the C<THAW> method, by using the stored classname, and will fail 664look up the C<THAW> method, by using the stored classname, and will fail
497if the method cannot be found. 665if the method cannot be found.
498 666
499After the lookup it will call the C<THAW> method with the stored classname 667After the lookup it will call the C<THAW> method with the stored classname
500as first argument, the constant string C<CBOR> as second argument, and all 668as first argument, the constant string C<CBOR> as second argument, and all
501values returned by C<FREEZE> as remaining arguments. 669values returned by C<FREEZE> as remaining arguments.
502 670
503=head4 EXAMPLES 671=head3 EXAMPLES
504 672
505Here is an example C<TO_CBOR> method: 673Here is an example C<TO_CBOR> method:
506 674
507 sub My::Object::TO_CBOR { 675 sub My::Object::TO_CBOR {
508 my ($obj) = @_; 676 my ($obj) = @_;
540 "$self" # encode url string 708 "$self" # encode url string
541 } 709 }
542 710
543 sub URI::THAW { 711 sub URI::THAW {
544 my ($class, $serialiser, $uri) = @_; 712 my ($class, $serialiser, $uri) = @_;
545
546 $class->new ($uri) 713 $class->new ($uri)
547 } 714 }
548 715
549Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For 716Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For
550example, a C<FREEZE> method that returns "type", "id" and "variant" values 717example, a C<FREEZE> method that returns "type", "id" and "variant" values
681additional tags (such as base64url). 848additional tags (such as base64url).
682 849
683=head2 ENFORCED TAGS 850=head2 ENFORCED TAGS
684 851
685These tags are always handled when decoding, and their handling cannot be 852These tags are always handled when decoding, and their handling cannot be
686overriden by the user. 853overridden by the user.
687 854
688=over 4 855=over 4
689 856
690=item 26 (perl-object, L<http://cbor.schmorp.de/perl-object>) 857=item 26 (perl-object, L<http://cbor.schmorp.de/perl-object>)
691 858
692These tags are automatically created (and decoded) for serialisable 859These tags are automatically created (and decoded) for serialisable
693objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object 860objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
694serialisation protocol). See L<OBJECT SERIALISATION> for details. 861serialisation protocol). See L<OBJECT SERIALISATION> for details.
695 862
696=item 28, 29 (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>) 863=item 28, 29 (shareable, sharedref, L<http://cbor.schmorp.de/value-sharing>)
697 864
698These tags are automatically decoded when encountered, resulting in 865These tags are automatically decoded when encountered (and they do not
866result in a cyclic data structure, see C<allow_cycles>), resulting in
699shared values in the decoded object. They are only encoded, however, when 867shared values in the decoded object. They are only encoded, however, when
700C<allow_sharable> is enabled. 868C<allow_sharing> is enabled.
701 869
870Not all shared values can be successfully decoded: values that reference
871themselves will I<currently> decode as C<undef> (this is not the same
872as a reference pointing to itself, which will be represented as a value
873that contains an indirect reference to itself - these will be decoded
874properly).
875
876Note that considerably more shared value data structures can be decoded
877than will be encoded - currently, only values pointed to by references
878will be shared, others will not. While non-reference shared values can be
879generated in Perl with some effort, they were considered too unimportant
880to be supported in the encoder. The decoder, however, will decode these
881values as shared values.
882
702=item 256, 25 (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>) 883=item 256, 25 (stringref-namespace, stringref, L<http://cbor.schmorp.de/stringref>)
703 884
704These tags are automatically decoded when encountered. They are only 885These tags are automatically decoded when encountered. They are only
705encoded, however, when C<pack_strings> is enabled. 886encoded, however, when C<pack_strings> is enabled.
706 887
707=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>) 888=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
708 889
709This tag is automatically generated when a reference are encountered (with 890This tag is automatically generated when a reference are encountered (with
710the exception of hash and array refernces). It is converted to a reference 891the exception of hash and array references). It is converted to a reference
711when decoding. 892when decoding.
712 893
713=item 55799 (self-describe CBOR, RFC 7049) 894=item 55799 (self-describe CBOR, RFC 7049)
714 895
715This value is not generated on encoding (unless explicitly requested by 896This value is not generated on encoding (unless explicitly requested by
718=back 899=back
719 900
720=head2 NON-ENFORCED TAGS 901=head2 NON-ENFORCED TAGS
721 902
722These tags have default filters provided when decoding. Their handling can 903These tags have default filters provided when decoding. Their handling can
723be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by 904be overridden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
724providing a custom C<filter> callback when decoding. 905providing a custom C<filter> callback when decoding.
725 906
726When they result in decoding into a specific Perl class, the module 907When they result in decoding into a specific Perl class, the module
727usually provides a corresponding C<TO_CBOR> method as well. 908usually provides a corresponding C<TO_CBOR> method as well.
728 909
731provide these modules. The decoding usually fails with an exception if the 912provide these modules. The decoding usually fails with an exception if the
732required module cannot be loaded. 913required module cannot be loaded.
733 914
734=over 4 915=over 4
735 916
917=item 0, 1 (date/time string, seconds since the epoch)
918
919These tags are decoded into L<Time::Piece> objects. The corresponding
920C<Time::Piece::TO_CBOR> method always encodes into tag 1 values currently.
921
922The L<Time::Piece> API is generally surprisingly bad, and fractional
923seconds are only accidentally kept intact, so watch out. On the plus side,
924the module comes with perl since 5.10, which has to count for something.
925
736=item 2, 3 (positive/negative bignum) 926=item 2, 3 (positive/negative bignum)
737 927
738These tags are decoded into L<Math::BigInt> objects. The corresponding 928These tags are decoded into L<Math::BigInt> objects. The corresponding
739C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR 929C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR
740integers, and others into positive/negative CBOR bignums. 930integers, and others into positive/negative CBOR bignums.
741 931
742=item 4, 5 (decimal fraction/bigfloat) 932=item 4, 5, 264, 265 (decimal fraction/bigfloat)
743 933
744Both decimal fractions and bigfloats are decoded into L<Math::BigFloat> 934Both decimal fractions and bigfloats are decoded into L<Math::BigFloat>
745objects. The corresponding C<Math::BigFloat::TO_CBOR> method I<always> 935objects. The corresponding C<Math::BigFloat::TO_CBOR> method I<always>
746encodes into a decimal fraction. 936encodes into a decimal fraction (either tag 4 or 264).
747 937
748CBOR cannot represent bigfloats with I<very> large exponents - conversion 938NaN and infinities are not encoded properly, as they cannot be represented
749of such big float objects is undefined. 939in CBOR.
750 940
751Also, NaN and infinities are not encoded properly. 941See L<BIGNUM SECURITY CONSIDERATIONS> for more info.
942
943=item 30 (rational numbers)
944
945These tags are decoded into L<Math::BigRat> objects. The corresponding
946C<Math::BigRat::TO_CBOR> method encodes rational numbers with denominator
947C<1> via their numerator only, i.e., they become normal integers or
948C<bignums>.
949
950See L<BIGNUM SECURITY CONSIDERATIONS> for more info.
752 951
753=item 21, 22, 23 (expected later JSON conversion) 952=item 21, 22, 23 (expected later JSON conversion)
754 953
755CBOR::XS is not a CBOR-to-JSON converter, and will simply ignore these 954CBOR::XS is not a CBOR-to-JSON converter, and will simply ignore these
756tags. 955tags.
761C<URI::TO_CBOR> method again results in a CBOR URI value. 960C<URI::TO_CBOR> method again results in a CBOR URI value.
762 961
763=back 962=back
764 963
765=cut 964=cut
766
767our %FILTER = (
768 # 0 # rfc4287 datetime, utf-8
769 # 1 # unix timestamp, any
770
771 2 => sub { # pos bigint
772 require Math::BigInt;
773 Math::BigInt->new ("0x" . unpack "H*", pop)
774 },
775
776 3 => sub { # neg bigint
777 require Math::BigInt;
778 -Math::BigInt->new ("0x" . unpack "H*", pop)
779 },
780
781 4 => sub { # decimal fraction, array
782 require Math::BigFloat;
783 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
784 },
785
786 5 => sub { # bigfloat, array
787 require Math::BigFloat;
788 scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
789 },
790
791 21 => sub { pop }, # expected conversion to base64url encoding
792 22 => sub { pop }, # expected conversion to base64 encoding
793 23 => sub { pop }, # expected conversion to base16 encoding
794
795 # 24 # embedded cbor, byte string
796
797 32 => sub {
798 require URI;
799 URI->new (pop)
800 },
801
802 # 33 # base64url rfc4648, utf-8
803 # 34 # base64 rfc46484, utf-8
804 # 35 # regex pcre/ecma262, utf-8
805 # 36 # mime message rfc2045, utf-8
806);
807
808 965
809=head1 CBOR and JSON 966=head1 CBOR and JSON
810 967
811CBOR is supposed to implement a superset of the JSON data model, and is, 968CBOR is supposed to implement a superset of the JSON data model, and is,
812with some coercion, able to represent all JSON texts (something that other 969with some coercion, able to represent all JSON texts (something that other
854Also keep in mind that CBOR::XS might leak contents of your Perl data 1011Also keep in mind that CBOR::XS might leak contents of your Perl data
855structures in its error messages, so when you serialise sensitive 1012structures in its error messages, so when you serialise sensitive
856information you might want to make sure that exceptions thrown by CBOR::XS 1013information you might want to make sure that exceptions thrown by CBOR::XS
857will not end up in front of untrusted eyes. 1014will not end up in front of untrusted eyes.
858 1015
1016
1017=head1 BIGNUM SECURITY CONSIDERATIONS
1018
1019CBOR::XS provides a C<TO_CBOR> method for both L<Math::BigInt> and
1020L<Math::BigFloat> that tries to encode the number in the simplest possible
1021way, that is, either a CBOR integer, a CBOR bigint/decimal fraction (tag
10224) or an arbitrary-exponent decimal fraction (tag 264). Rational numbers
1023(L<Math::BigRat>, tag 30) can also contain bignums as members.
1024
1025CBOR::XS will also understand base-2 bigfloat or arbitrary-exponent
1026bigfloats (tags 5 and 265), but it will never generate these on its own.
1027
1028Using the built-in L<Math::BigInt::Calc> support, encoding and decoding
1029decimal fractions is generally fast. Decoding bigints can be slow for very
1030big numbers (tens of thousands of digits, something that could potentially
1031be caught by limiting the size of CBOR texts), and decoding bigfloats or
1032arbitrary-exponent bigfloats can be I<extremely> slow (minutes, decades)
1033for large exponents (roughly 40 bit and longer).
1034
1035Additionally, L<Math::BigInt> can take advantage of other bignum
1036libraries, such as L<Math::GMP>, which cannot handle big floats with large
1037exponents, and might simply abort or crash your program, due to their code
1038quality.
1039
1040This can be a concern if you want to parse untrusted CBOR. If it is, you
1041might want to disable decoding of tag 2 (bigint) and 3 (negative bigint)
1042types. You should also disable types 5 and 265, as these can be slow even
1043without bigints.
1044
1045Disabling bigints will also partially or fully disable types that rely on
1046them, e.g. rational numbers that use bignums.
1047
1048
859=head1 CBOR IMPLEMENTATION NOTES 1049=head1 CBOR IMPLEMENTATION NOTES
860 1050
861This section contains some random implementation notes. They do not 1051This section contains some random implementation notes. They do not
862describe guaranteed behaviour, but merely behaviour as-is implemented 1052describe guaranteed behaviour, but merely behaviour as-is implemented
863right now. 1053right now.
871Only the double data type is supported for NV data types - when Perl uses 1061Only the double data type is supported for NV data types - when Perl uses
872long double to represent floating point values, they might not be encoded 1062long double to represent floating point values, they might not be encoded
873properly. Half precision types are accepted, but not encoded. 1063properly. Half precision types are accepted, but not encoded.
874 1064
875Strict mode and canonical mode are not implemented. 1065Strict mode and canonical mode are not implemented.
1066
1067
1068=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT
1069
1070On perls that were built without 64 bit integer support (these are rare
1071nowadays, even on 32 bit architectures, as all major Perl distributions
1072are built with 64 bit integer support), support for any kind of 64 bit
1073integer in CBOR is very limited - most likely, these 64 bit values will
1074be truncated, corrupted, or otherwise not decoded correctly. This also
1075includes string, array and map sizes that are stored as 64 bit integers.
876 1076
877 1077
878=head1 THREADS 1078=head1 THREADS
879 1079
880This module is I<not> guaranteed to be thread safe and there are no 1080This module is I<not> guaranteed to be thread safe and there are no
895service. I put the contact address into my modules for a reason. 1095service. I put the contact address into my modules for a reason.
896 1096
897=cut 1097=cut
898 1098
899our %FILTER = ( 1099our %FILTER = (
900 # 0 # rfc4287 datetime, utf-8 1100 0 => sub { # rfc4287 datetime, utf-8
901 # 1 # unix timestamp, any 1101 require Time::Piece;
1102 # Time::Piece::Strptime uses the "incredibly flexible date parsing routine"
1103 # from FreeBSD, which can't parse ISO 8601, RFC3339, RFC4287 or much of anything
1104 # else either. Whats incredibe over standard strptime totally escapes me.
1105 # doesn't do fractional times, either. sigh.
1106 # In fact, it's all a lie, it uses whatever strptime it wants, and of course,
1107 # they are all incompatible. The openbsd one simply ignores %z (but according to the
1108 # docs, it would be much more incredibly flexible indeed. If it worked, that is.).
1109 scalar eval {
1110 my $s = $_[1];
1111
1112 $s =~ s/Z$/+00:00/;
1113 $s =~ s/(\.[0-9]+)?([+-][0-9][0-9]):([0-9][0-9])$//
1114 or die;
1115
1116 my $b = $1 - ($2 * 60 + $3) * 60; # fractional part + offset. hopefully
1117 my $d = Time::Piece->strptime ($s, "%Y-%m-%dT%H:%M:%S");
1118
1119 Time::Piece::gmtime ($d->epoch + $b)
1120 } || die "corrupted CBOR date/time string ($_[0])";
1121 },
1122
1123 1 => sub { # seconds since the epoch, possibly fractional
1124 require Time::Piece;
1125 scalar Time::Piece::gmtime (pop)
1126 },
902 1127
903 2 => sub { # pos bigint 1128 2 => sub { # pos bigint
904 require Math::BigInt; 1129 require Math::BigInt;
905 Math::BigInt->new ("0x" . unpack "H*", pop) 1130 Math::BigInt->new ("0x" . unpack "H*", pop)
906 }, 1131 },
913 4 => sub { # decimal fraction, array 1138 4 => sub { # decimal fraction, array
914 require Math::BigFloat; 1139 require Math::BigFloat;
915 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0]) 1140 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
916 }, 1141 },
917 1142
1143 264 => sub { # decimal fraction with arbitrary exponent
1144 require Math::BigFloat;
1145 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
1146 },
1147
918 5 => sub { # bigfloat, array 1148 5 => sub { # bigfloat, array
919 require Math::BigFloat; 1149 require Math::BigFloat;
920 scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2) 1150 scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
1151 },
1152
1153 265 => sub { # bigfloat with arbitrary exponent
1154 require Math::BigFloat;
1155 scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
1156 },
1157
1158 30 => sub { # rational number
1159 require Math::BigRat;
1160 Math::BigRat->new ("$_[1][0]/$_[1][1]") # separate parameters only work in recent versons
921 }, 1161 },
922 1162
923 21 => sub { pop }, # expected conversion to base64url encoding 1163 21 => sub { pop }, # expected conversion to base64url encoding
924 22 => sub { pop }, # expected conversion to base64 encoding 1164 22 => sub { pop }, # expected conversion to base64 encoding
925 23 => sub { pop }, # expected conversion to base16 encoding 1165 23 => sub { pop }, # expected conversion to base16 encoding
942} 1182}
943 1183
944sub URI::TO_CBOR { 1184sub URI::TO_CBOR {
945 my $uri = $_[0]->as_string; 1185 my $uri = $_[0]->as_string;
946 utf8::upgrade $uri; 1186 utf8::upgrade $uri;
947 CBOR::XS::tag 32, $uri 1187 tag 32, $uri
948} 1188}
949 1189
950sub Math::BigInt::TO_CBOR { 1190sub Math::BigInt::TO_CBOR {
951 if ($_[0] >= -2147483648 && $_[0] <= 2147483647) { 1191 if (-2147483648 <= $_[0] && $_[0] <= 2147483647) {
952 $_[0]->numify 1192 $_[0]->numify
953 } else { 1193 } else {
954 my $hex = substr $_[0]->as_hex, 2; 1194 my $hex = substr $_[0]->as_hex, 2;
955 $hex = "0$hex" if 1 & length $hex; # sigh 1195 $hex = "0$hex" if 1 & length $hex; # sigh
956 CBOR::XS::tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex 1196 tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
957 } 1197 }
958} 1198}
959 1199
960sub Math::BigFloat::TO_CBOR { 1200sub Math::BigFloat::TO_CBOR {
961 my ($m, $e) = $_[0]->parts; 1201 my ($m, $e) = $_[0]->parts;
1202
1203 -9223372036854775808 <= $e && $e <= 18446744073709551615
962 CBOR::XS::tag 4, [$e->numify, $m] 1204 ? tag 4, [$e->numify, $m]
1205 : tag 264, [$e, $m]
1206}
1207
1208sub Math::BigRat::TO_CBOR {
1209 my ($n, $d) = $_[0]->parts;
1210
1211 # older versions of BigRat need *1, as they not always return numbers
1212
1213 $d*1 == 1
1214 ? $n*1
1215 : tag 30, [$n*1, $d*1]
1216}
1217
1218sub Time::Piece::TO_CBOR {
1219 tag 1, 0 + $_[0]->epoch
963} 1220}
964 1221
965XSLoader::load "CBOR::XS", $VERSION; 1222XSLoader::load "CBOR::XS", $VERSION;
966 1223
967=head1 SEE ALSO 1224=head1 SEE ALSO

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines