ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/XS.pm
(Generate patch)

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.25 by root, Thu Nov 28 12:08:07 2013 UTC vs.
Revision 1.51 by root, Sun Apr 24 19:16:15 2016 UTC

26 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string 26 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
27 } 27 }
28 28
29=head1 DESCRIPTION 29=head1 DESCRIPTION
30 30
31WARNING! This module is very new, and not very well tested (that's up
32to you to do). Furthermore, details of the implementation might change
33freely before version 1.0. And lastly, most extensions depend on an IANA
34assignment, and until that assignment is official, this implementation is
35not interoperable with other implementations (even future versions of this
36module) until the assignment is done.
37
38You are still invited to try out CBOR, and this module.
39
40This module converts Perl data structures to the Concise Binary Object 31This module converts Perl data structures to the Concise Binary Object
41Representation (CBOR) and vice versa. CBOR is a fast binary serialisation 32Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
42format that aims to use a superset of the JSON data model, i.e. when you 33format that aims to use an (almost) superset of the JSON data model, i.e.
43can represent something in JSON, you should be able to represent it in 34when you can represent something useful in JSON, you should be able to
44CBOR. 35represent it in CBOR.
45 36
46In short, CBOR is a faster and very compact binary alternative to JSON, 37In short, CBOR is a faster and quite compact binary alternative to JSON,
47with the added ability of supporting serialisation of Perl objects. (JSON 38with the added ability of supporting serialisation of Perl objects. (JSON
48often compresses better than CBOR though, so if you plan to compress the 39often compresses better than CBOR though, so if you plan to compress the
49data later you might want to compare both formats first). 40data later and speed is less important you might want to compare both
41formats first).
50 42
51To give you a general idea about speed, with texts in the megabyte range, 43To give you a general idea about speed, with texts in the megabyte range,
52C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or 44C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or
53L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the 45L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the
54data, the worse L<Storable> performs in comparison. 46data, the worse L<Storable> performs in comparison.
55 47
56As for compactness, C<CBOR::XS> encoded data structures are usually about 48Regarding compactness, C<CBOR::XS>-encoded data structures are usually
5720% smaller than the same data encoded as (compact) JSON or L<Storable>. 49about 20% smaller than the same data encoded as (compact) JSON or
50L<Storable>.
58 51
59In addition to the core CBOR data format, this module implements a number 52In addition to the core CBOR data format, this module implements a
60of extensions, to support cyclic and self-referencing data structures 53number of extensions, to support cyclic and shared data structures
61(see C<allow_sharing>), string deduplication (see C<pack_strings>) and 54(see C<allow_sharing> and C<allow_cycles>), string deduplication (see
62scalar references (always enabled). 55C<pack_strings>) and scalar references (always enabled).
63 56
64The primary goal of this module is to be I<correct> and the secondary goal 57The primary goal of this module is to be I<correct> and the secondary goal
65is to be I<fast>. To reach the latter goal it was written in C. 58is to be I<fast>. To reach the latter goal it was written in C.
66 59
67See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and 60See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
71 64
72package CBOR::XS; 65package CBOR::XS;
73 66
74use common::sense; 67use common::sense;
75 68
76our $VERSION = 0.09; 69our $VERSION = 1.41;
77our @ISA = qw(Exporter); 70our @ISA = qw(Exporter);
78 71
79our @EXPORT = qw(encode_cbor decode_cbor); 72our @EXPORT = qw(encode_cbor decode_cbor);
80 73
81use Exporter; 74use Exporter;
187reference to the earlier value. 180reference to the earlier value.
188 181
189This means that such values will only be encoded once, and will not result 182This means that such values will only be encoded once, and will not result
190in a deep cloning of the value on decode, in decoders supporting the value 183in a deep cloning of the value on decode, in decoders supporting the value
191sharing extension. This also makes it possible to encode cyclic data 184sharing extension. This also makes it possible to encode cyclic data
192structures. 185structures (which need C<allow_cycles> to ne enabled to be decoded by this
186module).
193 187
194It is recommended to leave it off unless you know your 188It is recommended to leave it off unless you know your
195communication partner supports the value sharing extensions to CBOR 189communication partner supports the value sharing extensions to CBOR
196(http://cbor.schmorp.de/value-sharing), as without decoder support, the 190(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the
197resulting data structure might be unusable. 191resulting data structure might be unusable.
198 192
199Detecting shared values incurs a runtime overhead when values are encoded 193Detecting shared values incurs a runtime overhead when values are encoded
200that have a reference counter large than one, and might unnecessarily 194that have a reference counter large than one, and might unnecessarily
201increase the encoded size, as potentially shared values are encode as 195increase the encoded size, as potentially shared values are encode as
202sharable whether or not they are actually shared. 196shareable whether or not they are actually shared.
203 197
204At the moment, only targets of references can be shared (e.g. scalars, 198At the moment, only targets of references can be shared (e.g. scalars,
205arrays or hashes pointed to by a reference). Weirder constructs, such as 199arrays or hashes pointed to by a reference). Weirder constructs, such as
206an array with multiple "copies" of the I<same> string, which are hard but 200an array with multiple "copies" of the I<same> string, which are hard but
207not impossible to create in Perl, are not supported (this is the same as 201not impossible to create in Perl, are not supported (this is the same as
212structures cannot be encoded in this mode. 206structures cannot be encoded in this mode.
213 207
214This option does not affect C<decode> in any way - shared values and 208This option does not affect C<decode> in any way - shared values and
215references will always be decoded properly if present. 209references will always be decoded properly if present.
216 210
211=item $cbor = $cbor->allow_cycles ([$enable])
212
213=item $enabled = $cbor->get_allow_cycles
214
215If C<$enable> is true (or missing), then C<decode> will happily decode
216self-referential (cyclic) data structures. By default these will not be
217decoded, as they need manual cleanup to avoid memory leaks, so code that
218isn't prepared for this will not leak memory.
219
220If C<$enable> is false (the default), then C<decode> will throw an error
221when it encounters a self-referential/cyclic data structure.
222
223FUTURE DIRECTION: the motivation behind this option is to avoid I<real>
224cycles - future versions of this module might chose to decode cyclic data
225structures using weak references when this option is off, instead of
226throwing an error.
227
228This option does not affect C<encode> in any way - shared values and
229references will always be encoded properly if present.
230
217=item $cbor = $cbor->pack_strings ([$enable]) 231=item $cbor = $cbor->pack_strings ([$enable])
218 232
219=item $enabled = $cbor->get_pack_strings 233=item $enabled = $cbor->get_pack_strings
220 234
221If C<$enable> is true (or missing), then C<encode> will try not to encode 235If C<$enable> is true (or missing), then C<encode> will try not to encode
224also results in a very large runtime overhead (expect encoding times to be 238also results in a very large runtime overhead (expect encoding times to be
2252-4 times as high as without). 2392-4 times as high as without).
226 240
227It is recommended to leave it off unless you know your 241It is recommended to leave it off unless you know your
228communications partner supports the stringref extension to CBOR 242communications partner supports the stringref extension to CBOR
229(http://cbor.schmorp.de/stringref), as without decoder support, the 243(L<http://cbor.schmorp.de/stringref>), as without decoder support, the
230resulting data structure might not be usable. 244resulting data structure might not be usable.
231 245
232If C<$enable> is false (the default), then C<encode> will encode strings 246If C<$enable> is false (the default), then C<encode> will encode strings
233the standard CBOR way. 247the standard CBOR way.
234 248
235This option does not affect C<decode> in any way - string references will 249This option does not affect C<decode> in any way - string references will
236always be decoded properly if present. 250always be decoded properly if present.
251
252=item $cbor = $cbor->validate_utf8 ([$enable])
253
254=item $enabled = $cbor->get_validate_utf8
255
256If C<$enable> is true (or missing), then C<decode> will validate that
257elements (text strings) containing UTF-8 data in fact contain valid UTF-8
258data (instead of blindly accepting it). This validation obviously takes
259extra time during decoding.
260
261The concept of "valid UTF-8" used is perl's concept, which is a superset
262of the official UTF-8.
263
264If C<$enable> is false (the default), then C<decode> will blindly accept
265UTF-8 data, marking them as valid UTF-8 in the resulting data structure
266regardless of whether that's true or not.
267
268Perl isn't too happy about corrupted UTF-8 in strings, but should
269generally not crash or do similarly evil things. Extensions might be not
270so forgiving, so it's recommended to turn on this setting if you receive
271untrusted CBOR.
272
273This option does not affect C<encode> in any way - strings that are
274supposedly valid UTF-8 will simply be dumped into the resulting CBOR
275string without checking whether that is, in fact, true or not.
237 276
238=item $cbor = $cbor->filter ([$cb->($tag, $value)]) 277=item $cbor = $cbor->filter ([$cb->($tag, $value)])
239 278
240=item $cb_or_undef = $cbor->get_filter 279=item $cb_or_undef = $cbor->get_filter
241 280
259function, C<CBOR::XS::default_filter>, is used. This function simply looks 298function, C<CBOR::XS::default_filter>, is used. This function simply looks
260up the tag in the C<%CBOR::XS::FILTER> hash. If an entry exists it must be 299up the tag in the C<%CBOR::XS::FILTER> hash. If an entry exists it must be
261a code reference that is called with tag and value, and is responsible for 300a code reference that is called with tag and value, and is responsible for
262decoding the value. If no entry exists, it returns no values. 301decoding the value. If no entry exists, it returns no values.
263 302
264Example: decode all tags not handled internally into CBOR::XS::Tagged 303Example: decode all tags not handled internally into C<CBOR::XS::Tagged>
265objects, with no other special handling (useful when working with 304objects, with no other special handling (useful when working with
266potentially "unsafe" CBOR data). 305potentially "unsafe" CBOR data).
267 306
268 CBOR::XS->new->filter (sub { })->decode ($cbor_data); 307 CBOR::XS->new->filter (sub { })->decode ($cbor_data);
269 308
296and you need to know where the first CBOR string ends amd the next one 335and you need to know where the first CBOR string ends amd the next one
297starts. 336starts.
298 337
299 CBOR::XS->new->decode_prefix ("......") 338 CBOR::XS->new->decode_prefix ("......")
300 => ("...", 3) 339 => ("...", 3)
340
341=back
342
343=head2 INCREMENTAL PARSING
344
345In some cases, there is the need for incremental parsing of JSON
346texts. While this module always has to keep both CBOR text and resulting
347Perl data structure in memory at one time, it does allow you to parse a
348CBOR stream incrementally, using a similar to using "decode_prefix" to see
349if a full CBOR object is available, but is much more efficient.
350
351It basically works by parsing as much of a CBOR string as possible - if
352the CBOR data is not complete yet, the pasrer will remember where it was,
353to be able to restart when more data has been accumulated. Once enough
354data is available to either decode a complete CBOR value or raise an
355error, a real decode will be attempted.
356
357A typical use case would be a network protocol that consists of sending
358and receiving CBOR-encoded messages. The solution that works with CBOR and
359about anything else is by prepending a length to every CBOR value, so the
360receiver knows how many octets to read. More compact (and slightly slower)
361would be to just send CBOR values back-to-back, as C<CBOR::XS> knows where
362a CBOR value ends, and doesn't need an explicit length.
363
364The following methods help with this:
365
366=over 4
367
368=item @decoded = $cbor->incr_parse ($buffer)
369
370This method attempts to decode exactly one CBOR value from the beginning
371of the given C<$buffer>. The value is removed from the C<$buffer> on
372success. When C<$buffer> doesn't contain a complete value yet, it returns
373nothing. Finally, when the C<$buffer> doesn't start with something
374that could ever be a valid CBOR value, it raises an exception, just as
375C<decode> would. In the latter case the decoder state is undefined and
376must be reset before being able to parse further.
377
378This method modifies the C<$buffer> in place. When no CBOR value can be
379decoded, the decoder stores the current string offset. On the next call,
380continues decoding at the place where it stopped before. For this to make
381sense, the C<$buffer> must begin with the same octets as on previous
382unsuccessful calls.
383
384You can call this method in scalar context, in which case it either
385returns a decoded value or C<undef>. This makes it impossible to
386distinguish between CBOR null values (which decode to C<undef>) and an
387unsuccessful decode, which is often acceptable.
388
389=item @decoded = $cbor->incr_parse_multiple ($buffer)
390
391Same as C<incr_parse>, but attempts to decode as many CBOR values as
392possible in one go, instead of at most one. Calls to C<incr_parse> and
393C<incr_parse_multiple> can be interleaved.
394
395=item $cbor->incr_reset
396
397Resets the incremental decoder. This throws away any saved state, so that
398subsequent calls to C<incr_parse> or C<incr_parse_multiple> start to parse
399a new CBOR value from the beginning of the C<$buffer> again.
400
401This method can be caled at any time, but it I<must> be called if you want
402to change your C<$buffer> or there was a decoding error and you want to
403reuse the C<$cbor> object for future incremental parsings.
301 404
302=back 405=back
303 406
304 407
305=head1 MAPPING 408=head1 MAPPING
323CBOR integers become (numeric) perl scalars. On perls without 64 bit 426CBOR integers become (numeric) perl scalars. On perls without 64 bit
324support, 64 bit integers will be truncated or otherwise corrupted. 427support, 64 bit integers will be truncated or otherwise corrupted.
325 428
326=item byte strings 429=item byte strings
327 430
328Byte strings will become octet strings in Perl (the byte values 0..255 431Byte strings will become octet strings in Perl (the Byte values 0..255
329will simply become characters of the same value in Perl). 432will simply become characters of the same value in Perl).
330 433
331=item UTF-8 strings 434=item UTF-8 strings
332 435
333UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be 436UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
356=item tagged values 459=item tagged values
357 460
358Tagged items consists of a numeric tag and another CBOR value. 461Tagged items consists of a numeric tag and another CBOR value.
359 462
360See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >> 463See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >>
361for details. 464for details on which tags are handled how.
362 465
363=item anything else 466=item anything else
364 467
365Anything else (e.g. unsupported simple values) will raise a decoding 468Anything else (e.g. unsupported simple values) will raise a decoding
366error. 469error.
369 472
370 473
371=head2 PERL -> CBOR 474=head2 PERL -> CBOR
372 475
373The mapping from Perl to CBOR is slightly more difficult, as Perl is a 476The mapping from Perl to CBOR is slightly more difficult, as Perl is a
374truly typeless language, so we can only guess which CBOR type is meant by 477typeless language. That means this module can only guess which CBOR type
375a Perl value. 478is meant by a perl value.
376 479
377=over 4 480=over 4
378 481
379=item hash references 482=item hash references
380 483
381Perl hash references become CBOR maps. As there is no inherent ordering in 484Perl hash references become CBOR maps. As there is no inherent ordering in
382hash keys (or CBOR maps), they will usually be encoded in a pseudo-random 485hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
383order. 486order. This order can be different each time a hash is encoded.
384 487
385Currently, tied hashes will use the indefinite-length format, while normal 488Currently, tied hashes will use the indefinite-length format, while normal
386hashes will use the fixed-length format. 489hashes will use the fixed-length format.
387 490
388=item array references 491=item array references
389 492
390Perl array references become fixed-length CBOR arrays. 493Perl array references become fixed-length CBOR arrays.
391 494
392=item other references 495=item other references
393 496
394Other unblessed references are generally not allowed and will cause an 497Other unblessed references will be represented using
395exception to be thrown, except for references to the integers C<0> and 498the indirection tag extension (tag value C<22098>,
396C<1>, which get turned into false and true in CBOR. 499L<http://cbor.schmorp.de/indirection>). CBOR decoders are guaranteed
500to be able to decode these values somehow, by either "doing the right
501thing", decoding into a generic tagged object, simply ignoring the tag, or
502something else.
397 503
398=item CBOR::XS::Tagged objects 504=item CBOR::XS::Tagged objects
399 505
400Objects of this type must be arrays consisting of a single C<[tag, value]> 506Objects of this type must be arrays consisting of a single C<[tag, value]>
401pair. The (numerical) tag will be encoded as a CBOR tag, the value will 507pair. The (numerical) tag will be encoded as a CBOR tag, the value will
402be encoded as appropriate for the value. You cna use C<CBOR::XS::tag> to 508be encoded as appropriate for the value. You must use C<CBOR::XS::tag> to
403create such objects. 509create such objects.
404 510
405=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error 511=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
406 512
407These special values become CBOR true, CBOR false and CBOR undefined 513These special values become CBOR true, CBOR false and CBOR undefined
424 # dump as number 530 # dump as number
425 encode_cbor [2] # yields [2] 531 encode_cbor [2] # yields [2]
426 encode_cbor [-3.0e17] # yields [-3e+17] 532 encode_cbor [-3.0e17] # yields [-3e+17]
427 my $value = 5; encode_cbor [$value] # yields [5] 533 my $value = 5; encode_cbor [$value] # yields [5]
428 534
429 # used as string, so dump as string 535 # used as string, so dump as string (either byte or text)
430 print $value; 536 print $value;
431 encode_cbor [$value] # yields ["5"] 537 encode_cbor [$value] # yields ["5"]
432 538
433 # undef becomes null 539 # undef becomes null
434 encode_cbor [undef] # yields [null] 540 encode_cbor [undef] # yields [null]
437 543
438 my $x = 3.1; # some variable containing a number 544 my $x = 3.1; # some variable containing a number
439 "$x"; # stringified 545 "$x"; # stringified
440 $x .= ""; # another, more awkward way to stringify 546 $x .= ""; # another, more awkward way to stringify
441 print $x; # perl does it for you, too, quite often 547 print $x; # perl does it for you, too, quite often
548
549You can force whether a string ie encoded as byte or text string by using
550C<utf8::upgrade> and C<utf8::downgrade>):
551
552 utf8::upgrade $x; # encode $x as text string
553 utf8::downgrade $x; # encode $x as byte string
554
555Perl doesn't define what operations up- and downgrade strings, so if the
556difference between byte and text is important, you should up- or downgrade
557your string as late as possible before encoding.
442 558
443You can force the type to be a CBOR number by numifying it: 559You can force the type to be a CBOR number by numifying it:
444 560
445 my $x = "3"; # some variable containing a string 561 my $x = "3"; # some variable containing a string
446 $x += 0; # numify it, ensuring it will be dumped as a number 562 $x += 0; # numify it, ensuring it will be dumped as a number
459 575
460=back 576=back
461 577
462=head2 OBJECT SERIALISATION 578=head2 OBJECT SERIALISATION
463 579
580This module implements both a CBOR-specific and the generic
581L<Types::Serialier> object serialisation protocol. The following
582subsections explain both methods.
583
584=head3 ENCODING
585
464This module knows two way to serialise a Perl object: The CBOR-specific 586This module knows two way to serialise a Perl object: The CBOR-specific
465way, and the generic way. 587way, and the generic way.
466 588
467Whenever the encoder encounters a Perl object that it cnanot serialise 589Whenever the encoder encounters a Perl object that it cannot serialise
468directly (most of them), it will first look up the C<TO_CBOR> method on 590directly (most of them), it will first look up the C<TO_CBOR> method on
469it. 591it.
470 592
471If it has a C<TO_CBOR> method, it will call it with the object as only 593If it has a C<TO_CBOR> method, it will call it with the object as only
472argument, and expects exactly one return value, which it will then 594argument, and expects exactly one return value, which it will then
478 600
479The C<FREEZE> method can return any number of values (i.e. zero or 601The C<FREEZE> method can return any number of values (i.e. zero or
480more). These will be encoded as CBOR perl object, together with the 602more). These will be encoded as CBOR perl object, together with the
481classname. 603classname.
482 604
605These methods I<MUST NOT> change the data structure that is being
606serialised. Failure to comply to this can result in memory corruption -
607and worse.
608
483If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail 609If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
484with an error. 610with an error.
485 611
612=head3 DECODING
613
486Objects encoded via C<TO_CBOR> cannot be automatically decoded, but 614Objects encoded via C<TO_CBOR> cannot (normally) be automatically decoded,
487objects encoded via C<FREEZE> can be decoded using the following protocol: 615but objects encoded via C<FREEZE> can be decoded using the following
616protocol:
488 617
489When an encoded CBOR perl object is encountered by the decoder, it will 618When an encoded CBOR perl object is encountered by the decoder, it will
490look up the C<THAW> method, by using the stored classname, and will fail 619look up the C<THAW> method, by using the stored classname, and will fail
491if the method cannot be found. 620if the method cannot be found.
492 621
493After the lookup it will call the C<THAW> method with the stored classname 622After the lookup it will call the C<THAW> method with the stored classname
494as first argument, the constant string C<CBOR> as second argument, and all 623as first argument, the constant string C<CBOR> as second argument, and all
495values returned by C<FREEZE> as remaining arguments. 624values returned by C<FREEZE> as remaining arguments.
496 625
497=head4 EXAMPLES 626=head3 EXAMPLES
498 627
499Here is an example C<TO_CBOR> method: 628Here is an example C<TO_CBOR> method:
500 629
501 sub My::Object::TO_CBOR { 630 sub My::Object::TO_CBOR {
502 my ($obj) = @_; 631 my ($obj) = @_;
513 642
514 sub URI::TO_CBOR { 643 sub URI::TO_CBOR {
515 my ($self) = @_; 644 my ($self) = @_;
516 my $uri = "$self"; # stringify uri 645 my $uri = "$self"; # stringify uri
517 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string 646 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
518 CBOR::XS::tagged 32, "$_[0]" 647 CBOR::XS::tag 32, "$_[0]"
519 } 648 }
520 649
521This will encode URIs as a UTF-8 string with tag 32, which indicates an 650This will encode URIs as a UTF-8 string with tag 32, which indicates an
522URI. 651URI.
523 652
675additional tags (such as base64url). 804additional tags (such as base64url).
676 805
677=head2 ENFORCED TAGS 806=head2 ENFORCED TAGS
678 807
679These tags are always handled when decoding, and their handling cannot be 808These tags are always handled when decoding, and their handling cannot be
680overriden by the user. 809overridden by the user.
681 810
682=over 4 811=over 4
683 812
684=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>) 813=item 26 (perl-object, L<http://cbor.schmorp.de/perl-object>)
685 814
686These tags are automatically created (and decoded) for serialisable 815These tags are automatically created (and decoded) for serialisable
687objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object 816objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
688serialisation protocol). See L<OBJECT SERIALISATION> for details. 817serialisation protocol). See L<OBJECT SERIALISATION> for details.
689 818
690=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>) 819=item 28, 29 (shareable, sharedref, L<http://cbor.schmorp.de/value-sharing>)
691 820
692These tags are automatically decoded when encountered, resulting in 821These tags are automatically decoded when encountered (and they do not
822result in a cyclic data structure, see C<allow_cycles>), resulting in
693shared values in the decoded object. They are only encoded, however, when 823shared values in the decoded object. They are only encoded, however, when
694C<allow_sharable> is enabled. 824C<allow_sharing> is enabled.
695 825
826Not all shared values can be successfully decoded: values that reference
827themselves will I<currently> decode as C<undef> (this is not the same
828as a reference pointing to itself, which will be represented as a value
829that contains an indirect reference to itself - these will be decoded
830properly).
831
832Note that considerably more shared value data structures can be decoded
833than will be encoded - currently, only values pointed to by references
834will be shared, others will not. While non-reference shared values can be
835generated in Perl with some effort, they were considered too unimportant
836to be supported in the encoder. The decoder, however, will decode these
837values as shared values.
838
696=item <unassigned>, <unassigned> (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>) 839=item 256, 25 (stringref-namespace, stringref, L<http://cbor.schmorp.de/stringref>)
697 840
698These tags are automatically decoded when encountered. They are only 841These tags are automatically decoded when encountered. They are only
699encoded, however, when C<pack_strings> is enabled. 842encoded, however, when C<pack_strings> is enabled.
700 843
701=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>) 844=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
702 845
703This tag is automatically generated when a reference are encountered (with 846This tag is automatically generated when a reference are encountered (with
704the exception of hash and array refernces). It is converted to a reference 847the exception of hash and array references). It is converted to a reference
705when decoding. 848when decoding.
706 849
707=item 55799 (self-describe CBOR, RFC 7049) 850=item 55799 (self-describe CBOR, RFC 7049)
708 851
709This value is not generated on encoding (unless explicitly requested by 852This value is not generated on encoding (unless explicitly requested by
712=back 855=back
713 856
714=head2 NON-ENFORCED TAGS 857=head2 NON-ENFORCED TAGS
715 858
716These tags have default filters provided when decoding. Their handling can 859These tags have default filters provided when decoding. Their handling can
717be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by 860be overridden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
718providing a custom C<filter> callback when decoding. 861providing a custom C<filter> callback when decoding.
719 862
720When they result in decoding into a specific Perl class, the module 863When they result in decoding into a specific Perl class, the module
721usually provides a corresponding C<TO_CBOR> method as well. 864usually provides a corresponding C<TO_CBOR> method as well.
722 865
725provide these modules. The decoding usually fails with an exception if the 868provide these modules. The decoding usually fails with an exception if the
726required module cannot be loaded. 869required module cannot be loaded.
727 870
728=over 4 871=over 4
729 872
873=item 0, 1 (date/time string, seconds since the epoch)
874
875These tags are decoded into L<Time::Piece> objects. The corresponding
876C<Time::Piece::TO_CBOR> method always encodes into tag 1 values currently.
877
878The L<Time::Piece> API is generally surprisingly bad, and fractional
879seconds are only accidentally kept intact, so watch out. On the plus side,
880the module comes with perl since 5.10, which has to count for something.
881
730=item 2, 3 (positive/negative bignum) 882=item 2, 3 (positive/negative bignum)
731 883
732These tags are decoded into L<Math::BigInt> objects. The corresponding 884These tags are decoded into L<Math::BigInt> objects. The corresponding
733C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR 885C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR
734integers, and others into positive/negative CBOR bignums. 886integers, and others into positive/negative CBOR bignums.
755C<URI::TO_CBOR> method again results in a CBOR URI value. 907C<URI::TO_CBOR> method again results in a CBOR URI value.
756 908
757=back 909=back
758 910
759=cut 911=cut
760
761our %FILTER = (
762 # 0 # rfc4287 datetime, utf-8
763 # 1 # unix timestamp, any
764
765 2 => sub { # pos bigint
766 require Math::BigInt;
767 Math::BigInt->new ("0x" . unpack "H*", pop)
768 },
769
770 3 => sub { # neg bigint
771 require Math::BigInt;
772 -Math::BigInt->new ("0x" . unpack "H*", pop)
773 },
774
775 4 => sub { # decimal fraction, array
776 require Math::BigFloat;
777 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
778 },
779
780 5 => sub { # bigfloat, array
781 require Math::BigFloat;
782 scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
783 },
784
785 21 => sub { pop }, # expected conversion to base64url encoding
786 22 => sub { pop }, # expected conversion to base64 encoding
787 23 => sub { pop }, # expected conversion to base16 encoding
788
789 # 24 # embedded cbor, byte string
790
791 32 => sub {
792 require URI;
793 URI->new (pop)
794 },
795
796 # 33 # base64url rfc4648, utf-8
797 # 34 # base64 rfc46484, utf-8
798 # 35 # regex pcre/ecma262, utf-8
799 # 36 # mime message rfc2045, utf-8
800);
801
802 912
803=head1 CBOR and JSON 913=head1 CBOR and JSON
804 914
805CBOR is supposed to implement a superset of the JSON data model, and is, 915CBOR is supposed to implement a superset of the JSON data model, and is,
806with some coercion, able to represent all JSON texts (something that other 916with some coercion, able to represent all JSON texts (something that other
867properly. Half precision types are accepted, but not encoded. 977properly. Half precision types are accepted, but not encoded.
868 978
869Strict mode and canonical mode are not implemented. 979Strict mode and canonical mode are not implemented.
870 980
871 981
982=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT
983
984On perls that were built without 64 bit integer support (these are rare
985nowadays, even on 32 bit architectures, as all major Perl distributions
986are built with 64 bit integer support), support for any kind of 64 bit
987integer in CBOR is very limited - most likely, these 64 bit values will
988be truncated, corrupted, or otherwise not decoded correctly. This also
989includes string, array and map sizes that are stored as 64 bit integers.
990
991
872=head1 THREADS 992=head1 THREADS
873 993
874This module is I<not> guaranteed to be thread safe and there are no 994This module is I<not> guaranteed to be thread safe and there are no
875plans to change this until Perl gets thread support (as opposed to the 995plans to change this until Perl gets thread support (as opposed to the
876horribly slow so-called "threads" which are simply slow and bloated 996horribly slow so-called "threads" which are simply slow and bloated
889service. I put the contact address into my modules for a reason. 1009service. I put the contact address into my modules for a reason.
890 1010
891=cut 1011=cut
892 1012
893our %FILTER = ( 1013our %FILTER = (
894 # 0 # rfc4287 datetime, utf-8 1014 0 => sub { # rfc4287 datetime, utf-8
895 # 1 # unix timestamp, any 1015 require Time::Piece;
1016 # Time::Piece::Strptime uses the "incredibly flexible date parsing routine"
1017 # from FreeBSD, which can't parse ISO 8601, RFC3339, RFC4287 or much of anything
1018 # else either. Whats incredibe over standard strptime totally escapes me.
1019 # doesn't do fractional times, either. sigh.
1020 # In fact, it's all a lie, it uses whatever strptime it wants, and of course,
1021 # they are all incompatible. The openbsd one simply ignores %z (but according to the
1022 # docs, it would be much more incredibly flexible indeed. If it worked, that is.).
1023 scalar eval {
1024 my $s = $_[1];
1025
1026 $s =~ s/Z$/+00:00/;
1027 $s =~ s/(\.[0-9]+)?([+-][0-9][0-9]):([0-9][0-9])$//
1028 or die;
1029
1030 my $b = $1 - ($2 * 60 + $3) * 60; # fractional part + offset. hopefully
1031 my $d = Time::Piece->strptime ($s, "%Y-%m-%dT%H:%M:%S");
1032
1033 Time::Piece::gmtime ($d->epoch + $b)
1034 } || die "corrupted CBOR date/time string ($_[0])";
1035 },
1036
1037 1 => sub { # seconds since the epoch, possibly fractional
1038 require Time::Piece;
1039 scalar Time::Piece::gmtime (pop)
1040 },
896 1041
897 2 => sub { # pos bigint 1042 2 => sub { # pos bigint
898 require Math::BigInt; 1043 require Math::BigInt;
899 Math::BigInt->new ("0x" . unpack "H*", pop) 1044 Math::BigInt->new ("0x" . unpack "H*", pop)
900 }, 1045 },
909 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0]) 1054 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
910 }, 1055 },
911 1056
912 5 => sub { # bigfloat, array 1057 5 => sub { # bigfloat, array
913 require Math::BigFloat; 1058 require Math::BigFloat;
914 scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2) 1059 scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
915 }, 1060 },
916 1061
917 21 => sub { pop }, # expected conversion to base64url encoding 1062 21 => sub { pop }, # expected conversion to base64url encoding
918 22 => sub { pop }, # expected conversion to base64 encoding 1063 22 => sub { pop }, # expected conversion to base64 encoding
919 23 => sub { pop }, # expected conversion to base16 encoding 1064 23 => sub { pop }, # expected conversion to base16 encoding
936} 1081}
937 1082
938sub URI::TO_CBOR { 1083sub URI::TO_CBOR {
939 my $uri = $_[0]->as_string; 1084 my $uri = $_[0]->as_string;
940 utf8::upgrade $uri; 1085 utf8::upgrade $uri;
941 CBOR::XS::tag 32, $uri 1086 tag 32, $uri
942} 1087}
943 1088
944sub Math::BigInt::TO_CBOR { 1089sub Math::BigInt::TO_CBOR {
945 if ($_[0] >= -2147483648 && $_[0] <= 2147483647) { 1090 if ($_[0] >= -2147483648 && $_[0] <= 2147483647) {
946 $_[0]->numify 1091 $_[0]->numify
947 } else { 1092 } else {
948 my $hex = substr $_[0]->as_hex, 2; 1093 my $hex = substr $_[0]->as_hex, 2;
949 $hex = "0$hex" if 1 & length $hex; # sigh 1094 $hex = "0$hex" if 1 & length $hex; # sigh
950 CBOR::XS::tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex 1095 tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
951 } 1096 }
952} 1097}
953 1098
954sub Math::BigFloat::TO_CBOR { 1099sub Math::BigFloat::TO_CBOR {
955 my ($m, $e) = $_[0]->parts; 1100 my ($m, $e) = $_[0]->parts;
956 CBOR::XS::tag 4, [$e->numify, $m] 1101 tag 4, [$e->numify, $m]
1102}
1103
1104sub Time::Piece::TO_CBOR {
1105 tag 1, 0 + $_[0]->epoch
957} 1106}
958 1107
959XSLoader::load "CBOR::XS", $VERSION; 1108XSLoader::load "CBOR::XS", $VERSION;
960 1109
961=head1 SEE ALSO 1110=head1 SEE ALSO

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines