ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/XS.pm
(Generate patch)

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.24 by root, Fri Nov 22 16:18:59 2013 UTC vs.
Revision 1.52 by root, Mon Apr 25 18:17:17 2016 UTC

26 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string 26 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
27 } 27 }
28 28
29=head1 DESCRIPTION 29=head1 DESCRIPTION
30 30
31WARNING! This module is very new, and not very well tested (that's up
32to you to do). Furthermore, details of the implementation might change
33freely before version 1.0. And lastly, most extensions depend on an IANA
34assignment, and until that assignment is official, this implementation is
35not interoperable with other implementations (even future versions of this
36module) until the assignment is done.
37
38You are still invited to try out CBOR, and this module.
39
40This module converts Perl data structures to the Concise Binary Object 31This module converts Perl data structures to the Concise Binary Object
41Representation (CBOR) and vice versa. CBOR is a fast binary serialisation 32Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
42format that aims to use a superset of the JSON data model, i.e. when you 33format that aims to use an (almost) superset of the JSON data model, i.e.
43can represent something in JSON, you should be able to represent it in 34when you can represent something useful in JSON, you should be able to
44CBOR. 35represent it in CBOR.
45 36
46In short, CBOR is a faster and very compact binary alternative to JSON, 37In short, CBOR is a faster and quite compact binary alternative to JSON,
47with the added ability of supporting serialisation of Perl objects. (JSON 38with the added ability of supporting serialisation of Perl objects. (JSON
48often compresses better than CBOR though, so if you plan to compress the 39often compresses better than CBOR though, so if you plan to compress the
49data later you might want to compare both formats first). 40data later and speed is less important you might want to compare both
41formats first).
50 42
51To give you a general idea about speed, with texts in the megabyte range, 43To give you a general idea about speed, with texts in the megabyte range,
52C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or 44C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or
53L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the 45L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the
54data, the worse L<Storable> performs in comparison. 46data, the worse L<Storable> performs in comparison.
55 47
56As for compactness, C<CBOR::XS> encoded data structures are usually about 48Regarding compactness, C<CBOR::XS>-encoded data structures are usually
5720% smaller than the same data encoded as (compact) JSON or L<Storable>. 49about 20% smaller than the same data encoded as (compact) JSON or
50L<Storable>.
58 51
59In addition to the core CBOR data format, this module implements a number 52In addition to the core CBOR data format, this module implements a
60of extensions, to support cyclic and self-referencing data structures 53number of extensions, to support cyclic and shared data structures
61(see C<allow_sharing>), string deduplication (see C<allow_stringref>) and 54(see C<allow_sharing> and C<allow_cycles>), string deduplication (see
62scalar references (always enabled). 55C<pack_strings>) and scalar references (always enabled).
63 56
64The primary goal of this module is to be I<correct> and the secondary goal 57The primary goal of this module is to be I<correct> and the secondary goal
65is to be I<fast>. To reach the latter goal it was written in C. 58is to be I<fast>. To reach the latter goal it was written in C.
66 59
67See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and 60See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
71 64
72package CBOR::XS; 65package CBOR::XS;
73 66
74use common::sense; 67use common::sense;
75 68
76our $VERSION = 0.09; 69our $VERSION = 1.41;
77our @ISA = qw(Exporter); 70our @ISA = qw(Exporter);
78 71
79our @EXPORT = qw(encode_cbor decode_cbor); 72our @EXPORT = qw(encode_cbor decode_cbor);
80 73
81use Exporter; 74use Exporter;
186as an array, is referenced multiple times), but instead will emit a 179as an array, is referenced multiple times), but instead will emit a
187reference to the earlier value. 180reference to the earlier value.
188 181
189This means that such values will only be encoded once, and will not result 182This means that such values will only be encoded once, and will not result
190in a deep cloning of the value on decode, in decoders supporting the value 183in a deep cloning of the value on decode, in decoders supporting the value
191sharing extension. 184sharing extension. This also makes it possible to encode cyclic data
185structures (which need C<allow_cycles> to ne enabled to be decoded by this
186module).
192 187
193It is recommended to leave it off unless you know your 188It is recommended to leave it off unless you know your
194communication partner supports the value sharing extensions to CBOR 189communication partner supports the value sharing extensions to CBOR
195(http://cbor.schmorp.de/value-sharing). 190(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the
191resulting data structure might be unusable.
196 192
197Detecting shared values incurs a runtime overhead when values are encoded 193Detecting shared values incurs a runtime overhead when values are encoded
198that have a reference counter large than one, and might unnecessarily 194that have a reference counter large than one, and might unnecessarily
199increase the encoded size, as potentially shared values are encode as 195increase the encoded size, as potentially shared values are encode as
200sharable whether or not they are actually shared. 196shareable whether or not they are actually shared.
201 197
202At the moment, only targets of references can be shared (e.g. scalars, 198At the moment, only targets of references can be shared (e.g. scalars,
203arrays or hashes pointed to by a reference). Weirder constructs, such as 199arrays or hashes pointed to by a reference). Weirder constructs, such as
204an array with multiple "copies" of the I<same> string, which are hard but 200an array with multiple "copies" of the I<same> string, which are hard but
205not impossible to create in Perl, are not supported (this is the same as 201not impossible to create in Perl, are not supported (this is the same as
206for L<Storable>). 202with L<Storable>).
207 203
208If C<$enable> is false (the default), then C<encode> will encode 204If C<$enable> is false (the default), then C<encode> will encode shared
209exception when it encounters anything it cannot encode as CBOR. 205data structures repeatedly, unsharing them in the process. Cyclic data
206structures cannot be encoded in this mode.
210 207
211This option does not affect C<decode> in any way - shared values and 208This option does not affect C<decode> in any way - shared values and
212references will always be decoded properly if present. 209references will always be decoded properly if present.
213 210
211=item $cbor = $cbor->allow_cycles ([$enable])
212
213=item $enabled = $cbor->get_allow_cycles
214
215If C<$enable> is true (or missing), then C<decode> will happily decode
216self-referential (cyclic) data structures. By default these will not be
217decoded, as they need manual cleanup to avoid memory leaks, so code that
218isn't prepared for this will not leak memory.
219
220If C<$enable> is false (the default), then C<decode> will throw an error
221when it encounters a self-referential/cyclic data structure.
222
223FUTURE DIRECTION: the motivation behind this option is to avoid I<real>
224cycles - future versions of this module might chose to decode cyclic data
225structures using weak references when this option is off, instead of
226throwing an error.
227
228This option does not affect C<encode> in any way - shared values and
229references will always be encoded properly if present.
230
214=item $cbor = $cbor->allow_stringref ([$enable]) 231=item $cbor = $cbor->pack_strings ([$enable])
215 232
216=item $enabled = $cbor->get_allow_stringref 233=item $enabled = $cbor->get_pack_strings
217 234
218If C<$enable> is true (or missing), then C<encode> will try not to encode 235If C<$enable> is true (or missing), then C<encode> will try not to encode
219the same string twice, but will instead encode a reference to the string 236the same string twice, but will instead encode a reference to the string
220instead. Depending on your data format. this can save a lot of space, but 237instead. Depending on your data format, this can save a lot of space, but
221also results in a very large runtime overhead (expect encoding times to be 238also results in a very large runtime overhead (expect encoding times to be
2222-4 times as high as without). 2392-4 times as high as without).
223 240
224It is recommended to leave it off unless you know your 241It is recommended to leave it off unless you know your
225communications partner supports the stringref extension to CBOR 242communications partner supports the stringref extension to CBOR
226(http://cbor.schmorp.de/stringref). 243(L<http://cbor.schmorp.de/stringref>), as without decoder support, the
244resulting data structure might not be usable.
227 245
228If C<$enable> is false (the default), then C<encode> will encode 246If C<$enable> is false (the default), then C<encode> will encode strings
229exception when it encounters anything it cannot encode as CBOR. 247the standard CBOR way.
230 248
231This option does not affect C<decode> in any way - string references will 249This option does not affect C<decode> in any way - string references will
232always be decoded properly if present. 250always be decoded properly if present.
251
252=item $cbor = $cbor->text_keys ([$enable])
253
254=item $enabled = $cbor->get_text_keys
255
256If C<$enabled> is true (or missing), then C<encode> will encode all
257perl hash keys as CBOR text strings/UTF-8 string, upgrading them as needed.
258
259If C<$enable> is false (the default), then C<encode> will encode hash keys
260normally - upgraded perl strings (strings internally encoded as UTF-8) as
261CBOR text strings, and downgraded perl strings as CBOR byte strings.
262
263This option does not affect C<decode> in any way.
264
265This option is useful for interoperability with CBOR decoders that don't
266treat byte strings as a form of text. It is especially useful as Perl
267gives very little control over hash keys.
268
269Enabling this option can be slow, as all downgraded hash keys that are
270encoded need to be scanned and converted to UTF-8.
271
272=item $cbor = $cbor->text_strings ([$enable])
273
274=item $enabled = $cbor->get_text_strings
275
276This option works similar to C<text_keys>, above, but works on all strings
277(including hash keys), so C<text_keys> has no further effect after
278enabling C<text_strings>.
279
280If C<$enabled> is true (or missing), then C<encode> will encode all perl
281strings as CBOR text strings/UTF-8 strings, upgrading them as needed.
282
283If C<$enable> is false (the default), then C<encode> will encode strings
284normally (but see C<text_keys>) - upgraded perl strings (strings
285internally encoded as UTF-8) as CBOR text strings, and downgraded perl
286strings as CBOR byte strings.
287
288This option does not affect C<decode> in any way.
289
290This option has similar advantages and disadvantages as C<text_keys>. In
291addition, this option effectively removes the ability to encode byte
292strings, which might break some C<FREEZE> and C<TO_CBOR> methods that rely
293on this, such as bignum encoding, so this option is mainly useful for very
294simple data.
295
296=item $cbor = $cbor->validate_utf8 ([$enable])
297
298=item $enabled = $cbor->get_validate_utf8
299
300If C<$enable> is true (or missing), then C<decode> will validate that
301elements (text strings) containing UTF-8 data in fact contain valid UTF-8
302data (instead of blindly accepting it). This validation obviously takes
303extra time during decoding.
304
305The concept of "valid UTF-8" used is perl's concept, which is a superset
306of the official UTF-8.
307
308If C<$enable> is false (the default), then C<decode> will blindly accept
309UTF-8 data, marking them as valid UTF-8 in the resulting data structure
310regardless of whether that's true or not.
311
312Perl isn't too happy about corrupted UTF-8 in strings, but should
313generally not crash or do similarly evil things. Extensions might be not
314so forgiving, so it's recommended to turn on this setting if you receive
315untrusted CBOR.
316
317This option does not affect C<encode> in any way - strings that are
318supposedly valid UTF-8 will simply be dumped into the resulting CBOR
319string without checking whether that is, in fact, true or not.
233 320
234=item $cbor = $cbor->filter ([$cb->($tag, $value)]) 321=item $cbor = $cbor->filter ([$cb->($tag, $value)])
235 322
236=item $cb_or_undef = $cbor->get_filter 323=item $cb_or_undef = $cbor->get_filter
237 324
255function, C<CBOR::XS::default_filter>, is used. This function simply looks 342function, C<CBOR::XS::default_filter>, is used. This function simply looks
256up the tag in the C<%CBOR::XS::FILTER> hash. If an entry exists it must be 343up the tag in the C<%CBOR::XS::FILTER> hash. If an entry exists it must be
257a code reference that is called with tag and value, and is responsible for 344a code reference that is called with tag and value, and is responsible for
258decoding the value. If no entry exists, it returns no values. 345decoding the value. If no entry exists, it returns no values.
259 346
260Example: decode all tags not handled internally into CBOR::XS::Tagged 347Example: decode all tags not handled internally into C<CBOR::XS::Tagged>
261objects, with no other special handling (useful when working with 348objects, with no other special handling (useful when working with
262potentially "unsafe" CBOR data). 349potentially "unsafe" CBOR data).
263 350
264 CBOR::XS->new->filter (sub { })->decode ($cbor_data); 351 CBOR::XS->new->filter (sub { })->decode ($cbor_data);
265 352
292and you need to know where the first CBOR string ends amd the next one 379and you need to know where the first CBOR string ends amd the next one
293starts. 380starts.
294 381
295 CBOR::XS->new->decode_prefix ("......") 382 CBOR::XS->new->decode_prefix ("......")
296 => ("...", 3) 383 => ("...", 3)
384
385=back
386
387=head2 INCREMENTAL PARSING
388
389In some cases, there is the need for incremental parsing of JSON
390texts. While this module always has to keep both CBOR text and resulting
391Perl data structure in memory at one time, it does allow you to parse a
392CBOR stream incrementally, using a similar to using "decode_prefix" to see
393if a full CBOR object is available, but is much more efficient.
394
395It basically works by parsing as much of a CBOR string as possible - if
396the CBOR data is not complete yet, the pasrer will remember where it was,
397to be able to restart when more data has been accumulated. Once enough
398data is available to either decode a complete CBOR value or raise an
399error, a real decode will be attempted.
400
401A typical use case would be a network protocol that consists of sending
402and receiving CBOR-encoded messages. The solution that works with CBOR and
403about anything else is by prepending a length to every CBOR value, so the
404receiver knows how many octets to read. More compact (and slightly slower)
405would be to just send CBOR values back-to-back, as C<CBOR::XS> knows where
406a CBOR value ends, and doesn't need an explicit length.
407
408The following methods help with this:
409
410=over 4
411
412=item @decoded = $cbor->incr_parse ($buffer)
413
414This method attempts to decode exactly one CBOR value from the beginning
415of the given C<$buffer>. The value is removed from the C<$buffer> on
416success. When C<$buffer> doesn't contain a complete value yet, it returns
417nothing. Finally, when the C<$buffer> doesn't start with something
418that could ever be a valid CBOR value, it raises an exception, just as
419C<decode> would. In the latter case the decoder state is undefined and
420must be reset before being able to parse further.
421
422This method modifies the C<$buffer> in place. When no CBOR value can be
423decoded, the decoder stores the current string offset. On the next call,
424continues decoding at the place where it stopped before. For this to make
425sense, the C<$buffer> must begin with the same octets as on previous
426unsuccessful calls.
427
428You can call this method in scalar context, in which case it either
429returns a decoded value or C<undef>. This makes it impossible to
430distinguish between CBOR null values (which decode to C<undef>) and an
431unsuccessful decode, which is often acceptable.
432
433=item @decoded = $cbor->incr_parse_multiple ($buffer)
434
435Same as C<incr_parse>, but attempts to decode as many CBOR values as
436possible in one go, instead of at most one. Calls to C<incr_parse> and
437C<incr_parse_multiple> can be interleaved.
438
439=item $cbor->incr_reset
440
441Resets the incremental decoder. This throws away any saved state, so that
442subsequent calls to C<incr_parse> or C<incr_parse_multiple> start to parse
443a new CBOR value from the beginning of the C<$buffer> again.
444
445This method can be caled at any time, but it I<must> be called if you want
446to change your C<$buffer> or there was a decoding error and you want to
447reuse the C<$cbor> object for future incremental parsings.
297 448
298=back 449=back
299 450
300 451
301=head1 MAPPING 452=head1 MAPPING
319CBOR integers become (numeric) perl scalars. On perls without 64 bit 470CBOR integers become (numeric) perl scalars. On perls without 64 bit
320support, 64 bit integers will be truncated or otherwise corrupted. 471support, 64 bit integers will be truncated or otherwise corrupted.
321 472
322=item byte strings 473=item byte strings
323 474
324Byte strings will become octet strings in Perl (the byte values 0..255 475Byte strings will become octet strings in Perl (the Byte values 0..255
325will simply become characters of the same value in Perl). 476will simply become characters of the same value in Perl).
326 477
327=item UTF-8 strings 478=item UTF-8 strings
328 479
329UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be 480UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
352=item tagged values 503=item tagged values
353 504
354Tagged items consists of a numeric tag and another CBOR value. 505Tagged items consists of a numeric tag and another CBOR value.
355 506
356See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >> 507See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >>
357for details. 508for details on which tags are handled how.
358 509
359=item anything else 510=item anything else
360 511
361Anything else (e.g. unsupported simple values) will raise a decoding 512Anything else (e.g. unsupported simple values) will raise a decoding
362error. 513error.
365 516
366 517
367=head2 PERL -> CBOR 518=head2 PERL -> CBOR
368 519
369The mapping from Perl to CBOR is slightly more difficult, as Perl is a 520The mapping from Perl to CBOR is slightly more difficult, as Perl is a
370truly typeless language, so we can only guess which CBOR type is meant by 521typeless language. That means this module can only guess which CBOR type
371a Perl value. 522is meant by a perl value.
372 523
373=over 4 524=over 4
374 525
375=item hash references 526=item hash references
376 527
377Perl hash references become CBOR maps. As there is no inherent ordering in 528Perl hash references become CBOR maps. As there is no inherent ordering in
378hash keys (or CBOR maps), they will usually be encoded in a pseudo-random 529hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
379order. 530order. This order can be different each time a hash is encoded.
380 531
381Currently, tied hashes will use the indefinite-length format, while normal 532Currently, tied hashes will use the indefinite-length format, while normal
382hashes will use the fixed-length format. 533hashes will use the fixed-length format.
383 534
384=item array references 535=item array references
385 536
386Perl array references become fixed-length CBOR arrays. 537Perl array references become fixed-length CBOR arrays.
387 538
388=item other references 539=item other references
389 540
390Other unblessed references are generally not allowed and will cause an 541Other unblessed references will be represented using
391exception to be thrown, except for references to the integers C<0> and 542the indirection tag extension (tag value C<22098>,
392C<1>, which get turned into false and true in CBOR. 543L<http://cbor.schmorp.de/indirection>). CBOR decoders are guaranteed
544to be able to decode these values somehow, by either "doing the right
545thing", decoding into a generic tagged object, simply ignoring the tag, or
546something else.
393 547
394=item CBOR::XS::Tagged objects 548=item CBOR::XS::Tagged objects
395 549
396Objects of this type must be arrays consisting of a single C<[tag, value]> 550Objects of this type must be arrays consisting of a single C<[tag, value]>
397pair. The (numerical) tag will be encoded as a CBOR tag, the value will 551pair. The (numerical) tag will be encoded as a CBOR tag, the value will
398be encoded as appropriate for the value. You cna use C<CBOR::XS::tag> to 552be encoded as appropriate for the value. You must use C<CBOR::XS::tag> to
399create such objects. 553create such objects.
400 554
401=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error 555=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
402 556
403These special values become CBOR true, CBOR false and CBOR undefined 557These special values become CBOR true, CBOR false and CBOR undefined
420 # dump as number 574 # dump as number
421 encode_cbor [2] # yields [2] 575 encode_cbor [2] # yields [2]
422 encode_cbor [-3.0e17] # yields [-3e+17] 576 encode_cbor [-3.0e17] # yields [-3e+17]
423 my $value = 5; encode_cbor [$value] # yields [5] 577 my $value = 5; encode_cbor [$value] # yields [5]
424 578
425 # used as string, so dump as string 579 # used as string, so dump as string (either byte or text)
426 print $value; 580 print $value;
427 encode_cbor [$value] # yields ["5"] 581 encode_cbor [$value] # yields ["5"]
428 582
429 # undef becomes null 583 # undef becomes null
430 encode_cbor [undef] # yields [null] 584 encode_cbor [undef] # yields [null]
433 587
434 my $x = 3.1; # some variable containing a number 588 my $x = 3.1; # some variable containing a number
435 "$x"; # stringified 589 "$x"; # stringified
436 $x .= ""; # another, more awkward way to stringify 590 $x .= ""; # another, more awkward way to stringify
437 print $x; # perl does it for you, too, quite often 591 print $x; # perl does it for you, too, quite often
592
593You can force whether a string ie encoded as byte or text string by using
594C<utf8::upgrade> and C<utf8::downgrade>):
595
596 utf8::upgrade $x; # encode $x as text string
597 utf8::downgrade $x; # encode $x as byte string
598
599Perl doesn't define what operations up- and downgrade strings, so if the
600difference between byte and text is important, you should up- or downgrade
601your string as late as possible before encoding.
438 602
439You can force the type to be a CBOR number by numifying it: 603You can force the type to be a CBOR number by numifying it:
440 604
441 my $x = "3"; # some variable containing a string 605 my $x = "3"; # some variable containing a string
442 $x += 0; # numify it, ensuring it will be dumped as a number 606 $x += 0; # numify it, ensuring it will be dumped as a number
455 619
456=back 620=back
457 621
458=head2 OBJECT SERIALISATION 622=head2 OBJECT SERIALISATION
459 623
624This module implements both a CBOR-specific and the generic
625L<Types::Serialier> object serialisation protocol. The following
626subsections explain both methods.
627
628=head3 ENCODING
629
460This module knows two way to serialise a Perl object: The CBOR-specific 630This module knows two way to serialise a Perl object: The CBOR-specific
461way, and the generic way. 631way, and the generic way.
462 632
463Whenever the encoder encounters a Perl object that it cnanot serialise 633Whenever the encoder encounters a Perl object that it cannot serialise
464directly (most of them), it will first look up the C<TO_CBOR> method on 634directly (most of them), it will first look up the C<TO_CBOR> method on
465it. 635it.
466 636
467If it has a C<TO_CBOR> method, it will call it with the object as only 637If it has a C<TO_CBOR> method, it will call it with the object as only
468argument, and expects exactly one return value, which it will then 638argument, and expects exactly one return value, which it will then
474 644
475The C<FREEZE> method can return any number of values (i.e. zero or 645The C<FREEZE> method can return any number of values (i.e. zero or
476more). These will be encoded as CBOR perl object, together with the 646more). These will be encoded as CBOR perl object, together with the
477classname. 647classname.
478 648
649These methods I<MUST NOT> change the data structure that is being
650serialised. Failure to comply to this can result in memory corruption -
651and worse.
652
479If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail 653If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
480with an error. 654with an error.
481 655
656=head3 DECODING
657
482Objects encoded via C<TO_CBOR> cannot be automatically decoded, but 658Objects encoded via C<TO_CBOR> cannot (normally) be automatically decoded,
483objects encoded via C<FREEZE> can be decoded using the following protocol: 659but objects encoded via C<FREEZE> can be decoded using the following
660protocol:
484 661
485When an encoded CBOR perl object is encountered by the decoder, it will 662When an encoded CBOR perl object is encountered by the decoder, it will
486look up the C<THAW> method, by using the stored classname, and will fail 663look up the C<THAW> method, by using the stored classname, and will fail
487if the method cannot be found. 664if the method cannot be found.
488 665
489After the lookup it will call the C<THAW> method with the stored classname 666After the lookup it will call the C<THAW> method with the stored classname
490as first argument, the constant string C<CBOR> as second argument, and all 667as first argument, the constant string C<CBOR> as second argument, and all
491values returned by C<FREEZE> as remaining arguments. 668values returned by C<FREEZE> as remaining arguments.
492 669
493=head4 EXAMPLES 670=head3 EXAMPLES
494 671
495Here is an example C<TO_CBOR> method: 672Here is an example C<TO_CBOR> method:
496 673
497 sub My::Object::TO_CBOR { 674 sub My::Object::TO_CBOR {
498 my ($obj) = @_; 675 my ($obj) = @_;
509 686
510 sub URI::TO_CBOR { 687 sub URI::TO_CBOR {
511 my ($self) = @_; 688 my ($self) = @_;
512 my $uri = "$self"; # stringify uri 689 my $uri = "$self"; # stringify uri
513 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string 690 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
514 CBOR::XS::tagged 32, "$_[0]" 691 CBOR::XS::tag 32, "$_[0]"
515 } 692 }
516 693
517This will encode URIs as a UTF-8 string with tag 32, which indicates an 694This will encode URIs as a UTF-8 string with tag 32, which indicates an
518URI. 695URI.
519 696
530 "$self" # encode url string 707 "$self" # encode url string
531 } 708 }
532 709
533 sub URI::THAW { 710 sub URI::THAW {
534 my ($class, $serialiser, $uri) = @_; 711 my ($class, $serialiser, $uri) = @_;
535
536 $class->new ($uri) 712 $class->new ($uri)
537 } 713 }
538 714
539Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For 715Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For
540example, a C<FREEZE> method that returns "type", "id" and "variant" values 716example, a C<FREEZE> method that returns "type", "id" and "variant" values
671additional tags (such as base64url). 847additional tags (such as base64url).
672 848
673=head2 ENFORCED TAGS 849=head2 ENFORCED TAGS
674 850
675These tags are always handled when decoding, and their handling cannot be 851These tags are always handled when decoding, and their handling cannot be
676overriden by the user. 852overridden by the user.
677 853
678=over 4 854=over 4
679 855
680=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>) 856=item 26 (perl-object, L<http://cbor.schmorp.de/perl-object>)
681 857
682These tags are automatically created (and decoded) for serialisable 858These tags are automatically created (and decoded) for serialisable
683objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object 859objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
684serialisation protocol). See L<OBJECT SERIALISATION> for details. 860serialisation protocol). See L<OBJECT SERIALISATION> for details.
685 861
686=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>) 862=item 28, 29 (shareable, sharedref, L<http://cbor.schmorp.de/value-sharing>)
687 863
688These tags are automatically decoded when encountered, resulting in 864These tags are automatically decoded when encountered (and they do not
865result in a cyclic data structure, see C<allow_cycles>), resulting in
689shared values in the decoded object. They are only encoded, however, when 866shared values in the decoded object. They are only encoded, however, when
690C<allow_sharable> is enabled. 867C<allow_sharing> is enabled.
691 868
869Not all shared values can be successfully decoded: values that reference
870themselves will I<currently> decode as C<undef> (this is not the same
871as a reference pointing to itself, which will be represented as a value
872that contains an indirect reference to itself - these will be decoded
873properly).
874
875Note that considerably more shared value data structures can be decoded
876than will be encoded - currently, only values pointed to by references
877will be shared, others will not. While non-reference shared values can be
878generated in Perl with some effort, they were considered too unimportant
879to be supported in the encoder. The decoder, however, will decode these
880values as shared values.
881
692=item <unassigned>, <unassigned> (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>) 882=item 256, 25 (stringref-namespace, stringref, L<http://cbor.schmorp.de/stringref>)
693 883
694These tags are automatically decoded when encountered. They are only 884These tags are automatically decoded when encountered. They are only
695encoded, however, when C<allow_stringref> is enabled. 885encoded, however, when C<pack_strings> is enabled.
696 886
697=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>) 887=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
698 888
699This tag is automatically generated when a reference are encountered (with 889This tag is automatically generated when a reference are encountered (with
700the exception of hash and array refernces). It is converted to a reference 890the exception of hash and array references). It is converted to a reference
701when decoding. 891when decoding.
702 892
703=item 55799 (self-describe CBOR, RFC 7049) 893=item 55799 (self-describe CBOR, RFC 7049)
704 894
705This value is not generated on encoding (unless explicitly requested by 895This value is not generated on encoding (unless explicitly requested by
708=back 898=back
709 899
710=head2 NON-ENFORCED TAGS 900=head2 NON-ENFORCED TAGS
711 901
712These tags have default filters provided when decoding. Their handling can 902These tags have default filters provided when decoding. Their handling can
713be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by 903be overridden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
714providing a custom C<filter> callback when decoding. 904providing a custom C<filter> callback when decoding.
715 905
716When they result in decoding into a specific Perl class, the module 906When they result in decoding into a specific Perl class, the module
717usually provides a corresponding C<TO_CBOR> method as well. 907usually provides a corresponding C<TO_CBOR> method as well.
718 908
721provide these modules. The decoding usually fails with an exception if the 911provide these modules. The decoding usually fails with an exception if the
722required module cannot be loaded. 912required module cannot be loaded.
723 913
724=over 4 914=over 4
725 915
916=item 0, 1 (date/time string, seconds since the epoch)
917
918These tags are decoded into L<Time::Piece> objects. The corresponding
919C<Time::Piece::TO_CBOR> method always encodes into tag 1 values currently.
920
921The L<Time::Piece> API is generally surprisingly bad, and fractional
922seconds are only accidentally kept intact, so watch out. On the plus side,
923the module comes with perl since 5.10, which has to count for something.
924
726=item 2, 3 (positive/negative bignum) 925=item 2, 3 (positive/negative bignum)
727 926
728These tags are decoded into L<Math::BigInt> objects. The corresponding 927These tags are decoded into L<Math::BigInt> objects. The corresponding
729C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR 928C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR
730integers, and others into positive/negative CBOR bignums. 929integers, and others into positive/negative CBOR bignums.
751C<URI::TO_CBOR> method again results in a CBOR URI value. 950C<URI::TO_CBOR> method again results in a CBOR URI value.
752 951
753=back 952=back
754 953
755=cut 954=cut
756
757our %FILTER = (
758 # 0 # rfc4287 datetime, utf-8
759 # 1 # unix timestamp, any
760
761 2 => sub { # pos bigint
762 require Math::BigInt;
763 Math::BigInt->new ("0x" . unpack "H*", pop)
764 },
765
766 3 => sub { # neg bigint
767 require Math::BigInt;
768 -Math::BigInt->new ("0x" . unpack "H*", pop)
769 },
770
771 4 => sub { # decimal fraction, array
772 require Math::BigFloat;
773 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
774 },
775
776 5 => sub { # bigfloat, array
777 require Math::BigFloat;
778 scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
779 },
780
781 21 => sub { pop }, # expected conversion to base64url encoding
782 22 => sub { pop }, # expected conversion to base64 encoding
783 23 => sub { pop }, # expected conversion to base16 encoding
784
785 # 24 # embedded cbor, byte string
786
787 32 => sub {
788 require URI;
789 URI->new (pop)
790 },
791
792 # 33 # base64url rfc4648, utf-8
793 # 34 # base64 rfc46484, utf-8
794 # 35 # regex pcre/ecma262, utf-8
795 # 36 # mime message rfc2045, utf-8
796);
797
798 955
799=head1 CBOR and JSON 956=head1 CBOR and JSON
800 957
801CBOR is supposed to implement a superset of the JSON data model, and is, 958CBOR is supposed to implement a superset of the JSON data model, and is,
802with some coercion, able to represent all JSON texts (something that other 959with some coercion, able to represent all JSON texts (something that other
863properly. Half precision types are accepted, but not encoded. 1020properly. Half precision types are accepted, but not encoded.
864 1021
865Strict mode and canonical mode are not implemented. 1022Strict mode and canonical mode are not implemented.
866 1023
867 1024
1025=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT
1026
1027On perls that were built without 64 bit integer support (these are rare
1028nowadays, even on 32 bit architectures, as all major Perl distributions
1029are built with 64 bit integer support), support for any kind of 64 bit
1030integer in CBOR is very limited - most likely, these 64 bit values will
1031be truncated, corrupted, or otherwise not decoded correctly. This also
1032includes string, array and map sizes that are stored as 64 bit integers.
1033
1034
868=head1 THREADS 1035=head1 THREADS
869 1036
870This module is I<not> guaranteed to be thread safe and there are no 1037This module is I<not> guaranteed to be thread safe and there are no
871plans to change this until Perl gets thread support (as opposed to the 1038plans to change this until Perl gets thread support (as opposed to the
872horribly slow so-called "threads" which are simply slow and bloated 1039horribly slow so-called "threads" which are simply slow and bloated
885service. I put the contact address into my modules for a reason. 1052service. I put the contact address into my modules for a reason.
886 1053
887=cut 1054=cut
888 1055
889our %FILTER = ( 1056our %FILTER = (
890 # 0 # rfc4287 datetime, utf-8 1057 0 => sub { # rfc4287 datetime, utf-8
891 # 1 # unix timestamp, any 1058 require Time::Piece;
1059 # Time::Piece::Strptime uses the "incredibly flexible date parsing routine"
1060 # from FreeBSD, which can't parse ISO 8601, RFC3339, RFC4287 or much of anything
1061 # else either. Whats incredibe over standard strptime totally escapes me.
1062 # doesn't do fractional times, either. sigh.
1063 # In fact, it's all a lie, it uses whatever strptime it wants, and of course,
1064 # they are all incompatible. The openbsd one simply ignores %z (but according to the
1065 # docs, it would be much more incredibly flexible indeed. If it worked, that is.).
1066 scalar eval {
1067 my $s = $_[1];
1068
1069 $s =~ s/Z$/+00:00/;
1070 $s =~ s/(\.[0-9]+)?([+-][0-9][0-9]):([0-9][0-9])$//
1071 or die;
1072
1073 my $b = $1 - ($2 * 60 + $3) * 60; # fractional part + offset. hopefully
1074 my $d = Time::Piece->strptime ($s, "%Y-%m-%dT%H:%M:%S");
1075
1076 Time::Piece::gmtime ($d->epoch + $b)
1077 } || die "corrupted CBOR date/time string ($_[0])";
1078 },
1079
1080 1 => sub { # seconds since the epoch, possibly fractional
1081 require Time::Piece;
1082 scalar Time::Piece::gmtime (pop)
1083 },
892 1084
893 2 => sub { # pos bigint 1085 2 => sub { # pos bigint
894 require Math::BigInt; 1086 require Math::BigInt;
895 Math::BigInt->new ("0x" . unpack "H*", pop) 1087 Math::BigInt->new ("0x" . unpack "H*", pop)
896 }, 1088 },
905 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0]) 1097 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
906 }, 1098 },
907 1099
908 5 => sub { # bigfloat, array 1100 5 => sub { # bigfloat, array
909 require Math::BigFloat; 1101 require Math::BigFloat;
910 scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2) 1102 scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
911 }, 1103 },
912 1104
913 21 => sub { pop }, # expected conversion to base64url encoding 1105 21 => sub { pop }, # expected conversion to base64url encoding
914 22 => sub { pop }, # expected conversion to base64 encoding 1106 22 => sub { pop }, # expected conversion to base64 encoding
915 23 => sub { pop }, # expected conversion to base16 encoding 1107 23 => sub { pop }, # expected conversion to base16 encoding
932} 1124}
933 1125
934sub URI::TO_CBOR { 1126sub URI::TO_CBOR {
935 my $uri = $_[0]->as_string; 1127 my $uri = $_[0]->as_string;
936 utf8::upgrade $uri; 1128 utf8::upgrade $uri;
937 CBOR::XS::tag 32, $uri 1129 tag 32, $uri
938} 1130}
939 1131
940sub Math::BigInt::TO_CBOR { 1132sub Math::BigInt::TO_CBOR {
941 if ($_[0] >= -2147483648 && $_[0] <= 2147483647) { 1133 if ($_[0] >= -2147483648 && $_[0] <= 2147483647) {
942 $_[0]->numify 1134 $_[0]->numify
943 } else { 1135 } else {
944 my $hex = substr $_[0]->as_hex, 2; 1136 my $hex = substr $_[0]->as_hex, 2;
945 $hex = "0$hex" if 1 & length $hex; # sigh 1137 $hex = "0$hex" if 1 & length $hex; # sigh
946 CBOR::XS::tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex 1138 tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
947 } 1139 }
948} 1140}
949 1141
950sub Math::BigFloat::TO_CBOR { 1142sub Math::BigFloat::TO_CBOR {
951 my ($m, $e) = $_[0]->parts; 1143 my ($m, $e) = $_[0]->parts;
952 CBOR::XS::tag 4, [$e->numify, $m] 1144 tag 4, [$e->numify, $m]
1145}
1146
1147sub Time::Piece::TO_CBOR {
1148 tag 1, 0 + $_[0]->epoch
953} 1149}
954 1150
955XSLoader::load "CBOR::XS", $VERSION; 1151XSLoader::load "CBOR::XS", $VERSION;
956 1152
957=head1 SEE ALSO 1153=head1 SEE ALSO

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines