ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/XS.pm
(Generate patch)

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.22 by root, Fri Nov 22 15:28:38 2013 UTC vs.
Revision 1.39 by root, Tue Dec 10 15:31:40 2013 UTC

26 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string 26 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
27 } 27 }
28 28
29=head1 DESCRIPTION 29=head1 DESCRIPTION
30 30
31WARNING! This module is very new, and not very well tested (that's up
32to you to do). Furthermore, details of the implementation might change
33freely before version 1.0. And lastly, most extensions depend on an IANA
34assignment, and until that assignment is official, this implementation is
35not interoperable with other implementations (even future versions of this
36module) until the assignment is done.
37
38You are still invited to try out CBOR, and this module.
39
40This module converts Perl data structures to the Concise Binary Object 31This module converts Perl data structures to the Concise Binary Object
41Representation (CBOR) and vice versa. CBOR is a fast binary serialisation 32Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
42format that aims to use a superset of the JSON data model, i.e. when you 33format that aims to use an (almost) superset of the JSON data model, i.e.
43can represent something in JSON, you should be able to represent it in 34when you can represent something useful in JSON, you should be able to
44CBOR. 35represent it in CBOR.
45 36
46In short, CBOR is a faster and very compact binary alternative to JSON, 37In short, CBOR is a faster and quite compact binary alternative to JSON,
47with the added ability of supporting serialisation of Perl objects. (JSON 38with the added ability of supporting serialisation of Perl objects. (JSON
48often compresses better than CBOR though, so if you plan to compress the 39often compresses better than CBOR though, so if you plan to compress the
49data later you might want to compare both formats first). 40data later and speed is less important you might want to compare both
41formats first).
50 42
51To give you a general idea about speed, with texts in the megabyte range, 43To give you a general idea about speed, with texts in the megabyte range,
52C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or 44C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or
53L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the 45L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the
54data, the worse L<Storable> performs in comparison. 46data, the worse L<Storable> performs in comparison.
55 47
56As for compactness, C<CBOR::XS> encoded data structures are usually about 48Regarding compactness, C<CBOR::XS>-encoded data structures are usually
5720% smaller than the same data encoded as (compact) JSON or L<Storable>. 49about 20% smaller than the same data encoded as (compact) JSON or
50L<Storable>.
58 51
59In addition to the core CBOR data format, this module implements a number 52In addition to the core CBOR data format, this module implements a
60of extensions, to support cyclic and self-referencing data structures 53number of extensions, to support cyclic and shared data structures
61(see C<allow_sharing>), string deduplication (see C<allow_stringref>) and 54(see C<allow_sharing> and C<allow_cycles>), string deduplication (see
62scalar references (always enabled). 55C<pack_strings>) and scalar references (always enabled).
63 56
64The primary goal of this module is to be I<correct> and the secondary goal 57The primary goal of this module is to be I<correct> and the secondary goal
65is to be I<fast>. To reach the latter goal it was written in C. 58is to be I<fast>. To reach the latter goal it was written in C.
66 59
67See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and 60See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
71 64
72package CBOR::XS; 65package CBOR::XS;
73 66
74use common::sense; 67use common::sense;
75 68
76our $VERSION = 0.08; 69our $VERSION = 1.12;
77our @ISA = qw(Exporter); 70our @ISA = qw(Exporter);
78 71
79our @EXPORT = qw(encode_cbor decode_cbor); 72our @EXPORT = qw(encode_cbor decode_cbor);
80 73
81use Exporter; 74use Exporter;
118strings. All boolean flags described below are by default I<disabled>. 111strings. All boolean flags described below are by default I<disabled>.
119 112
120The mutators for flags all return the CBOR object again and thus calls can 113The mutators for flags all return the CBOR object again and thus calls can
121be chained: 114be chained:
122 115
123#TODO
124 my $cbor = CBOR::XS->new->encode ({a => [1,2]}); 116 my $cbor = CBOR::XS->new->encode ({a => [1,2]});
125 117
126=item $cbor = $cbor->max_depth ([$maximum_nesting_depth]) 118=item $cbor = $cbor->max_depth ([$maximum_nesting_depth])
127 119
128=item $max_depth = $cbor->get_max_depth 120=item $max_depth = $cbor->get_max_depth
187as an array, is referenced multiple times), but instead will emit a 179as an array, is referenced multiple times), but instead will emit a
188reference to the earlier value. 180reference to the earlier value.
189 181
190This means that such values will only be encoded once, and will not result 182This means that such values will only be encoded once, and will not result
191in a deep cloning of the value on decode, in decoders supporting the value 183in a deep cloning of the value on decode, in decoders supporting the value
192sharing extension. 184sharing extension. This also makes it possible to encode cyclic data
185structures (which need C<allow_cycles> to ne enabled to be decoded by this
186module).
193 187
194It is recommended to leave it off unless you know your 188It is recommended to leave it off unless you know your
195communication partner supports the value sharing extensions to CBOR 189communication partner supports the value sharing extensions to CBOR
196(http://cbor.schmorp.de/value-sharing). 190(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the
191resulting data structure might be unusable.
197 192
198Detecting shared values incurs a runtime overhead when values are encoded 193Detecting shared values incurs a runtime overhead when values are encoded
199that have a reference counter large than one, and might unnecessarily 194that have a reference counter large than one, and might unnecessarily
200increase the encoded size, as potentially shared values are encode as 195increase the encoded size, as potentially shared values are encode as
201sharable whether or not they are actually shared. 196shareable whether or not they are actually shared.
202 197
203At the moment, only targets of references can be shared (e.g. scalars, 198At the moment, only targets of references can be shared (e.g. scalars,
204arrays or hashes pointed to by a reference). Weirder constructs, such as 199arrays or hashes pointed to by a reference). Weirder constructs, such as
205an array with multiple "copies" of the I<same> string, which are hard but 200an array with multiple "copies" of the I<same> string, which are hard but
206not impossible to create in Perl, are not supported (this is the same as 201not impossible to create in Perl, are not supported (this is the same as
207for L<Storable>). 202with L<Storable>).
208 203
209If C<$enable> is false (the default), then C<encode> will encode 204If C<$enable> is false (the default), then C<encode> will encode shared
210exception when it encounters anything it cannot encode as CBOR. 205data structures repeatedly, unsharing them in the process. Cyclic data
206structures cannot be encoded in this mode.
211 207
212This option does not affect C<decode> in any way - shared values and 208This option does not affect C<decode> in any way - shared values and
213references will always be decoded properly if present. 209references will always be decoded properly if present.
214 210
211=item $cbor = $cbor->allow_cycles ([$enable])
212
213=item $enabled = $cbor->get_allow_cycles
214
215If C<$enable> is true (or missing), then C<decode> will happily decode
216self-referential (cyclic) data structures. By default these will not be
217decoded, as they need manual cleanup to avoid memory leaks, so code that
218isn't prepared for this will not leak memory.
219
220If C<$enable> is false (the default), then C<decode> will throw an error
221when it encounters a self-referential/cyclic data structure.
222
223This option does not affect C<encode> in any way - shared values and
224references will always be decoded properly if present.
225
215=item $cbor = $cbor->allow_stringref ([$enable]) 226=item $cbor = $cbor->pack_strings ([$enable])
216 227
217=item $enabled = $cbor->get_allow_stringref 228=item $enabled = $cbor->get_pack_strings
218 229
219If C<$enable> is true (or missing), then C<encode> will try not to encode 230If C<$enable> is true (or missing), then C<encode> will try not to encode
220the same string twice, but will instead encode a reference to the string 231the same string twice, but will instead encode a reference to the string
221instead. Depending on your data format. this can save a lot of space, but 232instead. Depending on your data format, this can save a lot of space, but
222also results in a very large runtime overhead (expect encoding times to be 233also results in a very large runtime overhead (expect encoding times to be
2232-4 times as high as without). 2342-4 times as high as without).
224 235
225It is recommended to leave it off unless you know your 236It is recommended to leave it off unless you know your
226communications partner supports the stringref extension to CBOR 237communications partner supports the stringref extension to CBOR
227(http://cbor.schmorp.de/stringref). 238(L<http://cbor.schmorp.de/stringref>), as without decoder support, the
239resulting data structure might not be usable.
228 240
229If C<$enable> is false (the default), then C<encode> will encode 241If C<$enable> is false (the default), then C<encode> will encode strings
230exception when it encounters anything it cannot encode as CBOR. 242the standard CBOR way.
231 243
232This option does not affect C<decode> in any way - string references will 244This option does not affect C<decode> in any way - string references will
233always be decoded properly if present. 245always be decoded properly if present.
246
247=item $cbor = $cbor->validate_utf8 ([$enable])
248
249=item $enabled = $cbor->get_validate_utf8
250
251If C<$enable> is true (or missing), then C<decode> will validate that
252elements (text strings) containing UTF-8 data in fact contain valid UTF-8
253data (instead of blindly accepting it). This validation obviously takes
254extra time during decoding.
255
256The concept of "valid UTF-8" used is perl's concept, which is a superset
257of the official UTF-8.
258
259If C<$enable> is false (the default), then C<decode> will blindly accept
260UTF-8 data, marking them as valid UTF-8 in the resulting data structure
261regardless of whether thats true or not.
262
263Perl isn't too happy about corrupted UTF-8 in strings, but should
264generally not crash or do similarly evil things. Extensions might be not
265so forgiving, so it's recommended to turn on this setting if you receive
266untrusted CBOR.
267
268This option does not affect C<encode> in any way - strings that are
269supposedly valid UTF-8 will simply be dumped into the resulting CBOR
270string without checking whether that is, in fact, true or not.
271
272=item $cbor = $cbor->filter ([$cb->($tag, $value)])
273
274=item $cb_or_undef = $cbor->get_filter
275
276Sets or replaces the tagged value decoding filter (when C<$cb> is
277specified) or clears the filter (if no argument or C<undef> is provided).
278
279The filter callback is called only during decoding, when a non-enforced
280tagged value has been decoded (see L<TAG HANDLING AND EXTENSIONS> for a
281list of enforced tags). For specific tags, it's often better to provide a
282default converter using the C<%CBOR::XS::FILTER> hash (see below).
283
284The first argument is the numerical tag, the second is the (decoded) value
285that has been tagged.
286
287The filter function should return either exactly one value, which will
288replace the tagged value in the decoded data structure, or no values,
289which will result in default handling, which currently means the decoder
290creates a C<CBOR::XS::Tagged> object to hold the tag and the value.
291
292When the filter is cleared (the default state), the default filter
293function, C<CBOR::XS::default_filter>, is used. This function simply looks
294up the tag in the C<%CBOR::XS::FILTER> hash. If an entry exists it must be
295a code reference that is called with tag and value, and is responsible for
296decoding the value. If no entry exists, it returns no values.
297
298Example: decode all tags not handled internally into C<CBOR::XS::Tagged>
299objects, with no other special handling (useful when working with
300potentially "unsafe" CBOR data).
301
302 CBOR::XS->new->filter (sub { })->decode ($cbor_data);
303
304Example: provide a global filter for tag 1347375694, converting the value
305into some string form.
306
307 $CBOR::XS::FILTER{1347375694} = sub {
308 my ($tag, $value);
309
310 "tag 1347375694 value $value"
311 };
234 312
235=item $cbor_data = $cbor->encode ($perl_scalar) 313=item $cbor_data = $cbor->encode ($perl_scalar)
236 314
237Converts the given Perl data structure (a scalar value) to its CBOR 315Converts the given Perl data structure (a scalar value) to its CBOR
238representation. 316representation.
252and you need to know where the first CBOR string ends amd the next one 330and you need to know where the first CBOR string ends amd the next one
253starts. 331starts.
254 332
255 CBOR::XS->new->decode_prefix ("......") 333 CBOR::XS->new->decode_prefix ("......")
256 => ("...", 3) 334 => ("...", 3)
335
336=back
337
338=head2 INCREMENTAL PARSING
339
340In some cases, there is the need for incremental parsing of JSON
341texts. While this module always has to keep both CBOR text and resulting
342Perl data structure in memory at one time, it does allow you to parse a
343CBOR stream incrementally, using a similar to using "decode_prefix" to see
344if a full CBOR object is available, but is much more efficient.
345
346It basically works by parsing as much of a CBOR string as possible - if
347the CBOR data is not complete yet, the pasrer will remember where it was,
348to be able to restart when more data has been accumulated. Once enough
349data is available to either decode a complete CBOR value or raise an
350error, a real decode will be attempted.
351
352A typical use case would be a network protocol that consists of sending
353and receiving CBOR-encoded messages. The solution that works with CBOR and
354about anything else is by prepending a length to every CBOR value, so the
355receiver knows how many octets to read. More compact (and slightly slower)
356would be to just send CBOR values back-to-back, as C<CBOR::XS> knows where
357a CBOR value ends, and doesn't need an explicit length.
358
359The following methods help with this:
360
361=over 4
362
363=item @decoded = $cbor->incr_parse ($buffer)
364
365This method attempts to decode exactly one CBOR value from the beginning
366of the given C<$buffer>. The value is removed from the C<$buffer> on
367success. When C<$buffer> doesn't contain a complete value yet, it returns
368nothing. Finally, when the C<$buffer> doesn't start with something
369that could ever be a valid CBOR value, it raises an exception, just as
370C<decode> would. In the latter case the decoder state is undefined and
371must be reset before being able to parse further.
372
373This method modifies the C<$buffer> in place. When no CBOR value can be
374decoded, the decoder stores the current string offset. On the next call,
375continues decoding at the place where it stopped before. For this to make
376sense, the C<$buffer> must begin with the same octets as on previous
377unsuccessful calls.
378
379You can call this method in scalar context, in which case it either
380returns a decoded value or C<undef>. This makes it impossible to
381distinguish between CBOR null values (which decode to C<undef>) and an
382unsuccessful decode, which is often acceptable.
383
384=item @decoded = $cbor->incr_parse_multiple ($buffer)
385
386Same as C<incr_parse>, but attempts to decode as many CBOR values as
387possible in one go, instead of at most one. Calls to C<incr_parse> and
388C<incr_parse_multiple> can be interleaved.
389
390=item $cbor->incr_reset
391
392Resets the incremental decoder. This throws away any saved state, so that
393subsequent calls to C<incr_parse> or C<incr_parse_multiple> start to parse
394a new CBOR value from the beginning of the C<$buffer> again.
395
396This method can be caled at any time, but it I<must> be called if you want
397to change your C<$buffer> or there was a decoding error and you want to
398reuse the C<$cbor> object for future incremental parsings.
257 399
258=back 400=back
259 401
260 402
261=head1 MAPPING 403=head1 MAPPING
279CBOR integers become (numeric) perl scalars. On perls without 64 bit 421CBOR integers become (numeric) perl scalars. On perls without 64 bit
280support, 64 bit integers will be truncated or otherwise corrupted. 422support, 64 bit integers will be truncated or otherwise corrupted.
281 423
282=item byte strings 424=item byte strings
283 425
284Byte strings will become octet strings in Perl (the byte values 0..255 426Byte strings will become octet strings in Perl (the Byte values 0..255
285will simply become characters of the same value in Perl). 427will simply become characters of the same value in Perl).
286 428
287=item UTF-8 strings 429=item UTF-8 strings
288 430
289UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be 431UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
307C<Types:Serialiser::false> and C<Types::Serialiser::error>, 449C<Types:Serialiser::false> and C<Types::Serialiser::error>,
308respectively. They are overloaded to act almost exactly like the numbers 450respectively. They are overloaded to act almost exactly like the numbers
309C<1> and C<0> (for true and false) or to throw an exception on access (for 451C<1> and C<0> (for true and false) or to throw an exception on access (for
310error). See the L<Types::Serialiser> manpage for details. 452error). See the L<Types::Serialiser> manpage for details.
311 453
312=item CBOR tag 256 (perl object) 454=item tagged values
313 455
314The tag value C<256> (TODO: pending iana registration) will be used
315to deserialise a Perl object serialised with C<FREEZE>. See L<OBJECT
316SERIALISATION>, below, for details.
317
318=item CBOR tag 55799 (magic header)
319
320The tag 55799 is ignored (this tag implements the magic header).
321
322=item other CBOR tags
323
324Tagged items consists of a numeric tag and another CBOR value. Tags not 456Tagged items consists of a numeric tag and another CBOR value.
325handled internally are currently converted into a L<CBOR::XS::Tagged>
326object, which is simply a blessed array reference consisting of the
327numeric tag value followed by the (decoded) CBOR value.
328 457
329In the future, support for user-supplied conversions might get added. 458See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >>
459for details on which tags are handled how.
330 460
331=item anything else 461=item anything else
332 462
333Anything else (e.g. unsupported simple values) will raise a decoding 463Anything else (e.g. unsupported simple values) will raise a decoding
334error. 464error.
337 467
338 468
339=head2 PERL -> CBOR 469=head2 PERL -> CBOR
340 470
341The mapping from Perl to CBOR is slightly more difficult, as Perl is a 471The mapping from Perl to CBOR is slightly more difficult, as Perl is a
342truly typeless language, so we can only guess which CBOR type is meant by 472typeless language. That means this module can only guess which CBOR type
343a Perl value. 473is meant by a perl value.
344 474
345=over 4 475=over 4
346 476
347=item hash references 477=item hash references
348 478
349Perl hash references become CBOR maps. As there is no inherent ordering in 479Perl hash references become CBOR maps. As there is no inherent ordering in
350hash keys (or CBOR maps), they will usually be encoded in a pseudo-random 480hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
351order. 481order. This order can be different each time a hahs is encoded.
352 482
353Currently, tied hashes will use the indefinite-length format, while normal 483Currently, tied hashes will use the indefinite-length format, while normal
354hashes will use the fixed-length format. 484hashes will use the fixed-length format.
355 485
356=item array references 486=item array references
357 487
358Perl array references become fixed-length CBOR arrays. 488Perl array references become fixed-length CBOR arrays.
359 489
360=item other references 490=item other references
361 491
362Other unblessed references are generally not allowed and will cause an 492Other unblessed references will be represented using
363exception to be thrown, except for references to the integers C<0> and 493the indirection tag extension (tag value C<22098>,
364C<1>, which get turned into false and true in CBOR. 494L<http://cbor.schmorp.de/indirection>). CBOR decoders are guaranteed
495to be able to decode these values somehow, by either "doing the right
496thing", decoding into a generic tagged object, simply ignoring the tag, or
497something else.
365 498
366=item CBOR::XS::Tagged objects 499=item CBOR::XS::Tagged objects
367 500
368Objects of this type must be arrays consisting of a single C<[tag, value]> 501Objects of this type must be arrays consisting of a single C<[tag, value]>
369pair. The (numerical) tag will be encoded as a CBOR tag, the value will 502pair. The (numerical) tag will be encoded as a CBOR tag, the value will
370be encoded as appropriate for the value. You cna use C<CBOR::XS::tag> to 503be encoded as appropriate for the value. You must use C<CBOR::XS::tag> to
371create such objects. 504create such objects.
372 505
373=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error 506=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
374 507
375These special values become CBOR true, CBOR false and CBOR undefined 508These special values become CBOR true, CBOR false and CBOR undefined
377if you want. 510if you want.
378 511
379=item other blessed objects 512=item other blessed objects
380 513
381Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See 514Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
382L<OBJECT SERIALISATION>, below, for details. 515L<TAG HANDLING AND EXTENSIONS> for specific classes handled by this
516module, and L<OBJECT SERIALISATION> for generic object serialisation.
383 517
384=item simple scalars 518=item simple scalars
385 519
386TODO
387Simple Perl scalars (any scalar that is not a reference) are the most 520Simple Perl scalars (any scalar that is not a reference) are the most
388difficult objects to encode: CBOR::XS will encode undefined scalars as 521difficult objects to encode: CBOR::XS will encode undefined scalars as
389CBOR null values, scalars that have last been used in a string context 522CBOR null values, scalars that have last been used in a string context
390before encoding as CBOR strings, and anything else as number value: 523before encoding as CBOR strings, and anything else as number value:
391 524
392 # dump as number 525 # dump as number
393 encode_cbor [2] # yields [2] 526 encode_cbor [2] # yields [2]
394 encode_cbor [-3.0e17] # yields [-3e+17] 527 encode_cbor [-3.0e17] # yields [-3e+17]
395 my $value = 5; encode_cbor [$value] # yields [5] 528 my $value = 5; encode_cbor [$value] # yields [5]
396 529
397 # used as string, so dump as string 530 # used as string, so dump as string (either byte or text)
398 print $value; 531 print $value;
399 encode_cbor [$value] # yields ["5"] 532 encode_cbor [$value] # yields ["5"]
400 533
401 # undef becomes null 534 # undef becomes null
402 encode_cbor [undef] # yields [null] 535 encode_cbor [undef] # yields [null]
405 538
406 my $x = 3.1; # some variable containing a number 539 my $x = 3.1; # some variable containing a number
407 "$x"; # stringified 540 "$x"; # stringified
408 $x .= ""; # another, more awkward way to stringify 541 $x .= ""; # another, more awkward way to stringify
409 print $x; # perl does it for you, too, quite often 542 print $x; # perl does it for you, too, quite often
543
544You can force whether a string ie encoded as byte or text string by using
545C<utf8::upgrade> and C<utf8::downgrade>):
546
547 utf8::upgrade $x; # encode $x as text string
548 utf8::downgrade $x; # encode $x as byte string
549
550Perl doesn't define what operations up- and downgrade strings, so if the
551difference between byte and text is important, you should up- or downgrade
552your string as late as possible before encoding.
410 553
411You can force the type to be a CBOR number by numifying it: 554You can force the type to be a CBOR number by numifying it:
412 555
413 my $x = "3"; # some variable containing a string 556 my $x = "3"; # some variable containing a string
414 $x += 0; # numify it, ensuring it will be dumped as a number 557 $x += 0; # numify it, ensuring it will be dumped as a number
427 570
428=back 571=back
429 572
430=head2 OBJECT SERIALISATION 573=head2 OBJECT SERIALISATION
431 574
575This module implements both a CBOR-specific and the generic
576L<Types::Serialier> object serialisation protocol. The following
577subsections explain both methods.
578
579=head3 ENCODING
580
432This module knows two way to serialise a Perl object: The CBOR-specific 581This module knows two way to serialise a Perl object: The CBOR-specific
433way, and the generic way. 582way, and the generic way.
434 583
435Whenever the encoder encounters a Perl object that it cnanot serialise 584Whenever the encoder encounters a Perl object that it cannot serialise
436directly (most of them), it will first look up the C<TO_CBOR> method on 585directly (most of them), it will first look up the C<TO_CBOR> method on
437it. 586it.
438 587
439If it has a C<TO_CBOR> method, it will call it with the object as only 588If it has a C<TO_CBOR> method, it will call it with the object as only
440argument, and expects exactly one return value, which it will then 589argument, and expects exactly one return value, which it will then
446 595
447The C<FREEZE> method can return any number of values (i.e. zero or 596The C<FREEZE> method can return any number of values (i.e. zero or
448more). These will be encoded as CBOR perl object, together with the 597more). These will be encoded as CBOR perl object, together with the
449classname. 598classname.
450 599
600These methods I<MUST NOT> change the data structure that is being
601serialised. Failure to comply to this can result in memory corruption -
602and worse.
603
451If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail 604If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
452with an error. 605with an error.
453 606
607=head3 DECODING
608
454Objects encoded via C<TO_CBOR> cannot be automatically decoded, but 609Objects encoded via C<TO_CBOR> cannot (normally) be automatically decoded,
455objects encoded via C<FREEZE> can be decoded using the following protocol: 610but objects encoded via C<FREEZE> can be decoded using the following
611protocol:
456 612
457When an encoded CBOR perl object is encountered by the decoder, it will 613When an encoded CBOR perl object is encountered by the decoder, it will
458look up the C<THAW> method, by using the stored classname, and will fail 614look up the C<THAW> method, by using the stored classname, and will fail
459if the method cannot be found. 615if the method cannot be found.
460 616
461After the lookup it will call the C<THAW> method with the stored classname 617After the lookup it will call the C<THAW> method with the stored classname
462as first argument, the constant string C<CBOR> as second argument, and all 618as first argument, the constant string C<CBOR> as second argument, and all
463values returned by C<FREEZE> as remaining arguments. 619values returned by C<FREEZE> as remaining arguments.
464 620
465=head4 EXAMPLES 621=head3 EXAMPLES
466 622
467Here is an example C<TO_CBOR> method: 623Here is an example C<TO_CBOR> method:
468 624
469 sub My::Object::TO_CBOR { 625 sub My::Object::TO_CBOR {
470 my ($obj) = @_; 626 my ($obj) = @_;
481 637
482 sub URI::TO_CBOR { 638 sub URI::TO_CBOR {
483 my ($self) = @_; 639 my ($self) = @_;
484 my $uri = "$self"; # stringify uri 640 my $uri = "$self"; # stringify uri
485 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string 641 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
486 CBOR::XS::tagged 32, "$_[0]" 642 CBOR::XS::tag 32, "$_[0]"
487 } 643 }
488 644
489This will encode URIs as a UTF-8 string with tag 32, which indicates an 645This will encode URIs as a UTF-8 string with tag 32, which indicates an
490URI. 646URI.
491 647
633and extensions. If a tag is not mentioned here and no additional filters 789and extensions. If a tag is not mentioned here and no additional filters
634are provided for it, then the default handling applies (creating a 790are provided for it, then the default handling applies (creating a
635CBOR::XS::Tagged object on decoding, and only encoding the tag when 791CBOR::XS::Tagged object on decoding, and only encoding the tag when
636explicitly requested). 792explicitly requested).
637 793
794Tags not handled specifically are currently converted into a
795L<CBOR::XS::Tagged> object, which is simply a blessed array reference
796consisting of the numeric tag value followed by the (decoded) CBOR value.
797
638Future versions of this module reserve the right to special case 798Future versions of this module reserve the right to special case
639additional tags (such as base64url). 799additional tags (such as base64url).
640 800
641=head2 ENFORCED TAGS 801=head2 ENFORCED TAGS
642 802
643These tags are always handled when decoding, and their handling cannot be 803These tags are always handled when decoding, and their handling cannot be
644overriden by the user. 804overriden by the user.
645 805
646=over 4 806=over 4
647 807
648=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>) 808=item 26 (perl-object, L<http://cbor.schmorp.de/perl-object>)
649 809
650These tags are automatically created for serialisable objects using the 810These tags are automatically created (and decoded) for serialisable
651C<FREEZE/THAW> methods (the L<Types::Serialier> object serialisation 811objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
652protocol). 812serialisation protocol). See L<OBJECT SERIALISATION> for details.
653 813
654=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>) 814=item 28, 29 (shareable, sharedref, L <http://cbor.schmorp.de/value-sharing>)
655 815
656These tags are automatically decoded when encountered, resulting in 816These tags are automatically decoded when encountered (and they do not
817result in a cyclic data structure, see C<allow_cycles>), resulting in
657shared values in the decoded object. They are only encoded, however, when 818shared values in the decoded object. They are only encoded, however, when
658C<allow_sharable> is enabled. 819C<allow_sharing> is enabled.
659 820
821Not all shared values can be successfully decoded: values that reference
822themselves will I<currently> decode as C<undef> (this is not the same
823as a reference pointing to itself, which will be represented as a value
824that contains an indirect reference to itself - these will be decoded
825properly).
826
827Note that considerably more shared value data structures can be decoded
828than will be encoded - currently, only values pointed to by references
829will be shared, others will not. While non-reference shared values can be
830generated in Perl with some effort, they were considered too unimportant
831to be supported in the encoder. The decoder, however, will decode these
832values as shared values.
833
660=item <unassigned>, <unassigned> (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>) 834=item 256, 25 (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>)
661 835
662These tags are automatically decoded when encountered. They are only 836These tags are automatically decoded when encountered. They are only
663encoded, however, when C<allow_stringref> is enabled. 837encoded, however, when C<pack_strings> is enabled.
664 838
665=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>) 839=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
666 840
667This tag is automatically generated when a reference are encountered (with 841This tag is automatically generated when a reference are encountered (with
668the exception of hash and array refernces). It is converted to a reference 842the exception of hash and array refernces). It is converted to a reference
673This value is not generated on encoding (unless explicitly requested by 847This value is not generated on encoding (unless explicitly requested by
674the user), and is simply ignored when decoding. 848the user), and is simply ignored when decoding.
675 849
676=back 850=back
677 851
678=head2 OPTIONAL TAGS 852=head2 NON-ENFORCED TAGS
679 853
680These tags have default filters provided when decoding. Their handling can 854These tags have default filters provided when decoding. Their handling can
681be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by 855be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
682providing a custom C<filter> function when decoding. 856providing a custom C<filter> callback when decoding.
683 857
684When they result in decoding into a specific Perl class, the module 858When they result in decoding into a specific Perl class, the module
685usually provides a corresponding C<TO_CBOR> method as well. 859usually provides a corresponding C<TO_CBOR> method as well.
686 860
687When any of these need to load additional modules that are not part of the 861When any of these need to load additional modules that are not part of the
688perl core distribution (e.g. L<URI>), it is (currently) up to the user to 862perl core distribution (e.g. L<URI>), it is (currently) up to the user to
689provide these modules. The decoding usually fails with an exception if the 863provide these modules. The decoding usually fails with an exception if the
690required module cannot be loaded. 864required module cannot be loaded.
691 865
692=over 4 866=over 4
867
868=item 0, 1 (date/time string, seconds since the epoch)
869
870These tags are decoded into L<Time::Piece> objects. The corresponding
871C<Time::Piece::TO_CBOR> method always encodes into tag 1 values currently.
872
873The L<Time::Piece> API is generally surprisingly bad, and fractional
874seconds are only accidentally kept intact, so watch out. On the plus side,
875the module comes with perl since 5.10, which has to count for something.
693 876
694=item 2, 3 (positive/negative bignum) 877=item 2, 3 (positive/negative bignum)
695 878
696These tags are decoded into L<Math::BigInt> objects. The corresponding 879These tags are decoded into L<Math::BigInt> objects. The corresponding
697C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR 880C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR
831properly. Half precision types are accepted, but not encoded. 1014properly. Half precision types are accepted, but not encoded.
832 1015
833Strict mode and canonical mode are not implemented. 1016Strict mode and canonical mode are not implemented.
834 1017
835 1018
1019=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT
1020
1021On perls that were built without 64 bit integer support (these are rare
1022nowadays, even on 32 bit architectures), support for any kind of 64 bit
1023integer in CBOR is very limited - most likely, these 64 bit values will
1024be truncated, corrupted, or otherwise not decoded correctly. This also
1025includes string, array and map sizes that are stored as 64 bit integers.
1026
1027
836=head1 THREADS 1028=head1 THREADS
837 1029
838This module is I<not> guaranteed to be thread safe and there are no 1030This module is I<not> guaranteed to be thread safe and there are no
839plans to change this until Perl gets thread support (as opposed to the 1031plans to change this until Perl gets thread support (as opposed to the
840horribly slow so-called "threads" which are simply slow and bloated 1032horribly slow so-called "threads" which are simply slow and bloated
853service. I put the contact address into my modules for a reason. 1045service. I put the contact address into my modules for a reason.
854 1046
855=cut 1047=cut
856 1048
857our %FILTER = ( 1049our %FILTER = (
858 # 0 # rfc4287 datetime, utf-8 1050 0 => sub { # rfc4287 datetime, utf-8
859 # 1 # unix timestamp, any 1051 require Time::Piece;
1052 # Time::Piece::Strptime uses the "incredibly flexible date parsing routine"
1053 # from FreeBSD, which can't parse ISO 8601, RFC3339, RFC4287 or much of anything
1054 # else either. Whats incredibe over standard strptime totally escapes me.
1055 # doesn't do fractional times, either. sigh.
1056 # In fact, it's all a lie, it uses whatever strptime it wants, and of course,
1057 # they are all incomptible. The openbsd one simply ignores %z (but according to the
1058 # docs, it would be much more incredibly flexible indeed. If it worked, that is.).
1059 scalar eval {
1060 my $s = $_[1];
1061
1062 $s =~ s/Z$/+00:00/;
1063 $s =~ s/(\.[0-9]+)?([+-][0-9][0-9]):([0-9][0-9])$//
1064 or die;
1065
1066 my $b = $1 - ($2 * 60 + $3) * 60; # fractional part + offset. hopefully
1067 my $d = Time::Piece->strptime ($s, "%Y-%m-%dT%H:%M:%S");
1068
1069 Time::Piece::gmtime ($d->epoch + $b)
1070 } || die "corrupted CBOR date/time string ($_[0])";
1071 },
1072
1073 1 => sub { # seconds since the epoch, possibly fractional
1074 require Time::Piece;
1075 scalar Time::Piece::gmtime (pop)
1076 },
860 1077
861 2 => sub { # pos bigint 1078 2 => sub { # pos bigint
862 require Math::BigInt; 1079 require Math::BigInt;
863 Math::BigInt->new ("0x" . unpack "H*", pop) 1080 Math::BigInt->new ("0x" . unpack "H*", pop)
864 }, 1081 },
900} 1117}
901 1118
902sub URI::TO_CBOR { 1119sub URI::TO_CBOR {
903 my $uri = $_[0]->as_string; 1120 my $uri = $_[0]->as_string;
904 utf8::upgrade $uri; 1121 utf8::upgrade $uri;
905 CBOR::XS::tag 32, $uri 1122 tag 32, $uri
906} 1123}
907 1124
908sub Math::BigInt::TO_CBOR { 1125sub Math::BigInt::TO_CBOR {
909 if ($_[0] >= -2147483648 && $_[0] <= 2147483647) { 1126 if ($_[0] >= -2147483648 && $_[0] <= 2147483647) {
910 $_[0]->numify 1127 $_[0]->numify
911 } else { 1128 } else {
912 my $hex = substr $_[0]->as_hex, 2; 1129 my $hex = substr $_[0]->as_hex, 2;
913 $hex = "0$hex" if 1 & length $hex; # sigh 1130 $hex = "0$hex" if 1 & length $hex; # sigh
914 CBOR::XS::tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex 1131 tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
915 } 1132 }
916} 1133}
917 1134
918sub Math::BigFloat::TO_CBOR { 1135sub Math::BigFloat::TO_CBOR {
919 my ($m, $e) = $_[0]->parts; 1136 my ($m, $e) = $_[0]->parts;
920 CBOR::XS::tag 4, [$e->numify, $m] 1137 tag 4, [$e->numify, $m]
1138}
1139
1140sub Time::Piece::TO_CBOR {
1141 tag 1, $_[0]->epoch
921} 1142}
922 1143
923XSLoader::load "CBOR::XS", $VERSION; 1144XSLoader::load "CBOR::XS", $VERSION;
924 1145
925=head1 SEE ALSO 1146=head1 SEE ALSO

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines