ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/XS.pm
(Generate patch)

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.25 by root, Thu Nov 28 12:08:07 2013 UTC vs.
Revision 1.28 by root, Thu Nov 28 16:09:04 2013 UTC

… …
26 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string 26 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
27 } 27 }
28 28
29=head1 DESCRIPTION 29=head1 DESCRIPTION
30 30
31WARNING! This module is very new, and not very well tested (that's up
32to you to do). Furthermore, details of the implementation might change
33freely before version 1.0. And lastly, most extensions depend on an IANA
34assignment, and until that assignment is official, this implementation is
35not interoperable with other implementations (even future versions of this
36module) until the assignment is done.
37
38You are still invited to try out CBOR, and this module.
39
40This module converts Perl data structures to the Concise Binary Object 31This module converts Perl data structures to the Concise Binary Object
41Representation (CBOR) and vice versa. CBOR is a fast binary serialisation 32Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
42format that aims to use a superset of the JSON data model, i.e. when you 33format that aims to use an (almost) superset of the JSON data model, i.e.
43can represent something in JSON, you should be able to represent it in 34when you can represent something useful in JSON, you should be able to
44CBOR. 35represent it in CBOR.
45 36
46In short, CBOR is a faster and very compact binary alternative to JSON, 37In short, CBOR is a faster and quite compact binary alternative to JSON,
47with the added ability of supporting serialisation of Perl objects. (JSON 38with the added ability of supporting serialisation of Perl objects. (JSON
48often compresses better than CBOR though, so if you plan to compress the 39often compresses better than CBOR though, so if you plan to compress the
49data later you might want to compare both formats first). 40data later and speed is less important you might want to compare both
41formats first).
50 42
51To give you a general idea about speed, with texts in the megabyte range, 43To give you a general idea about speed, with texts in the megabyte range,
52C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or 44C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or
53L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the 45L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the
54data, the worse L<Storable> performs in comparison. 46data, the worse L<Storable> performs in comparison.
55 47
56As for compactness, C<CBOR::XS> encoded data structures are usually about 48Regarding compactness, C<CBOR::XS>-encoded data structures are usually
5720% smaller than the same data encoded as (compact) JSON or L<Storable>. 49about 20% smaller than the same data encoded as (compact) JSON or
50L<Storable>.
58 51
59In addition to the core CBOR data format, this module implements a number 52In addition to the core CBOR data format, this module implements a
60of extensions, to support cyclic and self-referencing data structures 53number of extensions, to support cyclic and shared data structures (see
61(see C<allow_sharing>), string deduplication (see C<pack_strings>) and 54C<allow_sharing>), string deduplication (see C<pack_strings>) and scalar
62scalar references (always enabled). 55references (always enabled).
63 56
64The primary goal of this module is to be I<correct> and the secondary goal 57The primary goal of this module is to be I<correct> and the secondary goal
65is to be I<fast>. To reach the latter goal it was written in C. 58is to be I<fast>. To reach the latter goal it was written in C.
66 59
67See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and 60See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
… …
71 64
72package CBOR::XS; 65package CBOR::XS;
73 66
74use common::sense; 67use common::sense;
75 68
76our $VERSION = 0.09; 69our $VERSION = '1.0';
77our @ISA = qw(Exporter); 70our @ISA = qw(Exporter);
78 71
79our @EXPORT = qw(encode_cbor decode_cbor); 72our @EXPORT = qw(encode_cbor decode_cbor);
80 73
81use Exporter; 74use Exporter;
… …
191sharing extension. This also makes it possible to encode cyclic data 184sharing extension. This also makes it possible to encode cyclic data
192structures. 185structures.
193 186
194It is recommended to leave it off unless you know your 187It is recommended to leave it off unless you know your
195communication partner supports the value sharing extensions to CBOR 188communication partner supports the value sharing extensions to CBOR
196(http://cbor.schmorp.de/value-sharing), as without decoder support, the 189(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the
197resulting data structure might be unusable. 190resulting data structure might be unusable.
198 191
199Detecting shared values incurs a runtime overhead when values are encoded 192Detecting shared values incurs a runtime overhead when values are encoded
200that have a reference counter large than one, and might unnecessarily 193that have a reference counter large than one, and might unnecessarily
201increase the encoded size, as potentially shared values are encode as 194increase the encoded size, as potentially shared values are encode as
… …
224also results in a very large runtime overhead (expect encoding times to be 217also results in a very large runtime overhead (expect encoding times to be
2252-4 times as high as without). 2182-4 times as high as without).
226 219
227It is recommended to leave it off unless you know your 220It is recommended to leave it off unless you know your
228communications partner supports the stringref extension to CBOR 221communications partner supports the stringref extension to CBOR
229(http://cbor.schmorp.de/stringref), as without decoder support, the 222(L<http://cbor.schmorp.de/stringref>), as without decoder support, the
230resulting data structure might not be usable. 223resulting data structure might not be usable.
231 224
232If C<$enable> is false (the default), then C<encode> will encode strings 225If C<$enable> is false (the default), then C<encode> will encode strings
233the standard CBOR way. 226the standard CBOR way.
234 227
… …
259function, C<CBOR::XS::default_filter>, is used. This function simply looks 252function, C<CBOR::XS::default_filter>, is used. This function simply looks
260up the tag in the C<%CBOR::XS::FILTER> hash. If an entry exists it must be 253up the tag in the C<%CBOR::XS::FILTER> hash. If an entry exists it must be
261a code reference that is called with tag and value, and is responsible for 254a code reference that is called with tag and value, and is responsible for
262decoding the value. If no entry exists, it returns no values. 255decoding the value. If no entry exists, it returns no values.
263 256
264Example: decode all tags not handled internally into CBOR::XS::Tagged 257Example: decode all tags not handled internally into C<CBOR::XS::Tagged>
265objects, with no other special handling (useful when working with 258objects, with no other special handling (useful when working with
266potentially "unsafe" CBOR data). 259potentially "unsafe" CBOR data).
267 260
268 CBOR::XS->new->filter (sub { })->decode ($cbor_data); 261 CBOR::XS->new->filter (sub { })->decode ($cbor_data);
269 262
… …
323CBOR integers become (numeric) perl scalars. On perls without 64 bit 316CBOR integers become (numeric) perl scalars. On perls without 64 bit
324support, 64 bit integers will be truncated or otherwise corrupted. 317support, 64 bit integers will be truncated or otherwise corrupted.
325 318
326=item byte strings 319=item byte strings
327 320
328Byte strings will become octet strings in Perl (the byte values 0..255 321Byte strings will become octet strings in Perl (the Byte values 0..255
329will simply become characters of the same value in Perl). 322will simply become characters of the same value in Perl).
330 323
331=item UTF-8 strings 324=item UTF-8 strings
332 325
333UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be 326UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
… …
356=item tagged values 349=item tagged values
357 350
358Tagged items consists of a numeric tag and another CBOR value. 351Tagged items consists of a numeric tag and another CBOR value.
359 352
360See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >> 353See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >>
361for details. 354for details on which tags are handled how.
362 355
363=item anything else 356=item anything else
364 357
365Anything else (e.g. unsupported simple values) will raise a decoding 358Anything else (e.g. unsupported simple values) will raise a decoding
366error. 359error.
… …
369 362
370 363
371=head2 PERL -> CBOR 364=head2 PERL -> CBOR
372 365
373The mapping from Perl to CBOR is slightly more difficult, as Perl is a 366The mapping from Perl to CBOR is slightly more difficult, as Perl is a
374truly typeless language, so we can only guess which CBOR type is meant by 367typeless language. That means this module can only guess which CBOR type
375a Perl value. 368is meant by a perl value.
376 369
377=over 4 370=over 4
378 371
379=item hash references 372=item hash references
380 373
381Perl hash references become CBOR maps. As there is no inherent ordering in 374Perl hash references become CBOR maps. As there is no inherent ordering in
382hash keys (or CBOR maps), they will usually be encoded in a pseudo-random 375hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
383order. 376order. This order can be different each time a hahs is encoded.
384 377
385Currently, tied hashes will use the indefinite-length format, while normal 378Currently, tied hashes will use the indefinite-length format, while normal
386hashes will use the fixed-length format. 379hashes will use the fixed-length format.
387 380
388=item array references 381=item array references
389 382
390Perl array references become fixed-length CBOR arrays. 383Perl array references become fixed-length CBOR arrays.
391 384
392=item other references 385=item other references
393 386
394Other unblessed references are generally not allowed and will cause an 387Other unblessed references will be represented using
395exception to be thrown, except for references to the integers C<0> and 388the indirection tag extension (tag value C<22098>,
396C<1>, which get turned into false and true in CBOR. 389L<http://cbor.schmorp.de/indirection>). CBOR decoders are guaranteed
390to be able to decode these values somehow, by either "doing the right
391thing", decoding into a generic tagged object, simply ignoring the tag, or
392something else.
397 393
398=item CBOR::XS::Tagged objects 394=item CBOR::XS::Tagged objects
399 395
400Objects of this type must be arrays consisting of a single C<[tag, value]> 396Objects of this type must be arrays consisting of a single C<[tag, value]>
401pair. The (numerical) tag will be encoded as a CBOR tag, the value will 397pair. The (numerical) tag will be encoded as a CBOR tag, the value will
402be encoded as appropriate for the value. You cna use C<CBOR::XS::tag> to 398be encoded as appropriate for the value. You must use C<CBOR::XS::tag> to
403create such objects. 399create such objects.
404 400
405=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error 401=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
406 402
407These special values become CBOR true, CBOR false and CBOR undefined 403These special values become CBOR true, CBOR false and CBOR undefined
… …
424 # dump as number 420 # dump as number
425 encode_cbor [2] # yields [2] 421 encode_cbor [2] # yields [2]
426 encode_cbor [-3.0e17] # yields [-3e+17] 422 encode_cbor [-3.0e17] # yields [-3e+17]
427 my $value = 5; encode_cbor [$value] # yields [5] 423 my $value = 5; encode_cbor [$value] # yields [5]
428 424
429 # used as string, so dump as string 425 # used as string, so dump as string (either byte or text)
430 print $value; 426 print $value;
431 encode_cbor [$value] # yields ["5"] 427 encode_cbor [$value] # yields ["5"]
432 428
433 # undef becomes null 429 # undef becomes null
434 encode_cbor [undef] # yields [null] 430 encode_cbor [undef] # yields [null]
… …
437 433
438 my $x = 3.1; # some variable containing a number 434 my $x = 3.1; # some variable containing a number
439 "$x"; # stringified 435 "$x"; # stringified
440 $x .= ""; # another, more awkward way to stringify 436 $x .= ""; # another, more awkward way to stringify
441 print $x; # perl does it for you, too, quite often 437 print $x; # perl does it for you, too, quite often
438
439You can force whether a string ie encoded as byte or text string by using
440C<utf8::upgrade> and C<utf8::downgrade>):
441
442 utf8::upgrade $x; # encode $x as text string
443 utf8::downgrade $x; # encode $x as byte string
444
445Perl doesn't define what operations up- and downgrade strings, so if the
446difference between byte and text is important, you should up- or downgrade
447your string as late as possible before encoding.
442 448
443You can force the type to be a CBOR number by numifying it: 449You can force the type to be a CBOR number by numifying it:
444 450
445 my $x = "3"; # some variable containing a string 451 my $x = "3"; # some variable containing a string
446 $x += 0; # numify it, ensuring it will be dumped as a number 452 $x += 0; # numify it, ensuring it will be dumped as a number
… …
513 519
514 sub URI::TO_CBOR { 520 sub URI::TO_CBOR {
515 my ($self) = @_; 521 my ($self) = @_;
516 my $uri = "$self"; # stringify uri 522 my $uri = "$self"; # stringify uri
517 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string 523 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
518 CBOR::XS::tagged 32, "$_[0]" 524 CBOR::XS::tag 32, "$_[0]"
519 } 525 }
520 526
521This will encode URIs as a UTF-8 string with tag 32, which indicates an 527This will encode URIs as a UTF-8 string with tag 32, which indicates an
522URI. 528URI.
523 529
… …
679These tags are always handled when decoding, and their handling cannot be 685These tags are always handled when decoding, and their handling cannot be
680overriden by the user. 686overriden by the user.
681 687
682=over 4 688=over 4
683 689
684=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>) 690=item 26 (perl-object, L<http://cbor.schmorp.de/perl-object>)
685 691
686These tags are automatically created (and decoded) for serialisable 692These tags are automatically created (and decoded) for serialisable
687objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object 693objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
688serialisation protocol). See L<OBJECT SERIALISATION> for details. 694serialisation protocol). See L<OBJECT SERIALISATION> for details.
689 695
690=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>) 696=item 28, 29 (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)
691 697
692These tags are automatically decoded when encountered, resulting in 698These tags are automatically decoded when encountered, resulting in
693shared values in the decoded object. They are only encoded, however, when 699shared values in the decoded object. They are only encoded, however, when
694C<allow_sharable> is enabled. 700C<allow_sharable> is enabled.
695 701
696=item <unassigned>, <unassigned> (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>) 702=item 256, 25 (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>)
697 703
698These tags are automatically decoded when encountered. They are only 704These tags are automatically decoded when encountered. They are only
699encoded, however, when C<pack_strings> is enabled. 705encoded, however, when C<pack_strings> is enabled.
700 706
701=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>) 707=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines