ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/XS.pm
(Generate patch)

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.40 by root, Sun Jan 5 14:24:54 2014 UTC vs.
Revision 1.53 by root, Mon Apr 25 18:20:22 2016 UTC

64 64
65package CBOR::XS; 65package CBOR::XS;
66 66
67use common::sense; 67use common::sense;
68 68
69our $VERSION = 1.25; 69our $VERSION = 1.41;
70our @ISA = qw(Exporter); 70our @ISA = qw(Exporter);
71 71
72our @EXPORT = qw(encode_cbor decode_cbor); 72our @EXPORT = qw(encode_cbor decode_cbor);
73 73
74use Exporter; 74use Exporter;
218isn't prepared for this will not leak memory. 218isn't prepared for this will not leak memory.
219 219
220If C<$enable> is false (the default), then C<decode> will throw an error 220If C<$enable> is false (the default), then C<decode> will throw an error
221when it encounters a self-referential/cyclic data structure. 221when it encounters a self-referential/cyclic data structure.
222 222
223FUTURE DIRECTION: the motivation behind this option is to avoid I<real>
224cycles - future versions of this module might chose to decode cyclic data
225structures using weak references when this option is off, instead of
226throwing an error.
227
223This option does not affect C<encode> in any way - shared values and 228This option does not affect C<encode> in any way - shared values and
224references will always be decoded properly if present. 229references will always be encoded properly if present.
225 230
226=item $cbor = $cbor->pack_strings ([$enable]) 231=item $cbor = $cbor->pack_strings ([$enable])
227 232
228=item $enabled = $cbor->get_pack_strings 233=item $enabled = $cbor->get_pack_strings
229 234
242the standard CBOR way. 247the standard CBOR way.
243 248
244This option does not affect C<decode> in any way - string references will 249This option does not affect C<decode> in any way - string references will
245always be decoded properly if present. 250always be decoded properly if present.
246 251
252=item $cbor = $cbor->text_keys ([$enable])
253
254=item $enabled = $cbor->get_text_keys
255
256If C<$enabled> is true (or missing), then C<encode> will encode all
257perl hash keys as CBOR text strings/UTF-8 string, upgrading them as needed.
258
259If C<$enable> is false (the default), then C<encode> will encode hash keys
260normally - upgraded perl strings (strings internally encoded as UTF-8) as
261CBOR text strings, and downgraded perl strings as CBOR byte strings.
262
263This option does not affect C<decode> in any way.
264
265This option is useful for interoperability with CBOR decoders that don't
266treat byte strings as a form of text. It is especially useful as Perl
267gives very little control over hash keys.
268
269Enabling this option can be slow, as all downgraded hash keys that are
270encoded need to be scanned and converted to UTF-8.
271
272=item $cbor = $cbor->text_strings ([$enable])
273
274=item $enabled = $cbor->get_text_strings
275
276This option works similar to C<text_keys>, above, but works on all strings
277(including hash keys), so C<text_keys> has no further effect after
278enabling C<text_strings>.
279
280If C<$enabled> is true (or missing), then C<encode> will encode all perl
281strings as CBOR text strings/UTF-8 strings, upgrading them as needed.
282
283If C<$enable> is false (the default), then C<encode> will encode strings
284normally (but see C<text_keys>) - upgraded perl strings (strings
285internally encoded as UTF-8) as CBOR text strings, and downgraded perl
286strings as CBOR byte strings.
287
288This option does not affect C<decode> in any way.
289
290This option has similar advantages and disadvantages as C<text_keys>. In
291addition, this option effectively removes the ability to encode byte
292strings, which might break some C<FREEZE> and C<TO_CBOR> methods that rely
293on this, such as bignum encoding, so this option is mainly useful for very
294simple data.
295
247=item $cbor = $cbor->validate_utf8 ([$enable]) 296=item $cbor = $cbor->validate_utf8 ([$enable])
248 297
249=item $enabled = $cbor->get_validate_utf8 298=item $enabled = $cbor->get_validate_utf8
250 299
251If C<$enable> is true (or missing), then C<decode> will validate that 300If C<$enable> is true (or missing), then C<decode> will validate that
256The concept of "valid UTF-8" used is perl's concept, which is a superset 305The concept of "valid UTF-8" used is perl's concept, which is a superset
257of the official UTF-8. 306of the official UTF-8.
258 307
259If C<$enable> is false (the default), then C<decode> will blindly accept 308If C<$enable> is false (the default), then C<decode> will blindly accept
260UTF-8 data, marking them as valid UTF-8 in the resulting data structure 309UTF-8 data, marking them as valid UTF-8 in the resulting data structure
261regardless of whether thats true or not. 310regardless of whether that's true or not.
262 311
263Perl isn't too happy about corrupted UTF-8 in strings, but should 312Perl isn't too happy about corrupted UTF-8 in strings, but should
264generally not crash or do similarly evil things. Extensions might be not 313generally not crash or do similarly evil things. Extensions might be not
265so forgiving, so it's recommended to turn on this setting if you receive 314so forgiving, so it's recommended to turn on this setting if you receive
266untrusted CBOR. 315untrusted CBOR.
476 525
477=item hash references 526=item hash references
478 527
479Perl hash references become CBOR maps. As there is no inherent ordering in 528Perl hash references become CBOR maps. As there is no inherent ordering in
480hash keys (or CBOR maps), they will usually be encoded in a pseudo-random 529hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
481order. This order can be different each time a hahs is encoded. 530order. This order can be different each time a hash is encoded.
482 531
483Currently, tied hashes will use the indefinite-length format, while normal 532Currently, tied hashes will use the indefinite-length format, while normal
484hashes will use the fixed-length format. 533hashes will use the fixed-length format.
485 534
486=item array references 535=item array references
539 my $x = 3.1; # some variable containing a number 588 my $x = 3.1; # some variable containing a number
540 "$x"; # stringified 589 "$x"; # stringified
541 $x .= ""; # another, more awkward way to stringify 590 $x .= ""; # another, more awkward way to stringify
542 print $x; # perl does it for you, too, quite often 591 print $x; # perl does it for you, too, quite often
543 592
544You can force whether a string ie encoded as byte or text string by using 593You can force whether a string is encoded as byte or text string by using
545C<utf8::upgrade> and C<utf8::downgrade>): 594C<utf8::upgrade> and C<utf8::downgrade> (if C<text_strings> is disabled):
546 595
547 utf8::upgrade $x; # encode $x as text string 596 utf8::upgrade $x; # encode $x as text string
548 utf8::downgrade $x; # encode $x as byte string 597 utf8::downgrade $x; # encode $x as byte string
549 598
550Perl doesn't define what operations up- and downgrade strings, so if the 599Perl doesn't define what operations up- and downgrade strings, so if the
551difference between byte and text is important, you should up- or downgrade 600difference between byte and text is important, you should up- or downgrade
552your string as late as possible before encoding. 601your string as late as possible before encoding. You can also force the
602use of CBOR text strings by using C<text_keys> or C<text_strings>.
553 603
554You can force the type to be a CBOR number by numifying it: 604You can force the type to be a CBOR number by numifying it:
555 605
556 my $x = "3"; # some variable containing a string 606 my $x = "3"; # some variable containing a string
557 $x += 0; # numify it, ensuring it will be dumped as a number 607 $x += 0; # numify it, ensuring it will be dumped as a number
658 "$self" # encode url string 708 "$self" # encode url string
659 } 709 }
660 710
661 sub URI::THAW { 711 sub URI::THAW {
662 my ($class, $serialiser, $uri) = @_; 712 my ($class, $serialiser, $uri) = @_;
663
664 $class->new ($uri) 713 $class->new ($uri)
665 } 714 }
666 715
667Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For 716Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For
668example, a C<FREEZE> method that returns "type", "id" and "variant" values 717example, a C<FREEZE> method that returns "type", "id" and "variant" values
799additional tags (such as base64url). 848additional tags (such as base64url).
800 849
801=head2 ENFORCED TAGS 850=head2 ENFORCED TAGS
802 851
803These tags are always handled when decoding, and their handling cannot be 852These tags are always handled when decoding, and their handling cannot be
804overriden by the user. 853overridden by the user.
805 854
806=over 4 855=over 4
807 856
808=item 26 (perl-object, L<http://cbor.schmorp.de/perl-object>) 857=item 26 (perl-object, L<http://cbor.schmorp.de/perl-object>)
809 858
810These tags are automatically created (and decoded) for serialisable 859These tags are automatically created (and decoded) for serialisable
811objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object 860objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
812serialisation protocol). See L<OBJECT SERIALISATION> for details. 861serialisation protocol). See L<OBJECT SERIALISATION> for details.
813 862
814=item 28, 29 (shareable, sharedref, L <http://cbor.schmorp.de/value-sharing>) 863=item 28, 29 (shareable, sharedref, L<http://cbor.schmorp.de/value-sharing>)
815 864
816These tags are automatically decoded when encountered (and they do not 865These tags are automatically decoded when encountered (and they do not
817result in a cyclic data structure, see C<allow_cycles>), resulting in 866result in a cyclic data structure, see C<allow_cycles>), resulting in
818shared values in the decoded object. They are only encoded, however, when 867shared values in the decoded object. They are only encoded, however, when
819C<allow_sharing> is enabled. 868C<allow_sharing> is enabled.
829will be shared, others will not. While non-reference shared values can be 878will be shared, others will not. While non-reference shared values can be
830generated in Perl with some effort, they were considered too unimportant 879generated in Perl with some effort, they were considered too unimportant
831to be supported in the encoder. The decoder, however, will decode these 880to be supported in the encoder. The decoder, however, will decode these
832values as shared values. 881values as shared values.
833 882
834=item 256, 25 (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>) 883=item 256, 25 (stringref-namespace, stringref, L<http://cbor.schmorp.de/stringref>)
835 884
836These tags are automatically decoded when encountered. They are only 885These tags are automatically decoded when encountered. They are only
837encoded, however, when C<pack_strings> is enabled. 886encoded, however, when C<pack_strings> is enabled.
838 887
839=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>) 888=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
840 889
841This tag is automatically generated when a reference are encountered (with 890This tag is automatically generated when a reference are encountered (with
842the exception of hash and array refernces). It is converted to a reference 891the exception of hash and array references). It is converted to a reference
843when decoding. 892when decoding.
844 893
845=item 55799 (self-describe CBOR, RFC 7049) 894=item 55799 (self-describe CBOR, RFC 7049)
846 895
847This value is not generated on encoding (unless explicitly requested by 896This value is not generated on encoding (unless explicitly requested by
850=back 899=back
851 900
852=head2 NON-ENFORCED TAGS 901=head2 NON-ENFORCED TAGS
853 902
854These tags have default filters provided when decoding. Their handling can 903These tags have default filters provided when decoding. Their handling can
855be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by 904be overridden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
856providing a custom C<filter> callback when decoding. 905providing a custom C<filter> callback when decoding.
857 906
858When they result in decoding into a specific Perl class, the module 907When they result in decoding into a specific Perl class, the module
859usually provides a corresponding C<TO_CBOR> method as well. 908usually provides a corresponding C<TO_CBOR> method as well.
860 909
902C<URI::TO_CBOR> method again results in a CBOR URI value. 951C<URI::TO_CBOR> method again results in a CBOR URI value.
903 952
904=back 953=back
905 954
906=cut 955=cut
907
908our %FILTER = (
909 # 0 # rfc4287 datetime, utf-8
910 # 1 # unix timestamp, any
911
912 2 => sub { # pos bigint
913 require Math::BigInt;
914 Math::BigInt->new ("0x" . unpack "H*", pop)
915 },
916
917 3 => sub { # neg bigint
918 require Math::BigInt;
919 -Math::BigInt->new ("0x" . unpack "H*", pop)
920 },
921
922 4 => sub { # decimal fraction, array
923 require Math::BigFloat;
924 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
925 },
926
927 5 => sub { # bigfloat, array
928 require Math::BigFloat;
929 scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
930 },
931
932 21 => sub { pop }, # expected conversion to base64url encoding
933 22 => sub { pop }, # expected conversion to base64 encoding
934 23 => sub { pop }, # expected conversion to base16 encoding
935
936 # 24 # embedded cbor, byte string
937
938 32 => sub {
939 require URI;
940 URI->new (pop)
941 },
942
943 # 33 # base64url rfc4648, utf-8
944 # 34 # base64 rfc46484, utf-8
945 # 35 # regex pcre/ecma262, utf-8
946 # 36 # mime message rfc2045, utf-8
947);
948
949 956
950=head1 CBOR and JSON 957=head1 CBOR and JSON
951 958
952CBOR is supposed to implement a superset of the JSON data model, and is, 959CBOR is supposed to implement a superset of the JSON data model, and is,
953with some coercion, able to represent all JSON texts (something that other 960with some coercion, able to represent all JSON texts (something that other
1017 1024
1018 1025
1019=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT 1026=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT
1020 1027
1021On perls that were built without 64 bit integer support (these are rare 1028On perls that were built without 64 bit integer support (these are rare
1022nowadays, even on 32 bit architectures), support for any kind of 64 bit 1029nowadays, even on 32 bit architectures, as all major Perl distributions
1030are built with 64 bit integer support), support for any kind of 64 bit
1023integer in CBOR is very limited - most likely, these 64 bit values will 1031integer in CBOR is very limited - most likely, these 64 bit values will
1024be truncated, corrupted, or otherwise not decoded correctly. This also 1032be truncated, corrupted, or otherwise not decoded correctly. This also
1025includes string, array and map sizes that are stored as 64 bit integers. 1033includes string, array and map sizes that are stored as 64 bit integers.
1026 1034
1027 1035
1052 # Time::Piece::Strptime uses the "incredibly flexible date parsing routine" 1060 # Time::Piece::Strptime uses the "incredibly flexible date parsing routine"
1053 # from FreeBSD, which can't parse ISO 8601, RFC3339, RFC4287 or much of anything 1061 # from FreeBSD, which can't parse ISO 8601, RFC3339, RFC4287 or much of anything
1054 # else either. Whats incredibe over standard strptime totally escapes me. 1062 # else either. Whats incredibe over standard strptime totally escapes me.
1055 # doesn't do fractional times, either. sigh. 1063 # doesn't do fractional times, either. sigh.
1056 # In fact, it's all a lie, it uses whatever strptime it wants, and of course, 1064 # In fact, it's all a lie, it uses whatever strptime it wants, and of course,
1057 # they are all incomptible. The openbsd one simply ignores %z (but according to the 1065 # they are all incompatible. The openbsd one simply ignores %z (but according to the
1058 # docs, it would be much more incredibly flexible indeed. If it worked, that is.). 1066 # docs, it would be much more incredibly flexible indeed. If it worked, that is.).
1059 scalar eval { 1067 scalar eval {
1060 my $s = $_[1]; 1068 my $s = $_[1];
1061 1069
1062 $s =~ s/Z$/+00:00/; 1070 $s =~ s/Z$/+00:00/;
1090 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0]) 1098 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
1091 }, 1099 },
1092 1100
1093 5 => sub { # bigfloat, array 1101 5 => sub { # bigfloat, array
1094 require Math::BigFloat; 1102 require Math::BigFloat;
1095 scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2) 1103 scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
1096 }, 1104 },
1097 1105
1098 21 => sub { pop }, # expected conversion to base64url encoding 1106 21 => sub { pop }, # expected conversion to base64url encoding
1099 22 => sub { pop }, # expected conversion to base64 encoding 1107 22 => sub { pop }, # expected conversion to base64 encoding
1100 23 => sub { pop }, # expected conversion to base16 encoding 1108 23 => sub { pop }, # expected conversion to base16 encoding

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines