[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.66 by root, Tue Jun 27 02:03:24 2017 UTC vs.
Revision 1.80 by root, Fri Dec 11 06:05:33 2020 UTC

…		…
38	with the added ability of supporting serialisation of Perl objects. (JSON	38	with the added ability of supporting serialisation of Perl objects. (JSON
39	often compresses better than CBOR though, so if you plan to compress the	39	often compresses better than CBOR though, so if you plan to compress the
40	data later and speed is less important you might want to compare both	40	data later and speed is less important you might want to compare both
41	formats first).	41	formats first).
42		42
		43	The primary goal of this module is to be I<correct> and the secondary goal
		44	is to be I<fast>. To reach the latter goal it was written in C.
		45
43	To give you a general idea about speed, with texts in the megabyte range,	46	To give you a general idea about speed, with texts in the megabyte range,
44	C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or	47	C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or
45	L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the	48	L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the
46	data, the worse L<Storable> performs in comparison.	49	data, the worse L<Storable> performs in comparison.
47		50
…		…
52	In addition to the core CBOR data format, this module implements a	55	In addition to the core CBOR data format, this module implements a
53	number of extensions, to support cyclic and shared data structures	56	number of extensions, to support cyclic and shared data structures
54	(see C<allow_sharing> and C<allow_cycles>), string deduplication (see	57	(see C<allow_sharing> and C<allow_cycles>), string deduplication (see
55	C<pack_strings>) and scalar references (always enabled).	58	C<pack_strings>) and scalar references (always enabled).
56		59
57	The primary goal of this module is to be I<correct> and the secondary goal
58	is to be I<fast>. To reach the latter goal it was written in C.
59
60	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	60	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
61	vice versa.	61	vice versa.
62		62
63	=cut	63	=cut
64		64
65	package CBOR::XS;	65	package CBOR::XS;
66		66
67	use common::sense;	67	use common::sense;
68		68
69	our $VERSION = 1.7;	69	our $VERSION = 1.83;
70	our @ISA = qw(Exporter);	70	our @ISA = qw(Exporter);
71		71
72	our @EXPORT = qw(encode_cbor decode_cbor);	72	our @EXPORT = qw(encode_cbor decode_cbor);
73		73
74	use Exporter;	74	use Exporter;
…		…
215	(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the	215	(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the
216	resulting data structure might be unusable.	216	resulting data structure might be unusable.
217		217
218	Detecting shared values incurs a runtime overhead when values are encoded	218	Detecting shared values incurs a runtime overhead when values are encoded
219	that have a reference counter large than one, and might unnecessarily	219	that have a reference counter large than one, and might unnecessarily
220	increase the encoded size, as potentially shared values are encode as	220	increase the encoded size, as potentially shared values are encoded as
221	shareable whether or not they are actually shared.	221	shareable whether or not they are actually shared.
222		222
223	At the moment, only targets of references can be shared (e.g. scalars,	223	At the moment, only targets of references can be shared (e.g. scalars,
224	arrays or hashes pointed to by a reference). Weirder constructs, such as	224	arrays or hashes pointed to by a reference). Weirder constructs, such as
225	an array with multiple "copies" of the I<same> string, which are hard but	225	an array with multiple "copies" of the I<same> string, which are hard but
…		…
330	strings as CBOR byte strings.	330	strings as CBOR byte strings.
331		331
332	This option does not affect C<decode> in any way.	332	This option does not affect C<decode> in any way.
333		333
334	This option has similar advantages and disadvantages as C<text_keys>. In	334	This option has similar advantages and disadvantages as C<text_keys>. In
335	addition, this option effectively removes the ability to encode byte	335	addition, this option effectively removes the ability to automatically
336	strings, which might break some C<FREEZE> and C<TO_CBOR> methods that rely	336	encode byte strings, which might break some C<FREEZE> and C<TO_CBOR>
337	on this, such as bignum encoding, so this option is mainly useful for very	337	methods that rely on this.
338	simple data.	338
		339	A workaround is to use explicit type casts, which are unaffected by this option.
339		340
340	=item $cbor = $cbor->validate_utf8 ([$enable])	341	=item $cbor = $cbor->validate_utf8 ([$enable])
341		342
342	=item $enabled = $cbor->get_validate_utf8	343	=item $enabled = $cbor->get_validate_utf8
343		344
…		…
453	when there is trailing garbage after the CBOR string, it will silently	454	when there is trailing garbage after the CBOR string, it will silently
454	stop parsing there and return the number of characters consumed so far.	455	stop parsing there and return the number of characters consumed so far.
455		456
456	This is useful if your CBOR texts are not delimited by an outer protocol	457	This is useful if your CBOR texts are not delimited by an outer protocol
457	and you need to know where the first CBOR string ends amd the next one	458	and you need to know where the first CBOR string ends amd the next one
458	starts.	459	starts - CBOR strings are self-delimited, so it is possible to concatenate
		460	CBOR strings without any delimiters or size fields and recover their data.
459		461
460	CBOR::XS->new->decode_prefix ("......")	462	CBOR::XS->new->decode_prefix ("......")
461	=> ("...", 3)	463	=> ("...", 3)
462		464
463	=back	465	=back
…		…
667	"$x"; # stringified	669	"$x"; # stringified
668	$x .= ""; # another, more awkward way to stringify	670	$x .= ""; # another, more awkward way to stringify
669	print $x; # perl does it for you, too, quite often	671	print $x; # perl does it for you, too, quite often
670		672
671	You can force whether a string is encoded as byte or text string by using	673	You can force whether a string is encoded as byte or text string by using
672	C<utf8::upgrade> and C<utf8::downgrade> (if C<text_strings> is disabled):	674	C<utf8::upgrade> and C<utf8::downgrade> (if C<text_strings> is disabled).
673		675
674	utf8::upgrade $x; # encode $x as text string	676	utf8::upgrade $x; # encode $x as text string
675	utf8::downgrade $x; # encode $x as byte string	677	utf8::downgrade $x; # encode $x as byte string
		678
		679	More options are available, see L<TYPE CASTS>, below, and the C<text_keys>
		680	and C<text_strings> options.
676		681
677	Perl doesn't define what operations up- and downgrade strings, so if the	682	Perl doesn't define what operations up- and downgrade strings, so if the
678	difference between byte and text is important, you should up- or downgrade	683	difference between byte and text is important, you should up- or downgrade
679	your string as late as possible before encoding. You can also force the	684	your string as late as possible before encoding. You can also force the
680	use of CBOR text strings by using C<text_keys> or C<text_strings>.	685	use of CBOR text strings by using C<text_keys> or C<text_strings>.
…		…
695	format will be used. Perls that use formats other than IEEE double to	700	format will be used. Perls that use formats other than IEEE double to
696	represent numerical values are supported, but might suffer loss of	701	represent numerical values are supported, but might suffer loss of
697	precision.	702	precision.
698		703
699	=back	704	=back
		705
		706	=head2 TYPE CASTS
		707
		708	B<EXPERIMENTAL>: As an experimental extension, C<CBOR::XS> allows you to
		709	force specific cbor types to be used when encoding. That allows you to
		710	encode types not normally accessible (e.g. half floats) as well as force
		711	string types even when C<text_strings> is in effect.
		712
		713	Type forcing is done by calling a special "cast" function which keeps a
		714	copy of the value and returns a new value that can be handed over to any
		715	CBOR encoder function.
		716
		717	The following casts are currently available (all of which are unary
		718	operators, that is, have a prototype of C<$>):
		719
		720	=over
		721
		722	=item CBOR::XS::as_int $value
		723
		724	Forces the value to be encoded as some form of (basic, not bignum) integer
		725	type.
		726
		727	=item CBOR::XS::as_text $value
		728
		729	Forces the value to be encoded as (UTF-8) text values.
		730
		731	=item CBOR::XS::as_bytes $value
		732
		733	Forces the value to be encoded as a (binary) string value.
		734
		735	Example: encode a perl string as binary even though C<text_strings> is in
		736	effect.
		737
		738	CBOR::XS->new->text_strings->encode ([4, "text", CBOR::XS::bytes "bytevalue"]);
		739
		740	=item CBOR::XS::as_bool $value
		741
		742	Converts a Perl boolean (which can be any kind of scalar) into a CBOR
		743	boolean. Strictly the same, but shorter to write, than:
		744
		745	$value ? Types::Serialiser::true : Types::Serialiser::false
		746
		747	=item CBOR::XS::as_float16 $value
		748
		749	Forces half-float (IEEE 754 binary16) encoding of the given value.
		750
		751	=item CBOR::XS::as_float32 $value
		752
		753	Forces single-float (IEEE 754 binary32) encoding of the given value.
		754
		755	=item CBOR::XS::as_float64 $value
		756
		757	Forces double-float (IEEE 754 binary64) encoding of the given value.
		758
		759	=item CBOR::XS::as_cbor $cbor_text
		760
		761	Not a type cast per-se, this type cast forces the argument to eb encoded
		762	as-is. This can be used to embed pre-encoded CBOR data.
		763
		764	Note that no checking on the validity of the C<$cbor_text> is done - it's
		765	the callers responsibility to correctly encode values.
		766
		767	=item CBOR::XS::as_map [key => value...]
		768
		769	Treat the array reference as key value pairs and output a CBOR map. This
		770	allows you to generate CBOR maps with arbitrary key types (or, if you
		771	don't care about semantics, duplicate keys or pairs in a custom order),
		772	which is otherwise hard to do with Perl.
		773
		774	The single argument must be an array reference with an even number of
		775	elements.
		776
		777	Note that only the reference to the array is copied, the array itself is
		778	not. Modifications done to the array before calling an encoding fuinction
		779	will be reflected in the encoded output.
		780
		781	Example: encode a CBOR map with a string and an integer as keys.
		782
		783	encode_cbor CBOR::XS::as_map [string => "value", 5 => "value"]
		784
		785	=back
		786
		787	=cut
		788
		789	sub CBOR::XS::as_cbor ($) { bless [$_[0], 0, undef], CBOR::XS::Tagged:: }
		790	sub CBOR::XS::as_int ($) { bless [$_[0], 1, undef], CBOR::XS::Tagged:: }
		791	sub CBOR::XS::as_bytes ($) { bless [$_[0], 2, undef], CBOR::XS::Tagged:: }
		792	sub CBOR::XS::as_text ($) { bless [$_[0], 3, undef], CBOR::XS::Tagged:: }
		793	sub CBOR::XS::as_float16 ($) { bless [$_[0], 4, undef], CBOR::XS::Tagged:: }
		794	sub CBOR::XS::as_float32 ($) { bless [$_[0], 5, undef], CBOR::XS::Tagged:: }
		795	sub CBOR::XS::as_float64 ($) { bless [$_[0], 6, undef], CBOR::XS::Tagged:: }
		796
		797	sub CBOR::XS::as_bool ($) { $_[0] ? $Types::Serialiser::true : $Types::Serialiser::false }
		798
		799	sub CBOR::XS::as_map ($) {
		800	ARRAY:: eq ref $_[0]
		801	and $#{ $_[0] } & 1
		802	or do { require Carp; Carp::croak ("CBOR::XS::as_map only acepts array references with an even number of elements, caught") };
		803
		804	bless [$_[0], 7, undef], CBOR::XS::Tagged::
		805	}
700		806
701	=head2 OBJECT SERIALISATION	807	=head2 OBJECT SERIALISATION
702		808
703	This module implements both a CBOR-specific and the generic	809	This module implements both a CBOR-specific and the generic
704	L<Types::Serialier> object serialisation protocol. The following	810	L<Types::Serialier> object serialisation protocol. The following
…		…
1057		1163
1058		1164
1059	=head1 SECURITY CONSIDERATIONS	1165	=head1 SECURITY CONSIDERATIONS
1060		1166
1061	Tl;dr... if you want to decode or encode CBOR from untrusted sources, you	1167	Tl;dr... if you want to decode or encode CBOR from untrusted sources, you
1062	should start with a coder object created via C<new_safe>:	1168	should start with a coder object created via C<new_safe> (which implements
		1169	the mitigations explained below):
1063		1170
1064	my $coder = CBOR::XS->new_safe;	1171	my $coder = CBOR::XS->new_safe;
1065		1172
1066	my $data = $coder->decode ($cbor_text);	1173	my $data = $coder->decode ($cbor_text);
1067	my $cbor = $coder->encode ($data);	1174	my $cbor = $coder->encode ($data);
…		…
1089	even if all your C<THAW> methods are secure, encoding data structures from	1196	even if all your C<THAW> methods are secure, encoding data structures from
1090	untrusted sources can invoke those and trigger bugs in those.	1197	untrusted sources can invoke those and trigger bugs in those.
1091		1198
1092	So, if you are not sure about the security of all the modules you	1199	So, if you are not sure about the security of all the modules you
1093	have loaded (you shouldn't), you should disable this part using	1200	have loaded (you shouldn't), you should disable this part using
1094	C<forbid_objects>.	1201	C<forbid_objects> or using C<new_safe>.
1095		1202
1096	=item CBOR can be extended with tags that call library code	1203	=item CBOR can be extended with tags that call library code
1097		1204
1098	CBOR can be extended with tags, and C<CBOR::XS> has a registry of	1205	CBOR can be extended with tags, and C<CBOR::XS> has a registry of
1099	conversion functions for many existing tags that can be extended via	1206	conversion functions for many existing tags that can be extended via
1100	third-party modules (see the C<filter> method).	1207	third-party modules (see the C<filter> method).
1101		1208
1102	If you don't trust these, you should configure the "safe" filter function,	1209	If you don't trust these, you should configure the "safe" filter function,
1103	C<CBOR::XS::safe_filter>, which by default only includes conversion	1210	C<CBOR::XS::safe_filter> (C<new_safe> does this), which by default only
1104	functions that are considered "safe" by the author (but again, they can be	1211	includes conversion functions that are considered "safe" by the author
1105	extended by third party modules).	1212	(but again, they can be extended by third party modules).
1106		1213
1107	Depending on your level of paranoia, you can use the "safe" filter:	1214	Depending on your level of paranoia, you can use the "safe" filter:
1108		1215
1109	$cbor->filter (\&CBOR::XS::safe_filter);	1216	$cbor->filter (\&CBOR::XS::safe_filter);
1110		1217
…		…
1125	the size of CBOR data you accept, or make sure then when your resources	1232	the size of CBOR data you accept, or make sure then when your resources
1126	run out, that's just fine (e.g. by using a separate process that can	1233	run out, that's just fine (e.g. by using a separate process that can
1127	crash safely). The size of a CBOR string in octets is usually a good	1234	crash safely). The size of a CBOR string in octets is usually a good
1128	indication of the size of the resources required to decode it into a Perl	1235	indication of the size of the resources required to decode it into a Perl
1129	structure. While CBOR::XS can check the size of the CBOR text (using	1236	structure. While CBOR::XS can check the size of the CBOR text (using
1130	C<max_size>), it might be too late when you already have it in memory, so	1237	C<max_size> - done by C<new_safe>), it might be too late when you already
1131	you might want to check the size before you accept the string.	1238	have it in memory, so you might want to check the size before you accept
		1239	the string.
1132		1240
1133	As for encoding, it is possible to construct data structures that are	1241	As for encoding, it is possible to construct data structures that are
1134	relatively small but result in large CBOR texts (for example by having an	1242	relatively small but result in large CBOR texts (for example by having an
1135	array full of references to the same big data structure, which will all be	1243	array full of references to the same big data structure, which will all be
1136	deep-cloned during encoding by default). This is rarely an actual issue	1244	deep-cloned during encoding by default). This is rarely an actual issue
…		…
1149	method.	1257	method.
1150		1258
1151	=item Resource-starving attacks: CPU en-/decoding complexity	1259	=item Resource-starving attacks: CPU en-/decoding complexity
1152		1260
1153	CBOR::XS will use the L<Math::BigInt>, L<Math::BigFloat> and	1261	CBOR::XS will use the L<Math::BigInt>, L<Math::BigFloat> and
1154	L<Math::BigRat> libraries to represent encode/decode bignums. These can	1262	L<Math::BigRat> libraries to represent encode/decode bignums. These can be
1155	be very slow (as in, centuries of CPU time) and can even crash your	1263	very slow (as in, centuries of CPU time) and can even crash your program
1156	program (and are generally not very trustworthy). See the next section for	1264	(and are generally not very trustworthy). See the next section on bignum
1157	details.	1265	security for details.
1158		1266
1159	=item Data breaches: leaking information in error messages	1267	=item Data breaches: leaking information in error messages
1160		1268
1161	CBOR::XS might leak contents of your Perl data structures in its error	1269	CBOR::XS might leak contents of your Perl data structures in its error
1162	messages, so when you serialise sensitive information you might want to	1270	messages, so when you serialise sensitive information you might want to
…		…
1225	=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT	1333	=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT
1226		1334
1227	On perls that were built without 64 bit integer support (these are rare	1335	On perls that were built without 64 bit integer support (these are rare
1228	nowadays, even on 32 bit architectures, as all major Perl distributions	1336	nowadays, even on 32 bit architectures, as all major Perl distributions
1229	are built with 64 bit integer support), support for any kind of 64 bit	1337	are built with 64 bit integer support), support for any kind of 64 bit
1230	integer in CBOR is very limited - most likely, these 64 bit values will	1338	value in CBOR is very limited - most likely, these 64 bit values will
1231	be truncated, corrupted, or otherwise not decoded correctly. This also	1339	be truncated, corrupted, or otherwise not decoded correctly. This also
1232	includes string, array and map sizes that are stored as 64 bit integers.	1340	includes string, float, array and map sizes that are stored as 64 bit
		1341	integers.
1233		1342
1234		1343
1235	=head1 THREADS	1344	=head1 THREADS
1236		1345
1237	This module is I<not> guaranteed to be thread safe and there are no	1346	This module is I<not> guaranteed to be thread safe and there are no

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.66 by root, Tue Jun 27 02:03:24 2017 UTC vs. Revision 1.80 by root, Fri Dec 11 06:05:33 2020 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.66 by root, Tue Jun 27 02:03:24 2017 UTC vs.
Revision 1.80 by root, Fri Dec 11 06:05:33 2020 UTC