[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.68 by root, Wed Jul 17 09:37:16 2019 UTC vs.
Revision 1.89 by root, Fri Sep 8 20:03:06 2023 UTC

…		…
64		64
65	package CBOR::XS;	65	package CBOR::XS;
66		66
67	use common::sense;	67	use common::sense;
68		68
69	our $VERSION = 1.71;	69	our $VERSION = 1.87;
70	our @ISA = qw(Exporter);	70	our @ISA = qw(Exporter);
71		71
72	our @EXPORT = qw(encode_cbor decode_cbor);	72	our @EXPORT = qw(encode_cbor decode_cbor);
73		73
74	use Exporter;	74	use Exporter;
…		…
121	but configures the coder object to be safe to use with untrusted	121	but configures the coder object to be safe to use with untrusted
122	data. Currently, this is equivalent to:	122	data. Currently, this is equivalent to:
123		123
124	my $cbor = CBOR::XS	124	my $cbor = CBOR::XS
125	->new	125	->new
		126	->validate_utf8
126	->forbid_objects	127	->forbid_objects
127	->filter (\&CBOR::XS::safe_filter)	128	->filter (\&CBOR::XS::safe_filter)
128	->max_size (1e8);	129	->max_size (1e8);
129		130
130	But is more future proof (it is better to crash because of a change than	131	But is more future proof (it is better to crash because of a change than
…		…
133	=cut	134	=cut
134		135
135	sub new_safe {	136	sub new_safe {
136	CBOR::XS	137	CBOR::XS
137	->new	138	->new
		139	->validate_utf8
138	->forbid_objects	140	->forbid_objects
139	->filter (\&CBOR::XS::safe_filter)	141	->filter (\&CBOR::XS::safe_filter)
140	->max_size (1e8)	142	->max_size (1e8)
141	}	143	}
142		144
…		…
214	communication partner supports the value sharing extensions to CBOR	216	communication partner supports the value sharing extensions to CBOR
215	(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the	217	(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the
216	resulting data structure might be unusable.	218	resulting data structure might be unusable.
217		219
218	Detecting shared values incurs a runtime overhead when values are encoded	220	Detecting shared values incurs a runtime overhead when values are encoded
219	that have a reference counter large than one, and might unnecessarily	221	that have a reference counter larger than one, and might unnecessarily
220	increase the encoded size, as potentially shared values are encode as	222	increase the encoded size, as potentially shared values are encoded as
221	shareable whether or not they are actually shared.	223	shareable whether or not they are actually shared.
222		224
223	At the moment, only targets of references can be shared (e.g. scalars,	225	At the moment, only targets of references can be shared (e.g. scalars,
224	arrays or hashes pointed to by a reference). Weirder constructs, such as	226	arrays or hashes pointed to by a reference). Weirder constructs, such as
225	an array with multiple "copies" of the I<same> string, which are hard but	227	an array with multiple "copies" of the I<same> string, which are hard but
…		…
243	isn't prepared for this will not leak memory.	245	isn't prepared for this will not leak memory.
244		246
245	If C<$enable> is false (the default), then C<decode> will throw an error	247	If C<$enable> is false (the default), then C<decode> will throw an error
246	when it encounters a self-referential/cyclic data structure.	248	when it encounters a self-referential/cyclic data structure.
247		249
248	FUTURE DIRECTION: the motivation behind this option is to avoid I<real>	250	This option does not affect C<encode> in any way - shared values and
249	cycles - future versions of this module might chose to decode cyclic data	251	references will always be encoded properly if present.
250	structures using weak references when this option is off, instead of	252
251	throwing an error.	253	=item $cbor = $cbor->allow_weak_cycles ([$enable])
		254
		255	=item $enabled = $cbor->get_allow_weak_cycles
		256
		257	This works like C<allow_cycles> in that it allows the resulting data
		258	structures to contain cycles, but unlike C<allow_cycles>, those cyclic
		259	rreferences will be weak. That means that code that recurrsively walks
		260	the data structure must be prepared with cycles, but at least not special
		261	precautions must be implemented to free these data structures.
		262
		263	Only those references leading to actual cycles will be weakened - other
		264	references, e.g. when the same hash or arrray is referenced multiple times
		265	in an arrray, will be normal references.
252		266
253	This option does not affect C<encode> in any way - shared values and	267	This option does not affect C<encode> in any way - shared values and
254	references will always be encoded properly if present.	268	references will always be encoded properly if present.
255		269
256	=item $cbor = $cbor->forbid_objects ([$enable])	270	=item $cbor = $cbor->forbid_objects ([$enable])
…		…
330	strings as CBOR byte strings.	344	strings as CBOR byte strings.
331		345
332	This option does not affect C<decode> in any way.	346	This option does not affect C<decode> in any way.
333		347
334	This option has similar advantages and disadvantages as C<text_keys>. In	348	This option has similar advantages and disadvantages as C<text_keys>. In
335	addition, this option effectively removes the ability to encode byte	349	addition, this option effectively removes the ability to automatically
336	strings, which might break some C<FREEZE> and C<TO_CBOR> methods that rely	350	encode byte strings, which might break some C<FREEZE> and C<TO_CBOR>
337	on this, such as bignum encoding, so this option is mainly useful for very	351	methods that rely on this.
338	simple data.	352
		353	A workaround is to use explicit type casts, which are unaffected by this option.
339		354
340	=item $cbor = $cbor->validate_utf8 ([$enable])	355	=item $cbor = $cbor->validate_utf8 ([$enable])
341		356
342	=item $enabled = $cbor->get_validate_utf8	357	=item $enabled = $cbor->get_validate_utf8
343		358
…		…
453	when there is trailing garbage after the CBOR string, it will silently	468	when there is trailing garbage after the CBOR string, it will silently
454	stop parsing there and return the number of characters consumed so far.	469	stop parsing there and return the number of characters consumed so far.
455		470
456	This is useful if your CBOR texts are not delimited by an outer protocol	471	This is useful if your CBOR texts are not delimited by an outer protocol
457	and you need to know where the first CBOR string ends amd the next one	472	and you need to know where the first CBOR string ends amd the next one
458	starts.	473	starts - CBOR strings are self-delimited, so it is possible to concatenate
		474	CBOR strings without any delimiters or size fields and recover their data.
459		475
460	CBOR::XS->new->decode_prefix ("......")	476	CBOR::XS->new->decode_prefix ("......")
461	=> ("...", 3)	477	=> ("...", 3)
462		478
463	=back	479	=back
…		…
469	Perl data structure in memory at one time, it does allow you to parse a	485	Perl data structure in memory at one time, it does allow you to parse a
470	CBOR stream incrementally, using a similar to using "decode_prefix" to see	486	CBOR stream incrementally, using a similar to using "decode_prefix" to see
471	if a full CBOR object is available, but is much more efficient.	487	if a full CBOR object is available, but is much more efficient.
472		488
473	It basically works by parsing as much of a CBOR string as possible - if	489	It basically works by parsing as much of a CBOR string as possible - if
474	the CBOR data is not complete yet, the pasrer will remember where it was,	490	the CBOR data is not complete yet, the parser will remember where it was,
475	to be able to restart when more data has been accumulated. Once enough	491	to be able to restart when more data has been accumulated. Once enough
476	data is available to either decode a complete CBOR value or raise an	492	data is available to either decode a complete CBOR value or raise an
477	error, a real decode will be attempted.	493	error, a real decode will be attempted.
478		494
479	A typical use case would be a network protocol that consists of sending	495	A typical use case would be a network protocol that consists of sending
…		…
631	create such objects.	647	create such objects.
632		648
633	=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error	649	=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
634		650
635	These special values become CBOR true, CBOR false and CBOR undefined	651	These special values become CBOR true, CBOR false and CBOR undefined
636	values, respectively. You can also use C<\1>, C<\0> and C<\undef> directly	652	values, respectively.
637	if you want.
638		653
639	=item other blessed objects	654	=item other blessed objects
640		655
641	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See	656	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
642	L<TAG HANDLING AND EXTENSIONS> for specific classes handled by this	657	L<TAG HANDLING AND EXTENSIONS> for specific classes handled by this
…		…
667	"$x"; # stringified	682	"$x"; # stringified
668	$x .= ""; # another, more awkward way to stringify	683	$x .= ""; # another, more awkward way to stringify
669	print $x; # perl does it for you, too, quite often	684	print $x; # perl does it for you, too, quite often
670		685
671	You can force whether a string is encoded as byte or text string by using	686	You can force whether a string is encoded as byte or text string by using
672	C<utf8::upgrade> and C<utf8::downgrade> (if C<text_strings> is disabled):	687	C<utf8::upgrade> and C<utf8::downgrade> (if C<text_strings> is disabled).
673		688
674	utf8::upgrade $x; # encode $x as text string	689	utf8::upgrade $x; # encode $x as text string
675	utf8::downgrade $x; # encode $x as byte string	690	utf8::downgrade $x; # encode $x as byte string
		691
		692	More options are available, see L<TYPE CASTS>, below, and the C<text_keys>
		693	and C<text_strings> options.
676		694
677	Perl doesn't define what operations up- and downgrade strings, so if the	695	Perl doesn't define what operations up- and downgrade strings, so if the
678	difference between byte and text is important, you should up- or downgrade	696	difference between byte and text is important, you should up- or downgrade
679	your string as late as possible before encoding. You can also force the	697	your string as late as possible before encoding. You can also force the
680	use of CBOR text strings by using C<text_keys> or C<text_strings>.	698	use of CBOR text strings by using C<text_keys> or C<text_strings>.
…		…
695	format will be used. Perls that use formats other than IEEE double to	713	format will be used. Perls that use formats other than IEEE double to
696	represent numerical values are supported, but might suffer loss of	714	represent numerical values are supported, but might suffer loss of
697	precision.	715	precision.
698		716
699	=back	717	=back
		718
		719	=head2 TYPE CASTS
		720
		721	B<EXPERIMENTAL>: As an experimental extension, C<CBOR::XS> allows you to
		722	force specific CBOR types to be used when encoding. That allows you to
		723	encode types not normally accessible (e.g. half floats) as well as force
		724	string types even when C<text_strings> is in effect.
		725
		726	Type forcing is done by calling a special "cast" function which keeps a
		727	copy of the value and returns a new value that can be handed over to any
		728	CBOR encoder function.
		729
		730	The following casts are currently available (all of which are unary
		731	operators, that is, have a prototype of C<$>):
		732
		733	=over
		734
		735	=item CBOR::XS::as_int $value
		736
		737	Forces the value to be encoded as some form of (basic, not bignum) integer
		738	type.
		739
		740	=item CBOR::XS::as_text $value
		741
		742	Forces the value to be encoded as (UTF-8) text values.
		743
		744	=item CBOR::XS::as_bytes $value
		745
		746	Forces the value to be encoded as a (binary) string value.
		747
		748	Example: encode a perl string as binary even though C<text_strings> is in
		749	effect.
		750
		751	CBOR::XS->new->text_strings->encode ([4, "text", CBOR::XS::bytes "bytevalue"]);
		752
		753	=item CBOR::XS::as_bool $value
		754
		755	Converts a Perl boolean (which can be any kind of scalar) into a CBOR
		756	boolean. Strictly the same, but shorter to write, than:
		757
		758	$value ? Types::Serialiser::true : Types::Serialiser::false
		759
		760	=item CBOR::XS::as_float16 $value
		761
		762	Forces half-float (IEEE 754 binary16) encoding of the given value.
		763
		764	=item CBOR::XS::as_float32 $value
		765
		766	Forces single-float (IEEE 754 binary32) encoding of the given value.
		767
		768	=item CBOR::XS::as_float64 $value
		769
		770	Forces double-float (IEEE 754 binary64) encoding of the given value.
		771
		772	=item CBOR::XS::as_cbor $cbor_text
		773
		774	Not a type cast per-se, this type cast forces the argument to be encoded
		775	as-is. This can be used to embed pre-encoded CBOR data.
		776
		777	Note that no checking on the validity of the C<$cbor_text> is done - it's
		778	the callers responsibility to correctly encode values.
		779
		780	=item CBOR::XS::as_map [key => value...]
		781
		782	Treat the array reference as key value pairs and output a CBOR map. This
		783	allows you to generate CBOR maps with arbitrary key types (or, if you
		784	don't care about semantics, duplicate keys or pairs in a custom order),
		785	which is otherwise hard to do with Perl.
		786
		787	The single argument must be an array reference with an even number of
		788	elements.
		789
		790	Note that only the reference to the array is copied, the array itself is
		791	not. Modifications done to the array before calling an encoding function
		792	will be reflected in the encoded output.
		793
		794	Example: encode a CBOR map with a string and an integer as keys.
		795
		796	encode_cbor CBOR::XS::as_map [string => "value", 5 => "value"]
		797
		798	=back
		799
		800	=cut
		801
		802	sub CBOR::XS::as_cbor ($) { bless [$_[0], 0, undef], CBOR::XS::Tagged:: }
		803	sub CBOR::XS::as_int ($) { bless [$_[0], 1, undef], CBOR::XS::Tagged:: }
		804	sub CBOR::XS::as_bytes ($) { bless [$_[0], 2, undef], CBOR::XS::Tagged:: }
		805	sub CBOR::XS::as_text ($) { bless [$_[0], 3, undef], CBOR::XS::Tagged:: }
		806	sub CBOR::XS::as_float16 ($) { bless [$_[0], 4, undef], CBOR::XS::Tagged:: }
		807	sub CBOR::XS::as_float32 ($) { bless [$_[0], 5, undef], CBOR::XS::Tagged:: }
		808	sub CBOR::XS::as_float64 ($) { bless [$_[0], 6, undef], CBOR::XS::Tagged:: }
		809
		810	sub CBOR::XS::as_bool ($) { $_[0] ? $Types::Serialiser::true : $Types::Serialiser::false }
		811
		812	sub CBOR::XS::as_map ($) {
		813	ARRAY:: eq ref $_[0]
		814	and $#{ $_[0] } & 1
		815	or do { require Carp; Carp::croak ("CBOR::XS::as_map only acepts array references with an even number of elements, caught") };
		816
		817	bless [$_[0], 7, undef], CBOR::XS::Tagged::
		818	}
700		819
701	=head2 OBJECT SERIALISATION	820	=head2 OBJECT SERIALISATION
702		821
703	This module implements both a CBOR-specific and the generic	822	This module implements both a CBOR-specific and the generic
704	L<Types::Serialier> object serialisation protocol. The following	823	L<Types::Serialier> object serialisation protocol. The following
…		…
1057		1176
1058		1177
1059	=head1 SECURITY CONSIDERATIONS	1178	=head1 SECURITY CONSIDERATIONS
1060		1179
1061	Tl;dr... if you want to decode or encode CBOR from untrusted sources, you	1180	Tl;dr... if you want to decode or encode CBOR from untrusted sources, you
1062	should start with a coder object created via C<new_safe>:	1181	should start with a coder object created via C<new_safe> (which implements
		1182	the mitigations explained below):
1063		1183
1064	my $coder = CBOR::XS->new_safe;	1184	my $coder = CBOR::XS->new_safe;
1065		1185
1066	my $data = $coder->decode ($cbor_text);	1186	my $data = $coder->decode ($cbor_text);
1067	my $cbor = $coder->encode ($data);	1187	my $cbor = $coder->encode ($data);
…		…
1089	even if all your C<THAW> methods are secure, encoding data structures from	1209	even if all your C<THAW> methods are secure, encoding data structures from
1090	untrusted sources can invoke those and trigger bugs in those.	1210	untrusted sources can invoke those and trigger bugs in those.
1091		1211
1092	So, if you are not sure about the security of all the modules you	1212	So, if you are not sure about the security of all the modules you
1093	have loaded (you shouldn't), you should disable this part using	1213	have loaded (you shouldn't), you should disable this part using
1094	C<forbid_objects>.	1214	C<forbid_objects> or using C<new_safe>.
1095		1215
1096	=item CBOR can be extended with tags that call library code	1216	=item CBOR can be extended with tags that call library code
1097		1217
1098	CBOR can be extended with tags, and C<CBOR::XS> has a registry of	1218	CBOR can be extended with tags, and C<CBOR::XS> has a registry of
1099	conversion functions for many existing tags that can be extended via	1219	conversion functions for many existing tags that can be extended via
1100	third-party modules (see the C<filter> method).	1220	third-party modules (see the C<filter> method).
1101		1221
1102	If you don't trust these, you should configure the "safe" filter function,	1222	If you don't trust these, you should configure the "safe" filter function,
1103	C<CBOR::XS::safe_filter>, which by default only includes conversion	1223	C<CBOR::XS::safe_filter> (C<new_safe> does this), which by default only
1104	functions that are considered "safe" by the author (but again, they can be	1224	includes conversion functions that are considered "safe" by the author
1105	extended by third party modules).	1225	(but again, they can be extended by third party modules).
1106		1226
1107	Depending on your level of paranoia, you can use the "safe" filter:	1227	Depending on your level of paranoia, you can use the "safe" filter:
1108		1228
1109	$cbor->filter (\&CBOR::XS::safe_filter);	1229	$cbor->filter (\&CBOR::XS::safe_filter);
1110		1230
…		…
1125	the size of CBOR data you accept, or make sure then when your resources	1245	the size of CBOR data you accept, or make sure then when your resources
1126	run out, that's just fine (e.g. by using a separate process that can	1246	run out, that's just fine (e.g. by using a separate process that can
1127	crash safely). The size of a CBOR string in octets is usually a good	1247	crash safely). The size of a CBOR string in octets is usually a good
1128	indication of the size of the resources required to decode it into a Perl	1248	indication of the size of the resources required to decode it into a Perl
1129	structure. While CBOR::XS can check the size of the CBOR text (using	1249	structure. While CBOR::XS can check the size of the CBOR text (using
1130	C<max_size>), it might be too late when you already have it in memory, so	1250	C<max_size> - done by C<new_safe>), it might be too late when you already
1131	you might want to check the size before you accept the string.	1251	have it in memory, so you might want to check the size before you accept
		1252	the string.
1132		1253
1133	As for encoding, it is possible to construct data structures that are	1254	As for encoding, it is possible to construct data structures that are
1134	relatively small but result in large CBOR texts (for example by having an	1255	relatively small but result in large CBOR texts (for example by having an
1135	array full of references to the same big data structure, which will all be	1256	array full of references to the same big data structure, which will all be
1136	deep-cloned during encoding by default). This is rarely an actual issue	1257	deep-cloned during encoding by default). This is rarely an actual issue
…		…
1149	method.	1270	method.
1150		1271
1151	=item Resource-starving attacks: CPU en-/decoding complexity	1272	=item Resource-starving attacks: CPU en-/decoding complexity
1152		1273
1153	CBOR::XS will use the L<Math::BigInt>, L<Math::BigFloat> and	1274	CBOR::XS will use the L<Math::BigInt>, L<Math::BigFloat> and
1154	L<Math::BigRat> libraries to represent encode/decode bignums. These can	1275	L<Math::BigRat> libraries to represent encode/decode bignums. These can be
1155	be very slow (as in, centuries of CPU time) and can even crash your	1276	very slow (as in, centuries of CPU time) and can even crash your program
1156	program (and are generally not very trustworthy). See the next section for	1277	(and are generally not very trustworthy). See the next section on bignum
1157	details.	1278	security for details.
1158		1279
1159	=item Data breaches: leaking information in error messages	1280	=item Data breaches: leaking information in error messages
1160		1281
1161	CBOR::XS might leak contents of your Perl data structures in its error	1282	CBOR::XS might leak contents of your Perl data structures in its error
1162	messages, so when you serialise sensitive information you might want to	1283	messages, so when you serialise sensitive information you might want to
…		…
1225	=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT	1346	=head1 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT
1226		1347
1227	On perls that were built without 64 bit integer support (these are rare	1348	On perls that were built without 64 bit integer support (these are rare
1228	nowadays, even on 32 bit architectures, as all major Perl distributions	1349	nowadays, even on 32 bit architectures, as all major Perl distributions
1229	are built with 64 bit integer support), support for any kind of 64 bit	1350	are built with 64 bit integer support), support for any kind of 64 bit
1230	integer in CBOR is very limited - most likely, these 64 bit values will	1351	value in CBOR is very limited - most likely, these 64 bit values will
1231	be truncated, corrupted, or otherwise not decoded correctly. This also	1352	be truncated, corrupted, or otherwise not decoded correctly. This also
1232	includes string, array and map sizes that are stored as 64 bit integers.	1353	includes string, float, array and map sizes that are stored as 64 bit
		1354	integers.
1233		1355
1234		1356
1235	=head1 THREADS	1357	=head1 THREADS
1236		1358
1237	This module is I<not> guaranteed to be thread safe and there are no	1359	This module is I<not> guaranteed to be thread safe and there are no

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.68 by root, Wed Jul 17 09:37:16 2019 UTC vs. Revision 1.89 by root, Fri Sep 8 20:03:06 2023 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.68 by root, Wed Jul 17 09:37:16 2019 UTC vs.
Revision 1.89 by root, Fri Sep 8 20:03:06 2023 UTC