[ViewVC] Diff of: cvs/CBOR-XS/README

Comparing CBOR-XS/README (file contents):
Revision 1.10 by root, Thu Nov 28 16:09:04 2013 UTC vs.
Revision 1.22 by root, Thu Oct 21 01:14:58 2021 UTC

…		…
33	with the added ability of supporting serialisation of Perl objects.	33	with the added ability of supporting serialisation of Perl objects.
34	(JSON often compresses better than CBOR though, so if you plan to	34	(JSON often compresses better than CBOR though, so if you plan to
35	compress the data later and speed is less important you might want to	35	compress the data later and speed is less important you might want to
36	compare both formats first).	36	compare both formats first).
37		37
		38	The primary goal of this module is to be correct and the secondary
		39	goal is to be fast. To reach the latter goal it was written in C.
		40
38	To give you a general idea about speed, with texts in the megabyte	41	To give you a general idea about speed, with texts in the megabyte
39	range, "CBOR::XS" usually encodes roughly twice as fast as Storable or	42	range, "CBOR::XS" usually encodes roughly twice as fast as Storable or
40	JSON::XS and decodes about 15%-30% faster than those. The shorter the	43	JSON::XS and decodes about 15%-30% faster than those. The shorter the
41	data, the worse Storable performs in comparison.	44	data, the worse Storable performs in comparison.
42		45
…		…
44	about 20% smaller than the same data encoded as (compact) JSON or	47	about 20% smaller than the same data encoded as (compact) JSON or
45	Storable.	48	Storable.
46		49
47	In addition to the core CBOR data format, this module implements a	50	In addition to the core CBOR data format, this module implements a
48	number of extensions, to support cyclic and shared data structures (see	51	number of extensions, to support cyclic and shared data structures (see
49	"allow_sharing"), string deduplication (see "pack_strings") and scalar	52	"allow_sharing" and "allow_cycles"), string deduplication (see
50	references (always enabled).	53	"pack_strings") and scalar references (always enabled).
51
52	The primary goal of this module is to be correct and the secondary
53	goal is to be fast. To reach the latter goal it was written in C.
54		54
55	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	55	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
56	vice versa.	56	vice versa.
57		57
58	FUNCTIONAL INTERFACE	58	FUNCTIONAL INTERFACE
…		…
78		78
79	The mutators for flags all return the CBOR object again and thus	79	The mutators for flags all return the CBOR object again and thus
80	calls can be chained:	80	calls can be chained:
81		81
82	my $cbor = CBOR::XS->new->encode ({a => [1,2]});	82	my $cbor = CBOR::XS->new->encode ({a => [1,2]});
		83
		84	$cbor = new_safe CBOR::XS
		85	Create a new, safe/secure CBOR::XS object. This is similar to "new",
		86	but configures the coder object to be safe to use with untrusted
		87	data. Currently, this is equivalent to:
		88
		89	my $cbor = CBOR::XS
		90	->new
		91	->validate_utf8
		92	->forbid_objects
		93	->filter (\&CBOR::XS::safe_filter)
		94	->max_size (1e8);
		95
		96	But is more future proof (it is better to crash because of a change
		97	than to be exploited in other ways).
83		98
84	$cbor = $cbor->max_depth ([$maximum_nesting_depth])	99	$cbor = $cbor->max_depth ([$maximum_nesting_depth])
85	$max_depth = $cbor->get_max_depth	100	$max_depth = $cbor->get_max_depth
86	Sets the maximum nesting level (default 512) accepted while encoding	101	Sets the maximum nesting level (default 512) accepted while encoding
87	or decoding. If a higher nesting level is detected in CBOR data or a	102	or decoding. If a higher nesting level is detected in CBOR data or a
…		…
101		116
102	Note that nesting is implemented by recursion in C. The default	117	Note that nesting is implemented by recursion in C. The default
103	value has been chosen to be as large as typical operating systems	118	value has been chosen to be as large as typical operating systems
104	allow without crashing.	119	allow without crashing.
105		120
106	See SECURITY CONSIDERATIONS, below, for more info on why this is	121	See "SECURITY CONSIDERATIONS", below, for more info on why this is
107	useful.	122	useful.
108		123
109	$cbor = $cbor->max_size ([$maximum_string_size])	124	$cbor = $cbor->max_size ([$maximum_string_size])
110	$max_size = $cbor->get_max_size	125	$max_size = $cbor->get_max_size
111	Set the maximum length a CBOR string may have (in bytes) where	126	Set the maximum length a CBOR string may have (in bytes) where
…		…
115	exception. This setting has no effect on "encode" (yet).	130	exception. This setting has no effect on "encode" (yet).
116		131
117	If no argument is given, the limit check will be deactivated (same	132	If no argument is given, the limit check will be deactivated (same
118	as when 0 is specified).	133	as when 0 is specified).
119		134
120	See SECURITY CONSIDERATIONS, below, for more info on why this is	135	See "SECURITY CONSIDERATIONS", below, for more info on why this is
121	useful.	136	useful.
122		137
123	$cbor = $cbor->allow_unknown ([$enable])	138	$cbor = $cbor->allow_unknown ([$enable])
124	$enabled = $cbor->get_allow_unknown	139	$enabled = $cbor->get_allow_unknown
125	If $enable is true (or missing), then "encode" will not throw an	140	If $enable is true (or missing), then "encode" will not throw an
…		…
141	instead will emit a reference to the earlier value.	156	instead will emit a reference to the earlier value.
142		157
143	This means that such values will only be encoded once, and will not	158	This means that such values will only be encoded once, and will not
144	result in a deep cloning of the value on decode, in decoders	159	result in a deep cloning of the value on decode, in decoders
145	supporting the value sharing extension. This also makes it possible	160	supporting the value sharing extension. This also makes it possible
146	to encode cyclic data structures.	161	to encode cyclic data structures (which need "allow_cycles" to be
		162	enabled to be decoded by this module).
147		163
148	It is recommended to leave it off unless you know your communication	164	It is recommended to leave it off unless you know your communication
149	partner supports the value sharing extensions to CBOR	165	partner supports the value sharing extensions to CBOR
150	(<http://cbor.schmorp.de/value-sharing>), as without decoder	166	(<http://cbor.schmorp.de/value-sharing>), as without decoder
151	support, the resulting data structure might be unusable.	167	support, the resulting data structure might be unusable.
152		168
153	Detecting shared values incurs a runtime overhead when values are	169	Detecting shared values incurs a runtime overhead when values are
154	encoded that have a reference counter large than one, and might	170	encoded that have a reference counter large than one, and might
155	unnecessarily increase the encoded size, as potentially shared	171	unnecessarily increase the encoded size, as potentially shared
156	values are encode as sharable whether or not they are actually	172	values are encoded as shareable whether or not they are actually
157	shared.	173	shared.
158		174
159	At the moment, only targets of references can be shared (e.g.	175	At the moment, only targets of references can be shared (e.g.
160	scalars, arrays or hashes pointed to by a reference). Weirder	176	scalars, arrays or hashes pointed to by a reference). Weirder
161	constructs, such as an array with multiple "copies" of the same	177	constructs, such as an array with multiple "copies" of the same
…		…
166	data structures repeatedly, unsharing them in the process. Cyclic	182	data structures repeatedly, unsharing them in the process. Cyclic
167	data structures cannot be encoded in this mode.	183	data structures cannot be encoded in this mode.
168		184
169	This option does not affect "decode" in any way - shared values and	185	This option does not affect "decode" in any way - shared values and
170	references will always be decoded properly if present.	186	references will always be decoded properly if present.
		187
		188	$cbor = $cbor->allow_cycles ([$enable])
		189	$enabled = $cbor->get_allow_cycles
		190	If $enable is true (or missing), then "decode" will happily decode
		191	self-referential (cyclic) data structures. By default these will not
		192	be decoded, as they need manual cleanup to avoid memory leaks, so
		193	code that isn't prepared for this will not leak memory.
		194
		195	If $enable is false (the default), then "decode" will throw an error
		196	when it encounters a self-referential/cyclic data structure.
		197
		198	FUTURE DIRECTION: the motivation behind this option is to avoid
		199	real cycles - future versions of this module might chose to decode
		200	cyclic data structures using weak references when this option is
		201	off, instead of throwing an error.
		202
		203	This option does not affect "encode" in any way - shared values and
		204	references will always be encoded properly if present.
		205
		206	$cbor = $cbor->forbid_objects ([$enable])
		207	$enabled = $cbor->get_forbid_objects
		208	Disables the use of the object serialiser protocol.
		209
		210	If $enable is true (or missing), then "encode" will will throw an
		211	exception when it encounters perl objects that would be encoded
		212	using the perl-object tag (26). When "decode" encounters such tags,
		213	it will fall back to the general filter/tagged logic as if this were
		214	an unknown tag (by default resulting in a "CBOR::XC::Tagged"
		215	object).
		216
		217	If $enable is false (the default), then "encode" will use the
		218	Types::Serialiser object serialisation protocol to serialise objects
		219	into perl-object tags, and "decode" will do the same to decode such
		220	tags.
		221
		222	See "SECURITY CONSIDERATIONS", below, for more info on why
		223	forbidding this protocol can be useful.
171		224
172	$cbor = $cbor->pack_strings ([$enable])	225	$cbor = $cbor->pack_strings ([$enable])
173	$enabled = $cbor->get_pack_strings	226	$enabled = $cbor->get_pack_strings
174	If $enable is true (or missing), then "encode" will try not to	227	If $enable is true (or missing), then "encode" will try not to
175	encode the same string twice, but will instead encode a reference to	228	encode the same string twice, but will instead encode a reference to
…		…
186	the standard CBOR way.	239	the standard CBOR way.
187		240
188	This option does not affect "decode" in any way - string references	241	This option does not affect "decode" in any way - string references
189	will always be decoded properly if present.	242	will always be decoded properly if present.
190		243
		244	$cbor = $cbor->text_keys ([$enable])
		245	$enabled = $cbor->get_text_keys
		246	If $enabled is true (or missing), then "encode" will encode all perl
		247	hash keys as CBOR text strings/UTF-8 string, upgrading them as
		248	needed.
		249
		250	If $enable is false (the default), then "encode" will encode hash
		251	keys normally - upgraded perl strings (strings internally encoded as
		252	UTF-8) as CBOR text strings, and downgraded perl strings as CBOR
		253	byte strings.
		254
		255	This option does not affect "decode" in any way.
		256
		257	This option is useful for interoperability with CBOR decoders that
		258	don't treat byte strings as a form of text. It is especially useful
		259	as Perl gives very little control over hash keys.
		260
		261	Enabling this option can be slow, as all downgraded hash keys that
		262	are encoded need to be scanned and converted to UTF-8.
		263
		264	$cbor = $cbor->text_strings ([$enable])
		265	$enabled = $cbor->get_text_strings
		266	This option works similar to "text_keys", above, but works on all
		267	strings (including hash keys), so "text_keys" has no further effect
		268	after enabling "text_strings".
		269
		270	If $enabled is true (or missing), then "encode" will encode all perl
		271	strings as CBOR text strings/UTF-8 strings, upgrading them as
		272	needed.
		273
		274	If $enable is false (the default), then "encode" will encode strings
		275	normally (but see "text_keys") - upgraded perl strings (strings
		276	internally encoded as UTF-8) as CBOR text strings, and downgraded
		277	perl strings as CBOR byte strings.
		278
		279	This option does not affect "decode" in any way.
		280
		281	This option has similar advantages and disadvantages as "text_keys".
		282	In addition, this option effectively removes the ability to
		283	automatically encode byte strings, which might break some "FREEZE"
		284	and "TO_CBOR" methods that rely on this.
		285
		286	A workaround is to use explicit type casts, which are unaffected by
		287	this option.
		288
		289	$cbor = $cbor->validate_utf8 ([$enable])
		290	$enabled = $cbor->get_validate_utf8
		291	If $enable is true (or missing), then "decode" will validate that
		292	elements (text strings) containing UTF-8 data in fact contain valid
		293	UTF-8 data (instead of blindly accepting it). This validation
		294	obviously takes extra time during decoding.
		295
		296	The concept of "valid UTF-8" used is perl's concept, which is a
		297	superset of the official UTF-8.
		298
		299	If $enable is false (the default), then "decode" will blindly accept
		300	UTF-8 data, marking them as valid UTF-8 in the resulting data
		301	structure regardless of whether that's true or not.
		302
		303	Perl isn't too happy about corrupted UTF-8 in strings, but should
		304	generally not crash or do similarly evil things. Extensions might be
		305	not so forgiving, so it's recommended to turn on this setting if you
		306	receive untrusted CBOR.
		307
		308	This option does not affect "encode" in any way - strings that are
		309	supposedly valid UTF-8 will simply be dumped into the resulting CBOR
		310	string without checking whether that is, in fact, true or not.
		311
191	$cbor = $cbor->filter ([$cb->($tag, $value)])	312	$cbor = $cbor->filter ([$cb->($tag, $value)])
192	$cb_or_undef = $cbor->get_filter	313	$cb_or_undef = $cbor->get_filter
193	Sets or replaces the tagged value decoding filter (when $cb is	314	Sets or replaces the tagged value decoding filter (when $cb is
194	specified) or clears the filter (if no argument or "undef" is	315	specified) or clears the filter (if no argument or "undef" is
195	provided).	316	provided).
…		…
212	When the filter is cleared (the default state), the default filter	333	When the filter is cleared (the default state), the default filter
213	function, "CBOR::XS::default_filter", is used. This function simply	334	function, "CBOR::XS::default_filter", is used. This function simply
214	looks up the tag in the %CBOR::XS::FILTER hash. If an entry exists	335	looks up the tag in the %CBOR::XS::FILTER hash. If an entry exists
215	it must be a code reference that is called with tag and value, and	336	it must be a code reference that is called with tag and value, and
216	is responsible for decoding the value. If no entry exists, it	337	is responsible for decoding the value. If no entry exists, it
217	returns no values.	338	returns no values. "CBOR::XS" provides a number of default filter
		339	functions already, the the %CBOR::XS::FILTER hash can be freely
		340	extended with more.
		341
		342	"CBOR::XS" additionally provides an alternative filter function that
		343	is supposed to be safe to use with untrusted data (which the default
		344	filter might not), called "CBOR::XS::safe_filter", which works the
		345	same as the "default_filter" but uses the %CBOR::XS::SAFE_FILTER
		346	variable instead. It is prepopulated with the tag decoding functions
		347	that are deemed safe (basically the same as %CBOR::XS::FILTER
		348	without all the bignum tags), and can be extended by user code as
		349	wlel, although, obviously, one should be very careful about adding
		350	decoding functions here, since the expectation is that they are safe
		351	to use on untrusted data, after all.
218		352
219	Example: decode all tags not handled internally into	353	Example: decode all tags not handled internally into
220	"CBOR::XS::Tagged" objects, with no other special handling (useful	354	"CBOR::XS::Tagged" objects, with no other special handling (useful
221	when working with potentially "unsafe" CBOR data).	355	when working with potentially "unsafe" CBOR data).
222		356
…		…
228	$CBOR::XS::FILTER{1347375694} = sub {	362	$CBOR::XS::FILTER{1347375694} = sub {
229	my ($tag, $value);	363	my ($tag, $value);
230		364
231	"tag 1347375694 value $value"	365	"tag 1347375694 value $value"
232	};	366	};
		367
		368	Example: provide your own filter function that looks up tags in your
		369	own hash:
		370
		371	my %my_filter = (
		372	998347484 => sub {
		373	my ($tag, $value);
		374
		375	"tag 998347484 value $value"
		376	};
		377	);
		378
		379	my $coder = CBOR::XS->new->filter (sub {
		380	&{ $my_filter{$_[0]} or return }
		381	});
		382
		383	Example: use the safe filter function (see "SECURITY CONSIDERATIONS"
		384	for more considerations on security).
		385
		386	CBOR::XS->new->filter (\&CBOR::XS::safe_filter)->decode ($cbor_data);
233		387
234	$cbor_data = $cbor->encode ($perl_scalar)	388	$cbor_data = $cbor->encode ($perl_scalar)
235	Converts the given Perl data structure (a scalar value) to its CBOR	389	Converts the given Perl data structure (a scalar value) to its CBOR
236	representation.	390	representation.
237		391
…		…
245	will silently stop parsing there and return the number of characters	399	will silently stop parsing there and return the number of characters
246	consumed so far.	400	consumed so far.
247		401
248	This is useful if your CBOR texts are not delimited by an outer	402	This is useful if your CBOR texts are not delimited by an outer
249	protocol and you need to know where the first CBOR string ends amd	403	protocol and you need to know where the first CBOR string ends amd
250	the next one starts.	404	the next one starts - CBOR strings are self-delimited, so it is
		405	possible to concatenate CBOR strings without any delimiters or size
		406	fields and recover their data.
251		407
252	CBOR::XS->new->decode_prefix ("......")	408	CBOR::XS->new->decode_prefix ("......")
253	=> ("...", 3)	409	=> ("...", 3)
		410
		411	INCREMENTAL PARSING
		412	In some cases, there is the need for incremental parsing of JSON texts.
		413	While this module always has to keep both CBOR text and resulting Perl
		414	data structure in memory at one time, it does allow you to parse a CBOR
		415	stream incrementally, using a similar to using "decode_prefix" to see if
		416	a full CBOR object is available, but is much more efficient.
		417
		418	It basically works by parsing as much of a CBOR string as possible - if
		419	the CBOR data is not complete yet, the pasrer will remember where it
		420	was, to be able to restart when more data has been accumulated. Once
		421	enough data is available to either decode a complete CBOR value or raise
		422	an error, a real decode will be attempted.
		423
		424	A typical use case would be a network protocol that consists of sending
		425	and receiving CBOR-encoded messages. The solution that works with CBOR
		426	and about anything else is by prepending a length to every CBOR value,
		427	so the receiver knows how many octets to read. More compact (and
		428	slightly slower) would be to just send CBOR values back-to-back, as
		429	"CBOR::XS" knows where a CBOR value ends, and doesn't need an explicit
		430	length.
		431
		432	The following methods help with this:
		433
		434	@decoded = $cbor->incr_parse ($buffer)
		435	This method attempts to decode exactly one CBOR value from the
		436	beginning of the given $buffer. The value is removed from the
		437	$buffer on success. When $buffer doesn't contain a complete value
		438	yet, it returns nothing. Finally, when the $buffer doesn't start
		439	with something that could ever be a valid CBOR value, it raises an
		440	exception, just as "decode" would. In the latter case the decoder
		441	state is undefined and must be reset before being able to parse
		442	further.
		443
		444	This method modifies the $buffer in place. When no CBOR value can be
		445	decoded, the decoder stores the current string offset. On the next
		446	call, continues decoding at the place where it stopped before. For
		447	this to make sense, the $buffer must begin with the same octets as
		448	on previous unsuccessful calls.
		449
		450	You can call this method in scalar context, in which case it either
		451	returns a decoded value or "undef". This makes it impossible to
		452	distinguish between CBOR null values (which decode to "undef") and
		453	an unsuccessful decode, which is often acceptable.
		454
		455	@decoded = $cbor->incr_parse_multiple ($buffer)
		456	Same as "incr_parse", but attempts to decode as many CBOR values as
		457	possible in one go, instead of at most one. Calls to "incr_parse"
		458	and "incr_parse_multiple" can be interleaved.
		459
		460	$cbor->incr_reset
		461	Resets the incremental decoder. This throws away any saved state, so
		462	that subsequent calls to "incr_parse" or "incr_parse_multiple" start
		463	to parse a new CBOR value from the beginning of the $buffer again.
		464
		465	This method can be called at any time, but it must be called if
		466	you want to change your $buffer or there was a decoding error and
		467	you want to reuse the $cbor object for future incremental parsings.
254		468
255	MAPPING	469	MAPPING
256	This section describes how CBOR::XS maps Perl values to CBOR values and	470	This section describes how CBOR::XS maps Perl values to CBOR values and
257	vice versa. These mappings are designed to "do the right thing" in most	471	vice versa. These mappings are designed to "do the right thing" in most
258	circumstances automatically, preserving round-tripping characteristics	472	circumstances automatically, preserving round-tripping characteristics
…		…
309		523
310	hash references	524	hash references
311	Perl hash references become CBOR maps. As there is no inherent	525	Perl hash references become CBOR maps. As there is no inherent
312	ordering in hash keys (or CBOR maps), they will usually be encoded	526	ordering in hash keys (or CBOR maps), they will usually be encoded
313	in a pseudo-random order. This order can be different each time a	527	in a pseudo-random order. This order can be different each time a
314	hahs is encoded.	528	hash is encoded.
315		529
316	Currently, tied hashes will use the indefinite-length format, while	530	Currently, tied hashes will use the indefinite-length format, while
317	normal hashes will use the fixed-length format.	531	normal hashes will use the fixed-length format.
318		532
319	array references	533	array references
…		…
368	my $x = 3.1; # some variable containing a number	582	my $x = 3.1; # some variable containing a number
369	"$x"; # stringified	583	"$x"; # stringified
370	$x .= ""; # another, more awkward way to stringify	584	$x .= ""; # another, more awkward way to stringify
371	print $x; # perl does it for you, too, quite often	585	print $x; # perl does it for you, too, quite often
372		586
373	You can force whether a string ie encoded as byte or text string by	587	You can force whether a string is encoded as byte or text string by
374	using "utf8::upgrade" and "utf8::downgrade"):	588	using "utf8::upgrade" and "utf8::downgrade" (if "text_strings" is
		589	disabled).
375		590
376	utf8::upgrade $x; # encode $x as text string	591	utf8::upgrade $x; # encode $x as text string
377	utf8::downgrade $x; # encode $x as byte string	592	utf8::downgrade $x; # encode $x as byte string
378		593
		594	More options are available, see "TYPE CASTS", below, and the
		595	"text_keys" and "text_strings" options.
		596
379	Perl doesn't define what operations up- and downgrade strings, so if	597	Perl doesn't define what operations up- and downgrade strings, so if
380	the difference between byte and text is important, you should up- or	598	the difference between byte and text is important, you should up- or
381	downgrade your string as late as possible before encoding.	599	downgrade your string as late as possible before encoding. You can
		600	also force the use of CBOR text strings by using "text_keys" or
		601	"text_strings".
382		602
383	You can force the type to be a CBOR number by numifying it:	603	You can force the type to be a CBOR number by numifying it:
384		604
385	my $x = "3"; # some variable containing a string	605	my $x = "3"; # some variable containing a string
386	$x += 0; # numify it, ensuring it will be dumped as a number	606	$x += 0; # numify it, ensuring it will be dumped as a number
…		…
395	IEEE single format if possible without loss of precision, otherwise	615	IEEE single format if possible without loss of precision, otherwise
396	the IEEE double format will be used. Perls that use formats other	616	the IEEE double format will be used. Perls that use formats other
397	than IEEE double to represent numerical values are supported, but	617	than IEEE double to represent numerical values are supported, but
398	might suffer loss of precision.	618	might suffer loss of precision.
399		619
		620	TYPE CASTS
		621	EXPERIMENTAL: As an experimental extension, "CBOR::XS" allows you to
		622	force specific CBOR types to be used when encoding. That allows you to
		623	encode types not normally accessible (e.g. half floats) as well as force
		624	string types even when "text_strings" is in effect.
		625
		626	Type forcing is done by calling a special "cast" function which keeps a
		627	copy of the value and returns a new value that can be handed over to any
		628	CBOR encoder function.
		629
		630	The following casts are currently available (all of which are unary
		631	operators, that is, have a prototype of "$"):
		632
		633	CBOR::XS::as_int $value
		634	Forces the value to be encoded as some form of (basic, not bignum)
		635	integer type.
		636
		637	CBOR::XS::as_text $value
		638	Forces the value to be encoded as (UTF-8) text values.
		639
		640	CBOR::XS::as_bytes $value
		641	Forces the value to be encoded as a (binary) string value.
		642
		643	Example: encode a perl string as binary even though "text_strings"
		644	is in effect.
		645
		646	CBOR::XS->new->text_strings->encode ([4, "text", CBOR::XS::bytes "bytevalue"]);
		647
		648	CBOR::XS::as_bool $value
		649	Converts a Perl boolean (which can be any kind of scalar) into a
		650	CBOR boolean. Strictly the same, but shorter to write, than:
		651
		652	$value ? Types::Serialiser::true : Types::Serialiser::false
		653
		654	CBOR::XS::as_float16 $value
		655	Forces half-float (IEEE 754 binary16) encoding of the given value.
		656
		657	CBOR::XS::as_float32 $value
		658	Forces single-float (IEEE 754 binary32) encoding of the given value.
		659
		660	CBOR::XS::as_float64 $value
		661	Forces double-float (IEEE 754 binary64) encoding of the given value.
		662
		663	CBOR::XS::as_cbor $cbor_text
		664	Not a type cast per-se, this type cast forces the argument to be
		665	encoded as-is. This can be used to embed pre-encoded CBOR data.
		666
		667	Note that no checking on the validity of the $cbor_text is done -
		668	it's the callers responsibility to correctly encode values.
		669
		670	CBOR::XS::as_map [key => value...]
		671	Treat the array reference as key value pairs and output a CBOR map.
		672	This allows you to generate CBOR maps with arbitrary key types (or,
		673	if you don't care about semantics, duplicate keys or pairs in a
		674	custom order), which is otherwise hard to do with Perl.
		675
		676	The single argument must be an array reference with an even number
		677	of elements.
		678
		679	Note that only the reference to the array is copied, the array
		680	itself is not. Modifications done to the array before calling an
		681	encoding function will be reflected in the encoded output.
		682
		683	Example: encode a CBOR map with a string and an integer as keys.
		684
		685	encode_cbor CBOR::XS::as_map [string => "value", 5 => "value"]
		686
400	OBJECT SERIALISATION	687	OBJECT SERIALISATION
		688	This module implements both a CBOR-specific and the generic
		689	Types::Serialier object serialisation protocol. The following
		690	subsections explain both methods.
		691
		692	ENCODING
401	This module knows two way to serialise a Perl object: The CBOR-specific	693	This module knows two way to serialise a Perl object: The CBOR-specific
402	way, and the generic way.	694	way, and the generic way.
403		695
404	Whenever the encoder encounters a Perl object that it cnanot serialise	696	Whenever the encoder encounters a Perl object that it cannot serialise
405	directly (most of them), it will first look up the "TO_CBOR" method on	697	directly (most of them), it will first look up the "TO_CBOR" method on
406	it.	698	it.
407		699
408	If it has a "TO_CBOR" method, it will call it with the object as only	700	If it has a "TO_CBOR" method, it will call it with the object as only
409	argument, and expects exactly one return value, which it will then	701	argument, and expects exactly one return value, which it will then
…		…
414	"CBOR" as the second argument, to distinguish it from other serialisers.	706	"CBOR" as the second argument, to distinguish it from other serialisers.
415		707
416	The "FREEZE" method can return any number of values (i.e. zero or more).	708	The "FREEZE" method can return any number of values (i.e. zero or more).
417	These will be encoded as CBOR perl object, together with the classname.	709	These will be encoded as CBOR perl object, together with the classname.
418		710
		711	These methods MUST NOT change the data structure that is being
		712	serialised. Failure to comply to this can result in memory corruption -
		713	and worse.
		714
419	If an object supports neither "TO_CBOR" nor "FREEZE", encoding will fail	715	If an object supports neither "TO_CBOR" nor "FREEZE", encoding will fail
420	with an error.	716	with an error.
421		717
		718	DECODING
422	Objects encoded via "TO_CBOR" cannot be automatically decoded, but	719	Objects encoded via "TO_CBOR" cannot (normally) be automatically
423	objects encoded via "FREEZE" can be decoded using the following	720	decoded, but objects encoded via "FREEZE" can be decoded using the
424	protocol:	721	following protocol:
425		722
426	When an encoded CBOR perl object is encountered by the decoder, it will	723	When an encoded CBOR perl object is encountered by the decoder, it will
427	look up the "THAW" method, by using the stored classname, and will fail	724	look up the "THAW" method, by using the stored classname, and will fail
428	if the method cannot be found.	725	if the method cannot be found.
429		726
…		…
471	"$self" # encode url string	768	"$self" # encode url string
472	}	769	}
473		770
474	sub URI::THAW {	771	sub URI::THAW {
475	my ($class, $serialiser, $uri) = @_;	772	my ($class, $serialiser, $uri) = @_;
476
477	$class->new ($uri)	773	$class->new ($uri)
478	}	774	}
479		775
480	Unlike "TO_CBOR", multiple values can be returned by "FREEZE". For	776	Unlike "TO_CBOR", multiple values can be returned by "FREEZE". For
481	example, a "FREEZE" method that returns "type", "id" and "variant"	777	example, a "FREEZE" method that returns "type", "id" and "variant"
…		…
577	Future versions of this module reserve the right to special case	873	Future versions of this module reserve the right to special case
578	additional tags (such as base64url).	874	additional tags (such as base64url).
579		875
580	ENFORCED TAGS	876	ENFORCED TAGS
581	These tags are always handled when decoding, and their handling cannot	877	These tags are always handled when decoding, and their handling cannot
582	be overriden by the user.	878	be overridden by the user.
583		879
584	26 (perl-object, <http://cbor.schmorp.de/perl-object>)	880	26 (perl-object, <http://cbor.schmorp.de/perl-object>)
585	These tags are automatically created (and decoded) for serialisable	881	These tags are automatically created (and decoded) for serialisable
586	objects using the "FREEZE/THAW" methods (the Types::Serialier object	882	objects using the "FREEZE/THAW" methods (the Types::Serialier object
587	serialisation protocol). See "OBJECT SERIALISATION" for details.	883	serialisation protocol). See "OBJECT SERIALISATION" for details.
588		884
589	28, 29 (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)	885	28, 29 (shareable, sharedref, <http://cbor.schmorp.de/value-sharing>)
590	These tags are automatically decoded when encountered, resulting in	886	These tags are automatically decoded when encountered (and they do
		887	not result in a cyclic data structure, see "allow_cycles"),
591	shared values in the decoded object. They are only encoded, however,	888	resulting in shared values in the decoded object. They are only
592	when "allow_sharable" is enabled.	889	encoded, however, when "allow_sharing" is enabled.
593		890
		891	Not all shared values can be successfully decoded: values that
		892	reference themselves will currently decode as "undef" (this is not
		893	the same as a reference pointing to itself, which will be
		894	represented as a value that contains an indirect reference to itself
		895	- these will be decoded properly).
		896
		897	Note that considerably more shared value data structures can be
		898	decoded than will be encoded - currently, only values pointed to by
		899	references will be shared, others will not. While non-reference
		900	shared values can be generated in Perl with some effort, they were
		901	considered too unimportant to be supported in the encoder. The
		902	decoder, however, will decode these values as shared values.
		903
594	256, 25 (stringref-namespace, stringref, L	904	256, 25 (stringref-namespace, stringref,
595	<http://cbor.schmorp.de/stringref>)	905	<http://cbor.schmorp.de/stringref>)
596	These tags are automatically decoded when encountered. They are only	906	These tags are automatically decoded when encountered. They are only
597	encoded, however, when "pack_strings" is enabled.	907	encoded, however, when "pack_strings" is enabled.
598		908
599	22098 (indirection, <http://cbor.schmorp.de/indirection>)	909	22098 (indirection, <http://cbor.schmorp.de/indirection>)
600	This tag is automatically generated when a reference are encountered	910	This tag is automatically generated when a reference are encountered
601	(with the exception of hash and array refernces). It is converted to	911	(with the exception of hash and array references). It is converted
602	a reference when decoding.	912	to a reference when decoding.
603		913
604	55799 (self-describe CBOR, RFC 7049)	914	55799 (self-describe CBOR, RFC 7049)
605	This value is not generated on encoding (unless explicitly requested	915	This value is not generated on encoding (unless explicitly requested
606	by the user), and is simply ignored when decoding.	916	by the user), and is simply ignored when decoding.
607		917
608	NON-ENFORCED TAGS	918	NON-ENFORCED TAGS
609	These tags have default filters provided when decoding. Their handling	919	These tags have default filters provided when decoding. Their handling
610	can be overriden by changing the %CBOR::XS::FILTER entry for the tag, or	920	can be overridden by changing the %CBOR::XS::FILTER entry for the tag,
611	by providing a custom "filter" callback when decoding.	921	or by providing a custom "filter" callback when decoding.
612		922
613	When they result in decoding into a specific Perl class, the module	923	When they result in decoding into a specific Perl class, the module
614	usually provides a corresponding "TO_CBOR" method as well.	924	usually provides a corresponding "TO_CBOR" method as well.
615		925
616	When any of these need to load additional modules that are not part of	926	When any of these need to load additional modules that are not part of
617	the perl core distribution (e.g. URI), it is (currently) up to the user	927	the perl core distribution (e.g. URI), it is (currently) up to the user
618	to provide these modules. The decoding usually fails with an exception	928	to provide these modules. The decoding usually fails with an exception
619	if the required module cannot be loaded.	929	if the required module cannot be loaded.
620		930
		931	0, 1 (date/time string, seconds since the epoch)
		932	These tags are decoded into Time::Piece objects. The corresponding
		933	"Time::Piece::TO_CBOR" method always encodes into tag 1 values
		934	currently.
		935
		936	The Time::Piece API is generally surprisingly bad, and fractional
		937	seconds are only accidentally kept intact, so watch out. On the plus
		938	side, the module comes with perl since 5.10, which has to count for
		939	something.
		940
621	2, 3 (positive/negative bignum)	941	2, 3 (positive/negative bignum)
622	These tags are decoded into Math::BigInt objects. The corresponding	942	These tags are decoded into Math::BigInt objects. The corresponding
623	"Math::BigInt::TO_CBOR" method encodes "small" bigints into normal	943	"Math::BigInt::TO_CBOR" method encodes "small" bigints into normal
624	CBOR integers, and others into positive/negative CBOR bignums.	944	CBOR integers, and others into positive/negative CBOR bignums.
625		945
626	4, 5 (decimal fraction/bigfloat)	946	4, 5, 264, 265 (decimal fraction/bigfloat)
627	Both decimal fractions and bigfloats are decoded into Math::BigFloat	947	Both decimal fractions and bigfloats are decoded into Math::BigFloat
628	objects. The corresponding "Math::BigFloat::TO_CBOR" method always	948	objects. The corresponding "Math::BigFloat::TO_CBOR" method always
629	encodes into a decimal fraction.	949	encodes into a decimal fraction (either tag 4 or 264).
630		950
631	CBOR cannot represent bigfloats with very large exponents -
632	conversion of such big float objects is undefined.
633
634	Also, NaN and infinities are not encoded properly.	951	NaN and infinities are not encoded properly, as they cannot be
		952	represented in CBOR.
		953
		954	See "BIGNUM SECURITY CONSIDERATIONS" for more info.
		955
		956	30 (rational numbers)
		957	These tags are decoded into Math::BigRat objects. The corresponding
		958	"Math::BigRat::TO_CBOR" method encodes rational numbers with
		959	denominator 1 via their numerator only, i.e., they become normal
		960	integers or "bignums".
		961
		962	See "BIGNUM SECURITY CONSIDERATIONS" for more info.
635		963
636	21, 22, 23 (expected later JSON conversion)	964	21, 22, 23 (expected later JSON conversion)
637	CBOR::XS is not a CBOR-to-JSON converter, and will simply ignore	965	CBOR::XS is not a CBOR-to-JSON converter, and will simply ignore
638	these tags.	966	these tags.
639		967
…		…
653	interoperability is improved in the future, then the goal will be to	981	interoperability is improved in the future, then the goal will be to
654	ensure that decoded JSON data will round-trip encoding and decoding to	982	ensure that decoded JSON data will round-trip encoding and decoding to
655	CBOR intact.	983	CBOR intact.
656		984
657	SECURITY CONSIDERATIONS	985	SECURITY CONSIDERATIONS
658	When you are using CBOR in a protocol, talking to untrusted potentially	986	Tl;dr... if you want to decode or encode CBOR from untrusted sources,
659	hostile creatures requires relatively few measures.	987	you should start with a coder object created via "new_safe" (which
		988	implements the mitigations explained below):
660		989
		990	my $coder = CBOR::XS->new_safe;
		991
		992	my $data = $coder->decode ($cbor_text);
		993	my $cbor = $coder->encode ($data);
		994
		995	Longer version: When you are using CBOR in a protocol, talking to
		996	untrusted potentially hostile creatures requires some thought:
		997
		998	Security of the CBOR decoder itself
661	First of all, your CBOR decoder should be secure, that is, should not	999	First and foremost, your CBOR decoder should be secure, that is,
662	have any buffer overflows. Obviously, this module should ensure that and	1000	should not have any buffer overflows or similar bugs that could
		1001	potentially be exploited. Obviously, this module should ensure that
663	I am trying hard on making that true, but you never know.	1002	and I am trying hard on making that true, but you never know.
664		1003
		1004	CBOR::XS can invoke almost arbitrary callbacks during decoding
		1005	CBOR::XS supports object serialisation - decoding CBOR can cause
		1006	calls to any "THAW" method in any package that exists in your
		1007	process (that is, CBOR::XS will not try to load modules, but any
		1008	existing "THAW" method or function can be called, so they all have
		1009	to be secure).
		1010
		1011	Less obviously, it will also invoke "TO_CBOR" and "FREEZE" methods -
		1012	even if all your "THAW" methods are secure, encoding data structures
		1013	from untrusted sources can invoke those and trigger bugs in those.
		1014
		1015	So, if you are not sure about the security of all the modules you
		1016	have loaded (you shouldn't), you should disable this part using
		1017	"forbid_objects" or using "new_safe".
		1018
		1019	CBOR can be extended with tags that call library code
		1020	CBOR can be extended with tags, and "CBOR::XS" has a registry of
		1021	conversion functions for many existing tags that can be extended via
		1022	third-party modules (see the "filter" method).
		1023
		1024	If you don't trust these, you should configure the "safe" filter
		1025	function, "CBOR::XS::safe_filter" ("new_safe" does this), which by
		1026	default only includes conversion functions that are considered
		1027	"safe" by the author (but again, they can be extended by third party
		1028	modules).
		1029
		1030	Depending on your level of paranoia, you can use the "safe" filter:
		1031
		1032	$cbor->filter (\&CBOR::XS::safe_filter);
		1033
		1034	... your own filter...
		1035
		1036	$cbor->filter (sub { ... do your stuffs here ... });
		1037
		1038	... or even no filter at all, disabling all tag decoding:
		1039
		1040	$cbor->filter (sub { });
		1041
		1042	This is never a problem for encoding, as the tag mechanism only
		1043	exists in CBOR texts.
		1044
		1045	Resource-starving attacks: object memory usage
665	Second, you need to avoid resource-starving attacks. That means you	1046	You need to avoid resource-starving attacks. That means you should
666	should limit the size of CBOR data you accept, or make sure then when	1047	limit the size of CBOR data you accept, or make sure then when your
667	your resources run out, that's just fine (e.g. by using a separate	1048	resources run out, that's just fine (e.g. by using a separate
668	process that can crash safely). The size of a CBOR string in octets is	1049	process that can crash safely). The size of a CBOR string in octets
669	usually a good indication of the size of the resources required to	1050	is usually a good indication of the size of the resources required
670	decode it into a Perl structure. While CBOR::XS can check the size of	1051	to decode it into a Perl structure. While CBOR::XS can check the
		1052	size of the CBOR text (using "max_size" - done by "new_safe"), it
671	the CBOR text, it might be too late when you already have it in memory,	1053	might be too late when you already have it in memory, so you might
672	so you might want to check the size before you accept the string.	1054	want to check the size before you accept the string.
673		1055
		1056	As for encoding, it is possible to construct data structures that
		1057	are relatively small but result in large CBOR texts (for example by
		1058	having an array full of references to the same big data structure,
		1059	which will all be deep-cloned during encoding by default). This is
		1060	rarely an actual issue (and the worst case is still just running out
		1061	of memory), but you can reduce this risk by using "allow_sharing".
		1062
		1063	Resource-starving attacks: stack overflows
674	Third, CBOR::XS recurses using the C stack when decoding objects and	1064	CBOR::XS recurses using the C stack when decoding objects and
675	arrays. The C stack is a limited resource: for instance, on my amd64	1065	arrays. The C stack is a limited resource: for instance, on my amd64
676	machine with 8MB of stack size I can decode around 180k nested arrays	1066	machine with 8MB of stack size I can decode around 180k nested
677	but only 14k nested CBOR objects (due to perl itself recursing deeply on	1067	arrays but only 14k nested CBOR objects (due to perl itself
678	croak to free the temporary). If that is exceeded, the program crashes.	1068	recursing deeply on croak to free the temporary). If that is
679	To be conservative, the default nesting limit is set to 512. If your	1069	exceeded, the program crashes. To be conservative, the default
680	process has a smaller stack, you should adjust this setting accordingly	1070	nesting limit is set to 512. If your process has a smaller stack,
681	with the "max_depth" method.	1071	you should adjust this setting accordingly with the "max_depth"
		1072	method.
682		1073
		1074	Resource-starving attacks: CPU en-/decoding complexity
		1075	CBOR::XS will use the Math::BigInt, Math::BigFloat and Math::BigRat
		1076	libraries to represent encode/decode bignums. These can be very slow
		1077	(as in, centuries of CPU time) and can even crash your program (and
		1078	are generally not very trustworthy). See the next section on bignum
		1079	security for details.
		1080
		1081	Data breaches: leaking information in error messages
		1082	CBOR::XS might leak contents of your Perl data structures in its
		1083	error messages, so when you serialise sensitive information you
		1084	might want to make sure that exceptions thrown by CBOR::XS will not
		1085	end up in front of untrusted eyes.
		1086
		1087	Something else...
683	Something else could bomb you, too, that I forgot to think of. In that	1088	Something else could bomb you, too, that I forgot to think of. In
684	case, you get to keep the pieces. I am always open for hints, though...	1089	that case, you get to keep the pieces. I am always open for hints,
		1090	though...
685		1091
686	Also keep in mind that CBOR::XS might leak contents of your Perl data	1092	BIGNUM SECURITY CONSIDERATIONS
687	structures in its error messages, so when you serialise sensitive	1093	CBOR::XS provides a "TO_CBOR" method for both Math::BigInt and
688	information you might want to make sure that exceptions thrown by	1094	Math::BigFloat that tries to encode the number in the simplest possible
689	CBOR::XS will not end up in front of untrusted eyes.	1095	way, that is, either a CBOR integer, a CBOR bigint/decimal fraction (tag
		1096	4) or an arbitrary-exponent decimal fraction (tag 264). Rational numbers
		1097	(Math::BigRat, tag 30) can also contain bignums as members.
		1098
		1099	CBOR::XS will also understand base-2 bigfloat or arbitrary-exponent
		1100	bigfloats (tags 5 and 265), but it will never generate these on its own.
		1101
		1102	Using the built-in Math::BigInt::Calc support, encoding and decoding
		1103	decimal fractions is generally fast. Decoding bigints can be slow for
		1104	very big numbers (tens of thousands of digits, something that could
		1105	potentially be caught by limiting the size of CBOR texts), and decoding
		1106	bigfloats or arbitrary-exponent bigfloats can be extremely slow
		1107	(minutes, decades) for large exponents (roughly 40 bit and longer).
		1108
		1109	Additionally, Math::BigInt can take advantage of other bignum libraries,
		1110	such as Math::GMP, which cannot handle big floats with large exponents,
		1111	and might simply abort or crash your program, due to their code quality.
		1112
		1113	This can be a concern if you want to parse untrusted CBOR. If it is, you
		1114	might want to disable decoding of tag 2 (bigint) and 3 (negative bigint)
		1115	types. You should also disable types 5 and 265, as these can be slow
		1116	even without bigints.
		1117
		1118	Disabling bigints will also partially or fully disable types that rely
		1119	on them, e.g. rational numbers that use bignums.
690		1120
691	CBOR IMPLEMENTATION NOTES	1121	CBOR IMPLEMENTATION NOTES
692	This section contains some random implementation notes. They do not	1122	This section contains some random implementation notes. They do not
693	describe guaranteed behaviour, but merely behaviour as-is implemented	1123	describe guaranteed behaviour, but merely behaviour as-is implemented
694	right now.	1124	right now.
…		…
702	Only the double data type is supported for NV data types - when Perl	1132	Only the double data type is supported for NV data types - when Perl
703	uses long double to represent floating point values, they might not be	1133	uses long double to represent floating point values, they might not be
704	encoded properly. Half precision types are accepted, but not encoded.	1134	encoded properly. Half precision types are accepted, but not encoded.
705		1135
706	Strict mode and canonical mode are not implemented.	1136	Strict mode and canonical mode are not implemented.
		1137
		1138	LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT
		1139	On perls that were built without 64 bit integer support (these are rare
		1140	nowadays, even on 32 bit architectures, as all major Perl distributions
		1141	are built with 64 bit integer support), support for any kind of 64 bit
		1142	value in CBOR is very limited - most likely, these 64 bit values will be
		1143	truncated, corrupted, or otherwise not decoded correctly. This also
		1144	includes string, float, array and map sizes that are stored as 64 bit
		1145	integers.
707		1146
708	THREADS	1147	THREADS
709	This module is not guaranteed to be thread safe and there are no plans	1148	This module is not guaranteed to be thread safe and there are no plans
710	to change this until Perl gets thread support (as opposed to the	1149	to change this until Perl gets thread support (as opposed to the
711	horribly slow so-called "threads" which are simply slow and bloated	1150	horribly slow so-called "threads" which are simply slow and bloated

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/README (file contents): Revision 1.10 by root, Thu Nov 28 16:09:04 2013 UTC vs. Revision 1.22 by root, Thu Oct 21 01:14:58 2021 UTC

Diff Legend

Comparing CBOR-XS/README (file contents):
Revision 1.10 by root, Thu Nov 28 16:09:04 2013 UTC vs.
Revision 1.22 by root, Thu Oct 21 01:14:58 2021 UTC