[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.17 by root, Wed Oct 30 10:11:04 2013 UTC vs.
Revision 1.24 by root, Fri Nov 22 16:18:59 2013 UTC

…		…
26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string	26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
27	}	27	}
28		28
29	=head1 DESCRIPTION	29	=head1 DESCRIPTION
30		30
31	WARNING! This module is very new, and not very well tested (that's up to	31	WARNING! This module is very new, and not very well tested (that's up
32	you to do). Furthermore, details of the implementation might change freely	32	to you to do). Furthermore, details of the implementation might change
33	before version 1.0. And lastly, the object serialisation protocol depends	33	freely before version 1.0. And lastly, most extensions depend on an IANA
34	on a pending IANA assignment, and until that assignment is official, this	34	assignment, and until that assignment is official, this implementation is
35	implementation is not interoperable with other implementations (even	35	not interoperable with other implementations (even future versions of this
36	future versions of this module) until the assignment is done.	36	module) until the assignment is done.
37		37
38	You are still invited to try out CBOR, and this module.	38	You are still invited to try out CBOR, and this module.
39		39
40	This module converts Perl data structures to the Concise Binary Object	40	This module converts Perl data structures to the Concise Binary Object
41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation	41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
…		…
54	data, the worse L<Storable> performs in comparison.	54	data, the worse L<Storable> performs in comparison.
55		55
56	As for compactness, C<CBOR::XS> encoded data structures are usually about	56	As for compactness, C<CBOR::XS> encoded data structures are usually about
57	20% smaller than the same data encoded as (compact) JSON or L<Storable>.	57	20% smaller than the same data encoded as (compact) JSON or L<Storable>.
58		58
		59	In addition to the core CBOR data format, this module implements a number
		60	of extensions, to support cyclic and self-referencing data structures
		61	(see C<allow_sharing>), string deduplication (see C<allow_stringref>) and
		62	scalar references (always enabled).
		63
59	The primary goal of this module is to be I<correct> and the secondary goal	64	The primary goal of this module is to be I<correct> and the secondary goal
60	is to be I<fast>. To reach the latter goal it was written in C.	65	is to be I<fast>. To reach the latter goal it was written in C.
61		66
62	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	67	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
63	vice versa.	68	vice versa.
…		…
66		71
67	package CBOR::XS;	72	package CBOR::XS;
68		73
69	use common::sense;	74	use common::sense;
70		75
71	our $VERSION = 0.08;	76	our $VERSION = 0.09;
72	our @ISA = qw(Exporter);	77	our @ISA = qw(Exporter);
73		78
74	our @EXPORT = qw(encode_cbor decode_cbor);	79	our @EXPORT = qw(encode_cbor decode_cbor);
75		80
76	use Exporter;	81	use Exporter;
…		…
113	strings. All boolean flags described below are by default I<disabled>.	118	strings. All boolean flags described below are by default I<disabled>.
114		119
115	The mutators for flags all return the CBOR object again and thus calls can	120	The mutators for flags all return the CBOR object again and thus calls can
116	be chained:	121	be chained:
117		122
118	#TODO
119	my $cbor = CBOR::XS->new->encode ({a => [1,2]});	123	my $cbor = CBOR::XS->new->encode ({a => [1,2]});
120		124
121	=item $cbor = $cbor->max_depth ([$maximum_nesting_depth])	125	=item $cbor = $cbor->max_depth ([$maximum_nesting_depth])
122		126
123	=item $max_depth = $cbor->get_max_depth	127	=item $max_depth = $cbor->get_max_depth
…		…
157	If no argument is given, the limit check will be deactivated (same as when	161	If no argument is given, the limit check will be deactivated (same as when
158	C<0> is specified).	162	C<0> is specified).
159		163
160	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.	164	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
161		165
		166	=item $cbor = $cbor->allow_unknown ([$enable])
		167
		168	=item $enabled = $cbor->get_allow_unknown
		169
		170	If C<$enable> is true (or missing), then C<encode> will I<not> throw an
		171	exception when it encounters values it cannot represent in CBOR (for
		172	example, filehandles) but instead will encode a CBOR C<error> value.
		173
		174	If C<$enable> is false (the default), then C<encode> will throw an
		175	exception when it encounters anything it cannot encode as CBOR.
		176
		177	This option does not affect C<decode> in any way, and it is recommended to
		178	leave it off unless you know your communications partner.
		179
		180	=item $cbor = $cbor->allow_sharing ([$enable])
		181
		182	=item $enabled = $cbor->get_allow_sharing
		183
		184	If C<$enable> is true (or missing), then C<encode> will not double-encode
		185	values that have been referenced before (e.g. when the same object, such
		186	as an array, is referenced multiple times), but instead will emit a
		187	reference to the earlier value.
		188
		189	This means that such values will only be encoded once, and will not result
		190	in a deep cloning of the value on decode, in decoders supporting the value
		191	sharing extension.
		192
		193	It is recommended to leave it off unless you know your
		194	communication partner supports the value sharing extensions to CBOR
		195	(http://cbor.schmorp.de/value-sharing).
		196
		197	Detecting shared values incurs a runtime overhead when values are encoded
		198	that have a reference counter large than one, and might unnecessarily
		199	increase the encoded size, as potentially shared values are encode as
		200	sharable whether or not they are actually shared.
		201
		202	At the moment, only targets of references can be shared (e.g. scalars,
		203	arrays or hashes pointed to by a reference). Weirder constructs, such as
		204	an array with multiple "copies" of the I<same> string, which are hard but
		205	not impossible to create in Perl, are not supported (this is the same as
		206	for L<Storable>).
		207
		208	If C<$enable> is false (the default), then C<encode> will encode
		209	exception when it encounters anything it cannot encode as CBOR.
		210
		211	This option does not affect C<decode> in any way - shared values and
		212	references will always be decoded properly if present.
		213
		214	=item $cbor = $cbor->allow_stringref ([$enable])
		215
		216	=item $enabled = $cbor->get_allow_stringref
		217
		218	If C<$enable> is true (or missing), then C<encode> will try not to encode
		219	the same string twice, but will instead encode a reference to the string
		220	instead. Depending on your data format. this can save a lot of space, but
		221	also results in a very large runtime overhead (expect encoding times to be
		222	2-4 times as high as without).
		223
		224	It is recommended to leave it off unless you know your
		225	communications partner supports the stringref extension to CBOR
		226	(http://cbor.schmorp.de/stringref).
		227
		228	If C<$enable> is false (the default), then C<encode> will encode
		229	exception when it encounters anything it cannot encode as CBOR.
		230
		231	This option does not affect C<decode> in any way - string references will
		232	always be decoded properly if present.
		233
		234	=item $cbor = $cbor->filter ([$cb->($tag, $value)])
		235
		236	=item $cb_or_undef = $cbor->get_filter
		237
		238	Sets or replaces the tagged value decoding filter (when C<$cb> is
		239	specified) or clears the filter (if no argument or C<undef> is provided).
		240
		241	The filter callback is called only during decoding, when a non-enforced
		242	tagged value has been decoded (see L<TAG HANDLING AND EXTENSIONS> for a
		243	list of enforced tags). For specific tags, it's often better to provide a
		244	default converter using the C<%CBOR::XS::FILTER> hash (see below).
		245
		246	The first argument is the numerical tag, the second is the (decoded) value
		247	that has been tagged.
		248
		249	The filter function should return either exactly one value, which will
		250	replace the tagged value in the decoded data structure, or no values,
		251	which will result in default handling, which currently means the decoder
		252	creates a C<CBOR::XS::Tagged> object to hold the tag and the value.
		253
		254	When the filter is cleared (the default state), the default filter
		255	function, C<CBOR::XS::default_filter>, is used. This function simply looks
		256	up the tag in the C<%CBOR::XS::FILTER> hash. If an entry exists it must be
		257	a code reference that is called with tag and value, and is responsible for
		258	decoding the value. If no entry exists, it returns no values.
		259
		260	Example: decode all tags not handled internally into CBOR::XS::Tagged
		261	objects, with no other special handling (useful when working with
		262	potentially "unsafe" CBOR data).
		263
		264	CBOR::XS->new->filter (sub { })->decode ($cbor_data);
		265
		266	Example: provide a global filter for tag 1347375694, converting the value
		267	into some string form.
		268
		269	$CBOR::XS::FILTER{1347375694} = sub {
		270	my ($tag, $value);
		271
		272	"tag 1347375694 value $value"
		273	};
		274
162	=item $cbor_data = $cbor->encode ($perl_scalar)	275	=item $cbor_data = $cbor->encode ($perl_scalar)
163		276
164	Converts the given Perl data structure (a scalar value) to its CBOR	277	Converts the given Perl data structure (a scalar value) to its CBOR
165	representation.	278	representation.
166		279
…		…
234	C<Types:Serialiser::false> and C<Types::Serialiser::error>,	347	C<Types:Serialiser::false> and C<Types::Serialiser::error>,
235	respectively. They are overloaded to act almost exactly like the numbers	348	respectively. They are overloaded to act almost exactly like the numbers
236	C<1> and C<0> (for true and false) or to throw an exception on access (for	349	C<1> and C<0> (for true and false) or to throw an exception on access (for
237	error). See the L<Types::Serialiser> manpage for details.	350	error). See the L<Types::Serialiser> manpage for details.
238		351
239	=item CBOR tag 256 (perl object)	352	=item tagged values
240		353
241	The tag value C<256> (TODO: pending iana registration) will be used
242	to deserialise a Perl object serialised with C<FREEZE>. See L<OBJECT
243	SERIALISATION>, below, for details.
244
245	=item CBOR tag 55799 (magic header)
246
247	The tag 55799 is ignored (this tag implements the magic header).
248
249	=item other CBOR tags
250
251	Tagged items consists of a numeric tag and another CBOR value. Tags not	354	Tagged items consists of a numeric tag and another CBOR value.
252	handled internally are currently converted into a L<CBOR::XS::Tagged>
253	object, which is simply a blessed array reference consisting of the
254	numeric tag value followed by the (decoded) CBOR value.
255		355
256	In the future, support for user-supplied conversions might get added.	356	See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >>
		357	for details.
257		358
258	=item anything else	359	=item anything else
259		360
260	Anything else (e.g. unsupported simple values) will raise a decoding	361	Anything else (e.g. unsupported simple values) will raise a decoding
261	error.	362	error.
…		…
304	if you want.	405	if you want.
305		406
306	=item other blessed objects	407	=item other blessed objects
307		408
308	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See	409	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
309	L<OBJECT SERIALISATION>, below, for details.	410	L<TAG HANDLING AND EXTENSIONS> for specific classes handled by this
		411	module, and L<OBJECT SERIALISATION> for generic object serialisation.
310		412
311	=item simple scalars	413	=item simple scalars
312		414
313	TODO
314	Simple Perl scalars (any scalar that is not a reference) are the most	415	Simple Perl scalars (any scalar that is not a reference) are the most
315	difficult objects to encode: CBOR::XS will encode undefined scalars as	416	difficult objects to encode: CBOR::XS will encode undefined scalars as
316	CBOR null values, scalars that have last been used in a string context	417	CBOR null values, scalars that have last been used in a string context
317	before encoding as CBOR strings, and anything else as number value:	418	before encoding as CBOR strings, and anything else as number value:
318		419
…		…
455	=head1 MAGIC HEADER	556	=head1 MAGIC HEADER
456		557
457	There is no way to distinguish CBOR from other formats	558	There is no way to distinguish CBOR from other formats
458	programmatically. To make it easier to distinguish CBOR from other	559	programmatically. To make it easier to distinguish CBOR from other
459	formats, the CBOR specification has a special "magic string" that can be	560	formats, the CBOR specification has a special "magic string" that can be
460	prepended to any CBOR string without changing it's meaning.	561	prepended to any CBOR string without changing its meaning.
461		562
462	This string is available as C<$CBOR::XS::MAGIC>. This module does not	563	This string is available as C<$CBOR::XS::MAGIC>. This module does not
463	prepend this string tot he CBOR data it generates, but it will ignroe it	564	prepend this string to the CBOR data it generates, but it will ignore it
464	if present, so users can prepend this string as a "file type" indicator as	565	if present, so users can prepend this string as a "file type" indicator as
465	required.	566	required.
466		567
467		568
468	=head1 THE CBOR::XS::Tagged CLASS	569	=head1 THE CBOR::XS::Tagged CLASS
…		…
551	Wrap CBOR data in CBOR:	652	Wrap CBOR data in CBOR:
552		653
553	my $cbor_cbor = encode_cbor	654	my $cbor_cbor = encode_cbor
554	CBOR::XS::tag 24,	655	CBOR::XS::tag 24,
555	encode_cbor [1, 2, 3];	656	encode_cbor [1, 2, 3];
		657
		658	=head1 TAG HANDLING AND EXTENSIONS
		659
		660	This section describes how this module handles specific tagged values
		661	and extensions. If a tag is not mentioned here and no additional filters
		662	are provided for it, then the default handling applies (creating a
		663	CBOR::XS::Tagged object on decoding, and only encoding the tag when
		664	explicitly requested).
		665
		666	Tags not handled specifically are currently converted into a
		667	L<CBOR::XS::Tagged> object, which is simply a blessed array reference
		668	consisting of the numeric tag value followed by the (decoded) CBOR value.
		669
		670	Future versions of this module reserve the right to special case
		671	additional tags (such as base64url).
		672
		673	=head2 ENFORCED TAGS
		674
		675	These tags are always handled when decoding, and their handling cannot be
		676	overriden by the user.
		677
		678	=over 4
		679
		680	=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>)
		681
		682	These tags are automatically created (and decoded) for serialisable
		683	objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
		684	serialisation protocol). See L<OBJECT SERIALISATION> for details.
		685
		686	=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)
		687
		688	These tags are automatically decoded when encountered, resulting in
		689	shared values in the decoded object. They are only encoded, however, when
		690	C<allow_sharable> is enabled.
		691
		692	=item <unassigned>, <unassigned> (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>)
		693
		694	These tags are automatically decoded when encountered. They are only
		695	encoded, however, when C<allow_stringref> is enabled.
		696
		697	=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
		698
		699	This tag is automatically generated when a reference are encountered (with
		700	the exception of hash and array refernces). It is converted to a reference
		701	when decoding.
		702
		703	=item 55799 (self-describe CBOR, RFC 7049)
		704
		705	This value is not generated on encoding (unless explicitly requested by
		706	the user), and is simply ignored when decoding.
		707
		708	=back
		709
		710	=head2 NON-ENFORCED TAGS
		711
		712	These tags have default filters provided when decoding. Their handling can
		713	be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
		714	providing a custom C<filter> callback when decoding.
		715
		716	When they result in decoding into a specific Perl class, the module
		717	usually provides a corresponding C<TO_CBOR> method as well.
		718
		719	When any of these need to load additional modules that are not part of the
		720	perl core distribution (e.g. L<URI>), it is (currently) up to the user to
		721	provide these modules. The decoding usually fails with an exception if the
		722	required module cannot be loaded.
		723
		724	=over 4
		725
		726	=item 2, 3 (positive/negative bignum)
		727
		728	These tags are decoded into L<Math::BigInt> objects. The corresponding
		729	C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR
		730	integers, and others into positive/negative CBOR bignums.
		731
		732	=item 4, 5 (decimal fraction/bigfloat)
		733
		734	Both decimal fractions and bigfloats are decoded into L<Math::BigFloat>
		735	objects. The corresponding C<Math::BigFloat::TO_CBOR> method I<always>
		736	encodes into a decimal fraction.
		737
		738	CBOR cannot represent bigfloats with I<very> large exponents - conversion
		739	of such big float objects is undefined.
		740
		741	Also, NaN and infinities are not encoded properly.
		742
		743	=item 21, 22, 23 (expected later JSON conversion)
		744
		745	CBOR::XS is not a CBOR-to-JSON converter, and will simply ignore these
		746	tags.
		747
		748	=item 32 (URI)
		749
		750	These objects decode into L<URI> objects. The corresponding
		751	C<URI::TO_CBOR> method again results in a CBOR URI value.
		752
		753	=back
		754
		755	=cut
		756
		757	our %FILTER = (
		758	# 0 # rfc4287 datetime, utf-8
		759	# 1 # unix timestamp, any
		760
		761	2 => sub { # pos bigint
		762	require Math::BigInt;
		763	Math::BigInt->new ("0x" . unpack "H*", pop)
		764	},
		765
		766	3 => sub { # neg bigint
		767	require Math::BigInt;
		768	-Math::BigInt->new ("0x" . unpack "H*", pop)
		769	},
		770
		771	4 => sub { # decimal fraction, array
		772	require Math::BigFloat;
		773	Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
		774	},
		775
		776	5 => sub { # bigfloat, array
		777	require Math::BigFloat;
		778	scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
		779	},
		780
		781	21 => sub { pop }, # expected conversion to base64url encoding
		782	22 => sub { pop }, # expected conversion to base64 encoding
		783	23 => sub { pop }, # expected conversion to base16 encoding
		784
		785	# 24 # embedded cbor, byte string
		786
		787	32 => sub {
		788	require URI;
		789	URI->new (pop)
		790	},
		791
		792	# 33 # base64url rfc4648, utf-8
		793	# 34 # base64 rfc46484, utf-8
		794	# 35 # regex pcre/ecma262, utf-8
		795	# 36 # mime message rfc2045, utf-8
		796	);
		797
556		798
557	=head1 CBOR and JSON	799	=head1 CBOR and JSON
558		800
559	CBOR is supposed to implement a superset of the JSON data model, and is,	801	CBOR is supposed to implement a superset of the JSON data model, and is,
560	with some coercion, able to represent all JSON texts (something that other	802	with some coercion, able to represent all JSON texts (something that other
…		…
642	Please refrain from using rt.cpan.org or any other bug reporting	884	Please refrain from using rt.cpan.org or any other bug reporting
643	service. I put the contact address into my modules for a reason.	885	service. I put the contact address into my modules for a reason.
644		886
645	=cut	887	=cut
646		888
		889	our %FILTER = (
		890	# 0 # rfc4287 datetime, utf-8
		891	# 1 # unix timestamp, any
		892
		893	2 => sub { # pos bigint
		894	require Math::BigInt;
		895	Math::BigInt->new ("0x" . unpack "H*", pop)
		896	},
		897
		898	3 => sub { # neg bigint
		899	require Math::BigInt;
		900	-Math::BigInt->new ("0x" . unpack "H*", pop)
		901	},
		902
		903	4 => sub { # decimal fraction, array
		904	require Math::BigFloat;
		905	Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
		906	},
		907
		908	5 => sub { # bigfloat, array
		909	require Math::BigFloat;
		910	scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
		911	},
		912
		913	21 => sub { pop }, # expected conversion to base64url encoding
		914	22 => sub { pop }, # expected conversion to base64 encoding
		915	23 => sub { pop }, # expected conversion to base16 encoding
		916
		917	# 24 # embedded cbor, byte string
		918
		919	32 => sub {
		920	require URI;
		921	URI->new (pop)
		922	},
		923
		924	# 33 # base64url rfc4648, utf-8
		925	# 34 # base64 rfc46484, utf-8
		926	# 35 # regex pcre/ecma262, utf-8
		927	# 36 # mime message rfc2045, utf-8
		928	);
		929
		930	sub CBOR::XS::default_filter {
		931	&{ $FILTER{$_[0]} or return }
		932	}
		933
		934	sub URI::TO_CBOR {
		935	my $uri = $_[0]->as_string;
		936	utf8::upgrade $uri;
		937	CBOR::XS::tag 32, $uri
		938	}
		939
		940	sub Math::BigInt::TO_CBOR {
		941	if ($_[0] >= -2147483648 && $_[0] <= 2147483647) {
		942	$_[0]->numify
		943	} else {
		944	my $hex = substr $_[0]->as_hex, 2;
		945	$hex = "0$hex" if 1 & length $hex; # sigh
		946	CBOR::XS::tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
		947	}
		948	}
		949
		950	sub Math::BigFloat::TO_CBOR {
		951	my ($m, $e) = $_[0]->parts;
		952	CBOR::XS::tag 4, [$e->numify, $m]
		953	}
		954
647	XSLoader::load "CBOR::XS", $VERSION;	955	XSLoader::load "CBOR::XS", $VERSION;
648		956
649	=head1 SEE ALSO	957	=head1 SEE ALSO
650		958
651	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,	959	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.17 by root, Wed Oct 30 10:11:04 2013 UTC vs. Revision 1.24 by root, Fri Nov 22 16:18:59 2013 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.17 by root, Wed Oct 30 10:11:04 2013 UTC vs.
Revision 1.24 by root, Fri Nov 22 16:18:59 2013 UTC