[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.20 by root, Wed Nov 20 02:03:08 2013 UTC vs.
Revision 1.24 by root, Fri Nov 22 16:18:59 2013 UTC

…		…
26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string	26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
27	}	27	}
28		28
29	=head1 DESCRIPTION	29	=head1 DESCRIPTION
30		30
31	WARNING! This module is very new, and not very well tested (that's up to	31	WARNING! This module is very new, and not very well tested (that's up
32	you to do). Furthermore, details of the implementation might change freely	32	to you to do). Furthermore, details of the implementation might change
33	before version 1.0. And lastly, the object serialisation protocol depends	33	freely before version 1.0. And lastly, most extensions depend on an IANA
34	on a pending IANA assignment, and until that assignment is official, this	34	assignment, and until that assignment is official, this implementation is
35	implementation is not interoperable with other implementations (even	35	not interoperable with other implementations (even future versions of this
36	future versions of this module) until the assignment is done.	36	module) until the assignment is done.
37		37
38	You are still invited to try out CBOR, and this module.	38	You are still invited to try out CBOR, and this module.
39		39
40	This module converts Perl data structures to the Concise Binary Object	40	This module converts Perl data structures to the Concise Binary Object
41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation	41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
…		…
54	data, the worse L<Storable> performs in comparison.	54	data, the worse L<Storable> performs in comparison.
55		55
56	As for compactness, C<CBOR::XS> encoded data structures are usually about	56	As for compactness, C<CBOR::XS> encoded data structures are usually about
57	20% smaller than the same data encoded as (compact) JSON or L<Storable>.	57	20% smaller than the same data encoded as (compact) JSON or L<Storable>.
58		58
		59	In addition to the core CBOR data format, this module implements a number
		60	of extensions, to support cyclic and self-referencing data structures
		61	(see C<allow_sharing>), string deduplication (see C<allow_stringref>) and
		62	scalar references (always enabled).
		63
59	The primary goal of this module is to be I<correct> and the secondary goal	64	The primary goal of this module is to be I<correct> and the secondary goal
60	is to be I<fast>. To reach the latter goal it was written in C.	65	is to be I<fast>. To reach the latter goal it was written in C.
61		66
62	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	67	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
63	vice versa.	68	vice versa.
…		…
66		71
67	package CBOR::XS;	72	package CBOR::XS;
68		73
69	use common::sense;	74	use common::sense;
70		75
71	our $VERSION = 0.08;	76	our $VERSION = 0.09;
72	our @ISA = qw(Exporter);	77	our @ISA = qw(Exporter);
73		78
74	our @EXPORT = qw(encode_cbor decode_cbor);	79	our @EXPORT = qw(encode_cbor decode_cbor);
75		80
76	use Exporter;	81	use Exporter;
…		…
113	strings. All boolean flags described below are by default I<disabled>.	118	strings. All boolean flags described below are by default I<disabled>.
114		119
115	The mutators for flags all return the CBOR object again and thus calls can	120	The mutators for flags all return the CBOR object again and thus calls can
116	be chained:	121	be chained:
117		122
118	#TODO
119	my $cbor = CBOR::XS->new->encode ({a => [1,2]});	123	my $cbor = CBOR::XS->new->encode ({a => [1,2]});
120		124
121	=item $cbor = $cbor->max_depth ([$maximum_nesting_depth])	125	=item $cbor = $cbor->max_depth ([$maximum_nesting_depth])
122		126
123	=item $max_depth = $cbor->get_max_depth	127	=item $max_depth = $cbor->get_max_depth
…		…
184		188
185	This means that such values will only be encoded once, and will not result	189	This means that such values will only be encoded once, and will not result
186	in a deep cloning of the value on decode, in decoders supporting the value	190	in a deep cloning of the value on decode, in decoders supporting the value
187	sharing extension.	191	sharing extension.
188		192
		193	It is recommended to leave it off unless you know your
		194	communication partner supports the value sharing extensions to CBOR
		195	(http://cbor.schmorp.de/value-sharing).
		196
189	Detecting shared values incurs a runtime overhead when values are encoded	197	Detecting shared values incurs a runtime overhead when values are encoded
190	that have a reference counter large than one, and might unnecessarily	198	that have a reference counter large than one, and might unnecessarily
191	increase the encoded size, as potentially shared values are encode as	199	increase the encoded size, as potentially shared values are encode as
192	sharable whether or not they are actually shared.	200	sharable whether or not they are actually shared.
193		201
…		…
199		207
200	If C<$enable> is false (the default), then C<encode> will encode	208	If C<$enable> is false (the default), then C<encode> will encode
201	exception when it encounters anything it cannot encode as CBOR.	209	exception when it encounters anything it cannot encode as CBOR.
202		210
203	This option does not affect C<decode> in any way - shared values and	211	This option does not affect C<decode> in any way - shared values and
204	references will always be decoded properly if present. It is recommended	212	references will always be decoded properly if present.
205	to leave it off unless you know your communications partner supports the	213
206	value sharing extensions to CBOR (http://cbor.schmorp.de/value-sharing).	214	=item $cbor = $cbor->allow_stringref ([$enable])
		215
		216	=item $enabled = $cbor->get_allow_stringref
		217
		218	If C<$enable> is true (or missing), then C<encode> will try not to encode
		219	the same string twice, but will instead encode a reference to the string
		220	instead. Depending on your data format. this can save a lot of space, but
		221	also results in a very large runtime overhead (expect encoding times to be
		222	2-4 times as high as without).
		223
		224	It is recommended to leave it off unless you know your
		225	communications partner supports the stringref extension to CBOR
		226	(http://cbor.schmorp.de/stringref).
		227
		228	If C<$enable> is false (the default), then C<encode> will encode
		229	exception when it encounters anything it cannot encode as CBOR.
		230
		231	This option does not affect C<decode> in any way - string references will
		232	always be decoded properly if present.
		233
		234	=item $cbor = $cbor->filter ([$cb->($tag, $value)])
		235
		236	=item $cb_or_undef = $cbor->get_filter
		237
		238	Sets or replaces the tagged value decoding filter (when C<$cb> is
		239	specified) or clears the filter (if no argument or C<undef> is provided).
		240
		241	The filter callback is called only during decoding, when a non-enforced
		242	tagged value has been decoded (see L<TAG HANDLING AND EXTENSIONS> for a
		243	list of enforced tags). For specific tags, it's often better to provide a
		244	default converter using the C<%CBOR::XS::FILTER> hash (see below).
		245
		246	The first argument is the numerical tag, the second is the (decoded) value
		247	that has been tagged.
		248
		249	The filter function should return either exactly one value, which will
		250	replace the tagged value in the decoded data structure, or no values,
		251	which will result in default handling, which currently means the decoder
		252	creates a C<CBOR::XS::Tagged> object to hold the tag and the value.
		253
		254	When the filter is cleared (the default state), the default filter
		255	function, C<CBOR::XS::default_filter>, is used. This function simply looks
		256	up the tag in the C<%CBOR::XS::FILTER> hash. If an entry exists it must be
		257	a code reference that is called with tag and value, and is responsible for
		258	decoding the value. If no entry exists, it returns no values.
		259
		260	Example: decode all tags not handled internally into CBOR::XS::Tagged
		261	objects, with no other special handling (useful when working with
		262	potentially "unsafe" CBOR data).
		263
		264	CBOR::XS->new->filter (sub { })->decode ($cbor_data);
		265
		266	Example: provide a global filter for tag 1347375694, converting the value
		267	into some string form.
		268
		269	$CBOR::XS::FILTER{1347375694} = sub {
		270	my ($tag, $value);
		271
		272	"tag 1347375694 value $value"
		273	};
207		274
208	=item $cbor_data = $cbor->encode ($perl_scalar)	275	=item $cbor_data = $cbor->encode ($perl_scalar)
209		276
210	Converts the given Perl data structure (a scalar value) to its CBOR	277	Converts the given Perl data structure (a scalar value) to its CBOR
211	representation.	278	representation.
…		…
280	C<Types:Serialiser::false> and C<Types::Serialiser::error>,	347	C<Types:Serialiser::false> and C<Types::Serialiser::error>,
281	respectively. They are overloaded to act almost exactly like the numbers	348	respectively. They are overloaded to act almost exactly like the numbers
282	C<1> and C<0> (for true and false) or to throw an exception on access (for	349	C<1> and C<0> (for true and false) or to throw an exception on access (for
283	error). See the L<Types::Serialiser> manpage for details.	350	error). See the L<Types::Serialiser> manpage for details.
284		351
285	=item CBOR tag 256 (perl object)	352	=item tagged values
286		353
287	The tag value C<256> (TODO: pending iana registration) will be used
288	to deserialise a Perl object serialised with C<FREEZE>. See L<OBJECT
289	SERIALISATION>, below, for details.
290
291	=item CBOR tag 55799 (magic header)
292
293	The tag 55799 is ignored (this tag implements the magic header).
294
295	=item other CBOR tags
296
297	Tagged items consists of a numeric tag and another CBOR value. Tags not	354	Tagged items consists of a numeric tag and another CBOR value.
298	handled internally are currently converted into a L<CBOR::XS::Tagged>
299	object, which is simply a blessed array reference consisting of the
300	numeric tag value followed by the (decoded) CBOR value.
301		355
302	In the future, support for user-supplied conversions might get added.	356	See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >>
		357	for details.
303		358
304	=item anything else	359	=item anything else
305		360
306	Anything else (e.g. unsupported simple values) will raise a decoding	361	Anything else (e.g. unsupported simple values) will raise a decoding
307	error.	362	error.
…		…
350	if you want.	405	if you want.
351		406
352	=item other blessed objects	407	=item other blessed objects
353		408
354	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See	409	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
355	L<OBJECT SERIALISATION>, below, for details.	410	L<TAG HANDLING AND EXTENSIONS> for specific classes handled by this
		411	module, and L<OBJECT SERIALISATION> for generic object serialisation.
356		412
357	=item simple scalars	413	=item simple scalars
358		414
359	TODO
360	Simple Perl scalars (any scalar that is not a reference) are the most	415	Simple Perl scalars (any scalar that is not a reference) are the most
361	difficult objects to encode: CBOR::XS will encode undefined scalars as	416	difficult objects to encode: CBOR::XS will encode undefined scalars as
362	CBOR null values, scalars that have last been used in a string context	417	CBOR null values, scalars that have last been used in a string context
363	before encoding as CBOR strings, and anything else as number value:	418	before encoding as CBOR strings, and anything else as number value:
364		419
…		…
600	CBOR::XS::tag 24,	655	CBOR::XS::tag 24,
601	encode_cbor [1, 2, 3];	656	encode_cbor [1, 2, 3];
602		657
603	=head1 TAG HANDLING AND EXTENSIONS	658	=head1 TAG HANDLING AND EXTENSIONS
604		659
605	This section describes how this module handles specific tagged values and	660	This section describes how this module handles specific tagged values
606	extensions. If a tag is not mentioned here, then the default handling	661	and extensions. If a tag is not mentioned here and no additional filters
		662	are provided for it, then the default handling applies (creating a
607	applies (creating a CBOR::XS::Tagged object on decoding, and only encoding	663	CBOR::XS::Tagged object on decoding, and only encoding the tag when
608	the tag when explicitly requested).	664	explicitly requested).
		665
		666	Tags not handled specifically are currently converted into a
		667	L<CBOR::XS::Tagged> object, which is simply a blessed array reference
		668	consisting of the numeric tag value followed by the (decoded) CBOR value.
609		669
610	Future versions of this module reserve the right to special case	670	Future versions of this module reserve the right to special case
611	additional tags (such as bigfloat or base64url).	671	additional tags (such as base64url).
		672
		673	=head2 ENFORCED TAGS
		674
		675	These tags are always handled when decoding, and their handling cannot be
		676	overriden by the user.
612		677
613	=over 4	678	=over 4
614		679
615	=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>)	680	=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>)
616		681
617	These tags are automatically created for serialisable objects using the	682	These tags are automatically created (and decoded) for serialisable
618	C<FREEZE/THAW> methods (the L<Types::Serialier> object serialisation	683	objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
619	protocol).	684	serialisation protocol). See L<OBJECT SERIALISATION> for details.
620		685
621	=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)	686	=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)
622		687
623	These tags are automatically decoded when encountered, resulting in	688	These tags are automatically decoded when encountered, resulting in
624	shared values in the decoded object. They are only encoded, however, when	689	shared values in the decoded object. They are only encoded, however, when
625	C<allow_sharable> is enabled.	690	C<allow_sharable> is enabled.
626		691
		692	=item <unassigned>, <unassigned> (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>)
		693
		694	These tags are automatically decoded when encountered. They are only
		695	encoded, however, when C<allow_stringref> is enabled.
		696
627	=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)	697	=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
628		698
629	This tag is automatically generated when a reference are encountered (with	699	This tag is automatically generated when a reference are encountered (with
630	the exception of hash and array refernces). It is converted to a reference	700	the exception of hash and array refernces). It is converted to a reference
631	when decoding.	701	when decoding.
…		…
634		704
635	This value is not generated on encoding (unless explicitly requested by	705	This value is not generated on encoding (unless explicitly requested by
636	the user), and is simply ignored when decoding.	706	the user), and is simply ignored when decoding.
637		707
638	=back	708	=back
		709
		710	=head2 NON-ENFORCED TAGS
		711
		712	These tags have default filters provided when decoding. Their handling can
		713	be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
		714	providing a custom C<filter> callback when decoding.
		715
		716	When they result in decoding into a specific Perl class, the module
		717	usually provides a corresponding C<TO_CBOR> method as well.
		718
		719	When any of these need to load additional modules that are not part of the
		720	perl core distribution (e.g. L<URI>), it is (currently) up to the user to
		721	provide these modules. The decoding usually fails with an exception if the
		722	required module cannot be loaded.
		723
		724	=over 4
		725
		726	=item 2, 3 (positive/negative bignum)
		727
		728	These tags are decoded into L<Math::BigInt> objects. The corresponding
		729	C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR
		730	integers, and others into positive/negative CBOR bignums.
		731
		732	=item 4, 5 (decimal fraction/bigfloat)
		733
		734	Both decimal fractions and bigfloats are decoded into L<Math::BigFloat>
		735	objects. The corresponding C<Math::BigFloat::TO_CBOR> method I<always>
		736	encodes into a decimal fraction.
		737
		738	CBOR cannot represent bigfloats with I<very> large exponents - conversion
		739	of such big float objects is undefined.
		740
		741	Also, NaN and infinities are not encoded properly.
		742
		743	=item 21, 22, 23 (expected later JSON conversion)
		744
		745	CBOR::XS is not a CBOR-to-JSON converter, and will simply ignore these
		746	tags.
		747
		748	=item 32 (URI)
		749
		750	These objects decode into L<URI> objects. The corresponding
		751	C<URI::TO_CBOR> method again results in a CBOR URI value.
		752
		753	=back
		754
		755	=cut
		756
		757	our %FILTER = (
		758	# 0 # rfc4287 datetime, utf-8
		759	# 1 # unix timestamp, any
		760
		761	2 => sub { # pos bigint
		762	require Math::BigInt;
		763	Math::BigInt->new ("0x" . unpack "H*", pop)
		764	},
		765
		766	3 => sub { # neg bigint
		767	require Math::BigInt;
		768	-Math::BigInt->new ("0x" . unpack "H*", pop)
		769	},
		770
		771	4 => sub { # decimal fraction, array
		772	require Math::BigFloat;
		773	Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
		774	},
		775
		776	5 => sub { # bigfloat, array
		777	require Math::BigFloat;
		778	scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
		779	},
		780
		781	21 => sub { pop }, # expected conversion to base64url encoding
		782	22 => sub { pop }, # expected conversion to base64 encoding
		783	23 => sub { pop }, # expected conversion to base16 encoding
		784
		785	# 24 # embedded cbor, byte string
		786
		787	32 => sub {
		788	require URI;
		789	URI->new (pop)
		790	},
		791
		792	# 33 # base64url rfc4648, utf-8
		793	# 34 # base64 rfc46484, utf-8
		794	# 35 # regex pcre/ecma262, utf-8
		795	# 36 # mime message rfc2045, utf-8
		796	);
639		797
640		798
641	=head1 CBOR and JSON	799	=head1 CBOR and JSON
642		800
643	CBOR is supposed to implement a superset of the JSON data model, and is,	801	CBOR is supposed to implement a superset of the JSON data model, and is,
…		…
726	Please refrain from using rt.cpan.org or any other bug reporting	884	Please refrain from using rt.cpan.org or any other bug reporting
727	service. I put the contact address into my modules for a reason.	885	service. I put the contact address into my modules for a reason.
728		886
729	=cut	887	=cut
730		888
		889	our %FILTER = (
		890	# 0 # rfc4287 datetime, utf-8
		891	# 1 # unix timestamp, any
		892
		893	2 => sub { # pos bigint
		894	require Math::BigInt;
		895	Math::BigInt->new ("0x" . unpack "H*", pop)
		896	},
		897
		898	3 => sub { # neg bigint
		899	require Math::BigInt;
		900	-Math::BigInt->new ("0x" . unpack "H*", pop)
		901	},
		902
		903	4 => sub { # decimal fraction, array
		904	require Math::BigFloat;
		905	Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
		906	},
		907
		908	5 => sub { # bigfloat, array
		909	require Math::BigFloat;
		910	scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
		911	},
		912
		913	21 => sub { pop }, # expected conversion to base64url encoding
		914	22 => sub { pop }, # expected conversion to base64 encoding
		915	23 => sub { pop }, # expected conversion to base16 encoding
		916
		917	# 24 # embedded cbor, byte string
		918
		919	32 => sub {
		920	require URI;
		921	URI->new (pop)
		922	},
		923
		924	# 33 # base64url rfc4648, utf-8
		925	# 34 # base64 rfc46484, utf-8
		926	# 35 # regex pcre/ecma262, utf-8
		927	# 36 # mime message rfc2045, utf-8
		928	);
		929
		930	sub CBOR::XS::default_filter {
		931	&{ $FILTER{$_[0]} or return }
		932	}
		933
		934	sub URI::TO_CBOR {
		935	my $uri = $_[0]->as_string;
		936	utf8::upgrade $uri;
		937	CBOR::XS::tag 32, $uri
		938	}
		939
		940	sub Math::BigInt::TO_CBOR {
		941	if ($_[0] >= -2147483648 && $_[0] <= 2147483647) {
		942	$_[0]->numify
		943	} else {
		944	my $hex = substr $_[0]->as_hex, 2;
		945	$hex = "0$hex" if 1 & length $hex; # sigh
		946	CBOR::XS::tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
		947	}
		948	}
		949
		950	sub Math::BigFloat::TO_CBOR {
		951	my ($m, $e) = $_[0]->parts;
		952	CBOR::XS::tag 4, [$e->numify, $m]
		953	}
		954
731	XSLoader::load "CBOR::XS", $VERSION;	955	XSLoader::load "CBOR::XS", $VERSION;
732		956
733	=head1 SEE ALSO	957	=head1 SEE ALSO
734		958
735	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,	959	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.20 by root, Wed Nov 20 02:03:08 2013 UTC vs. Revision 1.24 by root, Fri Nov 22 16:18:59 2013 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.20 by root, Wed Nov 20 02:03:08 2013 UTC vs.
Revision 1.24 by root, Fri Nov 22 16:18:59 2013 UTC