[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.19 by root, Wed Nov 20 01:09:46 2013 UTC vs.
Revision 1.23 by root, Fri Nov 22 16:00:30 2013 UTC

…		…
26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string	26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
27	}	27	}
28		28
29	=head1 DESCRIPTION	29	=head1 DESCRIPTION
30		30
31	WARNING! This module is very new, and not very well tested (that's up to	31	WARNING! This module is very new, and not very well tested (that's up
32	you to do). Furthermore, details of the implementation might change freely	32	to you to do). Furthermore, details of the implementation might change
33	before version 1.0. And lastly, the object serialisation protocol depends	33	freely before version 1.0. And lastly, most extensions depend on an IANA
34	on a pending IANA assignment, and until that assignment is official, this	34	assignment, and until that assignment is official, this implementation is
35	implementation is not interoperable with other implementations (even	35	not interoperable with other implementations (even future versions of this
36	future versions of this module) until the assignment is done.	36	module) until the assignment is done.
37		37
38	You are still invited to try out CBOR, and this module.	38	You are still invited to try out CBOR, and this module.
39		39
40	This module converts Perl data structures to the Concise Binary Object	40	This module converts Perl data structures to the Concise Binary Object
41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation	41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
…		…
54	data, the worse L<Storable> performs in comparison.	54	data, the worse L<Storable> performs in comparison.
55		55
56	As for compactness, C<CBOR::XS> encoded data structures are usually about	56	As for compactness, C<CBOR::XS> encoded data structures are usually about
57	20% smaller than the same data encoded as (compact) JSON or L<Storable>.	57	20% smaller than the same data encoded as (compact) JSON or L<Storable>.
58		58
		59	In addition to the core CBOR data format, this module implements a number
		60	of extensions, to support cyclic and self-referencing data structures
		61	(see C<allow_sharing>), string deduplication (see C<allow_stringref>) and
		62	scalar references (always enabled).
		63
59	The primary goal of this module is to be I<correct> and the secondary goal	64	The primary goal of this module is to be I<correct> and the secondary goal
60	is to be I<fast>. To reach the latter goal it was written in C.	65	is to be I<fast>. To reach the latter goal it was written in C.
61		66
62	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	67	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
63	vice versa.	68	vice versa.
…		…
113	strings. All boolean flags described below are by default I<disabled>.	118	strings. All boolean flags described below are by default I<disabled>.
114		119
115	The mutators for flags all return the CBOR object again and thus calls can	120	The mutators for flags all return the CBOR object again and thus calls can
116	be chained:	121	be chained:
117		122
118	#TODO
119	my $cbor = CBOR::XS->new->encode ({a => [1,2]});	123	my $cbor = CBOR::XS->new->encode ({a => [1,2]});
120		124
121	=item $cbor = $cbor->max_depth ([$maximum_nesting_depth])	125	=item $cbor = $cbor->max_depth ([$maximum_nesting_depth])
122		126
123	=item $max_depth = $cbor->get_max_depth	127	=item $max_depth = $cbor->get_max_depth
…		…
171	exception when it encounters anything it cannot encode as CBOR.	175	exception when it encounters anything it cannot encode as CBOR.
172		176
173	This option does not affect C<decode> in any way, and it is recommended to	177	This option does not affect C<decode> in any way, and it is recommended to
174	leave it off unless you know your communications partner.	178	leave it off unless you know your communications partner.
175		179
176	=item $cbor = $cbor->allow_sharable ([$enable])	180	=item $cbor = $cbor->allow_sharing ([$enable])
177		181
178	=item $enabled = $cbor->get_allow_sharable	182	=item $enabled = $cbor->get_allow_sharing
179		183
180	If C<$enable> is true (or missing), then C<encode> will not double-encode	184	If C<$enable> is true (or missing), then C<encode> will not double-encode
181	values that have been seen before (e.g. when the same object, such as an	185	values that have been referenced before (e.g. when the same object, such
182	array, is referenced multiple times), but instead will emit a reference to	186	as an array, is referenced multiple times), but instead will emit a
183	the earlier value.	187	reference to the earlier value.
184		188
185	This means that such values will only be encoded once, and will not result	189	This means that such values will only be encoded once, and will not result
186	in a deep cloning of the value on decode, in decoders supporting the value	190	in a deep cloning of the value on decode, in decoders supporting the value
187	sharing extension.	191	sharing extension.
		192
		193	It is recommended to leave it off unless you know your
		194	communication partner supports the value sharing extensions to CBOR
		195	(http://cbor.schmorp.de/value-sharing).
188		196
189	Detecting shared values incurs a runtime overhead when values are encoded	197	Detecting shared values incurs a runtime overhead when values are encoded
190	that have a reference counter large than one, and might unnecessarily	198	that have a reference counter large than one, and might unnecessarily
191	increase the encoded size, as potentially shared values are encode as	199	increase the encoded size, as potentially shared values are encode as
192	sharable whether or not they are actually shared.	200	sharable whether or not they are actually shared.
193		201
194	At the moment, all shared values will be detected, even weird and unusual	202	At the moment, only targets of references can be shared (e.g. scalars,
195	cases, such as an array with multiple "copies" of the I<same> scalar,	203	arrays or hashes pointed to by a reference). Weirder constructs, such as
196	which are hard but not impossible to create in Perl (L<Storable> for	204	an array with multiple "copies" of the I<same> string, which are hard but
197	example doesn't handle these cases). If this turns out ot be a performance	205	not impossible to create in Perl, are not supported (this is the same as
198	issue then future versions might limit the shared value detection to	206	for L<Storable>).
199	references only.
200		207
201	If C<$enable> is false (the default), then C<encode> will encode	208	If C<$enable> is false (the default), then C<encode> will encode
202	exception when it encounters anything it cannot encode as CBOR.	209	exception when it encounters anything it cannot encode as CBOR.
203		210
204	This option does not affect C<decode> in any way - shared values and	211	This option does not affect C<decode> in any way - shared values and
205	references will always be decoded properly if present. It is recommended	212	references will always be decoded properly if present.
206	to leave it off unless you know your communications partner supports the	213
207	value sharing extensions to CBOR (http://cbor.schmorp.de/value-sharing).	214	=item $cbor = $cbor->allow_stringref ([$enable])
		215
		216	=item $enabled = $cbor->get_allow_stringref
		217
		218	If C<$enable> is true (or missing), then C<encode> will try not to encode
		219	the same string twice, but will instead encode a reference to the string
		220	instead. Depending on your data format. this can save a lot of space, but
		221	also results in a very large runtime overhead (expect encoding times to be
		222	2-4 times as high as without).
		223
		224	It is recommended to leave it off unless you know your
		225	communications partner supports the stringref extension to CBOR
		226	(http://cbor.schmorp.de/stringref).
		227
		228	If C<$enable> is false (the default), then C<encode> will encode
		229	exception when it encounters anything it cannot encode as CBOR.
		230
		231	This option does not affect C<decode> in any way - string references will
		232	always be decoded properly if present.
		233
		234	=item $cbor = $cbor->filter ([$cb->($tag, $value)])
		235
		236	=item $cb_or_undef = $cbor->get_filter
		237
		238	TODO
208		239
209	=item $cbor_data = $cbor->encode ($perl_scalar)	240	=item $cbor_data = $cbor->encode ($perl_scalar)
210		241
211	Converts the given Perl data structure (a scalar value) to its CBOR	242	Converts the given Perl data structure (a scalar value) to its CBOR
212	representation.	243	representation.
…		…
281	C<Types:Serialiser::false> and C<Types::Serialiser::error>,	312	C<Types:Serialiser::false> and C<Types::Serialiser::error>,
282	respectively. They are overloaded to act almost exactly like the numbers	313	respectively. They are overloaded to act almost exactly like the numbers
283	C<1> and C<0> (for true and false) or to throw an exception on access (for	314	C<1> and C<0> (for true and false) or to throw an exception on access (for
284	error). See the L<Types::Serialiser> manpage for details.	315	error). See the L<Types::Serialiser> manpage for details.
285		316
286	=item CBOR tag 256 (perl object)	317	=item tagged values
287		318
288	The tag value C<256> (TODO: pending iana registration) will be used
289	to deserialise a Perl object serialised with C<FREEZE>. See L<OBJECT
290	SERIALISATION>, below, for details.
291
292	=item CBOR tag 55799 (magic header)
293
294	The tag 55799 is ignored (this tag implements the magic header).
295
296	=item other CBOR tags
297
298	Tagged items consists of a numeric tag and another CBOR value. Tags not	319	Tagged items consists of a numeric tag and another CBOR value.
299	handled internally are currently converted into a L<CBOR::XS::Tagged>
300	object, which is simply a blessed array reference consisting of the
301	numeric tag value followed by the (decoded) CBOR value.
302		320
303	In the future, support for user-supplied conversions might get added.	321	See L<TAG HANDLING AND EXTENSIONS> and the description of C<< ->filter >>
		322	for details.
304		323
305	=item anything else	324	=item anything else
306		325
307	Anything else (e.g. unsupported simple values) will raise a decoding	326	Anything else (e.g. unsupported simple values) will raise a decoding
308	error.	327	error.
…		…
351	if you want.	370	if you want.
352		371
353	=item other blessed objects	372	=item other blessed objects
354		373
355	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See	374	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
356	L<OBJECT SERIALISATION>, below, for details.	375	L<TAG HANDLING AND EXTENSIONS> for specific classes handled by this
		376	module, and L<OBJECT SERIALISATION> for generic object serialisation.
357		377
358	=item simple scalars	378	=item simple scalars
359		379
360	TODO
361	Simple Perl scalars (any scalar that is not a reference) are the most	380	Simple Perl scalars (any scalar that is not a reference) are the most
362	difficult objects to encode: CBOR::XS will encode undefined scalars as	381	difficult objects to encode: CBOR::XS will encode undefined scalars as
363	CBOR null values, scalars that have last been used in a string context	382	CBOR null values, scalars that have last been used in a string context
364	before encoding as CBOR strings, and anything else as number value:	383	before encoding as CBOR strings, and anything else as number value:
365		384
…		…
601	CBOR::XS::tag 24,	620	CBOR::XS::tag 24,
602	encode_cbor [1, 2, 3];	621	encode_cbor [1, 2, 3];
603		622
604	=head1 TAG HANDLING AND EXTENSIONS	623	=head1 TAG HANDLING AND EXTENSIONS
605		624
606	This section describes how this module handles specific tagged values and	625	This section describes how this module handles specific tagged values
607	extensions. If a tag is not mentioned here, then the default handling	626	and extensions. If a tag is not mentioned here and no additional filters
		627	are provided for it, then the default handling applies (creating a
608	applies (creating a CBOR::XS::Tagged object on decoding, and only encoding	628	CBOR::XS::Tagged object on decoding, and only encoding the tag when
609	the tag when explicitly requested).	629	explicitly requested).
		630
		631	Tags not handled specifically are currently converted into a
		632	L<CBOR::XS::Tagged> object, which is simply a blessed array reference
		633	consisting of the numeric tag value followed by the (decoded) CBOR value.
610		634
611	Future versions of this module reserve the right to special case	635	Future versions of this module reserve the right to special case
612	additional tags (such as bigfloat or base64url).	636	additional tags (such as base64url).
		637
		638	=head2 ENFORCED TAGS
		639
		640	These tags are always handled when decoding, and their handling cannot be
		641	overriden by the user.
613		642
614	=over 4	643	=over 4
615		644
616	=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>)	645	=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>)
617		646
618	These tags are automatically created for serialisable objects using the	647	These tags are automatically created (and decoded) for serialisable
619	C<FREEZE/THAW> methods (the L<Types::Serialier> object serialisation	648	objects using the C<FREEZE/THAW> methods (the L<Types::Serialier> object
620	protocol).	649	serialisation protocol). See L<OBJECT SERIALISATION> for details.
621		650
622	=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)	651	=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)
623		652
624	These tags are automatically decoded when encountered, resulting in	653	These tags are automatically decoded when encountered, resulting in
625	shared values in the decoded object. They are only encoded, however, when	654	shared values in the decoded object. They are only encoded, however, when
626	C<allow_sharable> is enabled.	655	C<allow_sharable> is enabled.
627		656
		657	=item <unassigned>, <unassigned> (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>)
		658
		659	These tags are automatically decoded when encountered. They are only
		660	encoded, however, when C<allow_stringref> is enabled.
		661
628	=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)	662	=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
629		663
630	This tag is automatically generated when a reference are encountered (with	664	This tag is automatically generated when a reference are encountered (with
631	the exception of hash and array refernces). It is converted to a reference	665	the exception of hash and array refernces). It is converted to a reference
632	when decoding.	666	when decoding.
…		…
635		669
636	This value is not generated on encoding (unless explicitly requested by	670	This value is not generated on encoding (unless explicitly requested by
637	the user), and is simply ignored when decoding.	671	the user), and is simply ignored when decoding.
638		672
639	=back	673	=back
		674
		675	=head2 OPTIONAL TAGS
		676
		677	These tags have default filters provided when decoding. Their handling can
		678	be overriden by changing the C<%CBOR::XS::FILTER> entry for the tag, or by
		679	providing a custom C<filter> function when decoding.
		680
		681	When they result in decoding into a specific Perl class, the module
		682	usually provides a corresponding C<TO_CBOR> method as well.
		683
		684	When any of these need to load additional modules that are not part of the
		685	perl core distribution (e.g. L<URI>), it is (currently) up to the user to
		686	provide these modules. The decoding usually fails with an exception if the
		687	required module cannot be loaded.
		688
		689	=over 4
		690
		691	=item 2, 3 (positive/negative bignum)
		692
		693	These tags are decoded into L<Math::BigInt> objects. The corresponding
		694	C<Math::BigInt::TO_CBOR> method encodes "small" bigints into normal CBOR
		695	integers, and others into positive/negative CBOR bignums.
		696
		697	=item 4, 5 (decimal fraction/bigfloat)
		698
		699	Both decimal fractions and bigfloats are decoded into L<Math::BigFloat>
		700	objects. The corresponding C<Math::BigFloat::TO_CBOR> method I<always>
		701	encodes into a decimal fraction.
		702
		703	CBOR cannot represent bigfloats with I<very> large exponents - conversion
		704	of such big float objects is undefined.
		705
		706	Also, NaN and infinities are not encoded properly.
		707
		708	=item 21, 22, 23 (expected later JSON conversion)
		709
		710	CBOR::XS is not a CBOR-to-JSON converter, and will simply ignore these
		711	tags.
		712
		713	=item 32 (URI)
		714
		715	These objects decode into L<URI> objects. The corresponding
		716	C<URI::TO_CBOR> method again results in a CBOR URI value.
		717
		718	=back
		719
		720	=cut
		721
		722	our %FILTER = (
		723	# 0 # rfc4287 datetime, utf-8
		724	# 1 # unix timestamp, any
		725
		726	2 => sub { # pos bigint
		727	require Math::BigInt;
		728	Math::BigInt->new ("0x" . unpack "H*", pop)
		729	},
		730
		731	3 => sub { # neg bigint
		732	require Math::BigInt;
		733	-Math::BigInt->new ("0x" . unpack "H*", pop)
		734	},
		735
		736	4 => sub { # decimal fraction, array
		737	require Math::BigFloat;
		738	Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
		739	},
		740
		741	5 => sub { # bigfloat, array
		742	require Math::BigFloat;
		743	scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
		744	},
		745
		746	21 => sub { pop }, # expected conversion to base64url encoding
		747	22 => sub { pop }, # expected conversion to base64 encoding
		748	23 => sub { pop }, # expected conversion to base16 encoding
		749
		750	# 24 # embedded cbor, byte string
		751
		752	32 => sub {
		753	require URI;
		754	URI->new (pop)
		755	},
		756
		757	# 33 # base64url rfc4648, utf-8
		758	# 34 # base64 rfc46484, utf-8
		759	# 35 # regex pcre/ecma262, utf-8
		760	# 36 # mime message rfc2045, utf-8
		761	);
640		762
641		763
642	=head1 CBOR and JSON	764	=head1 CBOR and JSON
643		765
644	CBOR is supposed to implement a superset of the JSON data model, and is,	766	CBOR is supposed to implement a superset of the JSON data model, and is,
…		…
727	Please refrain from using rt.cpan.org or any other bug reporting	849	Please refrain from using rt.cpan.org or any other bug reporting
728	service. I put the contact address into my modules for a reason.	850	service. I put the contact address into my modules for a reason.
729		851
730	=cut	852	=cut
731		853
		854	our %FILTER = (
		855	# 0 # rfc4287 datetime, utf-8
		856	# 1 # unix timestamp, any
		857
		858	2 => sub { # pos bigint
		859	require Math::BigInt;
		860	Math::BigInt->new ("0x" . unpack "H*", pop)
		861	},
		862
		863	3 => sub { # neg bigint
		864	require Math::BigInt;
		865	-Math::BigInt->new ("0x" . unpack "H*", pop)
		866	},
		867
		868	4 => sub { # decimal fraction, array
		869	require Math::BigFloat;
		870	Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
		871	},
		872
		873	5 => sub { # bigfloat, array
		874	require Math::BigFloat;
		875	scalar Math::BigFloat->new ($_[1][1])->blsft ($_[1][0], 2)
		876	},
		877
		878	21 => sub { pop }, # expected conversion to base64url encoding
		879	22 => sub { pop }, # expected conversion to base64 encoding
		880	23 => sub { pop }, # expected conversion to base16 encoding
		881
		882	# 24 # embedded cbor, byte string
		883
		884	32 => sub {
		885	require URI;
		886	URI->new (pop)
		887	},
		888
		889	# 33 # base64url rfc4648, utf-8
		890	# 34 # base64 rfc46484, utf-8
		891	# 35 # regex pcre/ecma262, utf-8
		892	# 36 # mime message rfc2045, utf-8
		893	);
		894
		895	sub CBOR::XS::default_filter {
		896	&{ $FILTER{$_[0]} or return }
		897	}
		898
		899	sub URI::TO_CBOR {
		900	my $uri = $_[0]->as_string;
		901	utf8::upgrade $uri;
		902	CBOR::XS::tag 32, $uri
		903	}
		904
		905	sub Math::BigInt::TO_CBOR {
		906	if ($_[0] >= -2147483648 && $_[0] <= 2147483647) {
		907	$_[0]->numify
		908	} else {
		909	my $hex = substr $_[0]->as_hex, 2;
		910	$hex = "0$hex" if 1 & length $hex; # sigh
		911	CBOR::XS::tag $_[0] >= 0 ? 2 : 3, pack "H*", $hex
		912	}
		913	}
		914
		915	sub Math::BigFloat::TO_CBOR {
		916	my ($m, $e) = $_[0]->parts;
		917	CBOR::XS::tag 4, [$e->numify, $m]
		918	}
		919
732	XSLoader::load "CBOR::XS", $VERSION;	920	XSLoader::load "CBOR::XS", $VERSION;
733		921
734	=head1 SEE ALSO	922	=head1 SEE ALSO
735		923
736	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,	924	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.19 by root, Wed Nov 20 01:09:46 2013 UTC vs. Revision 1.23 by root, Fri Nov 22 16:00:30 2013 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.19 by root, Wed Nov 20 01:09:46 2013 UTC vs.
Revision 1.23 by root, Fri Nov 22 16:00:30 2013 UTC