[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.4 by root, Sat Oct 26 22:25:47 2013 UTC vs.
Revision 1.21 by root, Wed Nov 20 16:29:02 2013 UTC

…		…
12	$perl_value = decode_cbor $binary_cbor_data;	12	$perl_value = decode_cbor $binary_cbor_data;
13		13
14	# OO-interface	14	# OO-interface
15		15
16	$coder = CBOR::XS->new;	16	$coder = CBOR::XS->new;
17	#TODO	17	$binary_cbor_data = $coder->encode ($perl_value);
		18	$perl_value = $coder->decode ($binary_cbor_data);
		19
		20	# prefix decoding
		21
		22	my $many_cbor_strings = ...;
		23	while (length $many_cbor_strings) {
		24	my ($data, $length) = $cbor->decode_prefix ($many_cbor_strings);
		25	# data was decoded
		26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
		27	}
18		28
19	=head1 DESCRIPTION	29	=head1 DESCRIPTION
20		30
21	WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA AND	31	WARNING! This module is very new, and not very well tested (that's up
22	EAT YOUR CHILDREN!	32	to you to do). Furthermore, details of the implementation might change
		33	freely before version 1.0. And lastly, most extensions depend on an IANA
		34	assignment, and until that assignment is official, this implementation is
		35	not interoperable with other implementations (even future versions of this
		36	module) until the assignment is done.
23		37
24	This module converts Perl data structures to CBOR and vice versa. Its	38	You are still invited to try out CBOR, and this module.
		39
		40	This module converts Perl data structures to the Concise Binary Object
		41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
		42	format that aims to use a superset of the JSON data model, i.e. when you
		43	can represent something in JSON, you should be able to represent it in
		44	CBOR.
		45
		46	In short, CBOR is a faster and very compact binary alternative to JSON,
		47	with the added ability of supporting serialisation of Perl objects. (JSON
		48	often compresses better than CBOR though, so if you plan to compress the
		49	data later you might want to compare both formats first).
		50
		51	To give you a general idea about speed, with texts in the megabyte range,
		52	C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or
		53	L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the
		54	data, the worse L<Storable> performs in comparison.
		55
		56	As for compactness, C<CBOR::XS> encoded data structures are usually about
		57	20% smaller than the same data encoded as (compact) JSON or L<Storable>.
		58
		59	In addition to the core CBOR data format, this module implements a number
		60	of extensions, to support cyclic and self-referencing data structures
		61	(see C<allow_sharing>), string deduplication (see C<allow_stringref>) and
		62	scalar references (always enabled).
		63
25	primary goal is to be I<correct> and its secondary goal is to be	64	The primary goal of this module is to be I<correct> and the secondary goal
26	I<fast>. To reach the latter goal it was written in C.	65	is to be I<fast>. To reach the latter goal it was written in C.
27		66
28	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	67	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
29	vice versa.	68	vice versa.
30		69
31	=cut	70	=cut
32		71
33	package CBOR::XS;	72	package CBOR::XS;
34		73
35	use common::sense;	74	use common::sense;
36		75
37	our $VERSION = 0.02;	76	our $VERSION = 0.08;
38	our @ISA = qw(Exporter);	77	our @ISA = qw(Exporter);
39		78
40	our @EXPORT = qw(encode_cbor decode_cbor);	79	our @EXPORT = qw(encode_cbor decode_cbor);
41		80
42	use Exporter;	81	use Exporter;
43	use XSLoader;	82	use XSLoader;
		83
		84	use Types::Serialiser;
44		85
45	our $MAGIC = "\xd9\xd9\xf7";	86	our $MAGIC = "\xd9\xd9\xf7";
46		87
47	=head1 FUNCTIONAL INTERFACE	88	=head1 FUNCTIONAL INTERFACE
48		89
…		…
121	If no argument is given, the limit check will be deactivated (same as when	162	If no argument is given, the limit check will be deactivated (same as when
122	C<0> is specified).	163	C<0> is specified).
123		164
124	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.	165	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
125		166
		167	=item $cbor = $cbor->allow_unknown ([$enable])
		168
		169	=item $enabled = $cbor->get_allow_unknown
		170
		171	If C<$enable> is true (or missing), then C<encode> will I<not> throw an
		172	exception when it encounters values it cannot represent in CBOR (for
		173	example, filehandles) but instead will encode a CBOR C<error> value.
		174
		175	If C<$enable> is false (the default), then C<encode> will throw an
		176	exception when it encounters anything it cannot encode as CBOR.
		177
		178	This option does not affect C<decode> in any way, and it is recommended to
		179	leave it off unless you know your communications partner.
		180
		181	=item $cbor = $cbor->allow_sharing ([$enable])
		182
		183	=item $enabled = $cbor->get_allow_sharing
		184
		185	If C<$enable> is true (or missing), then C<encode> will not double-encode
		186	values that have been referenced before (e.g. when the same object, such
		187	as an array, is referenced multiple times), but instead will emit a
		188	reference to the earlier value.
		189
		190	This means that such values will only be encoded once, and will not result
		191	in a deep cloning of the value on decode, in decoders supporting the value
		192	sharing extension.
		193
		194	It is recommended to leave it off unless you know your
		195	communication partner supports the value sharing extensions to CBOR
		196	(http://cbor.schmorp.de/value-sharing).
		197
		198	Detecting shared values incurs a runtime overhead when values are encoded
		199	that have a reference counter large than one, and might unnecessarily
		200	increase the encoded size, as potentially shared values are encode as
		201	sharable whether or not they are actually shared.
		202
		203	At the moment, only targets of references can be shared (e.g. scalars,
		204	arrays or hashes pointed to by a reference). Weirder constructs, such as
		205	an array with multiple "copies" of the I<same> string, which are hard but
		206	not impossible to create in Perl, are not supported (this is the same as
		207	for L<Storable>).
		208
		209	If C<$enable> is false (the default), then C<encode> will encode
		210	exception when it encounters anything it cannot encode as CBOR.
		211
		212	This option does not affect C<decode> in any way - shared values and
		213	references will always be decoded properly if present.
		214
		215	=item $cbor = $cbor->allow_stringref ([$enable])
		216
		217	=item $enabled = $cbor->get_allow_stringref
		218
		219	If C<$enable> is true (or missing), then C<encode> will try not to encode
		220	the same string twice, but will instead encode a reference to the string
		221	instead. Depending on your data format. this can save a lot of space, but
		222	also results in a very large runtime overhead (expect encoding times to be
		223	2-4 times as high as without).
		224
		225	It is recommended to leave it off unless you know your
		226	communications partner supports the stringref extension to CBOR
		227	(http://cbor.schmorp.de/stringref).
		228
		229	If C<$enable> is false (the default), then C<encode> will encode
		230	exception when it encounters anything it cannot encode as CBOR.
		231
		232	This option does not affect C<decode> in any way - string references will
		233	always be decoded properly if present.
		234
126	=item $cbor_data = $cbor->encode ($perl_scalar)	235	=item $cbor_data = $cbor->encode ($perl_scalar)
127		236
128	Converts the given Perl data structure (a scalar value) to its CBOR	237	Converts the given Perl data structure (a scalar value) to its CBOR
129	representation.	238	representation.
130		239
…		…
186		295
187	CBOR arrays and CBOR maps will be converted into references to a Perl	296	CBOR arrays and CBOR maps will be converted into references to a Perl
188	array or hash, respectively. The keys of the map will be stringified	297	array or hash, respectively. The keys of the map will be stringified
189	during this process.	298	during this process.
190		299
		300	=item null
		301
		302	CBOR null becomes C<undef> in Perl.
		303
191	=item true, false	304	=item true, false, undefined
192		305
193	These CBOR values become C<CBOR::XS::true> and C<CBOR::XS::false>,	306	These CBOR values become C<Types:Serialiser::true>,
		307	C<Types:Serialiser::false> and C<Types::Serialiser::error>,
194	respectively. They are overloaded to act almost exactly like the numbers	308	respectively. They are overloaded to act almost exactly like the numbers
195	C<1> and C<0>. You can check whether a scalar is a CBOR boolean by using	309	C<1> and C<0> (for true and false) or to throw an exception on access (for
196	the C<CBOR::XS::is_bool> function.	310	error). See the L<Types::Serialiser> manpage for details.
197		311
198	=item null, undefined	312	=item CBOR tag 256 (perl object)
199		313
200	CBOR null and undefined values becomes C<undef> in Perl (in the future,	314	The tag value C<256> (TODO: pending iana registration) will be used
201	Undefined may raise an exception or something else).	315	to deserialise a Perl object serialised with C<FREEZE>. See L<OBJECT
		316	SERIALISATION>, below, for details.
202		317
203	=item tags	318	=item CBOR tag 55799 (magic header)
204		319
		320	The tag 55799 is ignored (this tag implements the magic header).
		321
		322	=item other CBOR tags
		323
205	Tagged items consists of a numeric tag and another CBOR value. The tag	324	Tagged items consists of a numeric tag and another CBOR value. Tags not
206	55799 is ignored (this tag implements the magic header).	325	handled internally are currently converted into a L<CBOR::XS::Tagged>
207
208	All other tags are currently converted into a L<CBOR::XS::Tagged> object,
209	which is simply a blessed array reference consistsing of the numeric tag	326	object, which is simply a blessed array reference consisting of the
210	value followed by the (decoded) BOR value.	327	numeric tag value followed by the (decoded) CBOR value.
		328
		329	In the future, support for user-supplied conversions might get added.
211		330
212	=item anything else	331	=item anything else
213		332
214	Anything else (e.g. unsupported simple values) will raise a decoding	333	Anything else (e.g. unsupported simple values) will raise a decoding
215	error.	334	error.
…		…
245	C<1>, which get turned into false and true in CBOR.	364	C<1>, which get turned into false and true in CBOR.
246		365
247	=item CBOR::XS::Tagged objects	366	=item CBOR::XS::Tagged objects
248		367
249	Objects of this type must be arrays consisting of a single C<[tag, value]>	368	Objects of this type must be arrays consisting of a single C<[tag, value]>
250	pair. The (numerical) tag will be encoded as a CBOR tag, the value will be	369	pair. The (numerical) tag will be encoded as a CBOR tag, the value will
251	encoded as appropriate for the value.	370	be encoded as appropriate for the value. You cna use C<CBOR::XS::tag> to
		371	create such objects.
252		372
253	=item CBOR::XS::true, CBOR::XS::false	373	=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
254		374
255	These special values become CBOR true and CBOR false values,	375	These special values become CBOR true, CBOR false and CBOR undefined
256	respectively. You can also use C<\1> and C<\0> directly if you want.	376	values, respectively. You can also use C<\1>, C<\0> and C<\undef> directly
		377	if you want.
257		378
258	=item blessed objects	379	=item other blessed objects
259		380
260	Other blessed objects currently need to have a C<TO_CBOR> method. It	381	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
261	will be called on every object that is being serialised, and must return	382	L<OBJECT SERIALISATION>, below, for details.
262	something that can be encoded in CBOR.
263		383
264	=item simple scalars	384	=item simple scalars
265		385
266	TODO	386	TODO
267	Simple Perl scalars (any scalar that is not a reference) are the most	387	Simple Perl scalars (any scalar that is not a reference) are the most
…		…
305	represent numerical values are supported, but might suffer loss of	425	represent numerical values are supported, but might suffer loss of
306	precision.	426	precision.
307		427
308	=back	428	=back
309		429
		430	=head2 OBJECT SERIALISATION
310		431
		432	This module knows two way to serialise a Perl object: The CBOR-specific
		433	way, and the generic way.
		434
		435	Whenever the encoder encounters a Perl object that it cnanot serialise
		436	directly (most of them), it will first look up the C<TO_CBOR> method on
		437	it.
		438
		439	If it has a C<TO_CBOR> method, it will call it with the object as only
		440	argument, and expects exactly one return value, which it will then
		441	substitute and encode it in the place of the object.
		442
		443	Otherwise, it will look up the C<FREEZE> method. If it exists, it will
		444	call it with the object as first argument, and the constant string C<CBOR>
		445	as the second argument, to distinguish it from other serialisers.
		446
		447	The C<FREEZE> method can return any number of values (i.e. zero or
		448	more). These will be encoded as CBOR perl object, together with the
		449	classname.
		450
		451	If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
		452	with an error.
		453
		454	Objects encoded via C<TO_CBOR> cannot be automatically decoded, but
		455	objects encoded via C<FREEZE> can be decoded using the following protocol:
		456
		457	When an encoded CBOR perl object is encountered by the decoder, it will
		458	look up the C<THAW> method, by using the stored classname, and will fail
		459	if the method cannot be found.
		460
		461	After the lookup it will call the C<THAW> method with the stored classname
		462	as first argument, the constant string C<CBOR> as second argument, and all
		463	values returned by C<FREEZE> as remaining arguments.
		464
		465	=head4 EXAMPLES
		466
		467	Here is an example C<TO_CBOR> method:
		468
		469	sub My::Object::TO_CBOR {
		470	my ($obj) = @_;
		471
		472	["this is a serialised My::Object object", $obj->{id}]
		473	}
		474
		475	When a C<My::Object> is encoded to CBOR, it will instead encode a simple
		476	array with two members: a string, and the "object id". Decoding this CBOR
		477	string will yield a normal perl array reference in place of the object.
		478
		479	A more useful and practical example would be a serialisation method for
		480	the URI module. CBOR has a custom tag value for URIs, namely 32:
		481
		482	sub URI::TO_CBOR {
		483	my ($self) = @_;
		484	my $uri = "$self"; # stringify uri
		485	utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
		486	CBOR::XS::tagged 32, "$_[0]"
		487	}
		488
		489	This will encode URIs as a UTF-8 string with tag 32, which indicates an
		490	URI.
		491
		492	Decoding such an URI will not (currently) give you an URI object, but
		493	instead a CBOR::XS::Tagged object with tag number 32 and the string -
		494	exactly what was returned by C<TO_CBOR>.
		495
		496	To serialise an object so it can automatically be deserialised, you need
		497	to use C<FREEZE> and C<THAW>. To take the URI module as example, this
		498	would be a possible implementation:
		499
		500	sub URI::FREEZE {
		501	my ($self, $serialiser) = @_;
		502	"$self" # encode url string
		503	}
		504
		505	sub URI::THAW {
		506	my ($class, $serialiser, $uri) = @_;
		507
		508	$class->new ($uri)
		509	}
		510
		511	Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For
		512	example, a C<FREEZE> method that returns "type", "id" and "variant" values
		513	would cause an invocation of C<THAW> with 5 arguments:
		514
		515	sub My::Object::FREEZE {
		516	my ($self, $serialiser) = @_;
		517
		518	($self->{type}, $self->{id}, $self->{variant})
		519	}
		520
		521	sub My::Object::THAW {
		522	my ($class, $serialiser, $type, $id, $variant) = @_;
		523
		524	$class-<new (type => $type, id => $id, variant => $variant)
		525	}
		526
		527
311	=head2 MAGIC HEADER	528	=head1 MAGIC HEADER
312		529
313	There is no way to distinguish CBOR from other formats	530	There is no way to distinguish CBOR from other formats
314	programmatically. To make it easier to distinguish CBOR from other	531	programmatically. To make it easier to distinguish CBOR from other
315	formats, the CBOR specification has a special "magic string" that can be	532	formats, the CBOR specification has a special "magic string" that can be
316	prepended to any CBOR string without changing it's meaning.	533	prepended to any CBOR string without changing its meaning.
317		534
318	This string is available as C<$CBOR::XS::MAGIC>. This module does not	535	This string is available as C<$CBOR::XS::MAGIC>. This module does not
319	prepend this string tot he CBOR data it generates, but it will ignroe it	536	prepend this string to the CBOR data it generates, but it will ignore it
320	if present, so users can prepend this string as a "file type" indicator as	537	if present, so users can prepend this string as a "file type" indicator as
321	required.	538	required.
322		539
323		540
		541	=head1 THE CBOR::XS::Tagged CLASS
		542
		543	CBOR has the concept of tagged values - any CBOR value can be tagged with
		544	a numeric 64 bit number, which are centrally administered.
		545
		546	C<CBOR::XS> handles a few tags internally when en- or decoding. You can
		547	also create tags yourself by encoding C<CBOR::XS::Tagged> objects, and the
		548	decoder will create C<CBOR::XS::Tagged> objects itself when it hits an
		549	unknown tag.
		550
		551	These objects are simply blessed array references - the first member of
		552	the array being the numerical tag, the second being the value.
		553
		554	You can interact with C<CBOR::XS::Tagged> objects in the following ways:
		555
		556	=over 4
		557
		558	=item $tagged = CBOR::XS::tag $tag, $value
		559
		560	This function(!) creates a new C<CBOR::XS::Tagged> object using the given
		561	C<$tag> (0..2**64-1) to tag the given C<$value> (which can be any Perl
		562	value that can be encoded in CBOR, including serialisable Perl objects and
		563	C<CBOR::XS::Tagged> objects).
		564
		565	=item $tagged->[0]
		566
		567	=item $tagged->[0] = $new_tag
		568
		569	=item $tag = $tagged->tag
		570
		571	=item $new_tag = $tagged->tag ($new_tag)
		572
		573	Access/mutate the tag.
		574
		575	=item $tagged->[1]
		576
		577	=item $tagged->[1] = $new_value
		578
		579	=item $value = $tagged->value
		580
		581	=item $new_value = $tagged->value ($new_value)
		582
		583	Access/mutate the tagged value.
		584
		585	=back
		586
		587	=cut
		588
		589	sub tag($$) {
		590	bless [@_], CBOR::XS::Tagged::;
		591	}
		592
		593	sub CBOR::XS::Tagged::tag {
		594	$_[0][0] = $_[1] if $#_;
		595	$_[0][0]
		596	}
		597
		598	sub CBOR::XS::Tagged::value {
		599	$_[0][1] = $_[1] if $#_;
		600	$_[0][1]
		601	}
		602
		603	=head2 EXAMPLES
		604
		605	Here are some examples of C<CBOR::XS::Tagged> uses to tag objects.
		606
		607	You can look up CBOR tag value and emanings in the IANA registry at
		608	L<http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml>.
		609
		610	Prepend a magic header (C<$CBOR::XS::MAGIC>):
		611
		612	my $cbor = encode_cbor CBOR::XS::tag 55799, $value;
		613	# same as:
		614	my $cbor = $CBOR::XS::MAGIC . encode_cbor $value;
		615
		616	Serialise some URIs and a regex in an array:
		617
		618	my $cbor = encode_cbor [
		619	(CBOR::XS::tag 32, "http://www.nethype.de/"),
		620	(CBOR::XS::tag 32, "http://software.schmorp.de/"),
		621	(CBOR::XS::tag 35, "^[Pp][Ee][Rr][lL]\$"),
		622	];
		623
		624	Wrap CBOR data in CBOR:
		625
		626	my $cbor_cbor = encode_cbor
		627	CBOR::XS::tag 24,
		628	encode_cbor [1, 2, 3];
		629
		630	=head1 TAG HANDLING AND EXTENSIONS
		631
		632	This section describes how this module handles specific tagged values and
		633	extensions. If a tag is not mentioned here, then the default handling
		634	applies (creating a CBOR::XS::Tagged object on decoding, and only encoding
		635	the tag when explicitly requested).
		636
		637	Future versions of this module reserve the right to special case
		638	additional tags (such as bigfloat or base64url).
		639
		640	=over 4
		641
		642	=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>)
		643
		644	These tags are automatically created for serialisable objects using the
		645	C<FREEZE/THAW> methods (the L<Types::Serialier> object serialisation
		646	protocol).
		647
		648	=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)
		649
		650	These tags are automatically decoded when encountered, resulting in
		651	shared values in the decoded object. They are only encoded, however, when
		652	C<allow_sharable> is enabled.
		653
		654	=item <unassigned>, <unassigned> (stringref-namespace, stringref, L <http://cbor.schmorp.de/stringref>)
		655
		656	These tags are automatically decoded when encountered. They are only
		657	encoded, however, when C<allow_stringref> is enabled.
		658
		659	=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
		660
		661	This tag is automatically generated when a reference are encountered (with
		662	the exception of hash and array refernces). It is converted to a reference
		663	when decoding.
		664
		665	=item 55799 (self-describe CBOR, RFC 7049)
		666
		667	This value is not generated on encoding (unless explicitly requested by
		668	the user), and is simply ignored when decoding.
		669
		670	=back
		671
		672
324	=head2 CBOR and JSON	673	=head1 CBOR and JSON
325		674
326	CBOR is supposed to implement a superset of the JSON data model, and is,	675	CBOR is supposed to implement a superset of the JSON data model, and is,
327	with some coercion, able to represent all JSON texts (something that other	676	with some coercion, able to represent all JSON texts (something that other
328	"binary JSON" formats such as BSON generally do not support).	677	"binary JSON" formats such as BSON generally do not support).
329		678
…		…
409	Please refrain from using rt.cpan.org or any other bug reporting	758	Please refrain from using rt.cpan.org or any other bug reporting
410	service. I put the contact address into my modules for a reason.	759	service. I put the contact address into my modules for a reason.
411		760
412	=cut	761	=cut
413		762
414	our $true = do { bless \(my $dummy = 1), "CBOR::XS::Boolean" };
415	our $false = do { bless \(my $dummy = 0), "CBOR::XS::Boolean" };
416
417	sub true() { $true }
418	sub false() { $false }
419
420	sub is_bool($) {
421	UNIVERSAL::isa $_[0], "CBOR::XS::Boolean"
422	# or UNIVERSAL::isa $_[0], "CBOR::Literal"
423	}
424
425	XSLoader::load "CBOR::XS", $VERSION;	763	XSLoader::load "CBOR::XS", $VERSION;
426
427	package CBOR::XS::Boolean;
428
429	use overload
430	"0+" => sub { ${$_[0]} },
431	"++" => sub { $_[0] = ${$_[0]} + 1 },
432	"--" => sub { $_[0] = ${$_[0]} - 1 },
433	fallback => 1;
434
435	1;
436		764
437	=head1 SEE ALSO	765	=head1 SEE ALSO
438		766
439	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,	767	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,
440	serialisation.	768	serialisation.
441		769
		770	The L<Types::Serialiser> module provides the data model for true, false
		771	and error values.
		772
442	=head1 AUTHOR	773	=head1 AUTHOR
443		774
444	Marc Lehmann <schmorp@schmorp.de>	775	Marc Lehmann <schmorp@schmorp.de>
445	http://home.schmorp.de/	776	http://home.schmorp.de/
446		777
447	=cut	778	=cut
448		779
		780	1
		781

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.4 by root, Sat Oct 26 22:25:47 2013 UTC vs. Revision 1.21 by root, Wed Nov 20 16:29:02 2013 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.4 by root, Sat Oct 26 22:25:47 2013 UTC vs.
Revision 1.21 by root, Wed Nov 20 16:29:02 2013 UTC