[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.5 by root, Sat Oct 26 23:02:55 2013 UTC vs.
Revision 1.19 by root, Wed Nov 20 01:09:46 2013 UTC

…		…
12	$perl_value = decode_cbor $binary_cbor_data;	12	$perl_value = decode_cbor $binary_cbor_data;
13		13
14	# OO-interface	14	# OO-interface
15		15
16	$coder = CBOR::XS->new;	16	$coder = CBOR::XS->new;
17	#TODO	17	$binary_cbor_data = $coder->encode ($perl_value);
		18	$perl_value = $coder->decode ($binary_cbor_data);
		19
		20	# prefix decoding
		21
		22	my $many_cbor_strings = ...;
		23	while (length $many_cbor_strings) {
		24	my ($data, $length) = $cbor->decode_prefix ($many_cbor_strings);
		25	# data was decoded
		26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
		27	}
18		28
19	=head1 DESCRIPTION	29	=head1 DESCRIPTION
20		30
21	WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA	31	WARNING! This module is very new, and not very well tested (that's up to
22	AND EAT YOUR CHILDREN! (Actually, apart from being untested and a bit	32	you to do). Furthermore, details of the implementation might change freely
23	feature-limited, it might already be useful).	33	before version 1.0. And lastly, the object serialisation protocol depends
		34	on a pending IANA assignment, and until that assignment is official, this
		35	implementation is not interoperable with other implementations (even
		36	future versions of this module) until the assignment is done.
		37
		38	You are still invited to try out CBOR, and this module.
24		39
25	This module converts Perl data structures to the Concise Binary Object	40	This module converts Perl data structures to the Concise Binary Object
26	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation	41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
27	format that aims to use a superset of the JSON data model, i.e. when you	42	format that aims to use a superset of the JSON data model, i.e. when you
28	can represent something in JSON, you should be able to represent it in	43	can represent something in JSON, you should be able to represent it in
29	CBOR.	44	CBOR.
30		45
31	This makes it a faster and more compact binary alternative to JSON.	46	In short, CBOR is a faster and very compact binary alternative to JSON,
		47	with the added ability of supporting serialisation of Perl objects. (JSON
		48	often compresses better than CBOR though, so if you plan to compress the
		49	data later you might want to compare both formats first).
		50
		51	To give you a general idea about speed, with texts in the megabyte range,
		52	C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or
		53	L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the
		54	data, the worse L<Storable> performs in comparison.
		55
		56	As for compactness, C<CBOR::XS> encoded data structures are usually about
		57	20% smaller than the same data encoded as (compact) JSON or L<Storable>.
32		58
33	The primary goal of this module is to be I<correct> and the secondary goal	59	The primary goal of this module is to be I<correct> and the secondary goal
34	is to be I<fast>. To reach the latter goal it was written in C.	60	is to be I<fast>. To reach the latter goal it was written in C.
35		61
36	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	62	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
…		…
40		66
41	package CBOR::XS;	67	package CBOR::XS;
42		68
43	use common::sense;	69	use common::sense;
44		70
45	our $VERSION = 0.03;	71	our $VERSION = 0.08;
46	our @ISA = qw(Exporter);	72	our @ISA = qw(Exporter);
47		73
48	our @EXPORT = qw(encode_cbor decode_cbor);	74	our @EXPORT = qw(encode_cbor decode_cbor);
49		75
50	use Exporter;	76	use Exporter;
51	use XSLoader;	77	use XSLoader;
		78
		79	use Types::Serialiser;
52		80
53	our $MAGIC = "\xd9\xd9\xf7";	81	our $MAGIC = "\xd9\xd9\xf7";
54		82
55	=head1 FUNCTIONAL INTERFACE	83	=head1 FUNCTIONAL INTERFACE
56		84
…		…
129	If no argument is given, the limit check will be deactivated (same as when	157	If no argument is given, the limit check will be deactivated (same as when
130	C<0> is specified).	158	C<0> is specified).
131		159
132	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.	160	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
133		161
		162	=item $cbor = $cbor->allow_unknown ([$enable])
		163
		164	=item $enabled = $cbor->get_allow_unknown
		165
		166	If C<$enable> is true (or missing), then C<encode> will I<not> throw an
		167	exception when it encounters values it cannot represent in CBOR (for
		168	example, filehandles) but instead will encode a CBOR C<error> value.
		169
		170	If C<$enable> is false (the default), then C<encode> will throw an
		171	exception when it encounters anything it cannot encode as CBOR.
		172
		173	This option does not affect C<decode> in any way, and it is recommended to
		174	leave it off unless you know your communications partner.
		175
		176	=item $cbor = $cbor->allow_sharable ([$enable])
		177
		178	=item $enabled = $cbor->get_allow_sharable
		179
		180	If C<$enable> is true (or missing), then C<encode> will not double-encode
		181	values that have been seen before (e.g. when the same object, such as an
		182	array, is referenced multiple times), but instead will emit a reference to
		183	the earlier value.
		184
		185	This means that such values will only be encoded once, and will not result
		186	in a deep cloning of the value on decode, in decoders supporting the value
		187	sharing extension.
		188
		189	Detecting shared values incurs a runtime overhead when values are encoded
		190	that have a reference counter large than one, and might unnecessarily
		191	increase the encoded size, as potentially shared values are encode as
		192	sharable whether or not they are actually shared.
		193
		194	At the moment, all shared values will be detected, even weird and unusual
		195	cases, such as an array with multiple "copies" of the I<same> scalar,
		196	which are hard but not impossible to create in Perl (L<Storable> for
		197	example doesn't handle these cases). If this turns out ot be a performance
		198	issue then future versions might limit the shared value detection to
		199	references only.
		200
		201	If C<$enable> is false (the default), then C<encode> will encode
		202	exception when it encounters anything it cannot encode as CBOR.
		203
		204	This option does not affect C<decode> in any way - shared values and
		205	references will always be decoded properly if present. It is recommended
		206	to leave it off unless you know your communications partner supports the
		207	value sharing extensions to CBOR (http://cbor.schmorp.de/value-sharing).
		208
134	=item $cbor_data = $cbor->encode ($perl_scalar)	209	=item $cbor_data = $cbor->encode ($perl_scalar)
135		210
136	Converts the given Perl data structure (a scalar value) to its CBOR	211	Converts the given Perl data structure (a scalar value) to its CBOR
137	representation.	212	representation.
138		213
…		…
194		269
195	CBOR arrays and CBOR maps will be converted into references to a Perl	270	CBOR arrays and CBOR maps will be converted into references to a Perl
196	array or hash, respectively. The keys of the map will be stringified	271	array or hash, respectively. The keys of the map will be stringified
197	during this process.	272	during this process.
198		273
		274	=item null
		275
		276	CBOR null becomes C<undef> in Perl.
		277
199	=item true, false	278	=item true, false, undefined
200		279
201	These CBOR values become C<CBOR::XS::true> and C<CBOR::XS::false>,	280	These CBOR values become C<Types:Serialiser::true>,
		281	C<Types:Serialiser::false> and C<Types::Serialiser::error>,
202	respectively. They are overloaded to act almost exactly like the numbers	282	respectively. They are overloaded to act almost exactly like the numbers
203	C<1> and C<0>. You can check whether a scalar is a CBOR boolean by using	283	C<1> and C<0> (for true and false) or to throw an exception on access (for
204	the C<CBOR::XS::is_bool> function.	284	error). See the L<Types::Serialiser> manpage for details.
205		285
206	=item null, undefined	286	=item CBOR tag 256 (perl object)
207		287
208	CBOR null and undefined values becomes C<undef> in Perl (in the future,	288	The tag value C<256> (TODO: pending iana registration) will be used
209	Undefined may raise an exception or something else).	289	to deserialise a Perl object serialised with C<FREEZE>. See L<OBJECT
		290	SERIALISATION>, below, for details.
210		291
211	=item tags	292	=item CBOR tag 55799 (magic header)
212		293
		294	The tag 55799 is ignored (this tag implements the magic header).
		295
		296	=item other CBOR tags
		297
213	Tagged items consists of a numeric tag and another CBOR value. The tag	298	Tagged items consists of a numeric tag and another CBOR value. Tags not
214	55799 is ignored (this tag implements the magic header).	299	handled internally are currently converted into a L<CBOR::XS::Tagged>
215
216	All other tags are currently converted into a L<CBOR::XS::Tagged> object,
217	which is simply a blessed array reference consistsing of the numeric tag	300	object, which is simply a blessed array reference consisting of the
218	value followed by the (decoded) BOR value.	301	numeric tag value followed by the (decoded) CBOR value.
		302
		303	In the future, support for user-supplied conversions might get added.
219		304
220	=item anything else	305	=item anything else
221		306
222	Anything else (e.g. unsupported simple values) will raise a decoding	307	Anything else (e.g. unsupported simple values) will raise a decoding
223	error.	308	error.
…		…
253	C<1>, which get turned into false and true in CBOR.	338	C<1>, which get turned into false and true in CBOR.
254		339
255	=item CBOR::XS::Tagged objects	340	=item CBOR::XS::Tagged objects
256		341
257	Objects of this type must be arrays consisting of a single C<[tag, value]>	342	Objects of this type must be arrays consisting of a single C<[tag, value]>
258	pair. The (numerical) tag will be encoded as a CBOR tag, the value will be	343	pair. The (numerical) tag will be encoded as a CBOR tag, the value will
259	encoded as appropriate for the value.	344	be encoded as appropriate for the value. You cna use C<CBOR::XS::tag> to
		345	create such objects.
260		346
261	=item CBOR::XS::true, CBOR::XS::false	347	=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
262		348
263	These special values become CBOR true and CBOR false values,	349	These special values become CBOR true, CBOR false and CBOR undefined
264	respectively. You can also use C<\1> and C<\0> directly if you want.	350	values, respectively. You can also use C<\1>, C<\0> and C<\undef> directly
		351	if you want.
265		352
266	=item blessed objects	353	=item other blessed objects
267		354
268	Other blessed objects currently need to have a C<TO_CBOR> method. It	355	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
269	will be called on every object that is being serialised, and must return	356	L<OBJECT SERIALISATION>, below, for details.
270	something that can be encoded in CBOR.
271		357
272	=item simple scalars	358	=item simple scalars
273		359
274	TODO	360	TODO
275	Simple Perl scalars (any scalar that is not a reference) are the most	361	Simple Perl scalars (any scalar that is not a reference) are the most
…		…
313	represent numerical values are supported, but might suffer loss of	399	represent numerical values are supported, but might suffer loss of
314	precision.	400	precision.
315		401
316	=back	402	=back
317		403
		404	=head2 OBJECT SERIALISATION
318		405
		406	This module knows two way to serialise a Perl object: The CBOR-specific
		407	way, and the generic way.
		408
		409	Whenever the encoder encounters a Perl object that it cnanot serialise
		410	directly (most of them), it will first look up the C<TO_CBOR> method on
		411	it.
		412
		413	If it has a C<TO_CBOR> method, it will call it with the object as only
		414	argument, and expects exactly one return value, which it will then
		415	substitute and encode it in the place of the object.
		416
		417	Otherwise, it will look up the C<FREEZE> method. If it exists, it will
		418	call it with the object as first argument, and the constant string C<CBOR>
		419	as the second argument, to distinguish it from other serialisers.
		420
		421	The C<FREEZE> method can return any number of values (i.e. zero or
		422	more). These will be encoded as CBOR perl object, together with the
		423	classname.
		424
		425	If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
		426	with an error.
		427
		428	Objects encoded via C<TO_CBOR> cannot be automatically decoded, but
		429	objects encoded via C<FREEZE> can be decoded using the following protocol:
		430
		431	When an encoded CBOR perl object is encountered by the decoder, it will
		432	look up the C<THAW> method, by using the stored classname, and will fail
		433	if the method cannot be found.
		434
		435	After the lookup it will call the C<THAW> method with the stored classname
		436	as first argument, the constant string C<CBOR> as second argument, and all
		437	values returned by C<FREEZE> as remaining arguments.
		438
		439	=head4 EXAMPLES
		440
		441	Here is an example C<TO_CBOR> method:
		442
		443	sub My::Object::TO_CBOR {
		444	my ($obj) = @_;
		445
		446	["this is a serialised My::Object object", $obj->{id}]
		447	}
		448
		449	When a C<My::Object> is encoded to CBOR, it will instead encode a simple
		450	array with two members: a string, and the "object id". Decoding this CBOR
		451	string will yield a normal perl array reference in place of the object.
		452
		453	A more useful and practical example would be a serialisation method for
		454	the URI module. CBOR has a custom tag value for URIs, namely 32:
		455
		456	sub URI::TO_CBOR {
		457	my ($self) = @_;
		458	my $uri = "$self"; # stringify uri
		459	utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
		460	CBOR::XS::tagged 32, "$_[0]"
		461	}
		462
		463	This will encode URIs as a UTF-8 string with tag 32, which indicates an
		464	URI.
		465
		466	Decoding such an URI will not (currently) give you an URI object, but
		467	instead a CBOR::XS::Tagged object with tag number 32 and the string -
		468	exactly what was returned by C<TO_CBOR>.
		469
		470	To serialise an object so it can automatically be deserialised, you need
		471	to use C<FREEZE> and C<THAW>. To take the URI module as example, this
		472	would be a possible implementation:
		473
		474	sub URI::FREEZE {
		475	my ($self, $serialiser) = @_;
		476	"$self" # encode url string
		477	}
		478
		479	sub URI::THAW {
		480	my ($class, $serialiser, $uri) = @_;
		481
		482	$class->new ($uri)
		483	}
		484
		485	Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For
		486	example, a C<FREEZE> method that returns "type", "id" and "variant" values
		487	would cause an invocation of C<THAW> with 5 arguments:
		488
		489	sub My::Object::FREEZE {
		490	my ($self, $serialiser) = @_;
		491
		492	($self->{type}, $self->{id}, $self->{variant})
		493	}
		494
		495	sub My::Object::THAW {
		496	my ($class, $serialiser, $type, $id, $variant) = @_;
		497
		498	$class-<new (type => $type, id => $id, variant => $variant)
		499	}
		500
		501
319	=head2 MAGIC HEADER	502	=head1 MAGIC HEADER
320		503
321	There is no way to distinguish CBOR from other formats	504	There is no way to distinguish CBOR from other formats
322	programmatically. To make it easier to distinguish CBOR from other	505	programmatically. To make it easier to distinguish CBOR from other
323	formats, the CBOR specification has a special "magic string" that can be	506	formats, the CBOR specification has a special "magic string" that can be
324	prepended to any CBOR string without changing it's meaning.	507	prepended to any CBOR string without changing its meaning.
325		508
326	This string is available as C<$CBOR::XS::MAGIC>. This module does not	509	This string is available as C<$CBOR::XS::MAGIC>. This module does not
327	prepend this string tot he CBOR data it generates, but it will ignroe it	510	prepend this string to the CBOR data it generates, but it will ignore it
328	if present, so users can prepend this string as a "file type" indicator as	511	if present, so users can prepend this string as a "file type" indicator as
329	required.	512	required.
330		513
331		514
		515	=head1 THE CBOR::XS::Tagged CLASS
		516
		517	CBOR has the concept of tagged values - any CBOR value can be tagged with
		518	a numeric 64 bit number, which are centrally administered.
		519
		520	C<CBOR::XS> handles a few tags internally when en- or decoding. You can
		521	also create tags yourself by encoding C<CBOR::XS::Tagged> objects, and the
		522	decoder will create C<CBOR::XS::Tagged> objects itself when it hits an
		523	unknown tag.
		524
		525	These objects are simply blessed array references - the first member of
		526	the array being the numerical tag, the second being the value.
		527
		528	You can interact with C<CBOR::XS::Tagged> objects in the following ways:
		529
		530	=over 4
		531
		532	=item $tagged = CBOR::XS::tag $tag, $value
		533
		534	This function(!) creates a new C<CBOR::XS::Tagged> object using the given
		535	C<$tag> (0..2**64-1) to tag the given C<$value> (which can be any Perl
		536	value that can be encoded in CBOR, including serialisable Perl objects and
		537	C<CBOR::XS::Tagged> objects).
		538
		539	=item $tagged->[0]
		540
		541	=item $tagged->[0] = $new_tag
		542
		543	=item $tag = $tagged->tag
		544
		545	=item $new_tag = $tagged->tag ($new_tag)
		546
		547	Access/mutate the tag.
		548
		549	=item $tagged->[1]
		550
		551	=item $tagged->[1] = $new_value
		552
		553	=item $value = $tagged->value
		554
		555	=item $new_value = $tagged->value ($new_value)
		556
		557	Access/mutate the tagged value.
		558
		559	=back
		560
		561	=cut
		562
		563	sub tag($$) {
		564	bless [@_], CBOR::XS::Tagged::;
		565	}
		566
		567	sub CBOR::XS::Tagged::tag {
		568	$_[0][0] = $_[1] if $#_;
		569	$_[0][0]
		570	}
		571
		572	sub CBOR::XS::Tagged::value {
		573	$_[0][1] = $_[1] if $#_;
		574	$_[0][1]
		575	}
		576
		577	=head2 EXAMPLES
		578
		579	Here are some examples of C<CBOR::XS::Tagged> uses to tag objects.
		580
		581	You can look up CBOR tag value and emanings in the IANA registry at
		582	L<http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml>.
		583
		584	Prepend a magic header (C<$CBOR::XS::MAGIC>):
		585
		586	my $cbor = encode_cbor CBOR::XS::tag 55799, $value;
		587	# same as:
		588	my $cbor = $CBOR::XS::MAGIC . encode_cbor $value;
		589
		590	Serialise some URIs and a regex in an array:
		591
		592	my $cbor = encode_cbor [
		593	(CBOR::XS::tag 32, "http://www.nethype.de/"),
		594	(CBOR::XS::tag 32, "http://software.schmorp.de/"),
		595	(CBOR::XS::tag 35, "^[Pp][Ee][Rr][lL]\$"),
		596	];
		597
		598	Wrap CBOR data in CBOR:
		599
		600	my $cbor_cbor = encode_cbor
		601	CBOR::XS::tag 24,
		602	encode_cbor [1, 2, 3];
		603
		604	=head1 TAG HANDLING AND EXTENSIONS
		605
		606	This section describes how this module handles specific tagged values and
		607	extensions. If a tag is not mentioned here, then the default handling
		608	applies (creating a CBOR::XS::Tagged object on decoding, and only encoding
		609	the tag when explicitly requested).
		610
		611	Future versions of this module reserve the right to special case
		612	additional tags (such as bigfloat or base64url).
		613
		614	=over 4
		615
		616	=item <unassigned> (perl-object, L<http://cbor.schmorp.de/perl-object>)
		617
		618	These tags are automatically created for serialisable objects using the
		619	C<FREEZE/THAW> methods (the L<Types::Serialier> object serialisation
		620	protocol).
		621
		622	=item <unassigned>, <unassigned> (sharable, sharedref, L <http://cbor.schmorp.de/value-sharing>)
		623
		624	These tags are automatically decoded when encountered, resulting in
		625	shared values in the decoded object. They are only encoded, however, when
		626	C<allow_sharable> is enabled.
		627
		628	=item 22098 (indirection, L<http://cbor.schmorp.de/indirection>)
		629
		630	This tag is automatically generated when a reference are encountered (with
		631	the exception of hash and array refernces). It is converted to a reference
		632	when decoding.
		633
		634	=item 55799 (self-describe CBOR, RFC 7049)
		635
		636	This value is not generated on encoding (unless explicitly requested by
		637	the user), and is simply ignored when decoding.
		638
		639	=back
		640
		641
332	=head2 CBOR and JSON	642	=head1 CBOR and JSON
333		643
334	CBOR is supposed to implement a superset of the JSON data model, and is,	644	CBOR is supposed to implement a superset of the JSON data model, and is,
335	with some coercion, able to represent all JSON texts (something that other	645	with some coercion, able to represent all JSON texts (something that other
336	"binary JSON" formats such as BSON generally do not support).	646	"binary JSON" formats such as BSON generally do not support).
337		647
…		…
417	Please refrain from using rt.cpan.org or any other bug reporting	727	Please refrain from using rt.cpan.org or any other bug reporting
418	service. I put the contact address into my modules for a reason.	728	service. I put the contact address into my modules for a reason.
419		729
420	=cut	730	=cut
421		731
422	our $true = do { bless \(my $dummy = 1), "CBOR::XS::Boolean" };
423	our $false = do { bless \(my $dummy = 0), "CBOR::XS::Boolean" };
424
425	sub true() { $true }
426	sub false() { $false }
427
428	sub is_bool($) {
429	UNIVERSAL::isa $_[0], "CBOR::XS::Boolean"
430	# or UNIVERSAL::isa $_[0], "CBOR::Literal"
431	}
432
433	XSLoader::load "CBOR::XS", $VERSION;	732	XSLoader::load "CBOR::XS", $VERSION;
434
435	package CBOR::XS::Boolean;
436
437	use overload
438	"0+" => sub { ${$_[0]} },
439	"++" => sub { $_[0] = ${$_[0]} + 1 },
440	"--" => sub { $_[0] = ${$_[0]} - 1 },
441	fallback => 1;
442
443	1;
444		733
445	=head1 SEE ALSO	734	=head1 SEE ALSO
446		735
447	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,	736	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,
448	serialisation.	737	serialisation.
449		738
		739	The L<Types::Serialiser> module provides the data model for true, false
		740	and error values.
		741
450	=head1 AUTHOR	742	=head1 AUTHOR
451		743
452	Marc Lehmann <schmorp@schmorp.de>	744	Marc Lehmann <schmorp@schmorp.de>
453	http://home.schmorp.de/	745	http://home.schmorp.de/
454		746
455	=cut	747	=cut
456		748
		749	1
		750

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.5 by root, Sat Oct 26 23:02:55 2013 UTC vs. Revision 1.19 by root, Wed Nov 20 01:09:46 2013 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.5 by root, Sat Oct 26 23:02:55 2013 UTC vs.
Revision 1.19 by root, Wed Nov 20 01:09:46 2013 UTC