[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.1 by root, Fri Oct 25 23:09:45 2013 UTC vs.
Revision 1.18 by root, Sun Nov 17 05:26:14 2013 UTC

…		…
12	$perl_value = decode_cbor $binary_cbor_data;	12	$perl_value = decode_cbor $binary_cbor_data;
13		13
14	# OO-interface	14	# OO-interface
15		15
16	$coder = CBOR::XS->new;	16	$coder = CBOR::XS->new;
17	#TODO	17	$binary_cbor_data = $coder->encode ($perl_value);
		18	$perl_value = $coder->decode ($binary_cbor_data);
		19
		20	# prefix decoding
		21
		22	my $many_cbor_strings = ...;
		23	while (length $many_cbor_strings) {
		24	my ($data, $length) = $cbor->decode_prefix ($many_cbor_strings);
		25	# data was decoded
		26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
		27	}
18		28
19	=head1 DESCRIPTION	29	=head1 DESCRIPTION
20		30
21	WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA AND	31	WARNING! This module is very new, and not very well tested (that's up to
22	EAT YOUR CHILDREN!	32	you to do). Furthermore, details of the implementation might change freely
		33	before version 1.0. And lastly, the object serialisation protocol depends
		34	on a pending IANA assignment, and until that assignment is official, this
		35	implementation is not interoperable with other implementations (even
		36	future versions of this module) until the assignment is done.
23		37
24	This module converts Perl data structures to CBOR and vice versa. Its	38	You are still invited to try out CBOR, and this module.
		39
		40	This module converts Perl data structures to the Concise Binary Object
		41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
		42	format that aims to use a superset of the JSON data model, i.e. when you
		43	can represent something in JSON, you should be able to represent it in
		44	CBOR.
		45
		46	In short, CBOR is a faster and very compact binary alternative to JSON,
		47	with the added ability of supporting serialisation of Perl objects. (JSON
		48	often compresses better than CBOR though, so if you plan to compress the
		49	data later you might want to compare both formats first).
		50
		51	To give you a general idea about speed, with texts in the megabyte range,
		52	C<CBOR::XS> usually encodes roughly twice as fast as L<Storable> or
		53	L<JSON::XS> and decodes about 15%-30% faster than those. The shorter the
		54	data, the worse L<Storable> performs in comparison.
		55
		56	As for compactness, C<CBOR::XS> encoded data structures are usually about
		57	20% smaller than the same data encoded as (compact) JSON or L<Storable>.
		58
25	primary goal is to be I<correct> and its secondary goal is to be	59	The primary goal of this module is to be I<correct> and the secondary goal
26	I<fast>. To reach the latter goal it was written in C.	60	is to be I<fast>. To reach the latter goal it was written in C.
27		61
28	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	62	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
29	vice versa.	63	vice versa.
30		64
31	=cut	65	=cut
32		66
33	package CBOR::XS;	67	package CBOR::XS;
34		68
35	use common::sense;	69	use common::sense;
36		70
37	our $VERSION = 0.01;	71	our $VERSION = 0.08;
38	our @ISA = qw(Exporter);	72	our @ISA = qw(Exporter);
39		73
40	our @EXPORT = qw(encode_cbor decode_cbor);	74	our @EXPORT = qw(encode_cbor decode_cbor);
41		75
42	use Exporter;	76	use Exporter;
43	use XSLoader;	77	use XSLoader;
		78
		79	use Types::Serialiser;
		80
		81	our $MAGIC = "\xd9\xd9\xf7";
44		82
45	=head1 FUNCTIONAL INTERFACE	83	=head1 FUNCTIONAL INTERFACE
46		84
47	The following convenience methods are provided by this module. They are	85	The following convenience methods are provided by this module. They are
48	exported by default:	86	exported by default:
…		…
161		199
162	=head2 CBOR -> PERL	200	=head2 CBOR -> PERL
163		201
164	=over 4	202	=over 4
165		203
166	=item True, False	204	=item integers
167		205
168	These CBOR values become C<CBOR::XS::true> and C<CBOR::XS::false>,	206	CBOR integers become (numeric) perl scalars. On perls without 64 bit
		207	support, 64 bit integers will be truncated or otherwise corrupted.
		208
		209	=item byte strings
		210
		211	Byte strings will become octet strings in Perl (the byte values 0..255
		212	will simply become characters of the same value in Perl).
		213
		214	=item UTF-8 strings
		215
		216	UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
		217	decoded into proper Unicode code points. At the moment, the validity of
		218	the UTF-8 octets will not be validated - corrupt input will result in
		219	corrupted Perl strings.
		220
		221	=item arrays, maps
		222
		223	CBOR arrays and CBOR maps will be converted into references to a Perl
		224	array or hash, respectively. The keys of the map will be stringified
		225	during this process.
		226
		227	=item null
		228
		229	CBOR null becomes C<undef> in Perl.
		230
		231	=item true, false, undefined
		232
		233	These CBOR values become C<Types:Serialiser::true>,
		234	C<Types:Serialiser::false> and C<Types::Serialiser::error>,
169	respectively. They are overloaded to act almost exactly like the numbers	235	respectively. They are overloaded to act almost exactly like the numbers
170	C<1> and C<0>. You can check whether a scalar is a CBOR boolean by using	236	C<1> and C<0> (for true and false) or to throw an exception on access (for
171	the C<CBOR::XS::is_bool> function.	237	error). See the L<Types::Serialiser> manpage for details.
172		238
173	=item null	239	=item CBOR tag 256 (perl object)
174		240
175	A CBOR Null value becomes C<undef> in Perl.	241	The tag value C<256> (TODO: pending iana registration) will be used
		242	to deserialise a Perl object serialised with C<FREEZE>. See L<OBJECT
		243	SERIALISATION>, below, for details.
		244
		245	=item CBOR tag 55799 (magic header)
		246
		247	The tag 55799 is ignored (this tag implements the magic header).
		248
		249	=item other CBOR tags
		250
		251	Tagged items consists of a numeric tag and another CBOR value. Tags not
		252	handled internally are currently converted into a L<CBOR::XS::Tagged>
		253	object, which is simply a blessed array reference consisting of the
		254	numeric tag value followed by the (decoded) CBOR value.
		255
		256	In the future, support for user-supplied conversions might get added.
		257
		258	=item anything else
		259
		260	Anything else (e.g. unsupported simple values) will raise a decoding
		261	error.
176		262
177	=back	263	=back
178		264
179		265
180	=head2 PERL -> CBOR	266	=head2 PERL -> CBOR
…		…
185		271
186	=over 4	272	=over 4
187		273
188	=item hash references	274	=item hash references
189		275
190	Perl hash references become CBOR maps. As there is no inherent ordering	276	Perl hash references become CBOR maps. As there is no inherent ordering in
191	in hash keys (or CBOR maps), they will usually be encoded in a	277	hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
192	pseudo-random order.	278	order.
		279
		280	Currently, tied hashes will use the indefinite-length format, while normal
		281	hashes will use the fixed-length format.
193		282
194	=item array references	283	=item array references
195		284
196	Perl array references become CBOR arrays.	285	Perl array references become fixed-length CBOR arrays.
197		286
198	=item other references	287	=item other references
199		288
200	Other unblessed references are generally not allowed and will cause an	289	Other unblessed references are generally not allowed and will cause an
201	exception to be thrown, except for references to the integers C<0> and	290	exception to be thrown, except for references to the integers C<0> and
202	C<1>, which get turned into C<False> and C<True> in CBOR.	291	C<1>, which get turned into false and true in CBOR.
203		292
204	=item CBOR::XS::true, CBOR::XS::false	293	=item CBOR::XS::Tagged objects
205		294
		295	Objects of this type must be arrays consisting of a single C<[tag, value]>
		296	pair. The (numerical) tag will be encoded as a CBOR tag, the value will
		297	be encoded as appropriate for the value. You cna use C<CBOR::XS::tag> to
		298	create such objects.
		299
		300	=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
		301
206	These special values become CBOR True and CBOR False values,	302	These special values become CBOR true, CBOR false and CBOR undefined
207	respectively. You can also use C<\1> and C<\0> directly if you want.	303	values, respectively. You can also use C<\1>, C<\0> and C<\undef> directly
		304	if you want.
208		305
209	=item blessed objects	306	=item other blessed objects
210		307
211	Blessed objects are not directly representable in CBOR. TODO	308	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
212	See the	309	L<OBJECT SERIALISATION>, below, for details.
213	C<allow_blessed> and C<convert_blessed> methods on various options on
214	how to deal with this: basically, you can choose between throwing an
215	exception, encoding the reference as if it weren't blessed, or provide
216	your own serialiser method.
217		310
218	=item simple scalars	311	=item simple scalars
219		312
220	TODO	313	TODO
221	Simple Perl scalars (any scalar that is not a reference) are the most	314	Simple Perl scalars (any scalar that is not a reference) are the most
222	difficult objects to encode: CBOR::XS will encode undefined scalars as	315	difficult objects to encode: CBOR::XS will encode undefined scalars as
223	CBOR C<Null> values, scalars that have last been used in a string context	316	CBOR null values, scalars that have last been used in a string context
224	before encoding as CBOR strings, and anything else as number value:	317	before encoding as CBOR strings, and anything else as number value:
225		318
226	# dump as number	319	# dump as number
227	encode_cbor [2] # yields [2]	320	encode_cbor [2] # yields [2]
228	encode_cbor [-3.0e17] # yields [-3e+17]	321	encode_cbor [-3.0e17] # yields [-3e+17]
…		…
250		343
251	You can not currently force the type in other, less obscure, ways. Tell me	344	You can not currently force the type in other, less obscure, ways. Tell me
252	if you need this capability (but don't forget to explain why it's needed	345	if you need this capability (but don't forget to explain why it's needed
253	:).	346	:).
254		347
255	Note that numerical precision has the same meaning as under Perl (so	348	Perl values that seem to be integers generally use the shortest possible
256	binary to decimal conversion follows the same rules as in Perl, which	349	representation. Floating-point values will use either the IEEE single
257	can differ to other languages). Also, your perl interpreter might expose	350	format if possible without loss of precision, otherwise the IEEE double
258	extensions to the floating point numbers of your platform, such as	351	format will be used. Perls that use formats other than IEEE double to
259	infinities or NaN's - these cannot be represented in CBOR, and it is an	352	represent numerical values are supported, but might suffer loss of
260	error to pass those in.	353	precision.
261		354
262	=back	355	=back
263		356
		357	=head2 OBJECT SERIALISATION
264		358
		359	This module knows two way to serialise a Perl object: The CBOR-specific
		360	way, and the generic way.
		361
		362	Whenever the encoder encounters a Perl object that it cnanot serialise
		363	directly (most of them), it will first look up the C<TO_CBOR> method on
		364	it.
		365
		366	If it has a C<TO_CBOR> method, it will call it with the object as only
		367	argument, and expects exactly one return value, which it will then
		368	substitute and encode it in the place of the object.
		369
		370	Otherwise, it will look up the C<FREEZE> method. If it exists, it will
		371	call it with the object as first argument, and the constant string C<CBOR>
		372	as the second argument, to distinguish it from other serialisers.
		373
		374	The C<FREEZE> method can return any number of values (i.e. zero or
		375	more). These will be encoded as CBOR perl object, together with the
		376	classname.
		377
		378	If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
		379	with an error.
		380
		381	Objects encoded via C<TO_CBOR> cannot be automatically decoded, but
		382	objects encoded via C<FREEZE> can be decoded using the following protocol:
		383
		384	When an encoded CBOR perl object is encountered by the decoder, it will
		385	look up the C<THAW> method, by using the stored classname, and will fail
		386	if the method cannot be found.
		387
		388	After the lookup it will call the C<THAW> method with the stored classname
		389	as first argument, the constant string C<CBOR> as second argument, and all
		390	values returned by C<FREEZE> as remaining arguments.
		391
		392	=head4 EXAMPLES
		393
		394	Here is an example C<TO_CBOR> method:
		395
		396	sub My::Object::TO_CBOR {
		397	my ($obj) = @_;
		398
		399	["this is a serialised My::Object object", $obj->{id}]
		400	}
		401
		402	When a C<My::Object> is encoded to CBOR, it will instead encode a simple
		403	array with two members: a string, and the "object id". Decoding this CBOR
		404	string will yield a normal perl array reference in place of the object.
		405
		406	A more useful and practical example would be a serialisation method for
		407	the URI module. CBOR has a custom tag value for URIs, namely 32:
		408
		409	sub URI::TO_CBOR {
		410	my ($self) = @_;
		411	my $uri = "$self"; # stringify uri
		412	utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
		413	CBOR::XS::tagged 32, "$_[0]"
		414	}
		415
		416	This will encode URIs as a UTF-8 string with tag 32, which indicates an
		417	URI.
		418
		419	Decoding such an URI will not (currently) give you an URI object, but
		420	instead a CBOR::XS::Tagged object with tag number 32 and the string -
		421	exactly what was returned by C<TO_CBOR>.
		422
		423	To serialise an object so it can automatically be deserialised, you need
		424	to use C<FREEZE> and C<THAW>. To take the URI module as example, this
		425	would be a possible implementation:
		426
		427	sub URI::FREEZE {
		428	my ($self, $serialiser) = @_;
		429	"$self" # encode url string
		430	}
		431
		432	sub URI::THAW {
		433	my ($class, $serialiser, $uri) = @_;
		434
		435	$class->new ($uri)
		436	}
		437
		438	Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For
		439	example, a C<FREEZE> method that returns "type", "id" and "variant" values
		440	would cause an invocation of C<THAW> with 5 arguments:
		441
		442	sub My::Object::FREEZE {
		443	my ($self, $serialiser) = @_;
		444
		445	($self->{type}, $self->{id}, $self->{variant})
		446	}
		447
		448	sub My::Object::THAW {
		449	my ($class, $serialiser, $type, $id, $variant) = @_;
		450
		451	$class-<new (type => $type, id => $id, variant => $variant)
		452	}
		453
		454
		455	=head1 MAGIC HEADER
		456
		457	There is no way to distinguish CBOR from other formats
		458	programmatically. To make it easier to distinguish CBOR from other
		459	formats, the CBOR specification has a special "magic string" that can be
		460	prepended to any CBOR string without changing its meaning.
		461
		462	This string is available as C<$CBOR::XS::MAGIC>. This module does not
		463	prepend this string to the CBOR data it generates, but it will ignore it
		464	if present, so users can prepend this string as a "file type" indicator as
		465	required.
		466
		467
		468	=head1 THE CBOR::XS::Tagged CLASS
		469
		470	CBOR has the concept of tagged values - any CBOR value can be tagged with
		471	a numeric 64 bit number, which are centrally administered.
		472
		473	C<CBOR::XS> handles a few tags internally when en- or decoding. You can
		474	also create tags yourself by encoding C<CBOR::XS::Tagged> objects, and the
		475	decoder will create C<CBOR::XS::Tagged> objects itself when it hits an
		476	unknown tag.
		477
		478	These objects are simply blessed array references - the first member of
		479	the array being the numerical tag, the second being the value.
		480
		481	You can interact with C<CBOR::XS::Tagged> objects in the following ways:
		482
		483	=over 4
		484
		485	=item $tagged = CBOR::XS::tag $tag, $value
		486
		487	This function(!) creates a new C<CBOR::XS::Tagged> object using the given
		488	C<$tag> (0..2**64-1) to tag the given C<$value> (which can be any Perl
		489	value that can be encoded in CBOR, including serialisable Perl objects and
		490	C<CBOR::XS::Tagged> objects).
		491
		492	=item $tagged->[0]
		493
		494	=item $tagged->[0] = $new_tag
		495
		496	=item $tag = $tagged->tag
		497
		498	=item $new_tag = $tagged->tag ($new_tag)
		499
		500	Access/mutate the tag.
		501
		502	=item $tagged->[1]
		503
		504	=item $tagged->[1] = $new_value
		505
		506	=item $value = $tagged->value
		507
		508	=item $new_value = $tagged->value ($new_value)
		509
		510	Access/mutate the tagged value.
		511
		512	=back
		513
		514	=cut
		515
		516	sub tag($$) {
		517	bless [@_], CBOR::XS::Tagged::;
		518	}
		519
		520	sub CBOR::XS::Tagged::tag {
		521	$_[0][0] = $_[1] if $#_;
		522	$_[0][0]
		523	}
		524
		525	sub CBOR::XS::Tagged::value {
		526	$_[0][1] = $_[1] if $#_;
		527	$_[0][1]
		528	}
		529
		530	=head2 EXAMPLES
		531
		532	Here are some examples of C<CBOR::XS::Tagged> uses to tag objects.
		533
		534	You can look up CBOR tag value and emanings in the IANA registry at
		535	L<http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml>.
		536
		537	Prepend a magic header (C<$CBOR::XS::MAGIC>):
		538
		539	my $cbor = encode_cbor CBOR::XS::tag 55799, $value;
		540	# same as:
		541	my $cbor = $CBOR::XS::MAGIC . encode_cbor $value;
		542
		543	Serialise some URIs and a regex in an array:
		544
		545	my $cbor = encode_cbor [
		546	(CBOR::XS::tag 32, "http://www.nethype.de/"),
		547	(CBOR::XS::tag 32, "http://software.schmorp.de/"),
		548	(CBOR::XS::tag 35, "^[Pp][Ee][Rr][lL]\$"),
		549	];
		550
		551	Wrap CBOR data in CBOR:
		552
		553	my $cbor_cbor = encode_cbor
		554	CBOR::XS::tag 24,
		555	encode_cbor [1, 2, 3];
		556
265	=head2 CBOR and JSON	557	=head1 CBOR and JSON
266		558
267	TODO	559	CBOR is supposed to implement a superset of the JSON data model, and is,
		560	with some coercion, able to represent all JSON texts (something that other
		561	"binary JSON" formats such as BSON generally do not support).
		562
		563	CBOR implements some extra hints and support for JSON interoperability,
		564	and the spec offers further guidance for conversion between CBOR and
		565	JSON. None of this is currently implemented in CBOR, and the guidelines
		566	in the spec do not result in correct round-tripping of data. If JSON
		567	interoperability is improved in the future, then the goal will be to
		568	ensure that decoded JSON data will round-trip encoding and decoding to
		569	CBOR intact.
268		570
269		571
270	=head1 SECURITY CONSIDERATIONS	572	=head1 SECURITY CONSIDERATIONS
271		573
272	When you are using CBOR in a protocol, talking to untrusted potentially	574	When you are using CBOR in a protocol, talking to untrusted potentially
…		…
340	Please refrain from using rt.cpan.org or any other bug reporting	642	Please refrain from using rt.cpan.org or any other bug reporting
341	service. I put the contact address into my modules for a reason.	643	service. I put the contact address into my modules for a reason.
342		644
343	=cut	645	=cut
344		646
345	our $true = do { bless \(my $dummy = 1), "CBOR::XS::Boolean" };
346	our $false = do { bless \(my $dummy = 0), "CBOR::XS::Boolean" };
347
348	sub true() { $true }
349	sub false() { $false }
350
351	sub is_bool($) {
352	UNIVERSAL::isa $_[0], "CBOR::XS::Boolean"
353	# or UNIVERSAL::isa $_[0], "CBOR::Literal"
354	}
355
356	XSLoader::load "CBOR::XS", $VERSION;	647	XSLoader::load "CBOR::XS", $VERSION;
357
358	package CBOR::XS::Boolean;
359
360	use overload
361	"0+" => sub { ${$_[0]} },
362	"++" => sub { $_[0] = ${$_[0]} + 1 },
363	"--" => sub { $_[0] = ${$_[0]} - 1 },
364	fallback => 1;
365
366	1;
367		648
368	=head1 SEE ALSO	649	=head1 SEE ALSO
369		650
370	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,	651	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,
371	serialisation.	652	serialisation.
372		653
		654	The L<Types::Serialiser> module provides the data model for true, false
		655	and error values.
		656
373	=head1 AUTHOR	657	=head1 AUTHOR
374		658
375	Marc Lehmann <schmorp@schmorp.de>	659	Marc Lehmann <schmorp@schmorp.de>
376	http://home.schmorp.de/	660	http://home.schmorp.de/
377		661
378	=cut	662	=cut
379		663
		664	1
		665

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.1 by root, Fri Oct 25 23:09:45 2013 UTC vs. Revision 1.18 by root, Sun Nov 17 05:26:14 2013 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.1 by root, Fri Oct 25 23:09:45 2013 UTC vs.
Revision 1.18 by root, Sun Nov 17 05:26:14 2013 UTC