[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.1 by root, Fri Oct 25 23:09:45 2013 UTC vs.
Revision 1.13 by root, Tue Oct 29 15:56:31 2013 UTC

…		…
12	$perl_value = decode_cbor $binary_cbor_data;	12	$perl_value = decode_cbor $binary_cbor_data;
13		13
14	# OO-interface	14	# OO-interface
15		15
16	$coder = CBOR::XS->new;	16	$coder = CBOR::XS->new;
17	#TODO	17	$binary_cbor_data = $coder->encode ($perl_value);
		18	$perl_value = $coder->decode ($binary_cbor_data);
		19
		20	# prefix decoding
		21
		22	my $many_cbor_strings = ...;
		23	while (length $many_cbor_strings) {
		24	my ($data, $length) = $cbor->decode_prefix ($many_cbor_strings);
		25	# data was decoded
		26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
		27	}
18		28
19	=head1 DESCRIPTION	29	=head1 DESCRIPTION
20		30
21	WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA AND	31	WARNING! This module is very new, and not very well tested (that's up to
22	EAT YOUR CHILDREN!	32	you to do). Furthermore, details of the implementation might change freely
		33	before version 1.0. And lastly, the object serialisation protocol depends
		34	on a pending IANA assignment, and until that assignment is official, this
		35	implementation is not interoperable with other implementations (even
		36	future versions of this module) until the assignment is done.
23		37
24	This module converts Perl data structures to CBOR and vice versa. Its	38	You are still invited to try out CBOR, and this module.
		39
		40	This module converts Perl data structures to the Concise Binary Object
		41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
		42	format that aims to use a superset of the JSON data model, i.e. when you
		43	can represent something in JSON, you should be able to represent it in
		44	CBOR.
		45
		46	In short, CBOR is a faster and very compact binary alternative to JSON,
		47	with the added ability of supporting serialisation of Perl objects. (JSON
		48	often compresses better than CBOR though, so if you plan to compress the
		49	data later you might want to compare both formats first).
		50
25	primary goal is to be I<correct> and its secondary goal is to be	51	The primary goal of this module is to be I<correct> and the secondary goal
26	I<fast>. To reach the latter goal it was written in C.	52	is to be I<fast>. To reach the latter goal it was written in C.
27		53
28	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	54	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
29	vice versa.	55	vice versa.
30		56
31	=cut	57	=cut
32		58
33	package CBOR::XS;	59	package CBOR::XS;
34		60
35	use common::sense;	61	use common::sense;
36		62
37	our $VERSION = 0.01;	63	our $VERSION = 0.06;
38	our @ISA = qw(Exporter);	64	our @ISA = qw(Exporter);
39		65
40	our @EXPORT = qw(encode_cbor decode_cbor);	66	our @EXPORT = qw(encode_cbor decode_cbor);
41		67
42	use Exporter;	68	use Exporter;
43	use XSLoader;	69	use XSLoader;
		70
		71	use Types::Serialiser;
		72
		73	our $MAGIC = "\xd9\xd9\xf7";
44		74
45	=head1 FUNCTIONAL INTERFACE	75	=head1 FUNCTIONAL INTERFACE
46		76
47	The following convenience methods are provided by this module. They are	77	The following convenience methods are provided by this module. They are
48	exported by default:	78	exported by default:
…		…
161		191
162	=head2 CBOR -> PERL	192	=head2 CBOR -> PERL
163		193
164	=over 4	194	=over 4
165		195
166	=item True, False	196	=item integers
167		197
168	These CBOR values become C<CBOR::XS::true> and C<CBOR::XS::false>,	198	CBOR integers become (numeric) perl scalars. On perls without 64 bit
		199	support, 64 bit integers will be truncated or otherwise corrupted.
		200
		201	=item byte strings
		202
		203	Byte strings will become octet strings in Perl (the byte values 0..255
		204	will simply become characters of the same value in Perl).
		205
		206	=item UTF-8 strings
		207
		208	UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
		209	decoded into proper Unicode code points. At the moment, the validity of
		210	the UTF-8 octets will not be validated - corrupt input will result in
		211	corrupted Perl strings.
		212
		213	=item arrays, maps
		214
		215	CBOR arrays and CBOR maps will be converted into references to a Perl
		216	array or hash, respectively. The keys of the map will be stringified
		217	during this process.
		218
		219	=item null
		220
		221	CBOR null becomes C<undef> in Perl.
		222
		223	=item true, false, undefined
		224
		225	These CBOR values become C<Types:Serialiser::true>,
		226	C<Types:Serialiser::false> and C<Types::Serialiser::error>,
169	respectively. They are overloaded to act almost exactly like the numbers	227	respectively. They are overloaded to act almost exactly like the numbers
170	C<1> and C<0>. You can check whether a scalar is a CBOR boolean by using	228	C<1> and C<0> (for true and false) or to throw an exception on access (for
171	the C<CBOR::XS::is_bool> function.	229	error). See the L<Types::Serialiser> manpage for details.
172		230
173	=item null	231	=item CBOR tag 256 (perl object)
174		232
175	A CBOR Null value becomes C<undef> in Perl.	233	The tag value C<256> (TODO: pending iana registration) will be used
		234	to deserialise a Perl object serialised with C<FREEZE>. See L<OBJECT
		235	SERIALISATION>, below, for details.
		236
		237	=item CBOR tag 55799 (magic header)
		238
		239	The tag 55799 is ignored (this tag implements the magic header).
		240
		241	=item other CBOR tags
		242
		243	Tagged items consists of a numeric tag and another CBOR value. Tags not
		244	handled internally are currently converted into a L<CBOR::XS::Tagged>
		245	object, which is simply a blessed array reference consisting of the
		246	numeric tag value followed by the (decoded) CBOR value.
		247
		248	In the future, support for user-supplied conversions might get added.
		249
		250	=item anything else
		251
		252	Anything else (e.g. unsupported simple values) will raise a decoding
		253	error.
176		254
177	=back	255	=back
178		256
179		257
180	=head2 PERL -> CBOR	258	=head2 PERL -> CBOR
…		…
185		263
186	=over 4	264	=over 4
187		265
188	=item hash references	266	=item hash references
189		267
190	Perl hash references become CBOR maps. As there is no inherent ordering	268	Perl hash references become CBOR maps. As there is no inherent ordering in
191	in hash keys (or CBOR maps), they will usually be encoded in a	269	hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
192	pseudo-random order.	270	order.
		271
		272	Currently, tied hashes will use the indefinite-length format, while normal
		273	hashes will use the fixed-length format.
193		274
194	=item array references	275	=item array references
195		276
196	Perl array references become CBOR arrays.	277	Perl array references become fixed-length CBOR arrays.
197		278
198	=item other references	279	=item other references
199		280
200	Other unblessed references are generally not allowed and will cause an	281	Other unblessed references are generally not allowed and will cause an
201	exception to be thrown, except for references to the integers C<0> and	282	exception to be thrown, except for references to the integers C<0> and
202	C<1>, which get turned into C<False> and C<True> in CBOR.	283	C<1>, which get turned into false and true in CBOR.
203		284
204	=item CBOR::XS::true, CBOR::XS::false	285	=item CBOR::XS::Tagged objects
205		286
		287	Objects of this type must be arrays consisting of a single C<[tag, value]>
		288	pair. The (numerical) tag will be encoded as a CBOR tag, the value will
		289	be encoded as appropriate for the value. You cna use C<CBOR::XS::tag> to
		290	create such objects.
		291
		292	=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
		293
206	These special values become CBOR True and CBOR False values,	294	These special values become CBOR true, CBOR false and CBOR undefined
207	respectively. You can also use C<\1> and C<\0> directly if you want.	295	values, respectively. You can also use C<\1>, C<\0> and C<\undef> directly
		296	if you want.
208		297
209	=item blessed objects	298	=item other blessed objects
210		299
211	Blessed objects are not directly representable in CBOR. TODO	300	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
212	See the	301	L<OBJECT SERIALISATION>, below, for details.
213	C<allow_blessed> and C<convert_blessed> methods on various options on
214	how to deal with this: basically, you can choose between throwing an
215	exception, encoding the reference as if it weren't blessed, or provide
216	your own serialiser method.
217		302
218	=item simple scalars	303	=item simple scalars
219		304
220	TODO	305	TODO
221	Simple Perl scalars (any scalar that is not a reference) are the most	306	Simple Perl scalars (any scalar that is not a reference) are the most
222	difficult objects to encode: CBOR::XS will encode undefined scalars as	307	difficult objects to encode: CBOR::XS will encode undefined scalars as
223	CBOR C<Null> values, scalars that have last been used in a string context	308	CBOR null values, scalars that have last been used in a string context
224	before encoding as CBOR strings, and anything else as number value:	309	before encoding as CBOR strings, and anything else as number value:
225		310
226	# dump as number	311	# dump as number
227	encode_cbor [2] # yields [2]	312	encode_cbor [2] # yields [2]
228	encode_cbor [-3.0e17] # yields [-3e+17]	313	encode_cbor [-3.0e17] # yields [-3e+17]
…		…
250		335
251	You can not currently force the type in other, less obscure, ways. Tell me	336	You can not currently force the type in other, less obscure, ways. Tell me
252	if you need this capability (but don't forget to explain why it's needed	337	if you need this capability (but don't forget to explain why it's needed
253	:).	338	:).
254		339
255	Note that numerical precision has the same meaning as under Perl (so	340	Perl values that seem to be integers generally use the shortest possible
256	binary to decimal conversion follows the same rules as in Perl, which	341	representation. Floating-point values will use either the IEEE single
257	can differ to other languages). Also, your perl interpreter might expose	342	format if possible without loss of precision, otherwise the IEEE double
258	extensions to the floating point numbers of your platform, such as	343	format will be used. Perls that use formats other than IEEE double to
259	infinities or NaN's - these cannot be represented in CBOR, and it is an	344	represent numerical values are supported, but might suffer loss of
260	error to pass those in.	345	precision.
261		346
262	=back	347	=back
263		348
		349	=head2 OBJECT SERIALISATION
264		350
		351	This module knows two way to serialise a Perl object: The CBOR-specific
		352	way, and the generic way.
		353
		354	Whenever the encoder encounters a Perl object that it cnanot serialise
		355	directly (most of them), it will first look up the C<TO_CBOR> method on
		356	it.
		357
		358	If it has a C<TO_CBOR> method, it will call it with the object as only
		359	argument, and expects exactly one return value, which it will then
		360	substitute and encode it in the place of the object.
		361
		362	Otherwise, it will look up the C<FREEZE> method. If it exists, it will
		363	call it with the object as first argument, and the constant string C<CBOR>
		364	as the second argument, to distinguish it from other serialisers.
		365
		366	The C<FREEZE> method can return any number of values (i.e. zero or
		367	more). These will be encoded as CBOR perl object, together with the
		368	classname.
		369
		370	If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
		371	with an error.
		372
		373	Objects encoded via C<TO_CBOR> cannot be automatically decoded, but
		374	objects encoded via C<FREEZE> can be decoded using the following protocol:
		375
		376	When an encoded CBOR perl object is encountered by the decoder, it will
		377	look up the C<THAW> method, by using the stored classname, and will fail
		378	if the method cannot be found.
		379
		380	After the lookup it will call the C<THAW> method with the stored classname
		381	as first argument, the constant string C<CBOR> as second argument, and all
		382	values returned by C<FREEZE> as remaining arguments.
		383
		384	=head4 EXAMPLES
		385
		386	Here is an example C<TO_CBOR> method:
		387
		388	sub My::Object::TO_CBOR {
		389	my ($obj) = @_;
		390
		391	["this is a serialised My::Object object", $obj->{id}]
		392	}
		393
		394	When a C<My::Object> is encoded to CBOR, it will instead encode a simple
		395	array with two members: a string, and the "object id". Decoding this CBOR
		396	string will yield a normal perl array reference in place of the object.
		397
		398	A more useful and practical example would be a serialisation method for
		399	the URI module. CBOR has a custom tag value for URIs, namely 32:
		400
		401	sub URI::TO_CBOR {
		402	my ($self) = @_;
		403	my $uri = "$self"; # stringify uri
		404	utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
		405	CBOR::XS::tagged 32, "$_[0]"
		406	}
		407
		408	This will encode URIs as a UTF-8 string with tag 32, which indicates an
		409	URI.
		410
		411	Decoding such an URI will not (currently) give you an URI object, but
		412	instead a CBOR::XS::Tagged object with tag number 32 and the string -
		413	exactly what was returned by C<TO_CBOR>.
		414
		415	To serialise an object so it can automatically be deserialised, you need
		416	to use C<FREEZE> and C<THAW>. To take the URI module as example, this
		417	would be a possible implementation:
		418
		419	sub URI::FREEZE {
		420	my ($self, $serialiser) = @_;
		421	"$self" # encode url string
		422	}
		423
		424	sub URI::THAW {
		425	my ($class, $serialiser, $uri) = @_;
		426
		427	$class->new ($uri)
		428	}
		429
		430	Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For
		431	example, a C<FREEZE> method that returns "type", "id" and "variant" values
		432	would cause an invocation of C<THAW> with 5 arguments:
		433
		434	sub My::Object::FREEZE {
		435	my ($self, $serialiser) = @_;
		436
		437	($self->{type}, $self->{id}, $self->{variant})
		438	}
		439
		440	sub My::Object::THAW {
		441	my ($class, $serialiser, $type, $id, $variant) = @_;
		442
		443	$class-<new (type => $type, id => $id, variant => $variant)
		444	}
		445
		446
		447	=head1 MAGIC HEADER
		448
		449	There is no way to distinguish CBOR from other formats
		450	programmatically. To make it easier to distinguish CBOR from other
		451	formats, the CBOR specification has a special "magic string" that can be
		452	prepended to any CBOR string without changing it's meaning.
		453
		454	This string is available as C<$CBOR::XS::MAGIC>. This module does not
		455	prepend this string tot he CBOR data it generates, but it will ignroe it
		456	if present, so users can prepend this string as a "file type" indicator as
		457	required.
		458
		459
		460	=head1 THE CBOR::XS::Tagged CLASS
		461
		462	CBOR has the concept of tagged values - any CBOR value can be tagged with
		463	a numeric 64 bit number, which are centrally administered.
		464
		465	C<CBOR::XS> handles a few tags internally when en- or decoding. You can
		466	also create tags yourself by encoding C<CBOR::XS::Tagged> objects, and the
		467	decoder will create C<CBOR::XS::Tagged> objects itself when it hits an
		468	unknown tag.
		469
		470	These objects are simply blessed array references - the first member of
		471	the array being the numerical tag, the second being the value.
		472
		473	You can interact with C<CBOR::XS::Tagged> objects in the following ways:
		474
		475	=over 4
		476
		477	=item $tagged = CBOR::XS::tag $tag, $value
		478
		479	This function(!) creates a new C<CBOR::XS::Tagged> object using the given
		480	C<$tag> (0..2**64-1) to tag the given C<$value> (which can be any Perl
		481	value that can be encoded in CBOR, including serialisable Perl objects and
		482	C<CBOR::XS::Tagged> objects).
		483
		484	=item $tagged->[0]
		485
		486	=item $tagged->[0] = $new_tag
		487
		488	=item $tag = $tagged->tag
		489
		490	=item $new_tag = $tagged->tag ($new_tag)
		491
		492	Access/mutate the tag.
		493
		494	=item $tagged->[1]
		495
		496	=item $tagged->[1] = $new_value
		497
		498	=item $value = $tagged->value
		499
		500	=item $new_value = $tagged->value ($new_value)
		501
		502	Access/mutate the tagged value.
		503
		504	=back
		505
		506	=cut
		507
		508	sub tag($$) {
		509	bless [@_], CBOR::XS::Tagged::;
		510	}
		511
		512	sub CBOR::XS::Tagged::tag {
		513	$_[0][0] = $_[1] if $#_;
		514	$_[0][0]
		515	}
		516
		517	sub CBOR::XS::Tagged::value {
		518	$_[0][1] = $_[1] if $#_;
		519	$_[0][1]
		520	}
		521
		522	=head2 EXAMPLES
		523
		524	Here are some examples of C<CBOR::XS::Tagged> uses to tag objects.
		525
		526	You can look up CBOR tag value and emanings in the IANA registry at
		527	L<http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml>.
		528
		529	Prepend a magic header (C<$CBOR::XS::MAGIC>):
		530
		531	my $cbor = encode_cbor CBOR::XS::tag 55799, $value;
		532	# same as:
		533	my $cbor = $CBOR::XS::MAGIC . encode_cbor $value;
		534
		535	Serialise some URIs and a regex in an array:
		536
		537	my $cbor = encode_cbor [
		538	(CBOR::XS::tag 32, "http://www.nethype.de/"),
		539	(CBOR::XS::tag 32, "http://software.schmorp.de/"),
		540	(CBOR::XS::tag 35, "^[Pp][Ee][Rr][lL]\$"),
		541	];
		542
		543	Wrap CBOR data in CBOR:
		544
		545	my $cbor_cbor = encode_cbor
		546	CBOR::XS::tag 24,
		547	encode_cbor [1, 2, 3];
		548
265	=head2 CBOR and JSON	549	=head1 CBOR and JSON
266		550
267	TODO	551	CBOR is supposed to implement a superset of the JSON data model, and is,
		552	with some coercion, able to represent all JSON texts (something that other
		553	"binary JSON" formats such as BSON generally do not support).
		554
		555	CBOR implements some extra hints and support for JSON interoperability,
		556	and the spec offers further guidance for conversion between CBOR and
		557	JSON. None of this is currently implemented in CBOR, and the guidelines
		558	in the spec do not result in correct round-tripping of data. If JSON
		559	interoperability is improved in the future, then the goal will be to
		560	ensure that decoded JSON data will round-trip encoding and decoding to
		561	CBOR intact.
268		562
269		563
270	=head1 SECURITY CONSIDERATIONS	564	=head1 SECURITY CONSIDERATIONS
271		565
272	When you are using CBOR in a protocol, talking to untrusted potentially	566	When you are using CBOR in a protocol, talking to untrusted potentially
…		…
340	Please refrain from using rt.cpan.org or any other bug reporting	634	Please refrain from using rt.cpan.org or any other bug reporting
341	service. I put the contact address into my modules for a reason.	635	service. I put the contact address into my modules for a reason.
342		636
343	=cut	637	=cut
344		638
345	our $true = do { bless \(my $dummy = 1), "CBOR::XS::Boolean" };
346	our $false = do { bless \(my $dummy = 0), "CBOR::XS::Boolean" };
347
348	sub true() { $true }
349	sub false() { $false }
350
351	sub is_bool($) {
352	UNIVERSAL::isa $_[0], "CBOR::XS::Boolean"
353	# or UNIVERSAL::isa $_[0], "CBOR::Literal"
354	}
355
356	XSLoader::load "CBOR::XS", $VERSION;	639	XSLoader::load "CBOR::XS", $VERSION;
357
358	package CBOR::XS::Boolean;
359
360	use overload
361	"0+" => sub { ${$_[0]} },
362	"++" => sub { $_[0] = ${$_[0]} + 1 },
363	"--" => sub { $_[0] = ${$_[0]} - 1 },
364	fallback => 1;
365
366	1;
367		640
368	=head1 SEE ALSO	641	=head1 SEE ALSO
369		642
370	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,	643	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,
371	serialisation.	644	serialisation.
372		645
		646	The L<Types::Serialiser> module provides the data model for true, false
		647	and error values.
		648
373	=head1 AUTHOR	649	=head1 AUTHOR
374		650
375	Marc Lehmann <schmorp@schmorp.de>	651	Marc Lehmann <schmorp@schmorp.de>
376	http://home.schmorp.de/	652	http://home.schmorp.de/
377		653
378	=cut	654	=cut
379		655
		656	1
		657

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.1 by root, Fri Oct 25 23:09:45 2013 UTC vs. Revision 1.13 by root, Tue Oct 29 15:56:31 2013 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.1 by root, Fri Oct 25 23:09:45 2013 UTC vs.
Revision 1.13 by root, Tue Oct 29 15:56:31 2013 UTC