[ViewVC] Diff of: cvs/CBOR-XS/XS.pm

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.3 by root, Sat Oct 26 11:08:34 2013 UTC vs.
Revision 1.14 by root, Tue Oct 29 20:59:16 2013 UTC

…		…
12	$perl_value = decode_cbor $binary_cbor_data;	12	$perl_value = decode_cbor $binary_cbor_data;
13		13
14	# OO-interface	14	# OO-interface
15		15
16	$coder = CBOR::XS->new;	16	$coder = CBOR::XS->new;
17	#TODO	17	$binary_cbor_data = $coder->encode ($perl_value);
		18	$perl_value = $coder->decode ($binary_cbor_data);
		19
		20	# prefix decoding
		21
		22	my $many_cbor_strings = ...;
		23	while (length $many_cbor_strings) {
		24	my ($data, $length) = $cbor->decode_prefix ($many_cbor_strings);
		25	# data was decoded
		26	substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
		27	}
18		28
19	=head1 DESCRIPTION	29	=head1 DESCRIPTION
20		30
21	WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA AND	31	WARNING! This module is very new, and not very well tested (that's up to
22	EAT YOUR CHILDREN!	32	you to do). Furthermore, details of the implementation might change freely
		33	before version 1.0. And lastly, the object serialisation protocol depends
		34	on a pending IANA assignment, and until that assignment is official, this
		35	implementation is not interoperable with other implementations (even
		36	future versions of this module) until the assignment is done.
23		37
24	This module converts Perl data structures to CBOR and vice versa. Its	38	You are still invited to try out CBOR, and this module.
		39
		40	This module converts Perl data structures to the Concise Binary Object
		41	Representation (CBOR) and vice versa. CBOR is a fast binary serialisation
		42	format that aims to use a superset of the JSON data model, i.e. when you
		43	can represent something in JSON, you should be able to represent it in
		44	CBOR.
		45
		46	In short, CBOR is a faster and very compact binary alternative to JSON,
		47	with the added ability of supporting serialisation of Perl objects. (JSON
		48	often compresses better than CBOR though, so if you plan to compress the
		49	data later you might want to compare both formats first).
		50
		51	To give you a general idea, with texts in the megabyte range, C<CBOR::XS>
		52	usually encodes roughly twice as fast as L<Storable> or L<JSON::XS> and
		53	decodes about 15%-30% faster than those. The shorter the data, the worse
		54	L<Storable> performs in comparison.
		55
25	primary goal is to be I<correct> and its secondary goal is to be	56	The primary goal of this module is to be I<correct> and the secondary goal
26	I<fast>. To reach the latter goal it was written in C.	57	is to be I<fast>. To reach the latter goal it was written in C.
27		58
28	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and	59	See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
29	vice versa.	60	vice versa.
30		61
31	=cut	62	=cut
32		63
33	package CBOR::XS;	64	package CBOR::XS;
34		65
35	use common::sense;	66	use common::sense;
36		67
37	our $VERSION = 0.02;	68	our $VERSION = 0.06;
38	our @ISA = qw(Exporter);	69	our @ISA = qw(Exporter);
39		70
40	our @EXPORT = qw(encode_cbor decode_cbor);	71	our @EXPORT = qw(encode_cbor decode_cbor);
41		72
42	use Exporter;	73	use Exporter;
43	use XSLoader;	74	use XSLoader;
		75
		76	use Types::Serialiser;
44		77
45	our $MAGIC = "\xd9\xd9\xf7";	78	our $MAGIC = "\xd9\xd9\xf7";
46		79
47	=head1 FUNCTIONAL INTERFACE	80	=head1 FUNCTIONAL INTERFACE
48		81
…		…
163		196
164	=head2 CBOR -> PERL	197	=head2 CBOR -> PERL
165		198
166	=over 4	199	=over 4
167		200
168	=item True, False	201	=item integers
169		202
170	These CBOR values become C<CBOR::XS::true> and C<CBOR::XS::false>,	203	CBOR integers become (numeric) perl scalars. On perls without 64 bit
		204	support, 64 bit integers will be truncated or otherwise corrupted.
		205
		206	=item byte strings
		207
		208	Byte strings will become octet strings in Perl (the byte values 0..255
		209	will simply become characters of the same value in Perl).
		210
		211	=item UTF-8 strings
		212
		213	UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
		214	decoded into proper Unicode code points. At the moment, the validity of
		215	the UTF-8 octets will not be validated - corrupt input will result in
		216	corrupted Perl strings.
		217
		218	=item arrays, maps
		219
		220	CBOR arrays and CBOR maps will be converted into references to a Perl
		221	array or hash, respectively. The keys of the map will be stringified
		222	during this process.
		223
		224	=item null
		225
		226	CBOR null becomes C<undef> in Perl.
		227
		228	=item true, false, undefined
		229
		230	These CBOR values become C<Types:Serialiser::true>,
		231	C<Types:Serialiser::false> and C<Types::Serialiser::error>,
171	respectively. They are overloaded to act almost exactly like the numbers	232	respectively. They are overloaded to act almost exactly like the numbers
172	C<1> and C<0>. You can check whether a scalar is a CBOR boolean by using	233	C<1> and C<0> (for true and false) or to throw an exception on access (for
173	the C<CBOR::XS::is_bool> function.	234	error). See the L<Types::Serialiser> manpage for details.
174		235
175	=item Null, Undefined	236	=item CBOR tag 256 (perl object)
176		237
177	CBOR Null and Undefined values becomes C<undef> in Perl (in the future,	238	The tag value C<256> (TODO: pending iana registration) will be used
178	Undefined may raise an exception).	239	to deserialise a Perl object serialised with C<FREEZE>. See L<OBJECT
		240	SERIALISATION>, below, for details.
		241
		242	=item CBOR tag 55799 (magic header)
		243
		244	The tag 55799 is ignored (this tag implements the magic header).
		245
		246	=item other CBOR tags
		247
		248	Tagged items consists of a numeric tag and another CBOR value. Tags not
		249	handled internally are currently converted into a L<CBOR::XS::Tagged>
		250	object, which is simply a blessed array reference consisting of the
		251	numeric tag value followed by the (decoded) CBOR value.
		252
		253	In the future, support for user-supplied conversions might get added.
		254
		255	=item anything else
		256
		257	Anything else (e.g. unsupported simple values) will raise a decoding
		258	error.
179		259
180	=back	260	=back
181		261
182		262
183	=head2 PERL -> CBOR	263	=head2 PERL -> CBOR
…		…
188		268
189	=over 4	269	=over 4
190		270
191	=item hash references	271	=item hash references
192		272
193	Perl hash references become CBOR maps. As there is no inherent ordering	273	Perl hash references become CBOR maps. As there is no inherent ordering in
194	in hash keys (or CBOR maps), they will usually be encoded in a	274	hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
195	pseudo-random order.	275	order.
		276
		277	Currently, tied hashes will use the indefinite-length format, while normal
		278	hashes will use the fixed-length format.
196		279
197	=item array references	280	=item array references
198		281
199	Perl array references become CBOR arrays.	282	Perl array references become fixed-length CBOR arrays.
200		283
201	=item other references	284	=item other references
202		285
203	Other unblessed references are generally not allowed and will cause an	286	Other unblessed references are generally not allowed and will cause an
204	exception to be thrown, except for references to the integers C<0> and	287	exception to be thrown, except for references to the integers C<0> and
205	C<1>, which get turned into C<False> and C<True> in CBOR.	288	C<1>, which get turned into false and true in CBOR.
206		289
207	=item CBOR::XS::true, CBOR::XS::false	290	=item CBOR::XS::Tagged objects
208		291
		292	Objects of this type must be arrays consisting of a single C<[tag, value]>
		293	pair. The (numerical) tag will be encoded as a CBOR tag, the value will
		294	be encoded as appropriate for the value. You cna use C<CBOR::XS::tag> to
		295	create such objects.
		296
		297	=item Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error
		298
209	These special values become CBOR True and CBOR False values,	299	These special values become CBOR true, CBOR false and CBOR undefined
210	respectively. You can also use C<\1> and C<\0> directly if you want.	300	values, respectively. You can also use C<\1>, C<\0> and C<\undef> directly
		301	if you want.
211		302
212	=item blessed objects	303	=item other blessed objects
213		304
214	Blessed objects are not directly representable in CBOR. TODO	305	Other blessed objects are serialised via C<TO_CBOR> or C<FREEZE>. See
215	See the	306	L<OBJECT SERIALISATION>, below, for details.
216	C<allow_blessed> and C<convert_blessed> methods on various options on
217	how to deal with this: basically, you can choose between throwing an
218	exception, encoding the reference as if it weren't blessed, or provide
219	your own serialiser method.
220		307
221	=item simple scalars	308	=item simple scalars
222		309
223	TODO	310	TODO
224	Simple Perl scalars (any scalar that is not a reference) are the most	311	Simple Perl scalars (any scalar that is not a reference) are the most
225	difficult objects to encode: CBOR::XS will encode undefined scalars as	312	difficult objects to encode: CBOR::XS will encode undefined scalars as
226	CBOR C<Null> values, scalars that have last been used in a string context	313	CBOR null values, scalars that have last been used in a string context
227	before encoding as CBOR strings, and anything else as number value:	314	before encoding as CBOR strings, and anything else as number value:
228		315
229	# dump as number	316	# dump as number
230	encode_cbor [2] # yields [2]	317	encode_cbor [2] # yields [2]
231	encode_cbor [-3.0e17] # yields [-3e+17]	318	encode_cbor [-3.0e17] # yields [-3e+17]
…		…
253		340
254	You can not currently force the type in other, less obscure, ways. Tell me	341	You can not currently force the type in other, less obscure, ways. Tell me
255	if you need this capability (but don't forget to explain why it's needed	342	if you need this capability (but don't forget to explain why it's needed
256	:).	343	:).
257		344
258	Note that numerical precision has the same meaning as under Perl (so	345	Perl values that seem to be integers generally use the shortest possible
259	binary to decimal conversion follows the same rules as in Perl, which	346	representation. Floating-point values will use either the IEEE single
260	can differ to other languages). Also, your perl interpreter might expose	347	format if possible without loss of precision, otherwise the IEEE double
261	extensions to the floating point numbers of your platform, such as	348	format will be used. Perls that use formats other than IEEE double to
262	infinities or NaN's - these cannot be represented in CBOR, and it is an	349	represent numerical values are supported, but might suffer loss of
263	error to pass those in.	350	precision.
264		351
265	=back	352	=back
266		353
		354	=head2 OBJECT SERIALISATION
267		355
		356	This module knows two way to serialise a Perl object: The CBOR-specific
		357	way, and the generic way.
		358
		359	Whenever the encoder encounters a Perl object that it cnanot serialise
		360	directly (most of them), it will first look up the C<TO_CBOR> method on
		361	it.
		362
		363	If it has a C<TO_CBOR> method, it will call it with the object as only
		364	argument, and expects exactly one return value, which it will then
		365	substitute and encode it in the place of the object.
		366
		367	Otherwise, it will look up the C<FREEZE> method. If it exists, it will
		368	call it with the object as first argument, and the constant string C<CBOR>
		369	as the second argument, to distinguish it from other serialisers.
		370
		371	The C<FREEZE> method can return any number of values (i.e. zero or
		372	more). These will be encoded as CBOR perl object, together with the
		373	classname.
		374
		375	If an object supports neither C<TO_CBOR> nor C<FREEZE>, encoding will fail
		376	with an error.
		377
		378	Objects encoded via C<TO_CBOR> cannot be automatically decoded, but
		379	objects encoded via C<FREEZE> can be decoded using the following protocol:
		380
		381	When an encoded CBOR perl object is encountered by the decoder, it will
		382	look up the C<THAW> method, by using the stored classname, and will fail
		383	if the method cannot be found.
		384
		385	After the lookup it will call the C<THAW> method with the stored classname
		386	as first argument, the constant string C<CBOR> as second argument, and all
		387	values returned by C<FREEZE> as remaining arguments.
		388
		389	=head4 EXAMPLES
		390
		391	Here is an example C<TO_CBOR> method:
		392
		393	sub My::Object::TO_CBOR {
		394	my ($obj) = @_;
		395
		396	["this is a serialised My::Object object", $obj->{id}]
		397	}
		398
		399	When a C<My::Object> is encoded to CBOR, it will instead encode a simple
		400	array with two members: a string, and the "object id". Decoding this CBOR
		401	string will yield a normal perl array reference in place of the object.
		402
		403	A more useful and practical example would be a serialisation method for
		404	the URI module. CBOR has a custom tag value for URIs, namely 32:
		405
		406	sub URI::TO_CBOR {
		407	my ($self) = @_;
		408	my $uri = "$self"; # stringify uri
		409	utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
		410	CBOR::XS::tagged 32, "$_[0]"
		411	}
		412
		413	This will encode URIs as a UTF-8 string with tag 32, which indicates an
		414	URI.
		415
		416	Decoding such an URI will not (currently) give you an URI object, but
		417	instead a CBOR::XS::Tagged object with tag number 32 and the string -
		418	exactly what was returned by C<TO_CBOR>.
		419
		420	To serialise an object so it can automatically be deserialised, you need
		421	to use C<FREEZE> and C<THAW>. To take the URI module as example, this
		422	would be a possible implementation:
		423
		424	sub URI::FREEZE {
		425	my ($self, $serialiser) = @_;
		426	"$self" # encode url string
		427	}
		428
		429	sub URI::THAW {
		430	my ($class, $serialiser, $uri) = @_;
		431
		432	$class->new ($uri)
		433	}
		434
		435	Unlike C<TO_CBOR>, multiple values can be returned by C<FREEZE>. For
		436	example, a C<FREEZE> method that returns "type", "id" and "variant" values
		437	would cause an invocation of C<THAW> with 5 arguments:
		438
		439	sub My::Object::FREEZE {
		440	my ($self, $serialiser) = @_;
		441
		442	($self->{type}, $self->{id}, $self->{variant})
		443	}
		444
		445	sub My::Object::THAW {
		446	my ($class, $serialiser, $type, $id, $variant) = @_;
		447
		448	$class-<new (type => $type, id => $id, variant => $variant)
		449	}
		450
		451
268	=head2 MAGIC HEADER	452	=head1 MAGIC HEADER
269		453
270	There is no way to distinguish CBOR from other formats	454	There is no way to distinguish CBOR from other formats
271	programmatically. To make it easier to distinguish CBOR from other	455	programmatically. To make it easier to distinguish CBOR from other
272	formats, the CBOR specification has a special "magic string" that can be	456	formats, the CBOR specification has a special "magic string" that can be
273	prepended to any CBOR string without changing it's meaning.	457	prepended to any CBOR string without changing it's meaning.
…		…
276	prepend this string tot he CBOR data it generates, but it will ignroe it	460	prepend this string tot he CBOR data it generates, but it will ignroe it
277	if present, so users can prepend this string as a "file type" indicator as	461	if present, so users can prepend this string as a "file type" indicator as
278	required.	462	required.
279		463
280		464
		465	=head1 THE CBOR::XS::Tagged CLASS
		466
		467	CBOR has the concept of tagged values - any CBOR value can be tagged with
		468	a numeric 64 bit number, which are centrally administered.
		469
		470	C<CBOR::XS> handles a few tags internally when en- or decoding. You can
		471	also create tags yourself by encoding C<CBOR::XS::Tagged> objects, and the
		472	decoder will create C<CBOR::XS::Tagged> objects itself when it hits an
		473	unknown tag.
		474
		475	These objects are simply blessed array references - the first member of
		476	the array being the numerical tag, the second being the value.
		477
		478	You can interact with C<CBOR::XS::Tagged> objects in the following ways:
		479
		480	=over 4
		481
		482	=item $tagged = CBOR::XS::tag $tag, $value
		483
		484	This function(!) creates a new C<CBOR::XS::Tagged> object using the given
		485	C<$tag> (0..2**64-1) to tag the given C<$value> (which can be any Perl
		486	value that can be encoded in CBOR, including serialisable Perl objects and
		487	C<CBOR::XS::Tagged> objects).
		488
		489	=item $tagged->[0]
		490
		491	=item $tagged->[0] = $new_tag
		492
		493	=item $tag = $tagged->tag
		494
		495	=item $new_tag = $tagged->tag ($new_tag)
		496
		497	Access/mutate the tag.
		498
		499	=item $tagged->[1]
		500
		501	=item $tagged->[1] = $new_value
		502
		503	=item $value = $tagged->value
		504
		505	=item $new_value = $tagged->value ($new_value)
		506
		507	Access/mutate the tagged value.
		508
		509	=back
		510
		511	=cut
		512
		513	sub tag($$) {
		514	bless [@_], CBOR::XS::Tagged::;
		515	}
		516
		517	sub CBOR::XS::Tagged::tag {
		518	$_[0][0] = $_[1] if $#_;
		519	$_[0][0]
		520	}
		521
		522	sub CBOR::XS::Tagged::value {
		523	$_[0][1] = $_[1] if $#_;
		524	$_[0][1]
		525	}
		526
		527	=head2 EXAMPLES
		528
		529	Here are some examples of C<CBOR::XS::Tagged> uses to tag objects.
		530
		531	You can look up CBOR tag value and emanings in the IANA registry at
		532	L<http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml>.
		533
		534	Prepend a magic header (C<$CBOR::XS::MAGIC>):
		535
		536	my $cbor = encode_cbor CBOR::XS::tag 55799, $value;
		537	# same as:
		538	my $cbor = $CBOR::XS::MAGIC . encode_cbor $value;
		539
		540	Serialise some URIs and a regex in an array:
		541
		542	my $cbor = encode_cbor [
		543	(CBOR::XS::tag 32, "http://www.nethype.de/"),
		544	(CBOR::XS::tag 32, "http://software.schmorp.de/"),
		545	(CBOR::XS::tag 35, "^[Pp][Ee][Rr][lL]\$"),
		546	];
		547
		548	Wrap CBOR data in CBOR:
		549
		550	my $cbor_cbor = encode_cbor
		551	CBOR::XS::tag 24,
		552	encode_cbor [1, 2, 3];
		553
281	=head2 CBOR and JSON	554	=head1 CBOR and JSON
282		555
283	TODO	556	CBOR is supposed to implement a superset of the JSON data model, and is,
		557	with some coercion, able to represent all JSON texts (something that other
		558	"binary JSON" formats such as BSON generally do not support).
		559
		560	CBOR implements some extra hints and support for JSON interoperability,
		561	and the spec offers further guidance for conversion between CBOR and
		562	JSON. None of this is currently implemented in CBOR, and the guidelines
		563	in the spec do not result in correct round-tripping of data. If JSON
		564	interoperability is improved in the future, then the goal will be to
		565	ensure that decoded JSON data will round-trip encoding and decoding to
		566	CBOR intact.
284		567
285		568
286	=head1 SECURITY CONSIDERATIONS	569	=head1 SECURITY CONSIDERATIONS
287		570
288	When you are using CBOR in a protocol, talking to untrusted potentially	571	When you are using CBOR in a protocol, talking to untrusted potentially
…		…
356	Please refrain from using rt.cpan.org or any other bug reporting	639	Please refrain from using rt.cpan.org or any other bug reporting
357	service. I put the contact address into my modules for a reason.	640	service. I put the contact address into my modules for a reason.
358		641
359	=cut	642	=cut
360		643
361	our $true = do { bless \(my $dummy = 1), "CBOR::XS::Boolean" };
362	our $false = do { bless \(my $dummy = 0), "CBOR::XS::Boolean" };
363
364	sub true() { $true }
365	sub false() { $false }
366
367	sub is_bool($) {
368	UNIVERSAL::isa $_[0], "CBOR::XS::Boolean"
369	# or UNIVERSAL::isa $_[0], "CBOR::Literal"
370	}
371
372	XSLoader::load "CBOR::XS", $VERSION;	644	XSLoader::load "CBOR::XS", $VERSION;
373
374	package CBOR::XS::Boolean;
375
376	use overload
377	"0+" => sub { ${$_[0]} },
378	"++" => sub { $_[0] = ${$_[0]} + 1 },
379	"--" => sub { $_[0] = ${$_[0]} - 1 },
380	fallback => 1;
381
382	1;
383		645
384	=head1 SEE ALSO	646	=head1 SEE ALSO
385		647
386	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,	648	The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,
387	serialisation.	649	serialisation.
388		650
		651	The L<Types::Serialiser> module provides the data model for true, false
		652	and error values.
		653
389	=head1 AUTHOR	654	=head1 AUTHOR
390		655
391	Marc Lehmann <schmorp@schmorp.de>	656	Marc Lehmann <schmorp@schmorp.de>
392	http://home.schmorp.de/	657	http://home.schmorp.de/
393		658
394	=cut	659	=cut
395		660
		661	1
		662

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing CBOR-XS/XS.pm (file contents): Revision 1.3 by root, Sat Oct 26 11:08:34 2013 UTC vs. Revision 1.14 by root, Tue Oct 29 20:59:16 2013 UTC

Diff Legend

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.3 by root, Sat Oct 26 11:08:34 2013 UTC vs.
Revision 1.14 by root, Tue Oct 29 20:59:16 2013 UTC