ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/README
(Generate patch)

Comparing CBOR-XS/README (file contents):
Revision 1.3 by root, Sat Oct 26 11:08:34 2013 UTC vs.
Revision 1.8 by root, Tue Oct 29 22:04:52 2013 UTC

… …
8 $perl_value = decode_cbor $binary_cbor_data; 8 $perl_value = decode_cbor $binary_cbor_data;
9 9
10 # OO-interface 10 # OO-interface
11 11
12 $coder = CBOR::XS->new; 12 $coder = CBOR::XS->new;
13 #TODO 13 $binary_cbor_data = $coder->encode ($perl_value);
14 $perl_value = $coder->decode ($binary_cbor_data);
15
16 # prefix decoding
17
18 my $many_cbor_strings = ...;
19 while (length $many_cbor_strings) {
20 my ($data, $length) = $cbor->decode_prefix ($many_cbor_strings);
21 # data was decoded
22 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
23 }
14 24
15DESCRIPTION 25DESCRIPTION
16 WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA 26 WARNING! This module is very new, and not very well tested (that's up to
17 AND EAT YOUR CHILDREN! 27 you to do). Furthermore, details of the implementation might change
28 freely before version 1.0. And lastly, the object serialisation protocol
29 depends on a pending IANA assignment, and until that assignment is
30 official, this implementation is not interoperable with other
31 implementations (even future versions of this module) until the
32 assignment is done.
18 33
19 This module converts Perl data structures to CBOR and vice versa. Its 34 You are still invited to try out CBOR, and this module.
20 primary goal is to be *correct* and its secondary goal is to be *fast*. 35
36 This module converts Perl data structures to the Concise Binary Object
37 Representation (CBOR) and vice versa. CBOR is a fast binary
38 serialisation format that aims to use a superset of the JSON data model,
39 i.e. when you can represent something in JSON, you should be able to
40 represent it in CBOR.
41
42 In short, CBOR is a faster and very compact binary alternative to JSON,
43 with the added ability of supporting serialisation of Perl objects.
44 (JSON often compresses better than CBOR though, so if you plan to
45 compress the data later you might want to compare both formats first).
46
47 To give you a general idea about speed, with texts in the megabyte
48 range, "CBOR::XS" usually encodes roughly twice as fast as Storable or
49 JSON::XS and decodes about 15%-30% faster than those. The shorter the
50 data, the worse Storable performs in comparison.
51
52 As for compactness, "CBOR::XS" encoded data structures are usually about
53 20% smaller than the same data encoded as (compact) JSON or Storable.
54
55 The primary goal of this module is to be *correct* and the secondary
21 To reach the latter goal it was written in C. 56 goal is to be *fast*. To reach the latter goal it was written in C.
22 57
23 See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and 58 See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
24 vice versa. 59 vice versa.
25 60
26FUNCTIONAL INTERFACE 61FUNCTIONAL INTERFACE
… …
118 For the more enlightened: note that in the following descriptions, 153 For the more enlightened: note that in the following descriptions,
119 lowercase *perl* refers to the Perl interpreter, while uppercase *Perl* 154 lowercase *perl* refers to the Perl interpreter, while uppercase *Perl*
120 refers to the abstract Perl language itself. 155 refers to the abstract Perl language itself.
121 156
122 CBOR -> PERL 157 CBOR -> PERL
123 True, False 158 integers
124 These CBOR values become "CBOR::XS::true" and "CBOR::XS::false", 159 CBOR integers become (numeric) perl scalars. On perls without 64 bit
160 support, 64 bit integers will be truncated or otherwise corrupted.
161
162 byte strings
163 Byte strings will become octet strings in Perl (the byte values
164 0..255 will simply become characters of the same value in Perl).
165
166 UTF-8 strings
167 UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
168 decoded into proper Unicode code points. At the moment, the validity
169 of the UTF-8 octets will not be validated - corrupt input will
170 result in corrupted Perl strings.
171
172 arrays, maps
173 CBOR arrays and CBOR maps will be converted into references to a
174 Perl array or hash, respectively. The keys of the map will be
175 stringified during this process.
176
177 null
178 CBOR null becomes "undef" in Perl.
179
180 true, false, undefined
181 These CBOR values become "Types:Serialiser::true",
182 "Types:Serialiser::false" and "Types::Serialiser::error",
125 respectively. They are overloaded to act almost exactly like the 183 respectively. They are overloaded to act almost exactly like the
126 numbers 1 and 0. You can check whether a scalar is a CBOR boolean by 184 numbers 1 and 0 (for true and false) or to throw an exception on
127 using the "CBOR::XS::is_bool" function. 185 access (for error). See the Types::Serialiser manpage for details.
128 186
129 Null, Undefined 187 CBOR tag 256 (perl object)
130 CBOR Null and Undefined values becomes "undef" in Perl (in the 188 The tag value 256 (TODO: pending iana registration) will be used to
131 future, Undefined may raise an exception). 189 deserialise a Perl object serialised with "FREEZE". See OBJECT
190 SERIALISATION, below, for details.
191
192 CBOR tag 55799 (magic header)
193 The tag 55799 is ignored (this tag implements the magic header).
194
195 other CBOR tags
196 Tagged items consists of a numeric tag and another CBOR value. Tags
197 not handled internally are currently converted into a
198 CBOR::XS::Tagged object, which is simply a blessed array reference
199 consisting of the numeric tag value followed by the (decoded) CBOR
200 value.
201
202 In the future, support for user-supplied conversions might get
203 added.
204
205 anything else
206 Anything else (e.g. unsupported simple values) will raise a decoding
207 error.
132 208
133 PERL -> CBOR 209 PERL -> CBOR
134 The mapping from Perl to CBOR is slightly more difficult, as Perl is a 210 The mapping from Perl to CBOR is slightly more difficult, as Perl is a
135 truly typeless language, so we can only guess which CBOR type is meant 211 truly typeless language, so we can only guess which CBOR type is meant
136 by a Perl value. 212 by a Perl value.
… …
138 hash references 214 hash references
139 Perl hash references become CBOR maps. As there is no inherent 215 Perl hash references become CBOR maps. As there is no inherent
140 ordering in hash keys (or CBOR maps), they will usually be encoded 216 ordering in hash keys (or CBOR maps), they will usually be encoded
141 in a pseudo-random order. 217 in a pseudo-random order.
142 218
219 Currently, tied hashes will use the indefinite-length format, while
220 normal hashes will use the fixed-length format.
221
143 array references 222 array references
144 Perl array references become CBOR arrays. 223 Perl array references become fixed-length CBOR arrays.
145 224
146 other references 225 other references
147 Other unblessed references are generally not allowed and will cause 226 Other unblessed references are generally not allowed and will cause
148 an exception to be thrown, except for references to the integers 0 227 an exception to be thrown, except for references to the integers 0
149 and 1, which get turned into "False" and "True" in CBOR. 228 and 1, which get turned into false and true in CBOR.
150 229
151 CBOR::XS::true, CBOR::XS::false 230 CBOR::XS::Tagged objects
231 Objects of this type must be arrays consisting of a single "[tag,
232 value]" pair. The (numerical) tag will be encoded as a CBOR tag, the
233 value will be encoded as appropriate for the value. You cna use
234 "CBOR::XS::tag" to create such objects.
235
236 Types::Serialiser::true, Types::Serialiser::false,
237 Types::Serialiser::error
152 These special values become CBOR True and CBOR False values, 238 These special values become CBOR true, CBOR false and CBOR undefined
153 respectively. You can also use "\1" and "\0" directly if you want. 239 values, respectively. You can also use "\1", "\0" and "\undef"
240 directly if you want.
154 241
155 blessed objects 242 other blessed objects
156 Blessed objects are not directly representable in CBOR. TODO See the 243 Other blessed objects are serialised via "TO_CBOR" or "FREEZE". See
157 "allow_blessed" and "convert_blessed" methods on various options on 244 "OBJECT SERIALISATION", below, for details.
158 how to deal with this: basically, you can choose between throwing an
159 exception, encoding the reference as if it weren't blessed, or
160 provide your own serialiser method.
161 245
162 simple scalars 246 simple scalars
163 TODO Simple Perl scalars (any scalar that is not a reference) are 247 TODO Simple Perl scalars (any scalar that is not a reference) are
164 the most difficult objects to encode: CBOR::XS will encode undefined 248 the most difficult objects to encode: CBOR::XS will encode undefined
165 scalars as CBOR "Null" values, scalars that have last been used in a 249 scalars as CBOR null values, scalars that have last been used in a
166 string context before encoding as CBOR strings, and anything else as 250 string context before encoding as CBOR strings, and anything else as
167 number value: 251 number value:
168 252
169 # dump as number 253 # dump as number
170 encode_cbor [2] # yields [2] 254 encode_cbor [2] # yields [2]
… …
193 277
194 You can not currently force the type in other, less obscure, ways. 278 You can not currently force the type in other, less obscure, ways.
195 Tell me if you need this capability (but don't forget to explain why 279 Tell me if you need this capability (but don't forget to explain why
196 it's needed :). 280 it's needed :).
197 281
198 Note that numerical precision has the same meaning as under Perl (so 282 Perl values that seem to be integers generally use the shortest
199 binary to decimal conversion follows the same rules as in Perl, 283 possible representation. Floating-point values will use either the
200 which can differ to other languages). Also, your perl interpreter 284 IEEE single format if possible without loss of precision, otherwise
201 might expose extensions to the floating point numbers of your 285 the IEEE double format will be used. Perls that use formats other
202 platform, such as infinities or NaN's - these cannot be represented 286 than IEEE double to represent numerical values are supported, but
203 in CBOR, and it is an error to pass those in. 287 might suffer loss of precision.
204 288
289 OBJECT SERIALISATION
290 This module knows two way to serialise a Perl object: The CBOR-specific
291 way, and the generic way.
292
293 Whenever the encoder encounters a Perl object that it cnanot serialise
294 directly (most of them), it will first look up the "TO_CBOR" method on
295 it.
296
297 If it has a "TO_CBOR" method, it will call it with the object as only
298 argument, and expects exactly one return value, which it will then
299 substitute and encode it in the place of the object.
300
301 Otherwise, it will look up the "FREEZE" method. If it exists, it will
302 call it with the object as first argument, and the constant string
303 "CBOR" as the second argument, to distinguish it from other serialisers.
304
305 The "FREEZE" method can return any number of values (i.e. zero or more).
306 These will be encoded as CBOR perl object, together with the classname.
307
308 If an object supports neither "TO_CBOR" nor "FREEZE", encoding will fail
309 with an error.
310
311 Objects encoded via "TO_CBOR" cannot be automatically decoded, but
312 objects encoded via "FREEZE" can be decoded using the following
313 protocol:
314
315 When an encoded CBOR perl object is encountered by the decoder, it will
316 look up the "THAW" method, by using the stored classname, and will fail
317 if the method cannot be found.
318
319 After the lookup it will call the "THAW" method with the stored
320 classname as first argument, the constant string "CBOR" as second
321 argument, and all values returned by "FREEZE" as remaining arguments.
322
323 EXAMPLES
324 Here is an example "TO_CBOR" method:
325
326 sub My::Object::TO_CBOR {
327 my ($obj) = @_;
328
329 ["this is a serialised My::Object object", $obj->{id}]
330 }
331
332 When a "My::Object" is encoded to CBOR, it will instead encode a simple
333 array with two members: a string, and the "object id". Decoding this
334 CBOR string will yield a normal perl array reference in place of the
335 object.
336
337 A more useful and practical example would be a serialisation method for
338 the URI module. CBOR has a custom tag value for URIs, namely 32:
339
340 sub URI::TO_CBOR {
341 my ($self) = @_;
342 my $uri = "$self"; # stringify uri
343 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
344 CBOR::XS::tagged 32, "$_[0]"
345 }
346
347 This will encode URIs as a UTF-8 string with tag 32, which indicates an
348 URI.
349
350 Decoding such an URI will not (currently) give you an URI object, but
351 instead a CBOR::XS::Tagged object with tag number 32 and the string -
352 exactly what was returned by "TO_CBOR".
353
354 To serialise an object so it can automatically be deserialised, you need
355 to use "FREEZE" and "THAW". To take the URI module as example, this
356 would be a possible implementation:
357
358 sub URI::FREEZE {
359 my ($self, $serialiser) = @_;
360 "$self" # encode url string
361 }
362
363 sub URI::THAW {
364 my ($class, $serialiser, $uri) = @_;
365
366 $class->new ($uri)
367 }
368
369 Unlike "TO_CBOR", multiple values can be returned by "FREEZE". For
370 example, a "FREEZE" method that returns "type", "id" and "variant"
371 values would cause an invocation of "THAW" with 5 arguments:
372
373 sub My::Object::FREEZE {
374 my ($self, $serialiser) = @_;
375
376 ($self->{type}, $self->{id}, $self->{variant})
377 }
378
379 sub My::Object::THAW {
380 my ($class, $serialiser, $type, $id, $variant) = @_;
381
382 $class-<new (type => $type, id => $id, variant => $variant)
383 }
384
205 MAGIC HEADER 385MAGIC HEADER
206 There is no way to distinguish CBOR from other formats programmatically. 386 There is no way to distinguish CBOR from other formats programmatically.
207 To make it easier to distinguish CBOR from other formats, the CBOR 387 To make it easier to distinguish CBOR from other formats, the CBOR
208 specification has a special "magic string" that can be prepended to any 388 specification has a special "magic string" that can be prepended to any
209 CBOR string without changing it's meaning. 389 CBOR string without changing it's meaning.
210 390
211 This string is available as $CBOR::XS::MAGIC. This module does not 391 This string is available as $CBOR::XS::MAGIC. This module does not
212 prepend this string tot he CBOR data it generates, but it will ignroe it 392 prepend this string tot he CBOR data it generates, but it will ignroe it
213 if present, so users can prepend this string as a "file type" indicator 393 if present, so users can prepend this string as a "file type" indicator
214 as required. 394 as required.
215 395
396THE CBOR::XS::Tagged CLASS
397 CBOR has the concept of tagged values - any CBOR value can be tagged
398 with a numeric 64 bit number, which are centrally administered.
399
400 "CBOR::XS" handles a few tags internally when en- or decoding. You can
401 also create tags yourself by encoding "CBOR::XS::Tagged" objects, and
402 the decoder will create "CBOR::XS::Tagged" objects itself when it hits
403 an unknown tag.
404
405 These objects are simply blessed array references - the first member of
406 the array being the numerical tag, the second being the value.
407
408 You can interact with "CBOR::XS::Tagged" objects in the following ways:
409
410 $tagged = CBOR::XS::tag $tag, $value
411 This function(!) creates a new "CBOR::XS::Tagged" object using the
412 given $tag (0..2**64-1) to tag the given $value (which can be any
413 Perl value that can be encoded in CBOR, including serialisable Perl
414 objects and "CBOR::XS::Tagged" objects).
415
416 $tagged->[0]
417 $tagged->[0] = $new_tag
418 $tag = $tagged->tag
419 $new_tag = $tagged->tag ($new_tag)
420 Access/mutate the tag.
421
422 $tagged->[1]
423 $tagged->[1] = $new_value
424 $value = $tagged->value
425 $new_value = $tagged->value ($new_value)
426 Access/mutate the tagged value.
427
428 EXAMPLES
429 Here are some examples of "CBOR::XS::Tagged" uses to tag objects.
430
431 You can look up CBOR tag value and emanings in the IANA registry at
432 <http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml>.
433
434 Prepend a magic header ($CBOR::XS::MAGIC):
435
436 my $cbor = encode_cbor CBOR::XS::tag 55799, $value;
437 # same as:
438 my $cbor = $CBOR::XS::MAGIC . encode_cbor $value;
439
440 Serialise some URIs and a regex in an array:
441
442 my $cbor = encode_cbor [
443 (CBOR::XS::tag 32, "http://www.nethype.de/"),
444 (CBOR::XS::tag 32, "http://software.schmorp.de/"),
445 (CBOR::XS::tag 35, "^[Pp][Ee][Rr][lL]\$"),
446 ];
447
448 Wrap CBOR data in CBOR:
449
450 my $cbor_cbor = encode_cbor
451 CBOR::XS::tag 24,
452 encode_cbor [1, 2, 3];
453
216 CBOR and JSON 454CBOR and JSON
217 TODO 455 CBOR is supposed to implement a superset of the JSON data model, and is,
456 with some coercion, able to represent all JSON texts (something that
457 other "binary JSON" formats such as BSON generally do not support).
458
459 CBOR implements some extra hints and support for JSON interoperability,
460 and the spec offers further guidance for conversion between CBOR and
461 JSON. None of this is currently implemented in CBOR, and the guidelines
462 in the spec do not result in correct round-tripping of data. If JSON
463 interoperability is improved in the future, then the goal will be to
464 ensure that decoded JSON data will round-trip encoding and decoding to
465 CBOR intact.
218 466
219SECURITY CONSIDERATIONS 467SECURITY CONSIDERATIONS
220 When you are using CBOR in a protocol, talking to untrusted potentially 468 When you are using CBOR in a protocol, talking to untrusted potentially
221 hostile creatures requires relatively few measures. 469 hostile creatures requires relatively few measures.
222 470
… …
285 533
286SEE ALSO 534SEE ALSO
287 The JSON and JSON::XS modules that do similar, but human-readable, 535 The JSON and JSON::XS modules that do similar, but human-readable,
288 serialisation. 536 serialisation.
289 537
538 The Types::Serialiser module provides the data model for true, false and
539 error values.
540
290AUTHOR 541AUTHOR
291 Marc Lehmann <schmorp@schmorp.de> 542 Marc Lehmann <schmorp@schmorp.de>
292 http://home.schmorp.de/ 543 http://home.schmorp.de/
293 544

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines