ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/README
(Generate patch)

Comparing CBOR-XS/README (file contents):
Revision 1.3 by root, Sat Oct 26 11:08:34 2013 UTC vs.
Revision 1.7 by root, Tue Oct 29 15:56:31 2013 UTC

… …
8 $perl_value = decode_cbor $binary_cbor_data; 8 $perl_value = decode_cbor $binary_cbor_data;
9 9
10 # OO-interface 10 # OO-interface
11 11
12 $coder = CBOR::XS->new; 12 $coder = CBOR::XS->new;
13 #TODO 13 $binary_cbor_data = $coder->encode ($perl_value);
14 $perl_value = $coder->decode ($binary_cbor_data);
15
16 # prefix decoding
17
18 my $many_cbor_strings = ...;
19 while (length $many_cbor_strings) {
20 my ($data, $length) = $cbor->decode_prefix ($many_cbor_strings);
21 # data was decoded
22 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
23 }
14 24
15DESCRIPTION 25DESCRIPTION
16 WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA 26 WARNING! This module is very new, and not very well tested (that's up to
17 AND EAT YOUR CHILDREN! 27 you to do). Furthermore, details of the implementation might change
28 freely before version 1.0. And lastly, the object serialisation protocol
29 depends on a pending IANA assignment, and until that assignment is
30 official, this implementation is not interoperable with other
31 implementations (even future versions of this module) until the
32 assignment is done.
18 33
19 This module converts Perl data structures to CBOR and vice versa. Its 34 You are still invited to try out CBOR, and this module.
20 primary goal is to be *correct* and its secondary goal is to be *fast*. 35
36 This module converts Perl data structures to the Concise Binary Object
37 Representation (CBOR) and vice versa. CBOR is a fast binary
38 serialisation format that aims to use a superset of the JSON data model,
39 i.e. when you can represent something in JSON, you should be able to
40 represent it in CBOR.
41
42 In short, CBOR is a faster and very compact binary alternative to JSON,
43 with the added ability of supporting serialisation of Perl objects.
44 (JSON often compresses better than CBOR though, so if you plan to
45 compress the data later you might want to compare both formats first).
46
47 The primary goal of this module is to be *correct* and the secondary
21 To reach the latter goal it was written in C. 48 goal is to be *fast*. To reach the latter goal it was written in C.
22 49
23 See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and 50 See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
24 vice versa. 51 vice versa.
25 52
26FUNCTIONAL INTERFACE 53FUNCTIONAL INTERFACE
… …
118 For the more enlightened: note that in the following descriptions, 145 For the more enlightened: note that in the following descriptions,
119 lowercase *perl* refers to the Perl interpreter, while uppercase *Perl* 146 lowercase *perl* refers to the Perl interpreter, while uppercase *Perl*
120 refers to the abstract Perl language itself. 147 refers to the abstract Perl language itself.
121 148
122 CBOR -> PERL 149 CBOR -> PERL
123 True, False 150 integers
124 These CBOR values become "CBOR::XS::true" and "CBOR::XS::false", 151 CBOR integers become (numeric) perl scalars. On perls without 64 bit
152 support, 64 bit integers will be truncated or otherwise corrupted.
153
154 byte strings
155 Byte strings will become octet strings in Perl (the byte values
156 0..255 will simply become characters of the same value in Perl).
157
158 UTF-8 strings
159 UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
160 decoded into proper Unicode code points. At the moment, the validity
161 of the UTF-8 octets will not be validated - corrupt input will
162 result in corrupted Perl strings.
163
164 arrays, maps
165 CBOR arrays and CBOR maps will be converted into references to a
166 Perl array or hash, respectively. The keys of the map will be
167 stringified during this process.
168
169 null
170 CBOR null becomes "undef" in Perl.
171
172 true, false, undefined
173 These CBOR values become "Types:Serialiser::true",
174 "Types:Serialiser::false" and "Types::Serialiser::error",
125 respectively. They are overloaded to act almost exactly like the 175 respectively. They are overloaded to act almost exactly like the
126 numbers 1 and 0. You can check whether a scalar is a CBOR boolean by 176 numbers 1 and 0 (for true and false) or to throw an exception on
127 using the "CBOR::XS::is_bool" function. 177 access (for error). See the Types::Serialiser manpage for details.
128 178
129 Null, Undefined 179 CBOR tag 256 (perl object)
130 CBOR Null and Undefined values becomes "undef" in Perl (in the 180 The tag value 256 (TODO: pending iana registration) will be used to
131 future, Undefined may raise an exception). 181 deserialise a Perl object serialised with "FREEZE". See OBJECT
182 SERIALISATION, below, for details.
183
184 CBOR tag 55799 (magic header)
185 The tag 55799 is ignored (this tag implements the magic header).
186
187 other CBOR tags
188 Tagged items consists of a numeric tag and another CBOR value. Tags
189 not handled internally are currently converted into a
190 CBOR::XS::Tagged object, which is simply a blessed array reference
191 consisting of the numeric tag value followed by the (decoded) CBOR
192 value.
193
194 In the future, support for user-supplied conversions might get
195 added.
196
197 anything else
198 Anything else (e.g. unsupported simple values) will raise a decoding
199 error.
132 200
133 PERL -> CBOR 201 PERL -> CBOR
134 The mapping from Perl to CBOR is slightly more difficult, as Perl is a 202 The mapping from Perl to CBOR is slightly more difficult, as Perl is a
135 truly typeless language, so we can only guess which CBOR type is meant 203 truly typeless language, so we can only guess which CBOR type is meant
136 by a Perl value. 204 by a Perl value.
… …
138 hash references 206 hash references
139 Perl hash references become CBOR maps. As there is no inherent 207 Perl hash references become CBOR maps. As there is no inherent
140 ordering in hash keys (or CBOR maps), they will usually be encoded 208 ordering in hash keys (or CBOR maps), they will usually be encoded
141 in a pseudo-random order. 209 in a pseudo-random order.
142 210
211 Currently, tied hashes will use the indefinite-length format, while
212 normal hashes will use the fixed-length format.
213
143 array references 214 array references
144 Perl array references become CBOR arrays. 215 Perl array references become fixed-length CBOR arrays.
145 216
146 other references 217 other references
147 Other unblessed references are generally not allowed and will cause 218 Other unblessed references are generally not allowed and will cause
148 an exception to be thrown, except for references to the integers 0 219 an exception to be thrown, except for references to the integers 0
149 and 1, which get turned into "False" and "True" in CBOR. 220 and 1, which get turned into false and true in CBOR.
150 221
151 CBOR::XS::true, CBOR::XS::false 222 CBOR::XS::Tagged objects
223 Objects of this type must be arrays consisting of a single "[tag,
224 value]" pair. The (numerical) tag will be encoded as a CBOR tag, the
225 value will be encoded as appropriate for the value. You cna use
226 "CBOR::XS::tag" to create such objects.
227
228 Types::Serialiser::true, Types::Serialiser::false,
229 Types::Serialiser::error
152 These special values become CBOR True and CBOR False values, 230 These special values become CBOR true, CBOR false and CBOR undefined
153 respectively. You can also use "\1" and "\0" directly if you want. 231 values, respectively. You can also use "\1", "\0" and "\undef"
232 directly if you want.
154 233
155 blessed objects 234 other blessed objects
156 Blessed objects are not directly representable in CBOR. TODO See the 235 Other blessed objects are serialised via "TO_CBOR" or "FREEZE". See
157 "allow_blessed" and "convert_blessed" methods on various options on 236 "OBJECT SERIALISATION", below, for details.
158 how to deal with this: basically, you can choose between throwing an
159 exception, encoding the reference as if it weren't blessed, or
160 provide your own serialiser method.
161 237
162 simple scalars 238 simple scalars
163 TODO Simple Perl scalars (any scalar that is not a reference) are 239 TODO Simple Perl scalars (any scalar that is not a reference) are
164 the most difficult objects to encode: CBOR::XS will encode undefined 240 the most difficult objects to encode: CBOR::XS will encode undefined
165 scalars as CBOR "Null" values, scalars that have last been used in a 241 scalars as CBOR null values, scalars that have last been used in a
166 string context before encoding as CBOR strings, and anything else as 242 string context before encoding as CBOR strings, and anything else as
167 number value: 243 number value:
168 244
169 # dump as number 245 # dump as number
170 encode_cbor [2] # yields [2] 246 encode_cbor [2] # yields [2]
… …
193 269
194 You can not currently force the type in other, less obscure, ways. 270 You can not currently force the type in other, less obscure, ways.
195 Tell me if you need this capability (but don't forget to explain why 271 Tell me if you need this capability (but don't forget to explain why
196 it's needed :). 272 it's needed :).
197 273
198 Note that numerical precision has the same meaning as under Perl (so 274 Perl values that seem to be integers generally use the shortest
199 binary to decimal conversion follows the same rules as in Perl, 275 possible representation. Floating-point values will use either the
200 which can differ to other languages). Also, your perl interpreter 276 IEEE single format if possible without loss of precision, otherwise
201 might expose extensions to the floating point numbers of your 277 the IEEE double format will be used. Perls that use formats other
202 platform, such as infinities or NaN's - these cannot be represented 278 than IEEE double to represent numerical values are supported, but
203 in CBOR, and it is an error to pass those in. 279 might suffer loss of precision.
204 280
281 OBJECT SERIALISATION
282 This module knows two way to serialise a Perl object: The CBOR-specific
283 way, and the generic way.
284
285 Whenever the encoder encounters a Perl object that it cnanot serialise
286 directly (most of them), it will first look up the "TO_CBOR" method on
287 it.
288
289 If it has a "TO_CBOR" method, it will call it with the object as only
290 argument, and expects exactly one return value, which it will then
291 substitute and encode it in the place of the object.
292
293 Otherwise, it will look up the "FREEZE" method. If it exists, it will
294 call it with the object as first argument, and the constant string
295 "CBOR" as the second argument, to distinguish it from other serialisers.
296
297 The "FREEZE" method can return any number of values (i.e. zero or more).
298 These will be encoded as CBOR perl object, together with the classname.
299
300 If an object supports neither "TO_CBOR" nor "FREEZE", encoding will fail
301 with an error.
302
303 Objects encoded via "TO_CBOR" cannot be automatically decoded, but
304 objects encoded via "FREEZE" can be decoded using the following
305 protocol:
306
307 When an encoded CBOR perl object is encountered by the decoder, it will
308 look up the "THAW" method, by using the stored classname, and will fail
309 if the method cannot be found.
310
311 After the lookup it will call the "THAW" method with the stored
312 classname as first argument, the constant string "CBOR" as second
313 argument, and all values returned by "FREEZE" as remaining arguments.
314
315 EXAMPLES
316 Here is an example "TO_CBOR" method:
317
318 sub My::Object::TO_CBOR {
319 my ($obj) = @_;
320
321 ["this is a serialised My::Object object", $obj->{id}]
322 }
323
324 When a "My::Object" is encoded to CBOR, it will instead encode a simple
325 array with two members: a string, and the "object id". Decoding this
326 CBOR string will yield a normal perl array reference in place of the
327 object.
328
329 A more useful and practical example would be a serialisation method for
330 the URI module. CBOR has a custom tag value for URIs, namely 32:
331
332 sub URI::TO_CBOR {
333 my ($self) = @_;
334 my $uri = "$self"; # stringify uri
335 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
336 CBOR::XS::tagged 32, "$_[0]"
337 }
338
339 This will encode URIs as a UTF-8 string with tag 32, which indicates an
340 URI.
341
342 Decoding such an URI will not (currently) give you an URI object, but
343 instead a CBOR::XS::Tagged object with tag number 32 and the string -
344 exactly what was returned by "TO_CBOR".
345
346 To serialise an object so it can automatically be deserialised, you need
347 to use "FREEZE" and "THAW". To take the URI module as example, this
348 would be a possible implementation:
349
350 sub URI::FREEZE {
351 my ($self, $serialiser) = @_;
352 "$self" # encode url string
353 }
354
355 sub URI::THAW {
356 my ($class, $serialiser, $uri) = @_;
357
358 $class->new ($uri)
359 }
360
361 Unlike "TO_CBOR", multiple values can be returned by "FREEZE". For
362 example, a "FREEZE" method that returns "type", "id" and "variant"
363 values would cause an invocation of "THAW" with 5 arguments:
364
365 sub My::Object::FREEZE {
366 my ($self, $serialiser) = @_;
367
368 ($self->{type}, $self->{id}, $self->{variant})
369 }
370
371 sub My::Object::THAW {
372 my ($class, $serialiser, $type, $id, $variant) = @_;
373
374 $class-<new (type => $type, id => $id, variant => $variant)
375 }
376
205 MAGIC HEADER 377MAGIC HEADER
206 There is no way to distinguish CBOR from other formats programmatically. 378 There is no way to distinguish CBOR from other formats programmatically.
207 To make it easier to distinguish CBOR from other formats, the CBOR 379 To make it easier to distinguish CBOR from other formats, the CBOR
208 specification has a special "magic string" that can be prepended to any 380 specification has a special "magic string" that can be prepended to any
209 CBOR string without changing it's meaning. 381 CBOR string without changing it's meaning.
210 382
211 This string is available as $CBOR::XS::MAGIC. This module does not 383 This string is available as $CBOR::XS::MAGIC. This module does not
212 prepend this string tot he CBOR data it generates, but it will ignroe it 384 prepend this string tot he CBOR data it generates, but it will ignroe it
213 if present, so users can prepend this string as a "file type" indicator 385 if present, so users can prepend this string as a "file type" indicator
214 as required. 386 as required.
215 387
388THE CBOR::XS::Tagged CLASS
389 CBOR has the concept of tagged values - any CBOR value can be tagged
390 with a numeric 64 bit number, which are centrally administered.
391
392 "CBOR::XS" handles a few tags internally when en- or decoding. You can
393 also create tags yourself by encoding "CBOR::XS::Tagged" objects, and
394 the decoder will create "CBOR::XS::Tagged" objects itself when it hits
395 an unknown tag.
396
397 These objects are simply blessed array references - the first member of
398 the array being the numerical tag, the second being the value.
399
400 You can interact with "CBOR::XS::Tagged" objects in the following ways:
401
402 $tagged = CBOR::XS::tag $tag, $value
403 This function(!) creates a new "CBOR::XS::Tagged" object using the
404 given $tag (0..2**64-1) to tag the given $value (which can be any
405 Perl value that can be encoded in CBOR, including serialisable Perl
406 objects and "CBOR::XS::Tagged" objects).
407
408 $tagged->[0]
409 $tagged->[0] = $new_tag
410 $tag = $tagged->tag
411 $new_tag = $tagged->tag ($new_tag)
412 Access/mutate the tag.
413
414 $tagged->[1]
415 $tagged->[1] = $new_value
416 $value = $tagged->value
417 $new_value = $tagged->value ($new_value)
418 Access/mutate the tagged value.
419
420 EXAMPLES
421 Here are some examples of "CBOR::XS::Tagged" uses to tag objects.
422
423 You can look up CBOR tag value and emanings in the IANA registry at
424 <http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml>.
425
426 Prepend a magic header ($CBOR::XS::MAGIC):
427
428 my $cbor = encode_cbor CBOR::XS::tag 55799, $value;
429 # same as:
430 my $cbor = $CBOR::XS::MAGIC . encode_cbor $value;
431
432 Serialise some URIs and a regex in an array:
433
434 my $cbor = encode_cbor [
435 (CBOR::XS::tag 32, "http://www.nethype.de/"),
436 (CBOR::XS::tag 32, "http://software.schmorp.de/"),
437 (CBOR::XS::tag 35, "^[Pp][Ee][Rr][lL]\$"),
438 ];
439
440 Wrap CBOR data in CBOR:
441
442 my $cbor_cbor = encode_cbor
443 CBOR::XS::tag 24,
444 encode_cbor [1, 2, 3];
445
216 CBOR and JSON 446CBOR and JSON
217 TODO 447 CBOR is supposed to implement a superset of the JSON data model, and is,
448 with some coercion, able to represent all JSON texts (something that
449 other "binary JSON" formats such as BSON generally do not support).
450
451 CBOR implements some extra hints and support for JSON interoperability,
452 and the spec offers further guidance for conversion between CBOR and
453 JSON. None of this is currently implemented in CBOR, and the guidelines
454 in the spec do not result in correct round-tripping of data. If JSON
455 interoperability is improved in the future, then the goal will be to
456 ensure that decoded JSON data will round-trip encoding and decoding to
457 CBOR intact.
218 458
219SECURITY CONSIDERATIONS 459SECURITY CONSIDERATIONS
220 When you are using CBOR in a protocol, talking to untrusted potentially 460 When you are using CBOR in a protocol, talking to untrusted potentially
221 hostile creatures requires relatively few measures. 461 hostile creatures requires relatively few measures.
222 462
… …
285 525
286SEE ALSO 526SEE ALSO
287 The JSON and JSON::XS modules that do similar, but human-readable, 527 The JSON and JSON::XS modules that do similar, but human-readable,
288 serialisation. 528 serialisation.
289 529
530 The Types::Serialiser module provides the data model for true, false and
531 error values.
532
290AUTHOR 533AUTHOR
291 Marc Lehmann <schmorp@schmorp.de> 534 Marc Lehmann <schmorp@schmorp.de>
292 http://home.schmorp.de/ 535 http://home.schmorp.de/
293 536

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines