ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/README
(Generate patch)

Comparing CBOR-XS/README (file contents):
Revision 1.2 by root, Sat Oct 26 10:41:12 2013 UTC vs.
Revision 1.7 by root, Tue Oct 29 15:56:31 2013 UTC

… …
8 $perl_value = decode_cbor $binary_cbor_data; 8 $perl_value = decode_cbor $binary_cbor_data;
9 9
10 # OO-interface 10 # OO-interface
11 11
12 $coder = CBOR::XS->new; 12 $coder = CBOR::XS->new;
13 #TODO 13 $binary_cbor_data = $coder->encode ($perl_value);
14 $perl_value = $coder->decode ($binary_cbor_data);
15
16 # prefix decoding
17
18 my $many_cbor_strings = ...;
19 while (length $many_cbor_strings) {
20 my ($data, $length) = $cbor->decode_prefix ($many_cbor_strings);
21 # data was decoded
22 substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
23 }
14 24
15DESCRIPTION 25DESCRIPTION
16 WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA 26 WARNING! This module is very new, and not very well tested (that's up to
17 AND EAT YOUR CHILDREN! 27 you to do). Furthermore, details of the implementation might change
28 freely before version 1.0. And lastly, the object serialisation protocol
29 depends on a pending IANA assignment, and until that assignment is
30 official, this implementation is not interoperable with other
31 implementations (even future versions of this module) until the
32 assignment is done.
18 33
19 This module converts Perl data structures to CBOR and vice versa. Its 34 You are still invited to try out CBOR, and this module.
20 primary goal is to be *correct* and its secondary goal is to be *fast*. 35
36 This module converts Perl data structures to the Concise Binary Object
37 Representation (CBOR) and vice versa. CBOR is a fast binary
38 serialisation format that aims to use a superset of the JSON data model,
39 i.e. when you can represent something in JSON, you should be able to
40 represent it in CBOR.
41
42 In short, CBOR is a faster and very compact binary alternative to JSON,
43 with the added ability of supporting serialisation of Perl objects.
44 (JSON often compresses better than CBOR though, so if you plan to
45 compress the data later you might want to compare both formats first).
46
47 The primary goal of this module is to be *correct* and the secondary
21 To reach the latter goal it was written in C. 48 goal is to be *fast*. To reach the latter goal it was written in C.
22 49
23 See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and 50 See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
24 vice versa. 51 vice versa.
25 52
26FUNCTIONAL INTERFACE 53FUNCTIONAL INTERFACE
… …
118 For the more enlightened: note that in the following descriptions, 145 For the more enlightened: note that in the following descriptions,
119 lowercase *perl* refers to the Perl interpreter, while uppercase *Perl* 146 lowercase *perl* refers to the Perl interpreter, while uppercase *Perl*
120 refers to the abstract Perl language itself. 147 refers to the abstract Perl language itself.
121 148
122 CBOR -> PERL 149 CBOR -> PERL
123 True, False 150 integers
124 These CBOR values become "CBOR::XS::true" and "CBOR::XS::false", 151 CBOR integers become (numeric) perl scalars. On perls without 64 bit
152 support, 64 bit integers will be truncated or otherwise corrupted.
153
154 byte strings
155 Byte strings will become octet strings in Perl (the byte values
156 0..255 will simply become characters of the same value in Perl).
157
158 UTF-8 strings
159 UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
160 decoded into proper Unicode code points. At the moment, the validity
161 of the UTF-8 octets will not be validated - corrupt input will
162 result in corrupted Perl strings.
163
164 arrays, maps
165 CBOR arrays and CBOR maps will be converted into references to a
166 Perl array or hash, respectively. The keys of the map will be
167 stringified during this process.
168
169 null
170 CBOR null becomes "undef" in Perl.
171
172 true, false, undefined
173 These CBOR values become "Types:Serialiser::true",
174 "Types:Serialiser::false" and "Types::Serialiser::error",
125 respectively. They are overloaded to act almost exactly like the 175 respectively. They are overloaded to act almost exactly like the
126 numbers 1 and 0. You can check whether a scalar is a CBOR boolean by 176 numbers 1 and 0 (for true and false) or to throw an exception on
127 using the "CBOR::XS::is_bool" function. 177 access (for error). See the Types::Serialiser manpage for details.
128 178
129 null 179 CBOR tag 256 (perl object)
130 A CBOR Null value becomes "undef" in Perl. 180 The tag value 256 (TODO: pending iana registration) will be used to
181 deserialise a Perl object serialised with "FREEZE". See OBJECT
182 SERIALISATION, below, for details.
183
184 CBOR tag 55799 (magic header)
185 The tag 55799 is ignored (this tag implements the magic header).
186
187 other CBOR tags
188 Tagged items consists of a numeric tag and another CBOR value. Tags
189 not handled internally are currently converted into a
190 CBOR::XS::Tagged object, which is simply a blessed array reference
191 consisting of the numeric tag value followed by the (decoded) CBOR
192 value.
193
194 In the future, support for user-supplied conversions might get
195 added.
196
197 anything else
198 Anything else (e.g. unsupported simple values) will raise a decoding
199 error.
131 200
132 PERL -> CBOR 201 PERL -> CBOR
133 The mapping from Perl to CBOR is slightly more difficult, as Perl is a 202 The mapping from Perl to CBOR is slightly more difficult, as Perl is a
134 truly typeless language, so we can only guess which CBOR type is meant 203 truly typeless language, so we can only guess which CBOR type is meant
135 by a Perl value. 204 by a Perl value.
… …
137 hash references 206 hash references
138 Perl hash references become CBOR maps. As there is no inherent 207 Perl hash references become CBOR maps. As there is no inherent
139 ordering in hash keys (or CBOR maps), they will usually be encoded 208 ordering in hash keys (or CBOR maps), they will usually be encoded
140 in a pseudo-random order. 209 in a pseudo-random order.
141 210
211 Currently, tied hashes will use the indefinite-length format, while
212 normal hashes will use the fixed-length format.
213
142 array references 214 array references
143 Perl array references become CBOR arrays. 215 Perl array references become fixed-length CBOR arrays.
144 216
145 other references 217 other references
146 Other unblessed references are generally not allowed and will cause 218 Other unblessed references are generally not allowed and will cause
147 an exception to be thrown, except for references to the integers 0 219 an exception to be thrown, except for references to the integers 0
148 and 1, which get turned into "False" and "True" in CBOR. 220 and 1, which get turned into false and true in CBOR.
149 221
150 CBOR::XS::true, CBOR::XS::false 222 CBOR::XS::Tagged objects
223 Objects of this type must be arrays consisting of a single "[tag,
224 value]" pair. The (numerical) tag will be encoded as a CBOR tag, the
225 value will be encoded as appropriate for the value. You cna use
226 "CBOR::XS::tag" to create such objects.
227
228 Types::Serialiser::true, Types::Serialiser::false,
229 Types::Serialiser::error
151 These special values become CBOR True and CBOR False values, 230 These special values become CBOR true, CBOR false and CBOR undefined
152 respectively. You can also use "\1" and "\0" directly if you want. 231 values, respectively. You can also use "\1", "\0" and "\undef"
232 directly if you want.
153 233
154 blessed objects 234 other blessed objects
155 Blessed objects are not directly representable in CBOR. TODO See the 235 Other blessed objects are serialised via "TO_CBOR" or "FREEZE". See
156 "allow_blessed" and "convert_blessed" methods on various options on 236 "OBJECT SERIALISATION", below, for details.
157 how to deal with this: basically, you can choose between throwing an
158 exception, encoding the reference as if it weren't blessed, or
159 provide your own serialiser method.
160 237
161 simple scalars 238 simple scalars
162 TODO Simple Perl scalars (any scalar that is not a reference) are 239 TODO Simple Perl scalars (any scalar that is not a reference) are
163 the most difficult objects to encode: CBOR::XS will encode undefined 240 the most difficult objects to encode: CBOR::XS will encode undefined
164 scalars as CBOR "Null" values, scalars that have last been used in a 241 scalars as CBOR null values, scalars that have last been used in a
165 string context before encoding as CBOR strings, and anything else as 242 string context before encoding as CBOR strings, and anything else as
166 number value: 243 number value:
167 244
168 # dump as number 245 # dump as number
169 encode_cbor [2] # yields [2] 246 encode_cbor [2] # yields [2]
… …
192 269
193 You can not currently force the type in other, less obscure, ways. 270 You can not currently force the type in other, less obscure, ways.
194 Tell me if you need this capability (but don't forget to explain why 271 Tell me if you need this capability (but don't forget to explain why
195 it's needed :). 272 it's needed :).
196 273
197 Note that numerical precision has the same meaning as under Perl (so 274 Perl values that seem to be integers generally use the shortest
198 binary to decimal conversion follows the same rules as in Perl, 275 possible representation. Floating-point values will use either the
199 which can differ to other languages). Also, your perl interpreter 276 IEEE single format if possible without loss of precision, otherwise
200 might expose extensions to the floating point numbers of your 277 the IEEE double format will be used. Perls that use formats other
201 platform, such as infinities or NaN's - these cannot be represented 278 than IEEE double to represent numerical values are supported, but
202 in CBOR, and it is an error to pass those in. 279 might suffer loss of precision.
203 280
281 OBJECT SERIALISATION
282 This module knows two way to serialise a Perl object: The CBOR-specific
283 way, and the generic way.
284
285 Whenever the encoder encounters a Perl object that it cnanot serialise
286 directly (most of them), it will first look up the "TO_CBOR" method on
287 it.
288
289 If it has a "TO_CBOR" method, it will call it with the object as only
290 argument, and expects exactly one return value, which it will then
291 substitute and encode it in the place of the object.
292
293 Otherwise, it will look up the "FREEZE" method. If it exists, it will
294 call it with the object as first argument, and the constant string
295 "CBOR" as the second argument, to distinguish it from other serialisers.
296
297 The "FREEZE" method can return any number of values (i.e. zero or more).
298 These will be encoded as CBOR perl object, together with the classname.
299
300 If an object supports neither "TO_CBOR" nor "FREEZE", encoding will fail
301 with an error.
302
303 Objects encoded via "TO_CBOR" cannot be automatically decoded, but
304 objects encoded via "FREEZE" can be decoded using the following
305 protocol:
306
307 When an encoded CBOR perl object is encountered by the decoder, it will
308 look up the "THAW" method, by using the stored classname, and will fail
309 if the method cannot be found.
310
311 After the lookup it will call the "THAW" method with the stored
312 classname as first argument, the constant string "CBOR" as second
313 argument, and all values returned by "FREEZE" as remaining arguments.
314
315 EXAMPLES
316 Here is an example "TO_CBOR" method:
317
318 sub My::Object::TO_CBOR {
319 my ($obj) = @_;
320
321 ["this is a serialised My::Object object", $obj->{id}]
322 }
323
324 When a "My::Object" is encoded to CBOR, it will instead encode a simple
325 array with two members: a string, and the "object id". Decoding this
326 CBOR string will yield a normal perl array reference in place of the
327 object.
328
329 A more useful and practical example would be a serialisation method for
330 the URI module. CBOR has a custom tag value for URIs, namely 32:
331
332 sub URI::TO_CBOR {
333 my ($self) = @_;
334 my $uri = "$self"; # stringify uri
335 utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
336 CBOR::XS::tagged 32, "$_[0]"
337 }
338
339 This will encode URIs as a UTF-8 string with tag 32, which indicates an
340 URI.
341
342 Decoding such an URI will not (currently) give you an URI object, but
343 instead a CBOR::XS::Tagged object with tag number 32 and the string -
344 exactly what was returned by "TO_CBOR".
345
346 To serialise an object so it can automatically be deserialised, you need
347 to use "FREEZE" and "THAW". To take the URI module as example, this
348 would be a possible implementation:
349
350 sub URI::FREEZE {
351 my ($self, $serialiser) = @_;
352 "$self" # encode url string
353 }
354
355 sub URI::THAW {
356 my ($class, $serialiser, $uri) = @_;
357
358 $class->new ($uri)
359 }
360
361 Unlike "TO_CBOR", multiple values can be returned by "FREEZE". For
362 example, a "FREEZE" method that returns "type", "id" and "variant"
363 values would cause an invocation of "THAW" with 5 arguments:
364
365 sub My::Object::FREEZE {
366 my ($self, $serialiser) = @_;
367
368 ($self->{type}, $self->{id}, $self->{variant})
369 }
370
371 sub My::Object::THAW {
372 my ($class, $serialiser, $type, $id, $variant) = @_;
373
374 $class-<new (type => $type, id => $id, variant => $variant)
375 }
376
377MAGIC HEADER
378 There is no way to distinguish CBOR from other formats programmatically.
379 To make it easier to distinguish CBOR from other formats, the CBOR
380 specification has a special "magic string" that can be prepended to any
381 CBOR string without changing it's meaning.
382
383 This string is available as $CBOR::XS::MAGIC. This module does not
384 prepend this string tot he CBOR data it generates, but it will ignroe it
385 if present, so users can prepend this string as a "file type" indicator
386 as required.
387
388THE CBOR::XS::Tagged CLASS
389 CBOR has the concept of tagged values - any CBOR value can be tagged
390 with a numeric 64 bit number, which are centrally administered.
391
392 "CBOR::XS" handles a few tags internally when en- or decoding. You can
393 also create tags yourself by encoding "CBOR::XS::Tagged" objects, and
394 the decoder will create "CBOR::XS::Tagged" objects itself when it hits
395 an unknown tag.
396
397 These objects are simply blessed array references - the first member of
398 the array being the numerical tag, the second being the value.
399
400 You can interact with "CBOR::XS::Tagged" objects in the following ways:
401
402 $tagged = CBOR::XS::tag $tag, $value
403 This function(!) creates a new "CBOR::XS::Tagged" object using the
404 given $tag (0..2**64-1) to tag the given $value (which can be any
405 Perl value that can be encoded in CBOR, including serialisable Perl
406 objects and "CBOR::XS::Tagged" objects).
407
408 $tagged->[0]
409 $tagged->[0] = $new_tag
410 $tag = $tagged->tag
411 $new_tag = $tagged->tag ($new_tag)
412 Access/mutate the tag.
413
414 $tagged->[1]
415 $tagged->[1] = $new_value
416 $value = $tagged->value
417 $new_value = $tagged->value ($new_value)
418 Access/mutate the tagged value.
419
420 EXAMPLES
421 Here are some examples of "CBOR::XS::Tagged" uses to tag objects.
422
423 You can look up CBOR tag value and emanings in the IANA registry at
424 <http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml>.
425
426 Prepend a magic header ($CBOR::XS::MAGIC):
427
428 my $cbor = encode_cbor CBOR::XS::tag 55799, $value;
429 # same as:
430 my $cbor = $CBOR::XS::MAGIC . encode_cbor $value;
431
432 Serialise some URIs and a regex in an array:
433
434 my $cbor = encode_cbor [
435 (CBOR::XS::tag 32, "http://www.nethype.de/"),
436 (CBOR::XS::tag 32, "http://software.schmorp.de/"),
437 (CBOR::XS::tag 35, "^[Pp][Ee][Rr][lL]\$"),
438 ];
439
440 Wrap CBOR data in CBOR:
441
442 my $cbor_cbor = encode_cbor
443 CBOR::XS::tag 24,
444 encode_cbor [1, 2, 3];
445
204 CBOR and JSON 446CBOR and JSON
205 TODO 447 CBOR is supposed to implement a superset of the JSON data model, and is,
448 with some coercion, able to represent all JSON texts (something that
449 other "binary JSON" formats such as BSON generally do not support).
450
451 CBOR implements some extra hints and support for JSON interoperability,
452 and the spec offers further guidance for conversion between CBOR and
453 JSON. None of this is currently implemented in CBOR, and the guidelines
454 in the spec do not result in correct round-tripping of data. If JSON
455 interoperability is improved in the future, then the goal will be to
456 ensure that decoded JSON data will round-trip encoding and decoding to
457 CBOR intact.
206 458
207SECURITY CONSIDERATIONS 459SECURITY CONSIDERATIONS
208 When you are using CBOR in a protocol, talking to untrusted potentially 460 When you are using CBOR in a protocol, talking to untrusted potentially
209 hostile creatures requires relatively few measures. 461 hostile creatures requires relatively few measures.
210 462
… …
273 525
274SEE ALSO 526SEE ALSO
275 The JSON and JSON::XS modules that do similar, but human-readable, 527 The JSON and JSON::XS modules that do similar, but human-readable,
276 serialisation. 528 serialisation.
277 529
530 The Types::Serialiser module provides the data model for true, false and
531 error values.
532
278AUTHOR 533AUTHOR
279 Marc Lehmann <schmorp@schmorp.de> 534 Marc Lehmann <schmorp@schmorp.de>
280 http://home.schmorp.de/ 535 http://home.schmorp.de/
281 536

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines