ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/XS.pm
Revision: 1.4
Committed: Sat Oct 26 22:25:47 2013 UTC (10 years, 6 months ago) by root
Branch: MAIN
Changes since 1.3: +77 -24 lines
Log Message:
*** empty log message ***

File Contents

# Content
1 =head1 NAME
2
3 CBOR::XS - Concise Binary Object Representation (CBOR, RFC7049)
4
5 =encoding utf-8
6
7 =head1 SYNOPSIS
8
9 use CBOR::XS;
10
11 $binary_cbor_data = encode_cbor $perl_value;
12 $perl_value = decode_cbor $binary_cbor_data;
13
14 # OO-interface
15
16 $coder = CBOR::XS->new;
17 #TODO
18
19 =head1 DESCRIPTION
20
21 WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA AND
22 EAT YOUR CHILDREN!
23
24 This module converts Perl data structures to CBOR and vice versa. Its
25 primary goal is to be I<correct> and its secondary goal is to be
26 I<fast>. To reach the latter goal it was written in C.
27
28 See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
29 vice versa.
30
31 =cut
32
33 package CBOR::XS;
34
35 use common::sense;
36
37 our $VERSION = 0.02;
38 our @ISA = qw(Exporter);
39
40 our @EXPORT = qw(encode_cbor decode_cbor);
41
42 use Exporter;
43 use XSLoader;
44
45 our $MAGIC = "\xd9\xd9\xf7";
46
47 =head1 FUNCTIONAL INTERFACE
48
49 The following convenience methods are provided by this module. They are
50 exported by default:
51
52 =over 4
53
54 =item $cbor_data = encode_cbor $perl_scalar
55
56 Converts the given Perl data structure to CBOR representation. Croaks on
57 error.
58
59 =item $perl_scalar = decode_cbor $cbor_data
60
61 The opposite of C<encode_cbor>: expects a valid CBOR string to parse,
62 returning the resulting perl scalar. Croaks on error.
63
64 =back
65
66
67 =head1 OBJECT-ORIENTED INTERFACE
68
69 The object oriented interface lets you configure your own encoding or
70 decoding style, within the limits of supported formats.
71
72 =over 4
73
74 =item $cbor = new CBOR::XS
75
76 Creates a new CBOR::XS object that can be used to de/encode CBOR
77 strings. All boolean flags described below are by default I<disabled>.
78
79 The mutators for flags all return the CBOR object again and thus calls can
80 be chained:
81
82 #TODO
83 my $cbor = CBOR::XS->new->encode ({a => [1,2]});
84
85 =item $cbor = $cbor->max_depth ([$maximum_nesting_depth])
86
87 =item $max_depth = $cbor->get_max_depth
88
89 Sets the maximum nesting level (default C<512>) accepted while encoding
90 or decoding. If a higher nesting level is detected in CBOR data or a Perl
91 data structure, then the encoder and decoder will stop and croak at that
92 point.
93
94 Nesting level is defined by number of hash- or arrayrefs that the encoder
95 needs to traverse to reach a given point or the number of C<{> or C<[>
96 characters without their matching closing parenthesis crossed to reach a
97 given character in a string.
98
99 Setting the maximum depth to one disallows any nesting, so that ensures
100 that the object is only a single hash/object or array.
101
102 If no argument is given, the highest possible setting will be used, which
103 is rarely useful.
104
105 Note that nesting is implemented by recursion in C. The default value has
106 been chosen to be as large as typical operating systems allow without
107 crashing.
108
109 See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
110
111 =item $cbor = $cbor->max_size ([$maximum_string_size])
112
113 =item $max_size = $cbor->get_max_size
114
115 Set the maximum length a CBOR string may have (in bytes) where decoding
116 is being attempted. The default is C<0>, meaning no limit. When C<decode>
117 is called on a string that is longer then this many bytes, it will not
118 attempt to decode the string but throw an exception. This setting has no
119 effect on C<encode> (yet).
120
121 If no argument is given, the limit check will be deactivated (same as when
122 C<0> is specified).
123
124 See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
125
126 =item $cbor_data = $cbor->encode ($perl_scalar)
127
128 Converts the given Perl data structure (a scalar value) to its CBOR
129 representation.
130
131 =item $perl_scalar = $cbor->decode ($cbor_data)
132
133 The opposite of C<encode>: expects CBOR data and tries to parse it,
134 returning the resulting simple scalar or reference. Croaks on error.
135
136 =item ($perl_scalar, $octets) = $cbor->decode_prefix ($cbor_data)
137
138 This works like the C<decode> method, but instead of raising an exception
139 when there is trailing garbage after the CBOR string, it will silently
140 stop parsing there and return the number of characters consumed so far.
141
142 This is useful if your CBOR texts are not delimited by an outer protocol
143 and you need to know where the first CBOR string ends amd the next one
144 starts.
145
146 CBOR::XS->new->decode_prefix ("......")
147 => ("...", 3)
148
149 =back
150
151
152 =head1 MAPPING
153
154 This section describes how CBOR::XS maps Perl values to CBOR values and
155 vice versa. These mappings are designed to "do the right thing" in most
156 circumstances automatically, preserving round-tripping characteristics
157 (what you put in comes out as something equivalent).
158
159 For the more enlightened: note that in the following descriptions,
160 lowercase I<perl> refers to the Perl interpreter, while uppercase I<Perl>
161 refers to the abstract Perl language itself.
162
163
164 =head2 CBOR -> PERL
165
166 =over 4
167
168 =item integers
169
170 CBOR integers become (numeric) perl scalars. On perls without 64 bit
171 support, 64 bit integers will be truncated or otherwise corrupted.
172
173 =item byte strings
174
175 Byte strings will become octet strings in Perl (the byte values 0..255
176 will simply become characters of the same value in Perl).
177
178 =item UTF-8 strings
179
180 UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
181 decoded into proper Unicode code points. At the moment, the validity of
182 the UTF-8 octets will not be validated - corrupt input will result in
183 corrupted Perl strings.
184
185 =item arrays, maps
186
187 CBOR arrays and CBOR maps will be converted into references to a Perl
188 array or hash, respectively. The keys of the map will be stringified
189 during this process.
190
191 =item true, false
192
193 These CBOR values become C<CBOR::XS::true> and C<CBOR::XS::false>,
194 respectively. They are overloaded to act almost exactly like the numbers
195 C<1> and C<0>. You can check whether a scalar is a CBOR boolean by using
196 the C<CBOR::XS::is_bool> function.
197
198 =item null, undefined
199
200 CBOR null and undefined values becomes C<undef> in Perl (in the future,
201 Undefined may raise an exception or something else).
202
203 =item tags
204
205 Tagged items consists of a numeric tag and another CBOR value. The tag
206 55799 is ignored (this tag implements the magic header).
207
208 All other tags are currently converted into a L<CBOR::XS::Tagged> object,
209 which is simply a blessed array reference consistsing of the numeric tag
210 value followed by the (decoded) BOR value.
211
212 =item anything else
213
214 Anything else (e.g. unsupported simple values) will raise a decoding
215 error.
216
217 =back
218
219
220 =head2 PERL -> CBOR
221
222 The mapping from Perl to CBOR is slightly more difficult, as Perl is a
223 truly typeless language, so we can only guess which CBOR type is meant by
224 a Perl value.
225
226 =over 4
227
228 =item hash references
229
230 Perl hash references become CBOR maps. As there is no inherent ordering in
231 hash keys (or CBOR maps), they will usually be encoded in a pseudo-random
232 order.
233
234 Currently, tied hashes will use the indefinite-length format, while normal
235 hashes will use the fixed-length format.
236
237 =item array references
238
239 Perl array references become fixed-length CBOR arrays.
240
241 =item other references
242
243 Other unblessed references are generally not allowed and will cause an
244 exception to be thrown, except for references to the integers C<0> and
245 C<1>, which get turned into false and true in CBOR.
246
247 =item CBOR::XS::Tagged objects
248
249 Objects of this type must be arrays consisting of a single C<[tag, value]>
250 pair. The (numerical) tag will be encoded as a CBOR tag, the value will be
251 encoded as appropriate for the value.
252
253 =item CBOR::XS::true, CBOR::XS::false
254
255 These special values become CBOR true and CBOR false values,
256 respectively. You can also use C<\1> and C<\0> directly if you want.
257
258 =item blessed objects
259
260 Other blessed objects currently need to have a C<TO_CBOR> method. It
261 will be called on every object that is being serialised, and must return
262 something that can be encoded in CBOR.
263
264 =item simple scalars
265
266 TODO
267 Simple Perl scalars (any scalar that is not a reference) are the most
268 difficult objects to encode: CBOR::XS will encode undefined scalars as
269 CBOR null values, scalars that have last been used in a string context
270 before encoding as CBOR strings, and anything else as number value:
271
272 # dump as number
273 encode_cbor [2] # yields [2]
274 encode_cbor [-3.0e17] # yields [-3e+17]
275 my $value = 5; encode_cbor [$value] # yields [5]
276
277 # used as string, so dump as string
278 print $value;
279 encode_cbor [$value] # yields ["5"]
280
281 # undef becomes null
282 encode_cbor [undef] # yields [null]
283
284 You can force the type to be a CBOR string by stringifying it:
285
286 my $x = 3.1; # some variable containing a number
287 "$x"; # stringified
288 $x .= ""; # another, more awkward way to stringify
289 print $x; # perl does it for you, too, quite often
290
291 You can force the type to be a CBOR number by numifying it:
292
293 my $x = "3"; # some variable containing a string
294 $x += 0; # numify it, ensuring it will be dumped as a number
295 $x *= 1; # same thing, the choice is yours.
296
297 You can not currently force the type in other, less obscure, ways. Tell me
298 if you need this capability (but don't forget to explain why it's needed
299 :).
300
301 Perl values that seem to be integers generally use the shortest possible
302 representation. Floating-point values will use either the IEEE single
303 format if possible without loss of precision, otherwise the IEEE double
304 format will be used. Perls that use formats other than IEEE double to
305 represent numerical values are supported, but might suffer loss of
306 precision.
307
308 =back
309
310
311 =head2 MAGIC HEADER
312
313 There is no way to distinguish CBOR from other formats
314 programmatically. To make it easier to distinguish CBOR from other
315 formats, the CBOR specification has a special "magic string" that can be
316 prepended to any CBOR string without changing it's meaning.
317
318 This string is available as C<$CBOR::XS::MAGIC>. This module does not
319 prepend this string tot he CBOR data it generates, but it will ignroe it
320 if present, so users can prepend this string as a "file type" indicator as
321 required.
322
323
324 =head2 CBOR and JSON
325
326 CBOR is supposed to implement a superset of the JSON data model, and is,
327 with some coercion, able to represent all JSON texts (something that other
328 "binary JSON" formats such as BSON generally do not support).
329
330 CBOR implements some extra hints and support for JSON interoperability,
331 and the spec offers further guidance for conversion between CBOR and
332 JSON. None of this is currently implemented in CBOR, and the guidelines
333 in the spec do not result in correct round-tripping of data. If JSON
334 interoperability is improved in the future, then the goal will be to
335 ensure that decoded JSON data will round-trip encoding and decoding to
336 CBOR intact.
337
338
339 =head1 SECURITY CONSIDERATIONS
340
341 When you are using CBOR in a protocol, talking to untrusted potentially
342 hostile creatures requires relatively few measures.
343
344 First of all, your CBOR decoder should be secure, that is, should not have
345 any buffer overflows. Obviously, this module should ensure that and I am
346 trying hard on making that true, but you never know.
347
348 Second, you need to avoid resource-starving attacks. That means you should
349 limit the size of CBOR data you accept, or make sure then when your
350 resources run out, that's just fine (e.g. by using a separate process that
351 can crash safely). The size of a CBOR string in octets is usually a good
352 indication of the size of the resources required to decode it into a Perl
353 structure. While CBOR::XS can check the size of the CBOR text, it might be
354 too late when you already have it in memory, so you might want to check
355 the size before you accept the string.
356
357 Third, CBOR::XS recurses using the C stack when decoding objects and
358 arrays. The C stack is a limited resource: for instance, on my amd64
359 machine with 8MB of stack size I can decode around 180k nested arrays but
360 only 14k nested CBOR objects (due to perl itself recursing deeply on croak
361 to free the temporary). If that is exceeded, the program crashes. To be
362 conservative, the default nesting limit is set to 512. If your process
363 has a smaller stack, you should adjust this setting accordingly with the
364 C<max_depth> method.
365
366 Something else could bomb you, too, that I forgot to think of. In that
367 case, you get to keep the pieces. I am always open for hints, though...
368
369 Also keep in mind that CBOR::XS might leak contents of your Perl data
370 structures in its error messages, so when you serialise sensitive
371 information you might want to make sure that exceptions thrown by CBOR::XS
372 will not end up in front of untrusted eyes.
373
374 =head1 CBOR IMPLEMENTATION NOTES
375
376 This section contains some random implementation notes. They do not
377 describe guaranteed behaviour, but merely behaviour as-is implemented
378 right now.
379
380 64 bit integers are only properly decoded when Perl was built with 64 bit
381 support.
382
383 Strings and arrays are encoded with a definite length. Hashes as well,
384 unless they are tied (or otherwise magical).
385
386 Only the double data type is supported for NV data types - when Perl uses
387 long double to represent floating point values, they might not be encoded
388 properly. Half precision types are accepted, but not encoded.
389
390 Strict mode and canonical mode are not implemented.
391
392
393 =head1 THREADS
394
395 This module is I<not> guaranteed to be thread safe and there are no
396 plans to change this until Perl gets thread support (as opposed to the
397 horribly slow so-called "threads" which are simply slow and bloated
398 process simulations - use fork, it's I<much> faster, cheaper, better).
399
400 (It might actually work, but you have been warned).
401
402
403 =head1 BUGS
404
405 While the goal of this module is to be correct, that unfortunately does
406 not mean it's bug-free, only that I think its design is bug-free. If you
407 keep reporting bugs they will be fixed swiftly, though.
408
409 Please refrain from using rt.cpan.org or any other bug reporting
410 service. I put the contact address into my modules for a reason.
411
412 =cut
413
414 our $true = do { bless \(my $dummy = 1), "CBOR::XS::Boolean" };
415 our $false = do { bless \(my $dummy = 0), "CBOR::XS::Boolean" };
416
417 sub true() { $true }
418 sub false() { $false }
419
420 sub is_bool($) {
421 UNIVERSAL::isa $_[0], "CBOR::XS::Boolean"
422 # or UNIVERSAL::isa $_[0], "CBOR::Literal"
423 }
424
425 XSLoader::load "CBOR::XS", $VERSION;
426
427 package CBOR::XS::Boolean;
428
429 use overload
430 "0+" => sub { ${$_[0]} },
431 "++" => sub { $_[0] = ${$_[0]} + 1 },
432 "--" => sub { $_[0] = ${$_[0]} - 1 },
433 fallback => 1;
434
435 1;
436
437 =head1 SEE ALSO
438
439 The L<JSON> and L<JSON::XS> modules that do similar, but human-readable,
440 serialisation.
441
442 =head1 AUTHOR
443
444 Marc Lehmann <schmorp@schmorp.de>
445 http://home.schmorp.de/
446
447 =cut
448