ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/README
Revision: 1.4
Committed: Sat Oct 26 23:02:55 2013 UTC (10 years, 6 months ago) by root
Branch: MAIN
CVS Tags: rel-0_03
Changes since 1.3: +79 -24 lines
Log Message:
0.03

File Contents

# Content
1 NAME
2 CBOR::XS - Concise Binary Object Representation (CBOR, RFC7049)
3
4 SYNOPSIS
5 use CBOR::XS;
6
7 $binary_cbor_data = encode_cbor $perl_value;
8 $perl_value = decode_cbor $binary_cbor_data;
9
10 # OO-interface
11
12 $coder = CBOR::XS->new;
13 #TODO
14
15 DESCRIPTION
16 WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA
17 AND EAT YOUR CHILDREN! (Actually, apart from being untested and a bit
18 feature-limited, it might already be useful).
19
20 This module converts Perl data structures to the Concise Binary Object
21 Representation (CBOR) and vice versa. CBOR is a fast binary
22 serialisation format that aims to use a superset of the JSON data model,
23 i.e. when you can represent something in JSON, you should be able to
24 represent it in CBOR.
25
26 This makes it a faster and more compact binary alternative to JSON.
27
28 The primary goal of this module is to be *correct* and the secondary
29 goal is to be *fast*. To reach the latter goal it was written in C.
30
31 See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
32 vice versa.
33
34 FUNCTIONAL INTERFACE
35 The following convenience methods are provided by this module. They are
36 exported by default:
37
38 $cbor_data = encode_cbor $perl_scalar
39 Converts the given Perl data structure to CBOR representation.
40 Croaks on error.
41
42 $perl_scalar = decode_cbor $cbor_data
43 The opposite of "encode_cbor": expects a valid CBOR string to parse,
44 returning the resulting perl scalar. Croaks on error.
45
46 OBJECT-ORIENTED INTERFACE
47 The object oriented interface lets you configure your own encoding or
48 decoding style, within the limits of supported formats.
49
50 $cbor = new CBOR::XS
51 Creates a new CBOR::XS object that can be used to de/encode CBOR
52 strings. All boolean flags described below are by default
53 *disabled*.
54
55 The mutators for flags all return the CBOR object again and thus
56 calls can be chained:
57
58 #TODO my $cbor = CBOR::XS->new->encode ({a => [1,2]});
59
60 $cbor = $cbor->max_depth ([$maximum_nesting_depth])
61 $max_depth = $cbor->get_max_depth
62 Sets the maximum nesting level (default 512) accepted while encoding
63 or decoding. If a higher nesting level is detected in CBOR data or a
64 Perl data structure, then the encoder and decoder will stop and
65 croak at that point.
66
67 Nesting level is defined by number of hash- or arrayrefs that the
68 encoder needs to traverse to reach a given point or the number of
69 "{" or "[" characters without their matching closing parenthesis
70 crossed to reach a given character in a string.
71
72 Setting the maximum depth to one disallows any nesting, so that
73 ensures that the object is only a single hash/object or array.
74
75 If no argument is given, the highest possible setting will be used,
76 which is rarely useful.
77
78 Note that nesting is implemented by recursion in C. The default
79 value has been chosen to be as large as typical operating systems
80 allow without crashing.
81
82 See SECURITY CONSIDERATIONS, below, for more info on why this is
83 useful.
84
85 $cbor = $cbor->max_size ([$maximum_string_size])
86 $max_size = $cbor->get_max_size
87 Set the maximum length a CBOR string may have (in bytes) where
88 decoding is being attempted. The default is 0, meaning no limit.
89 When "decode" is called on a string that is longer then this many
90 bytes, it will not attempt to decode the string but throw an
91 exception. This setting has no effect on "encode" (yet).
92
93 If no argument is given, the limit check will be deactivated (same
94 as when 0 is specified).
95
96 See SECURITY CONSIDERATIONS, below, for more info on why this is
97 useful.
98
99 $cbor_data = $cbor->encode ($perl_scalar)
100 Converts the given Perl data structure (a scalar value) to its CBOR
101 representation.
102
103 $perl_scalar = $cbor->decode ($cbor_data)
104 The opposite of "encode": expects CBOR data and tries to parse it,
105 returning the resulting simple scalar or reference. Croaks on error.
106
107 ($perl_scalar, $octets) = $cbor->decode_prefix ($cbor_data)
108 This works like the "decode" method, but instead of raising an
109 exception when there is trailing garbage after the CBOR string, it
110 will silently stop parsing there and return the number of characters
111 consumed so far.
112
113 This is useful if your CBOR texts are not delimited by an outer
114 protocol and you need to know where the first CBOR string ends amd
115 the next one starts.
116
117 CBOR::XS->new->decode_prefix ("......")
118 => ("...", 3)
119
120 MAPPING
121 This section describes how CBOR::XS maps Perl values to CBOR values and
122 vice versa. These mappings are designed to "do the right thing" in most
123 circumstances automatically, preserving round-tripping characteristics
124 (what you put in comes out as something equivalent).
125
126 For the more enlightened: note that in the following descriptions,
127 lowercase *perl* refers to the Perl interpreter, while uppercase *Perl*
128 refers to the abstract Perl language itself.
129
130 CBOR -> PERL
131 integers
132 CBOR integers become (numeric) perl scalars. On perls without 64 bit
133 support, 64 bit integers will be truncated or otherwise corrupted.
134
135 byte strings
136 Byte strings will become octet strings in Perl (the byte values
137 0..255 will simply become characters of the same value in Perl).
138
139 UTF-8 strings
140 UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
141 decoded into proper Unicode code points. At the moment, the validity
142 of the UTF-8 octets will not be validated - corrupt input will
143 result in corrupted Perl strings.
144
145 arrays, maps
146 CBOR arrays and CBOR maps will be converted into references to a
147 Perl array or hash, respectively. The keys of the map will be
148 stringified during this process.
149
150 true, false
151 These CBOR values become "CBOR::XS::true" and "CBOR::XS::false",
152 respectively. They are overloaded to act almost exactly like the
153 numbers 1 and 0. You can check whether a scalar is a CBOR boolean by
154 using the "CBOR::XS::is_bool" function.
155
156 null, undefined
157 CBOR null and undefined values becomes "undef" in Perl (in the
158 future, Undefined may raise an exception or something else).
159
160 tags
161 Tagged items consists of a numeric tag and another CBOR value. The
162 tag 55799 is ignored (this tag implements the magic header).
163
164 All other tags are currently converted into a CBOR::XS::Tagged
165 object, which is simply a blessed array reference consistsing of the
166 numeric tag value followed by the (decoded) BOR value.
167
168 anything else
169 Anything else (e.g. unsupported simple values) will raise a decoding
170 error.
171
172 PERL -> CBOR
173 The mapping from Perl to CBOR is slightly more difficult, as Perl is a
174 truly typeless language, so we can only guess which CBOR type is meant
175 by a Perl value.
176
177 hash references
178 Perl hash references become CBOR maps. As there is no inherent
179 ordering in hash keys (or CBOR maps), they will usually be encoded
180 in a pseudo-random order.
181
182 Currently, tied hashes will use the indefinite-length format, while
183 normal hashes will use the fixed-length format.
184
185 array references
186 Perl array references become fixed-length CBOR arrays.
187
188 other references
189 Other unblessed references are generally not allowed and will cause
190 an exception to be thrown, except for references to the integers 0
191 and 1, which get turned into false and true in CBOR.
192
193 CBOR::XS::Tagged objects
194 Objects of this type must be arrays consisting of a single "[tag,
195 value]" pair. The (numerical) tag will be encoded as a CBOR tag, the
196 value will be encoded as appropriate for the value.
197
198 CBOR::XS::true, CBOR::XS::false
199 These special values become CBOR true and CBOR false values,
200 respectively. You can also use "\1" and "\0" directly if you want.
201
202 blessed objects
203 Other blessed objects currently need to have a "TO_CBOR" method. It
204 will be called on every object that is being serialised, and must
205 return something that can be encoded in CBOR.
206
207 simple scalars
208 TODO Simple Perl scalars (any scalar that is not a reference) are
209 the most difficult objects to encode: CBOR::XS will encode undefined
210 scalars as CBOR null values, scalars that have last been used in a
211 string context before encoding as CBOR strings, and anything else as
212 number value:
213
214 # dump as number
215 encode_cbor [2] # yields [2]
216 encode_cbor [-3.0e17] # yields [-3e+17]
217 my $value = 5; encode_cbor [$value] # yields [5]
218
219 # used as string, so dump as string
220 print $value;
221 encode_cbor [$value] # yields ["5"]
222
223 # undef becomes null
224 encode_cbor [undef] # yields [null]
225
226 You can force the type to be a CBOR string by stringifying it:
227
228 my $x = 3.1; # some variable containing a number
229 "$x"; # stringified
230 $x .= ""; # another, more awkward way to stringify
231 print $x; # perl does it for you, too, quite often
232
233 You can force the type to be a CBOR number by numifying it:
234
235 my $x = "3"; # some variable containing a string
236 $x += 0; # numify it, ensuring it will be dumped as a number
237 $x *= 1; # same thing, the choice is yours.
238
239 You can not currently force the type in other, less obscure, ways.
240 Tell me if you need this capability (but don't forget to explain why
241 it's needed :).
242
243 Perl values that seem to be integers generally use the shortest
244 possible representation. Floating-point values will use either the
245 IEEE single format if possible without loss of precision, otherwise
246 the IEEE double format will be used. Perls that use formats other
247 than IEEE double to represent numerical values are supported, but
248 might suffer loss of precision.
249
250 MAGIC HEADER
251 There is no way to distinguish CBOR from other formats programmatically.
252 To make it easier to distinguish CBOR from other formats, the CBOR
253 specification has a special "magic string" that can be prepended to any
254 CBOR string without changing it's meaning.
255
256 This string is available as $CBOR::XS::MAGIC. This module does not
257 prepend this string tot he CBOR data it generates, but it will ignroe it
258 if present, so users can prepend this string as a "file type" indicator
259 as required.
260
261 CBOR and JSON
262 CBOR is supposed to implement a superset of the JSON data model, and is,
263 with some coercion, able to represent all JSON texts (something that
264 other "binary JSON" formats such as BSON generally do not support).
265
266 CBOR implements some extra hints and support for JSON interoperability,
267 and the spec offers further guidance for conversion between CBOR and
268 JSON. None of this is currently implemented in CBOR, and the guidelines
269 in the spec do not result in correct round-tripping of data. If JSON
270 interoperability is improved in the future, then the goal will be to
271 ensure that decoded JSON data will round-trip encoding and decoding to
272 CBOR intact.
273
274 SECURITY CONSIDERATIONS
275 When you are using CBOR in a protocol, talking to untrusted potentially
276 hostile creatures requires relatively few measures.
277
278 First of all, your CBOR decoder should be secure, that is, should not
279 have any buffer overflows. Obviously, this module should ensure that and
280 I am trying hard on making that true, but you never know.
281
282 Second, you need to avoid resource-starving attacks. That means you
283 should limit the size of CBOR data you accept, or make sure then when
284 your resources run out, that's just fine (e.g. by using a separate
285 process that can crash safely). The size of a CBOR string in octets is
286 usually a good indication of the size of the resources required to
287 decode it into a Perl structure. While CBOR::XS can check the size of
288 the CBOR text, it might be too late when you already have it in memory,
289 so you might want to check the size before you accept the string.
290
291 Third, CBOR::XS recurses using the C stack when decoding objects and
292 arrays. The C stack is a limited resource: for instance, on my amd64
293 machine with 8MB of stack size I can decode around 180k nested arrays
294 but only 14k nested CBOR objects (due to perl itself recursing deeply on
295 croak to free the temporary). If that is exceeded, the program crashes.
296 To be conservative, the default nesting limit is set to 512. If your
297 process has a smaller stack, you should adjust this setting accordingly
298 with the "max_depth" method.
299
300 Something else could bomb you, too, that I forgot to think of. In that
301 case, you get to keep the pieces. I am always open for hints, though...
302
303 Also keep in mind that CBOR::XS might leak contents of your Perl data
304 structures in its error messages, so when you serialise sensitive
305 information you might want to make sure that exceptions thrown by
306 CBOR::XS will not end up in front of untrusted eyes.
307
308 CBOR IMPLEMENTATION NOTES
309 This section contains some random implementation notes. They do not
310 describe guaranteed behaviour, but merely behaviour as-is implemented
311 right now.
312
313 64 bit integers are only properly decoded when Perl was built with 64
314 bit support.
315
316 Strings and arrays are encoded with a definite length. Hashes as well,
317 unless they are tied (or otherwise magical).
318
319 Only the double data type is supported for NV data types - when Perl
320 uses long double to represent floating point values, they might not be
321 encoded properly. Half precision types are accepted, but not encoded.
322
323 Strict mode and canonical mode are not implemented.
324
325 THREADS
326 This module is *not* guaranteed to be thread safe and there are no plans
327 to change this until Perl gets thread support (as opposed to the
328 horribly slow so-called "threads" which are simply slow and bloated
329 process simulations - use fork, it's *much* faster, cheaper, better).
330
331 (It might actually work, but you have been warned).
332
333 BUGS
334 While the goal of this module is to be correct, that unfortunately does
335 not mean it's bug-free, only that I think its design is bug-free. If you
336 keep reporting bugs they will be fixed swiftly, though.
337
338 Please refrain from using rt.cpan.org or any other bug reporting
339 service. I put the contact address into my modules for a reason.
340
341 SEE ALSO
342 The JSON and JSON::XS modules that do similar, but human-readable,
343 serialisation.
344
345 AUTHOR
346 Marc Lehmann <schmorp@schmorp.de>
347 http://home.schmorp.de/
348