ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/README
Revision: 1.4
Committed: Sat Oct 26 23:02:55 2013 UTC (10 years, 6 months ago) by root
Branch: MAIN
CVS Tags: rel-0_03
Changes since 1.3: +79 -24 lines
Log Message:
0.03

File Contents

# User Rev Content
1 root 1.2 NAME
2     CBOR::XS - Concise Binary Object Representation (CBOR, RFC7049)
3    
4     SYNOPSIS
5     use CBOR::XS;
6    
7     $binary_cbor_data = encode_cbor $perl_value;
8     $perl_value = decode_cbor $binary_cbor_data;
9    
10     # OO-interface
11    
12     $coder = CBOR::XS->new;
13     #TODO
14    
15     DESCRIPTION
16     WARNING! THIS IS A PRE-ALPHA RELEASE! IT WILL CRASH, CORRUPT YOUR DATA
17 root 1.4 AND EAT YOUR CHILDREN! (Actually, apart from being untested and a bit
18     feature-limited, it might already be useful).
19 root 1.2
20 root 1.4 This module converts Perl data structures to the Concise Binary Object
21     Representation (CBOR) and vice versa. CBOR is a fast binary
22     serialisation format that aims to use a superset of the JSON data model,
23     i.e. when you can represent something in JSON, you should be able to
24     represent it in CBOR.
25    
26     This makes it a faster and more compact binary alternative to JSON.
27    
28     The primary goal of this module is to be *correct* and the secondary
29     goal is to be *fast*. To reach the latter goal it was written in C.
30 root 1.2
31     See MAPPING, below, on how CBOR::XS maps perl values to CBOR values and
32     vice versa.
33    
34     FUNCTIONAL INTERFACE
35     The following convenience methods are provided by this module. They are
36     exported by default:
37    
38     $cbor_data = encode_cbor $perl_scalar
39     Converts the given Perl data structure to CBOR representation.
40     Croaks on error.
41    
42     $perl_scalar = decode_cbor $cbor_data
43     The opposite of "encode_cbor": expects a valid CBOR string to parse,
44     returning the resulting perl scalar. Croaks on error.
45    
46     OBJECT-ORIENTED INTERFACE
47     The object oriented interface lets you configure your own encoding or
48     decoding style, within the limits of supported formats.
49    
50     $cbor = new CBOR::XS
51     Creates a new CBOR::XS object that can be used to de/encode CBOR
52     strings. All boolean flags described below are by default
53     *disabled*.
54    
55     The mutators for flags all return the CBOR object again and thus
56     calls can be chained:
57    
58     #TODO my $cbor = CBOR::XS->new->encode ({a => [1,2]});
59    
60     $cbor = $cbor->max_depth ([$maximum_nesting_depth])
61     $max_depth = $cbor->get_max_depth
62     Sets the maximum nesting level (default 512) accepted while encoding
63     or decoding. If a higher nesting level is detected in CBOR data or a
64     Perl data structure, then the encoder and decoder will stop and
65     croak at that point.
66    
67     Nesting level is defined by number of hash- or arrayrefs that the
68     encoder needs to traverse to reach a given point or the number of
69     "{" or "[" characters without their matching closing parenthesis
70     crossed to reach a given character in a string.
71    
72     Setting the maximum depth to one disallows any nesting, so that
73     ensures that the object is only a single hash/object or array.
74    
75     If no argument is given, the highest possible setting will be used,
76     which is rarely useful.
77    
78     Note that nesting is implemented by recursion in C. The default
79     value has been chosen to be as large as typical operating systems
80     allow without crashing.
81    
82     See SECURITY CONSIDERATIONS, below, for more info on why this is
83     useful.
84    
85     $cbor = $cbor->max_size ([$maximum_string_size])
86     $max_size = $cbor->get_max_size
87     Set the maximum length a CBOR string may have (in bytes) where
88     decoding is being attempted. The default is 0, meaning no limit.
89     When "decode" is called on a string that is longer then this many
90     bytes, it will not attempt to decode the string but throw an
91     exception. This setting has no effect on "encode" (yet).
92    
93     If no argument is given, the limit check will be deactivated (same
94     as when 0 is specified).
95    
96     See SECURITY CONSIDERATIONS, below, for more info on why this is
97     useful.
98    
99     $cbor_data = $cbor->encode ($perl_scalar)
100     Converts the given Perl data structure (a scalar value) to its CBOR
101     representation.
102    
103     $perl_scalar = $cbor->decode ($cbor_data)
104     The opposite of "encode": expects CBOR data and tries to parse it,
105     returning the resulting simple scalar or reference. Croaks on error.
106    
107     ($perl_scalar, $octets) = $cbor->decode_prefix ($cbor_data)
108     This works like the "decode" method, but instead of raising an
109     exception when there is trailing garbage after the CBOR string, it
110     will silently stop parsing there and return the number of characters
111     consumed so far.
112    
113     This is useful if your CBOR texts are not delimited by an outer
114     protocol and you need to know where the first CBOR string ends amd
115     the next one starts.
116    
117     CBOR::XS->new->decode_prefix ("......")
118     => ("...", 3)
119    
120     MAPPING
121     This section describes how CBOR::XS maps Perl values to CBOR values and
122     vice versa. These mappings are designed to "do the right thing" in most
123     circumstances automatically, preserving round-tripping characteristics
124     (what you put in comes out as something equivalent).
125    
126     For the more enlightened: note that in the following descriptions,
127     lowercase *perl* refers to the Perl interpreter, while uppercase *Perl*
128     refers to the abstract Perl language itself.
129    
130     CBOR -> PERL
131 root 1.4 integers
132     CBOR integers become (numeric) perl scalars. On perls without 64 bit
133     support, 64 bit integers will be truncated or otherwise corrupted.
134    
135     byte strings
136     Byte strings will become octet strings in Perl (the byte values
137     0..255 will simply become characters of the same value in Perl).
138    
139     UTF-8 strings
140     UTF-8 strings in CBOR will be decoded, i.e. the UTF-8 octets will be
141     decoded into proper Unicode code points. At the moment, the validity
142     of the UTF-8 octets will not be validated - corrupt input will
143     result in corrupted Perl strings.
144    
145     arrays, maps
146     CBOR arrays and CBOR maps will be converted into references to a
147     Perl array or hash, respectively. The keys of the map will be
148     stringified during this process.
149    
150     true, false
151 root 1.2 These CBOR values become "CBOR::XS::true" and "CBOR::XS::false",
152     respectively. They are overloaded to act almost exactly like the
153     numbers 1 and 0. You can check whether a scalar is a CBOR boolean by
154     using the "CBOR::XS::is_bool" function.
155    
156 root 1.4 null, undefined
157     CBOR null and undefined values becomes "undef" in Perl (in the
158     future, Undefined may raise an exception or something else).
159    
160     tags
161     Tagged items consists of a numeric tag and another CBOR value. The
162     tag 55799 is ignored (this tag implements the magic header).
163    
164     All other tags are currently converted into a CBOR::XS::Tagged
165     object, which is simply a blessed array reference consistsing of the
166     numeric tag value followed by the (decoded) BOR value.
167    
168     anything else
169     Anything else (e.g. unsupported simple values) will raise a decoding
170     error.
171 root 1.2
172     PERL -> CBOR
173     The mapping from Perl to CBOR is slightly more difficult, as Perl is a
174     truly typeless language, so we can only guess which CBOR type is meant
175     by a Perl value.
176    
177     hash references
178     Perl hash references become CBOR maps. As there is no inherent
179     ordering in hash keys (or CBOR maps), they will usually be encoded
180     in a pseudo-random order.
181    
182 root 1.4 Currently, tied hashes will use the indefinite-length format, while
183     normal hashes will use the fixed-length format.
184    
185 root 1.2 array references
186 root 1.4 Perl array references become fixed-length CBOR arrays.
187 root 1.2
188     other references
189     Other unblessed references are generally not allowed and will cause
190     an exception to be thrown, except for references to the integers 0
191 root 1.4 and 1, which get turned into false and true in CBOR.
192    
193     CBOR::XS::Tagged objects
194     Objects of this type must be arrays consisting of a single "[tag,
195     value]" pair. The (numerical) tag will be encoded as a CBOR tag, the
196     value will be encoded as appropriate for the value.
197 root 1.2
198     CBOR::XS::true, CBOR::XS::false
199 root 1.4 These special values become CBOR true and CBOR false values,
200 root 1.2 respectively. You can also use "\1" and "\0" directly if you want.
201    
202     blessed objects
203 root 1.4 Other blessed objects currently need to have a "TO_CBOR" method. It
204     will be called on every object that is being serialised, and must
205     return something that can be encoded in CBOR.
206 root 1.2
207     simple scalars
208     TODO Simple Perl scalars (any scalar that is not a reference) are
209     the most difficult objects to encode: CBOR::XS will encode undefined
210 root 1.4 scalars as CBOR null values, scalars that have last been used in a
211 root 1.2 string context before encoding as CBOR strings, and anything else as
212     number value:
213    
214     # dump as number
215     encode_cbor [2] # yields [2]
216     encode_cbor [-3.0e17] # yields [-3e+17]
217     my $value = 5; encode_cbor [$value] # yields [5]
218    
219     # used as string, so dump as string
220     print $value;
221     encode_cbor [$value] # yields ["5"]
222    
223     # undef becomes null
224     encode_cbor [undef] # yields [null]
225    
226     You can force the type to be a CBOR string by stringifying it:
227    
228     my $x = 3.1; # some variable containing a number
229     "$x"; # stringified
230     $x .= ""; # another, more awkward way to stringify
231     print $x; # perl does it for you, too, quite often
232    
233     You can force the type to be a CBOR number by numifying it:
234    
235     my $x = "3"; # some variable containing a string
236     $x += 0; # numify it, ensuring it will be dumped as a number
237     $x *= 1; # same thing, the choice is yours.
238    
239     You can not currently force the type in other, less obscure, ways.
240     Tell me if you need this capability (but don't forget to explain why
241     it's needed :).
242    
243 root 1.4 Perl values that seem to be integers generally use the shortest
244     possible representation. Floating-point values will use either the
245     IEEE single format if possible without loss of precision, otherwise
246     the IEEE double format will be used. Perls that use formats other
247     than IEEE double to represent numerical values are supported, but
248     might suffer loss of precision.
249 root 1.2
250 root 1.3 MAGIC HEADER
251     There is no way to distinguish CBOR from other formats programmatically.
252     To make it easier to distinguish CBOR from other formats, the CBOR
253     specification has a special "magic string" that can be prepended to any
254     CBOR string without changing it's meaning.
255    
256     This string is available as $CBOR::XS::MAGIC. This module does not
257     prepend this string tot he CBOR data it generates, but it will ignroe it
258     if present, so users can prepend this string as a "file type" indicator
259     as required.
260    
261 root 1.2 CBOR and JSON
262 root 1.4 CBOR is supposed to implement a superset of the JSON data model, and is,
263     with some coercion, able to represent all JSON texts (something that
264     other "binary JSON" formats such as BSON generally do not support).
265    
266     CBOR implements some extra hints and support for JSON interoperability,
267     and the spec offers further guidance for conversion between CBOR and
268     JSON. None of this is currently implemented in CBOR, and the guidelines
269     in the spec do not result in correct round-tripping of data. If JSON
270     interoperability is improved in the future, then the goal will be to
271     ensure that decoded JSON data will round-trip encoding and decoding to
272     CBOR intact.
273 root 1.2
274     SECURITY CONSIDERATIONS
275     When you are using CBOR in a protocol, talking to untrusted potentially
276     hostile creatures requires relatively few measures.
277    
278     First of all, your CBOR decoder should be secure, that is, should not
279     have any buffer overflows. Obviously, this module should ensure that and
280     I am trying hard on making that true, but you never know.
281    
282     Second, you need to avoid resource-starving attacks. That means you
283     should limit the size of CBOR data you accept, or make sure then when
284     your resources run out, that's just fine (e.g. by using a separate
285     process that can crash safely). The size of a CBOR string in octets is
286     usually a good indication of the size of the resources required to
287     decode it into a Perl structure. While CBOR::XS can check the size of
288     the CBOR text, it might be too late when you already have it in memory,
289     so you might want to check the size before you accept the string.
290    
291     Third, CBOR::XS recurses using the C stack when decoding objects and
292     arrays. The C stack is a limited resource: for instance, on my amd64
293     machine with 8MB of stack size I can decode around 180k nested arrays
294     but only 14k nested CBOR objects (due to perl itself recursing deeply on
295     croak to free the temporary). If that is exceeded, the program crashes.
296     To be conservative, the default nesting limit is set to 512. If your
297     process has a smaller stack, you should adjust this setting accordingly
298     with the "max_depth" method.
299    
300     Something else could bomb you, too, that I forgot to think of. In that
301     case, you get to keep the pieces. I am always open for hints, though...
302    
303     Also keep in mind that CBOR::XS might leak contents of your Perl data
304     structures in its error messages, so when you serialise sensitive
305     information you might want to make sure that exceptions thrown by
306     CBOR::XS will not end up in front of untrusted eyes.
307    
308     CBOR IMPLEMENTATION NOTES
309     This section contains some random implementation notes. They do not
310     describe guaranteed behaviour, but merely behaviour as-is implemented
311     right now.
312    
313     64 bit integers are only properly decoded when Perl was built with 64
314     bit support.
315    
316     Strings and arrays are encoded with a definite length. Hashes as well,
317     unless they are tied (or otherwise magical).
318    
319     Only the double data type is supported for NV data types - when Perl
320     uses long double to represent floating point values, they might not be
321     encoded properly. Half precision types are accepted, but not encoded.
322    
323     Strict mode and canonical mode are not implemented.
324    
325     THREADS
326     This module is *not* guaranteed to be thread safe and there are no plans
327     to change this until Perl gets thread support (as opposed to the
328     horribly slow so-called "threads" which are simply slow and bloated
329     process simulations - use fork, it's *much* faster, cheaper, better).
330    
331     (It might actually work, but you have been warned).
332    
333     BUGS
334     While the goal of this module is to be correct, that unfortunately does
335     not mean it's bug-free, only that I think its design is bug-free. If you
336     keep reporting bugs they will be fixed swiftly, though.
337    
338     Please refrain from using rt.cpan.org or any other bug reporting
339     service. I put the contact address into my modules for a reason.
340    
341     SEE ALSO
342     The JSON and JSON::XS modules that do similar, but human-readable,
343     serialisation.
344    
345     AUTHOR
346     Marc Lehmann <schmorp@schmorp.de>
347     http://home.schmorp.de/
348