ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
Revision: 1.139
Committed: Thu May 23 09:31:32 2013 UTC (10 years, 11 months ago) by root
Branch: MAIN
CVS Tags: rel-2_34
Changes since 1.138: +21 -2 lines
Log Message:
2.34

File Contents

# Content
1 =head1 NAME
2
3 JSON::XS - JSON serialising/deserialising, done correctly and fast
4
5 =encoding utf-8
6
7 JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ
8 (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html)
9
10 =head1 SYNOPSIS
11
12 use JSON::XS;
13
14 # exported functions, they croak on error
15 # and expect/generate UTF-8
16
17 $utf8_encoded_json_text = encode_json $perl_hash_or_arrayref;
18 $perl_hash_or_arrayref = decode_json $utf8_encoded_json_text;
19
20 # OO-interface
21
22 $coder = JSON::XS->new->ascii->pretty->allow_nonref;
23 $pretty_printed_unencoded = $coder->encode ($perl_scalar);
24 $perl_scalar = $coder->decode ($unicode_json_text);
25
26 # Note that JSON version 2.0 and above will automatically use JSON::XS
27 # if available, at virtually no speed overhead either, so you should
28 # be able to just:
29
30 use JSON;
31
32 # and do the same things, except that you have a pure-perl fallback now.
33
34 =head1 DESCRIPTION
35
36 This module converts Perl data structures to JSON and vice versa. Its
37 primary goal is to be I<correct> and its secondary goal is to be
38 I<fast>. To reach the latter goal it was written in C.
39
40 Beginning with version 2.0 of the JSON module, when both JSON and
41 JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42 overridden) with no overhead due to emulation (by inheriting constructor
43 and methods). If JSON::XS is not available, it will fall back to the
44 compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45 gives you a portable JSON API that can be fast when you need and doesn't
46 require a C compiler when that is a problem.
47
48 As this is the n-th-something JSON module on CPAN, what was the reason
49 to write yet another JSON module? While it seems there are many JSON
50 modules, none of them correctly handle all corner cases, and in most cases
51 their maintainers are unresponsive, gone missing, or not listening to bug
52 reports for other reasons.
53
54 See MAPPING, below, on how JSON::XS maps perl values to JSON values and
55 vice versa.
56
57 =head2 FEATURES
58
59 =over 4
60
61 =item * correct Unicode handling
62
63 This module knows how to handle Unicode, documents how and when it does
64 so, and even documents what "correct" means.
65
66 =item * round-trip integrity
67
68 When you serialise a perl data structure using only data types supported
69 by JSON and Perl, the deserialised data structure is identical on the Perl
70 level. (e.g. the string "2.0" doesn't suddenly become "2" just because
71 it looks like a number). There I<are> minor exceptions to this, read the
72 MAPPING section below to learn about those.
73
74 =item * strict checking of JSON correctness
75
76 There is no guessing, no generating of illegal JSON texts by default,
77 and only JSON is accepted as input by default (the latter is a security
78 feature).
79
80 =item * fast
81
82 Compared to other JSON modules and other serialisers such as Storable,
83 this module usually compares favourably in terms of speed, too.
84
85 =item * simple to use
86
87 This module has both a simple functional interface as well as an object
88 oriented interface interface.
89
90 =item * reasonably versatile output formats
91
92 You can choose between the most compact guaranteed-single-line format
93 possible (nice for simple line-based protocols), a pure-ASCII format
94 (for when your transport is not 8-bit clean, still supports the whole
95 Unicode range), or a pretty-printed format (for when you want to read that
96 stuff). Or you can combine those features in whatever way you like.
97
98 =back
99
100 =cut
101
102 package JSON::XS;
103
104 use common::sense;
105
106 our $VERSION = 2.34;
107 our @ISA = qw(Exporter);
108
109 our @EXPORT = qw(encode_json decode_json to_json from_json);
110
111 sub to_json($) {
112 require Carp;
113 Carp::croak ("JSON::XS::to_json has been renamed to encode_json, either downgrade to pre-2.0 versions of JSON::XS or rename the call");
114 }
115
116 sub from_json($) {
117 require Carp;
118 Carp::croak ("JSON::XS::from_json has been renamed to decode_json, either downgrade to pre-2.0 versions of JSON::XS or rename the call");
119 }
120
121 use Exporter;
122 use XSLoader;
123
124 =head1 FUNCTIONAL INTERFACE
125
126 The following convenience methods are provided by this module. They are
127 exported by default:
128
129 =over 4
130
131 =item $json_text = encode_json $perl_scalar
132
133 Converts the given Perl data structure to a UTF-8 encoded, binary string
134 (that is, the string contains octets only). Croaks on error.
135
136 This function call is functionally identical to:
137
138 $json_text = JSON::XS->new->utf8->encode ($perl_scalar)
139
140 Except being faster.
141
142 =item $perl_scalar = decode_json $json_text
143
144 The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries
145 to parse that as an UTF-8 encoded JSON text, returning the resulting
146 reference. Croaks on error.
147
148 This function call is functionally identical to:
149
150 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
151
152 Except being faster.
153
154 =item $is_boolean = JSON::XS::is_bool $scalar
155
156 Returns true if the passed scalar represents either JSON::XS::true or
157 JSON::XS::false, two constants that act like C<1> and C<0>, respectively
158 and are used to represent JSON C<true> and C<false> values in Perl.
159
160 See MAPPING, below, for more information on how JSON values are mapped to
161 Perl.
162
163 =back
164
165
166 =head1 A FEW NOTES ON UNICODE AND PERL
167
168 Since this often leads to confusion, here are a few very clear words on
169 how Unicode works in Perl, modulo bugs.
170
171 =over 4
172
173 =item 1. Perl strings can store characters with ordinal values > 255.
174
175 This enables you to store Unicode characters as single characters in a
176 Perl string - very natural.
177
178 =item 2. Perl does I<not> associate an encoding with your strings.
179
180 ... until you force it to, e.g. when matching it against a regex, or
181 printing the scalar to a file, in which case Perl either interprets your
182 string as locale-encoded text, octets/binary, or as Unicode, depending
183 on various settings. In no case is an encoding stored together with your
184 data, it is I<use> that decides encoding, not any magical meta data.
185
186 =item 3. The internal utf-8 flag has no meaning with regards to the
187 encoding of your string.
188
189 Just ignore that flag unless you debug a Perl bug, a module written in
190 XS or want to dive into the internals of perl. Otherwise it will only
191 confuse you, as, despite the name, it says nothing about how your string
192 is encoded. You can have Unicode strings with that flag set, with that
193 flag clear, and you can have binary data with that flag set and that flag
194 clear. Other possibilities exist, too.
195
196 If you didn't know about that flag, just the better, pretend it doesn't
197 exist.
198
199 =item 4. A "Unicode String" is simply a string where each character can be
200 validly interpreted as a Unicode code point.
201
202 If you have UTF-8 encoded data, it is no longer a Unicode string, but a
203 Unicode string encoded in UTF-8, giving you a binary string.
204
205 =item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string.
206
207 It's a fact. Learn to live with it.
208
209 =back
210
211 I hope this helps :)
212
213
214 =head1 OBJECT-ORIENTED INTERFACE
215
216 The object oriented interface lets you configure your own encoding or
217 decoding style, within the limits of supported formats.
218
219 =over 4
220
221 =item $json = new JSON::XS
222
223 Creates a new JSON::XS object that can be used to de/encode JSON
224 strings. All boolean flags described below are by default I<disabled>.
225
226 The mutators for flags all return the JSON object again and thus calls can
227 be chained:
228
229 my $json = JSON::XS->new->utf8->space_after->encode ({a => [1,2]})
230 => {"a": [1, 2]}
231
232 =item $json = $json->ascii ([$enable])
233
234 =item $enabled = $json->get_ascii
235
236 If C<$enable> is true (or missing), then the C<encode> method will not
237 generate characters outside the code range C<0..127> (which is ASCII). Any
238 Unicode characters outside that range will be escaped using either a
239 single \uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence,
240 as per RFC4627. The resulting encoded JSON text can be treated as a native
241 Unicode string, an ascii-encoded, latin1-encoded or UTF-8 encoded string,
242 or any other superset of ASCII.
243
244 If C<$enable> is false, then the C<encode> method will not escape Unicode
245 characters unless required by the JSON syntax or other flags. This results
246 in a faster and more compact format.
247
248 See also the section I<ENCODING/CODESET FLAG NOTES> later in this
249 document.
250
251 The main use for this flag is to produce JSON texts that can be
252 transmitted over a 7-bit channel, as the encoded JSON texts will not
253 contain any 8 bit characters.
254
255 JSON::XS->new->ascii (1)->encode ([chr 0x10401])
256 => ["\ud801\udc01"]
257
258 =item $json = $json->latin1 ([$enable])
259
260 =item $enabled = $json->get_latin1
261
262 If C<$enable> is true (or missing), then the C<encode> method will encode
263 the resulting JSON text as latin1 (or iso-8859-1), escaping any characters
264 outside the code range C<0..255>. The resulting string can be treated as a
265 latin1-encoded JSON text or a native Unicode string. The C<decode> method
266 will not be affected in any way by this flag, as C<decode> by default
267 expects Unicode, which is a strict superset of latin1.
268
269 If C<$enable> is false, then the C<encode> method will not escape Unicode
270 characters unless required by the JSON syntax or other flags.
271
272 See also the section I<ENCODING/CODESET FLAG NOTES> later in this
273 document.
274
275 The main use for this flag is efficiently encoding binary data as JSON
276 text, as most octets will not be escaped, resulting in a smaller encoded
277 size. The disadvantage is that the resulting JSON text is encoded
278 in latin1 (and must correctly be treated as such when storing and
279 transferring), a rare encoding for JSON. It is therefore most useful when
280 you want to store data structures known to contain binary data efficiently
281 in files or databases, not when talking to other JSON encoders/decoders.
282
283 JSON::XS->new->latin1->encode (["\x{89}\x{abc}"]
284 => ["\x{89}\\u0abc"] # (perl syntax, U+abc escaped, U+89 not)
285
286 =item $json = $json->utf8 ([$enable])
287
288 =item $enabled = $json->get_utf8
289
290 If C<$enable> is true (or missing), then the C<encode> method will encode
291 the JSON result into UTF-8, as required by many protocols, while the
292 C<decode> method expects to be handled an UTF-8-encoded string. Please
293 note that UTF-8-encoded strings do not contain any characters outside the
294 range C<0..255>, they are thus useful for bytewise/binary I/O. In future
295 versions, enabling this option might enable autodetection of the UTF-16
296 and UTF-32 encoding families, as described in RFC4627.
297
298 If C<$enable> is false, then the C<encode> method will return the JSON
299 string as a (non-encoded) Unicode string, while C<decode> expects thus a
300 Unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs
301 to be done yourself, e.g. using the Encode module.
302
303 See also the section I<ENCODING/CODESET FLAG NOTES> later in this
304 document.
305
306 Example, output UTF-16BE-encoded JSON:
307
308 use Encode;
309 $jsontext = encode "UTF-16BE", JSON::XS->new->encode ($object);
310
311 Example, decode UTF-32LE-encoded JSON:
312
313 use Encode;
314 $object = JSON::XS->new->decode (decode "UTF-32LE", $jsontext);
315
316 =item $json = $json->pretty ([$enable])
317
318 This enables (or disables) all of the C<indent>, C<space_before> and
319 C<space_after> (and in the future possibly more) flags in one call to
320 generate the most readable (or most compact) form possible.
321
322 Example, pretty-print some simple structure:
323
324 my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
325 =>
326 {
327 "a" : [
328 1,
329 2
330 ]
331 }
332
333 =item $json = $json->indent ([$enable])
334
335 =item $enabled = $json->get_indent
336
337 If C<$enable> is true (or missing), then the C<encode> method will use a multiline
338 format as output, putting every array member or object/hash key-value pair
339 into its own line, indenting them properly.
340
341 If C<$enable> is false, no newlines or indenting will be produced, and the
342 resulting JSON text is guaranteed not to contain any C<newlines>.
343
344 This setting has no effect when decoding JSON texts.
345
346 =item $json = $json->space_before ([$enable])
347
348 =item $enabled = $json->get_space_before
349
350 If C<$enable> is true (or missing), then the C<encode> method will add an extra
351 optional space before the C<:> separating keys from values in JSON objects.
352
353 If C<$enable> is false, then the C<encode> method will not add any extra
354 space at those places.
355
356 This setting has no effect when decoding JSON texts. You will also
357 most likely combine this setting with C<space_after>.
358
359 Example, space_before enabled, space_after and indent disabled:
360
361 {"key" :"value"}
362
363 =item $json = $json->space_after ([$enable])
364
365 =item $enabled = $json->get_space_after
366
367 If C<$enable> is true (or missing), then the C<encode> method will add an extra
368 optional space after the C<:> separating keys from values in JSON objects
369 and extra whitespace after the C<,> separating key-value pairs and array
370 members.
371
372 If C<$enable> is false, then the C<encode> method will not add any extra
373 space at those places.
374
375 This setting has no effect when decoding JSON texts.
376
377 Example, space_before and indent disabled, space_after enabled:
378
379 {"key": "value"}
380
381 =item $json = $json->relaxed ([$enable])
382
383 =item $enabled = $json->get_relaxed
384
385 If C<$enable> is true (or missing), then C<decode> will accept some
386 extensions to normal JSON syntax (see below). C<encode> will not be
387 affected in anyway. I<Be aware that this option makes you accept invalid
388 JSON texts as if they were valid!>. I suggest only to use this option to
389 parse application-specific files written by humans (configuration files,
390 resource files etc.)
391
392 If C<$enable> is false (the default), then C<decode> will only accept
393 valid JSON texts.
394
395 Currently accepted extensions are:
396
397 =over 4
398
399 =item * list items can have an end-comma
400
401 JSON I<separates> array elements and key-value pairs with commas. This
402 can be annoying if you write JSON texts manually and want to be able to
403 quickly append elements, so this extension accepts comma at the end of
404 such items not just between them:
405
406 [
407 1,
408 2, <- this comma not normally allowed
409 ]
410 {
411 "k1": "v1",
412 "k2": "v2", <- this comma not normally allowed
413 }
414
415 =item * shell-style '#'-comments
416
417 Whenever JSON allows whitespace, shell-style comments are additionally
418 allowed. They are terminated by the first carriage-return or line-feed
419 character, after which more white-space and comments are allowed.
420
421 [
422 1, # this comment not allowed in JSON
423 # neither this one...
424 ]
425
426 =back
427
428 =item $json = $json->canonical ([$enable])
429
430 =item $enabled = $json->get_canonical
431
432 If C<$enable> is true (or missing), then the C<encode> method will output JSON objects
433 by sorting their keys. This is adding a comparatively high overhead.
434
435 If C<$enable> is false, then the C<encode> method will output key-value
436 pairs in the order Perl stores them (which will likely change between runs
437 of the same script, and can change even within the same run from 5.18
438 onwards).
439
440 This option is useful if you want the same data structure to be encoded as
441 the same JSON text (given the same overall settings). If it is disabled,
442 the same hash might be encoded differently even if contains the same data,
443 as key-value pairs have no inherent ordering in Perl.
444
445 This setting has no effect when decoding JSON texts.
446
447 This setting has currently no effect on tied hashes.
448
449 =item $json = $json->allow_nonref ([$enable])
450
451 =item $enabled = $json->get_allow_nonref
452
453 If C<$enable> is true (or missing), then the C<encode> method can convert a
454 non-reference into its corresponding string, number or null JSON value,
455 which is an extension to RFC4627. Likewise, C<decode> will accept those JSON
456 values instead of croaking.
457
458 If C<$enable> is false, then the C<encode> method will croak if it isn't
459 passed an arrayref or hashref, as JSON texts must either be an object
460 or array. Likewise, C<decode> will croak if given something that is not a
461 JSON object or array.
462
463 Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>,
464 resulting in an invalid JSON text:
465
466 JSON::XS->new->allow_nonref->encode ("Hello, World!")
467 => "Hello, World!"
468
469 =item $json = $json->allow_unknown ([$enable])
470
471 =item $enabled = $json->get_allow_unknown
472
473 If C<$enable> is true (or missing), then C<encode> will I<not> throw an
474 exception when it encounters values it cannot represent in JSON (for
475 example, filehandles) but instead will encode a JSON C<null> value. Note
476 that blessed objects are not included here and are handled separately by
477 c<allow_nonref>.
478
479 If C<$enable> is false (the default), then C<encode> will throw an
480 exception when it encounters anything it cannot encode as JSON.
481
482 This option does not affect C<decode> in any way, and it is recommended to
483 leave it off unless you know your communications partner.
484
485 =item $json = $json->allow_blessed ([$enable])
486
487 =item $enabled = $json->get_allow_blessed
488
489 If C<$enable> is true (or missing), then the C<encode> method will not
490 barf when it encounters a blessed reference. Instead, the value of the
491 B<convert_blessed> option will decide whether C<null> (C<convert_blessed>
492 disabled or no C<TO_JSON> method found) or a representation of the
493 object (C<convert_blessed> enabled and C<TO_JSON> method found) is being
494 encoded. Has no effect on C<decode>.
495
496 If C<$enable> is false (the default), then C<encode> will throw an
497 exception when it encounters a blessed object.
498
499 =item $json = $json->convert_blessed ([$enable])
500
501 =item $enabled = $json->get_convert_blessed
502
503 If C<$enable> is true (or missing), then C<encode>, upon encountering a
504 blessed object, will check for the availability of the C<TO_JSON> method
505 on the object's class. If found, it will be called in scalar context
506 and the resulting scalar will be encoded instead of the object. If no
507 C<TO_JSON> method is found, the value of C<allow_blessed> will decide what
508 to do.
509
510 The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON>
511 returns other blessed objects, those will be handled in the same
512 way. C<TO_JSON> must take care of not causing an endless recursion cycle
513 (== crash) in this case. The name of C<TO_JSON> was chosen because other
514 methods called by the Perl core (== not by the user of the object) are
515 usually in upper case letters and to avoid collisions with any C<to_json>
516 function or method.
517
518 This setting does not yet influence C<decode> in any way, but in the
519 future, global hooks might get installed that influence C<decode> and are
520 enabled by this setting.
521
522 If C<$enable> is false, then the C<allow_blessed> setting will decide what
523 to do when a blessed object is found.
524
525 =item $json = $json->filter_json_object ([$coderef->($hashref)])
526
527 When C<$coderef> is specified, it will be called from C<decode> each
528 time it decodes a JSON object. The only argument is a reference to the
529 newly-created hash. If the code references returns a single scalar (which
530 need not be a reference), this value (i.e. a copy of that scalar to avoid
531 aliasing) is inserted into the deserialised data structure. If it returns
532 an empty list (NOTE: I<not> C<undef>, which is a valid scalar), the
533 original deserialised hash will be inserted. This setting can slow down
534 decoding considerably.
535
536 When C<$coderef> is omitted or undefined, any existing callback will
537 be removed and C<decode> will not change the deserialised hash in any
538 way.
539
540 Example, convert all JSON objects into the integer 5:
541
542 my $js = JSON::XS->new->filter_json_object (sub { 5 });
543 # returns [5]
544 $js->decode ('[{}]')
545 # throw an exception because allow_nonref is not enabled
546 # so a lone 5 is not allowed.
547 $js->decode ('{"a":1, "b":2}');
548
549 =item $json = $json->filter_json_single_key_object ($key [=> $coderef->($value)])
550
551 Works remotely similar to C<filter_json_object>, but is only called for
552 JSON objects having a single key named C<$key>.
553
554 This C<$coderef> is called before the one specified via
555 C<filter_json_object>, if any. It gets passed the single value in the JSON
556 object. If it returns a single value, it will be inserted into the data
557 structure. If it returns nothing (not even C<undef> but the empty list),
558 the callback from C<filter_json_object> will be called next, as if no
559 single-key callback were specified.
560
561 If C<$coderef> is omitted or undefined, the corresponding callback will be
562 disabled. There can only ever be one callback for a given key.
563
564 As this callback gets called less often then the C<filter_json_object>
565 one, decoding speed will not usually suffer as much. Therefore, single-key
566 objects make excellent targets to serialise Perl objects into, especially
567 as single-key JSON objects are as close to the type-tagged value concept
568 as JSON gets (it's basically an ID/VALUE tuple). Of course, JSON does not
569 support this in any way, so you need to make sure your data never looks
570 like a serialised Perl hash.
571
572 Typical names for the single object key are C<__class_whatever__>, or
573 C<$__dollars_are_rarely_used__$> or C<}ugly_brace_placement>, or even
574 things like C<__class_md5sum(classname)__>, to reduce the risk of clashing
575 with real hashes.
576
577 Example, decode JSON objects of the form C<< { "__widget__" => <id> } >>
578 into the corresponding C<< $WIDGET{<id>} >> object:
579
580 # return whatever is in $WIDGET{5}:
581 JSON::XS
582 ->new
583 ->filter_json_single_key_object (__widget__ => sub {
584 $WIDGET{ $_[0] }
585 })
586 ->decode ('{"__widget__": 5')
587
588 # this can be used with a TO_JSON method in some "widget" class
589 # for serialisation to json:
590 sub WidgetBase::TO_JSON {
591 my ($self) = @_;
592
593 unless ($self->{id}) {
594 $self->{id} = ..get..some..id..;
595 $WIDGET{$self->{id}} = $self;
596 }
597
598 { __widget__ => $self->{id} }
599 }
600
601 =item $json = $json->shrink ([$enable])
602
603 =item $enabled = $json->get_shrink
604
605 Perl usually over-allocates memory a bit when allocating space for
606 strings. This flag optionally resizes strings generated by either
607 C<encode> or C<decode> to their minimum size possible. This can save
608 memory when your JSON texts are either very very long or you have many
609 short strings. It will also try to downgrade any strings to octet-form
610 if possible: perl stores strings internally either in an encoding called
611 UTF-X or in octet-form. The latter cannot store everything but uses less
612 space in general (and some buggy Perl or C code might even rely on that
613 internal representation being used).
614
615 The actual definition of what shrink does might change in future versions,
616 but it will always try to save space at the expense of time.
617
618 If C<$enable> is true (or missing), the string returned by C<encode> will
619 be shrunk-to-fit, while all strings generated by C<decode> will also be
620 shrunk-to-fit.
621
622 If C<$enable> is false, then the normal perl allocation algorithms are used.
623 If you work with your data, then this is likely to be faster.
624
625 In the future, this setting might control other things, such as converting
626 strings that look like integers or floats into integers or floats
627 internally (there is no difference on the Perl level), saving space.
628
629 =item $json = $json->max_depth ([$maximum_nesting_depth])
630
631 =item $max_depth = $json->get_max_depth
632
633 Sets the maximum nesting level (default C<512>) accepted while encoding
634 or decoding. If a higher nesting level is detected in JSON text or a Perl
635 data structure, then the encoder and decoder will stop and croak at that
636 point.
637
638 Nesting level is defined by number of hash- or arrayrefs that the encoder
639 needs to traverse to reach a given point or the number of C<{> or C<[>
640 characters without their matching closing parenthesis crossed to reach a
641 given character in a string.
642
643 Setting the maximum depth to one disallows any nesting, so that ensures
644 that the object is only a single hash/object or array.
645
646 If no argument is given, the highest possible setting will be used, which
647 is rarely useful.
648
649 Note that nesting is implemented by recursion in C. The default value has
650 been chosen to be as large as typical operating systems allow without
651 crashing.
652
653 See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
654
655 =item $json = $json->max_size ([$maximum_string_size])
656
657 =item $max_size = $json->get_max_size
658
659 Set the maximum length a JSON text may have (in bytes) where decoding is
660 being attempted. The default is C<0>, meaning no limit. When C<decode>
661 is called on a string that is longer then this many bytes, it will not
662 attempt to decode the string but throw an exception. This setting has no
663 effect on C<encode> (yet).
664
665 If no argument is given, the limit check will be deactivated (same as when
666 C<0> is specified).
667
668 See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
669
670 =item $json_text = $json->encode ($perl_scalar)
671
672 Converts the given Perl data structure (a simple scalar or a reference
673 to a hash or array) to its JSON representation. Simple scalars will be
674 converted into JSON string or number sequences, while references to arrays
675 become JSON arrays and references to hashes become JSON objects. Undefined
676 Perl values (e.g. C<undef>) become JSON C<null> values. Neither C<true>
677 nor C<false> values will be generated.
678
679 =item $perl_scalar = $json->decode ($json_text)
680
681 The opposite of C<encode>: expects a JSON text and tries to parse it,
682 returning the resulting simple scalar or reference. Croaks on error.
683
684 JSON numbers and strings become simple Perl scalars. JSON arrays become
685 Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
686 C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.
687
688 =item ($perl_scalar, $characters) = $json->decode_prefix ($json_text)
689
690 This works like the C<decode> method, but instead of raising an exception
691 when there is trailing garbage after the first JSON object, it will
692 silently stop parsing there and return the number of characters consumed
693 so far.
694
695 This is useful if your JSON texts are not delimited by an outer protocol
696 (which is not the brightest thing to do in the first place) and you need
697 to know where the JSON text ends.
698
699 JSON::XS->new->decode_prefix ("[1] the tail")
700 => ([], 3)
701
702 =back
703
704
705 =head1 INCREMENTAL PARSING
706
707 In some cases, there is the need for incremental parsing of JSON
708 texts. While this module always has to keep both JSON text and resulting
709 Perl data structure in memory at one time, it does allow you to parse a
710 JSON stream incrementally. It does so by accumulating text until it has
711 a full JSON object, which it then can decode. This process is similar to
712 using C<decode_prefix> to see if a full JSON object is available, but
713 is much more efficient (and can be implemented with a minimum of method
714 calls).
715
716 JSON::XS will only attempt to parse the JSON text once it is sure it
717 has enough text to get a decisive result, using a very simple but
718 truly incremental parser. This means that it sometimes won't stop as
719 early as the full parser, for example, it doesn't detect mismatched
720 parentheses. The only thing it guarantees is that it starts decoding as
721 soon as a syntactically valid JSON text has been seen. This means you need
722 to set resource limits (e.g. C<max_size>) to ensure the parser will stop
723 parsing in the presence if syntax errors.
724
725 The following methods implement this incremental parser.
726
727 =over 4
728
729 =item [void, scalar or list context] = $json->incr_parse ([$string])
730
731 This is the central parsing function. It can both append new text and
732 extract objects from the stream accumulated so far (both of these
733 functions are optional).
734
735 If C<$string> is given, then this string is appended to the already
736 existing JSON fragment stored in the C<$json> object.
737
738 After that, if the function is called in void context, it will simply
739 return without doing anything further. This can be used to add more text
740 in as many chunks as you want.
741
742 If the method is called in scalar context, then it will try to extract
743 exactly I<one> JSON object. If that is successful, it will return this
744 object, otherwise it will return C<undef>. If there is a parse error,
745 this method will croak just as C<decode> would do (one can then use
746 C<incr_skip> to skip the errornous part). This is the most common way of
747 using the method.
748
749 And finally, in list context, it will try to extract as many objects
750 from the stream as it can find and return them, or the empty list
751 otherwise. For this to work, there must be no separators between the JSON
752 objects or arrays, instead they must be concatenated back-to-back. If
753 an error occurs, an exception will be raised as in the scalar context
754 case. Note that in this case, any previously-parsed JSON texts will be
755 lost.
756
757 Example: Parse some JSON arrays/objects in a given string and return
758 them.
759
760 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
761
762 =item $lvalue_string = $json->incr_text
763
764 This method returns the currently stored JSON fragment as an lvalue, that
765 is, you can manipulate it. This I<only> works when a preceding call to
766 C<incr_parse> in I<scalar context> successfully returned an object. Under
767 all other circumstances you must not call this function (I mean it.
768 although in simple tests it might actually work, it I<will> fail under
769 real world conditions). As a special exception, you can also call this
770 method before having parsed anything.
771
772 This function is useful in two cases: a) finding the trailing text after a
773 JSON object or b) parsing multiple JSON objects separated by non-JSON text
774 (such as commas).
775
776 =item $json->incr_skip
777
778 This will reset the state of the incremental parser and will remove
779 the parsed text from the input buffer so far. This is useful after
780 C<incr_parse> died, in which case the input buffer and incremental parser
781 state is left unchanged, to skip the text parsed so far and to reset the
782 parse state.
783
784 The difference to C<incr_reset> is that only text until the parse error
785 occured is removed.
786
787 =item $json->incr_reset
788
789 This completely resets the incremental parser, that is, after this call,
790 it will be as if the parser had never parsed anything.
791
792 This is useful if you want to repeatedly parse JSON objects and want to
793 ignore any trailing data, which means you have to reset the parser after
794 each successful decode.
795
796 =back
797
798 =head2 LIMITATIONS
799
800 All options that affect decoding are supported, except
801 C<allow_nonref>. The reason for this is that it cannot be made to
802 work sensibly: JSON objects and arrays are self-delimited, i.e. you can concatenate
803 them back to back and still decode them perfectly. This does not hold true
804 for JSON numbers, however.
805
806 For example, is the string C<1> a single JSON number, or is it simply the
807 start of C<12>? Or is C<12> a single JSON number, or the concatenation
808 of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS
809 takes the conservative route and disallows this case.
810
811 =head2 EXAMPLES
812
813 Some examples will make all this clearer. First, a simple example that
814 works similarly to C<decode_prefix>: We want to decode the JSON object at
815 the start of a string and identify the portion after the JSON object:
816
817 my $text = "[1,2,3] hello";
818
819 my $json = new JSON::XS;
820
821 my $obj = $json->incr_parse ($text)
822 or die "expected JSON object or array at beginning of string";
823
824 my $tail = $json->incr_text;
825 # $tail now contains " hello"
826
827 Easy, isn't it?
828
829 Now for a more complicated example: Imagine a hypothetical protocol where
830 you read some requests from a TCP stream, and each request is a JSON
831 array, without any separation between them (in fact, it is often useful to
832 use newlines as "separators", as these get interpreted as whitespace at
833 the start of the JSON text, which makes it possible to test said protocol
834 with C<telnet>...).
835
836 Here is how you'd do it (it is trivial to write this in an event-based
837 manner):
838
839 my $json = new JSON::XS;
840
841 # read some data from the socket
842 while (sysread $socket, my $buf, 4096) {
843
844 # split and decode as many requests as possible
845 for my $request ($json->incr_parse ($buf)) {
846 # act on the $request
847 }
848 }
849
850 Another complicated example: Assume you have a string with JSON objects
851 or arrays, all separated by (optional) comma characters (e.g. C<[1],[2],
852 [3]>). To parse them, we have to skip the commas between the JSON texts,
853 and here is where the lvalue-ness of C<incr_text> comes in useful:
854
855 my $text = "[1],[2], [3]";
856 my $json = new JSON::XS;
857
858 # void context, so no parsing done
859 $json->incr_parse ($text);
860
861 # now extract as many objects as possible. note the
862 # use of scalar context so incr_text can be called.
863 while (my $obj = $json->incr_parse) {
864 # do something with $obj
865
866 # now skip the optional comma
867 $json->incr_text =~ s/^ \s* , //x;
868 }
869
870 Now lets go for a very complex example: Assume that you have a gigantic
871 JSON array-of-objects, many gigabytes in size, and you want to parse it,
872 but you cannot load it into memory fully (this has actually happened in
873 the real world :).
874
875 Well, you lost, you have to implement your own JSON parser. But JSON::XS
876 can still help you: You implement a (very simple) array parser and let
877 JSON decode the array elements, which are all full JSON objects on their
878 own (this wouldn't work if the array elements could be JSON numbers, for
879 example):
880
881 my $json = new JSON::XS;
882
883 # open the monster
884 open my $fh, "<bigfile.json"
885 or die "bigfile: $!";
886
887 # first parse the initial "["
888 for (;;) {
889 sysread $fh, my $buf, 65536
890 or die "read error: $!";
891 $json->incr_parse ($buf); # void context, so no parsing
892
893 # Exit the loop once we found and removed(!) the initial "[".
894 # In essence, we are (ab-)using the $json object as a simple scalar
895 # we append data to.
896 last if $json->incr_text =~ s/^ \s* \[ //x;
897 }
898
899 # now we have the skipped the initial "[", so continue
900 # parsing all the elements.
901 for (;;) {
902 # in this loop we read data until we got a single JSON object
903 for (;;) {
904 if (my $obj = $json->incr_parse) {
905 # do something with $obj
906 last;
907 }
908
909 # add more data
910 sysread $fh, my $buf, 65536
911 or die "read error: $!";
912 $json->incr_parse ($buf); # void context, so no parsing
913 }
914
915 # in this loop we read data until we either found and parsed the
916 # separating "," between elements, or the final "]"
917 for (;;) {
918 # first skip whitespace
919 $json->incr_text =~ s/^\s*//;
920
921 # if we find "]", we are done
922 if ($json->incr_text =~ s/^\]//) {
923 print "finished.\n";
924 exit;
925 }
926
927 # if we find ",", we can continue with the next element
928 if ($json->incr_text =~ s/^,//) {
929 last;
930 }
931
932 # if we find anything else, we have a parse error!
933 if (length $json->incr_text) {
934 die "parse error near ", $json->incr_text;
935 }
936
937 # else add more data
938 sysread $fh, my $buf, 65536
939 or die "read error: $!";
940 $json->incr_parse ($buf); # void context, so no parsing
941 }
942
943 This is a complex example, but most of the complexity comes from the fact
944 that we are trying to be correct (bear with me if I am wrong, I never ran
945 the above example :).
946
947
948
949 =head1 MAPPING
950
951 This section describes how JSON::XS maps Perl values to JSON values and
952 vice versa. These mappings are designed to "do the right thing" in most
953 circumstances automatically, preserving round-tripping characteristics
954 (what you put in comes out as something equivalent).
955
956 For the more enlightened: note that in the following descriptions,
957 lowercase I<perl> refers to the Perl interpreter, while uppercase I<Perl>
958 refers to the abstract Perl language itself.
959
960
961 =head2 JSON -> PERL
962
963 =over 4
964
965 =item object
966
967 A JSON object becomes a reference to a hash in Perl. No ordering of object
968 keys is preserved (JSON does not preserve object key ordering itself).
969
970 =item array
971
972 A JSON array becomes a reference to an array in Perl.
973
974 =item string
975
976 A JSON string becomes a string scalar in Perl - Unicode codepoints in JSON
977 are represented by the same codepoints in the Perl string, so no manual
978 decoding is necessary.
979
980 =item number
981
982 A JSON number becomes either an integer, numeric (floating point) or
983 string scalar in perl, depending on its range and any fractional parts. On
984 the Perl level, there is no difference between those as Perl handles all
985 the conversion details, but an integer may take slightly less memory and
986 might represent more values exactly than floating point numbers.
987
988 If the number consists of digits only, JSON::XS will try to represent
989 it as an integer value. If that fails, it will try to represent it as
990 a numeric (floating point) value if that is possible without loss of
991 precision. Otherwise it will preserve the number as a string value (in
992 which case you lose roundtripping ability, as the JSON number will be
993 re-encoded toa JSON string).
994
995 Numbers containing a fractional or exponential part will always be
996 represented as numeric (floating point) values, possibly at a loss of
997 precision (in which case you might lose perfect roundtripping ability, but
998 the JSON number will still be re-encoded as a JSON number).
999
1000 Note that precision is not accuracy - binary floating point values cannot
1001 represent most decimal fractions exactly, and when converting from and to
1002 floating point, JSON::XS only guarantees precision up to but not including
1003 the leats significant bit.
1004
1005 =item true, false
1006
1007 These JSON atoms become C<JSON::XS::true> and C<JSON::XS::false>,
1008 respectively. They are overloaded to act almost exactly like the numbers
1009 C<1> and C<0>. You can check whether a scalar is a JSON boolean by using
1010 the C<JSON::XS::is_bool> function.
1011
1012 =item null
1013
1014 A JSON null atom becomes C<undef> in Perl.
1015
1016 =back
1017
1018
1019 =head2 PERL -> JSON
1020
1021 The mapping from Perl to JSON is slightly more difficult, as Perl is a
1022 truly typeless language, so we can only guess which JSON type is meant by
1023 a Perl value.
1024
1025 =over 4
1026
1027 =item hash references
1028
1029 Perl hash references become JSON objects. As there is no inherent ordering
1030 in hash keys (or JSON objects), they will usually be encoded in a
1031 pseudo-random order that can change between runs of the same program but
1032 stays generally the same within a single run of a program. JSON::XS can
1033 optionally sort the hash keys (determined by the I<canonical> flag), so
1034 the same datastructure will serialise to the same JSON text (given same
1035 settings and version of JSON::XS), but this incurs a runtime overhead
1036 and is only rarely useful, e.g. when you want to compare some JSON text
1037 against another for equality.
1038
1039 =item array references
1040
1041 Perl array references become JSON arrays.
1042
1043 =item other references
1044
1045 Other unblessed references are generally not allowed and will cause an
1046 exception to be thrown, except for references to the integers C<0> and
1047 C<1>, which get turned into C<false> and C<true> atoms in JSON. You can
1048 also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
1049
1050 encode_json [\0, JSON::XS::true] # yields [false,true]
1051
1052 =item JSON::XS::true, JSON::XS::false
1053
1054 These special values become JSON true and JSON false values,
1055 respectively. You can also use C<\1> and C<\0> directly if you want.
1056
1057 =item blessed objects
1058
1059 Blessed objects are not directly representable in JSON. See the
1060 C<allow_blessed> and C<convert_blessed> methods on various options on
1061 how to deal with this: basically, you can choose between throwing an
1062 exception, encoding the reference as if it weren't blessed, or provide
1063 your own serialiser method.
1064
1065 =item simple scalars
1066
1067 Simple Perl scalars (any scalar that is not a reference) are the most
1068 difficult objects to encode: JSON::XS will encode undefined scalars as
1069 JSON C<null> values, scalars that have last been used in a string context
1070 before encoding as JSON strings, and anything else as number value:
1071
1072 # dump as number
1073 encode_json [2] # yields [2]
1074 encode_json [-3.0e17] # yields [-3e+17]
1075 my $value = 5; encode_json [$value] # yields [5]
1076
1077 # used as string, so dump as string
1078 print $value;
1079 encode_json [$value] # yields ["5"]
1080
1081 # undef becomes null
1082 encode_json [undef] # yields [null]
1083
1084 You can force the type to be a JSON string by stringifying it:
1085
1086 my $x = 3.1; # some variable containing a number
1087 "$x"; # stringified
1088 $x .= ""; # another, more awkward way to stringify
1089 print $x; # perl does it for you, too, quite often
1090
1091 You can force the type to be a JSON number by numifying it:
1092
1093 my $x = "3"; # some variable containing a string
1094 $x += 0; # numify it, ensuring it will be dumped as a number
1095 $x *= 1; # same thing, the choice is yours.
1096
1097 You can not currently force the type in other, less obscure, ways. Tell me
1098 if you need this capability (but don't forget to explain why it's needed
1099 :).
1100
1101 Note that numerical precision has the same meaning as under Perl (so
1102 binary to decimal conversion follows the same rules as in Perl, which
1103 can differ to other languages). Also, your perl interpreter might expose
1104 extensions to the floating point numbers of your platform, such as
1105 infinities or NaN's - these cannot be represented in JSON, and it is an
1106 error to pass those in.
1107
1108 =back
1109
1110
1111 =head1 ENCODING/CODESET FLAG NOTES
1112
1113 The interested reader might have seen a number of flags that signify
1114 encodings or codesets - C<utf8>, C<latin1> and C<ascii>. There seems to be
1115 some confusion on what these do, so here is a short comparison:
1116
1117 C<utf8> controls whether the JSON text created by C<encode> (and expected
1118 by C<decode>) is UTF-8 encoded or not, while C<latin1> and C<ascii> only
1119 control whether C<encode> escapes character values outside their respective
1120 codeset range. Neither of these flags conflict with each other, although
1121 some combinations make less sense than others.
1122
1123 Care has been taken to make all flags symmetrical with respect to
1124 C<encode> and C<decode>, that is, texts encoded with any combination of
1125 these flag values will be correctly decoded when the same flags are used
1126 - in general, if you use different flag settings while encoding vs. when
1127 decoding you likely have a bug somewhere.
1128
1129 Below comes a verbose discussion of these flags. Note that a "codeset" is
1130 simply an abstract set of character-codepoint pairs, while an encoding
1131 takes those codepoint numbers and I<encodes> them, in our case into
1132 octets. Unicode is (among other things) a codeset, UTF-8 is an encoding,
1133 and ISO-8859-1 (= latin 1) and ASCII are both codesets I<and> encodings at
1134 the same time, which can be confusing.
1135
1136 =over 4
1137
1138 =item C<utf8> flag disabled
1139
1140 When C<utf8> is disabled (the default), then C<encode>/C<decode> generate
1141 and expect Unicode strings, that is, characters with high ordinal Unicode
1142 values (> 255) will be encoded as such characters, and likewise such
1143 characters are decoded as-is, no canges to them will be done, except
1144 "(re-)interpreting" them as Unicode codepoints or Unicode characters,
1145 respectively (to Perl, these are the same thing in strings unless you do
1146 funny/weird/dumb stuff).
1147
1148 This is useful when you want to do the encoding yourself (e.g. when you
1149 want to have UTF-16 encoded JSON texts) or when some other layer does
1150 the encoding for you (for example, when printing to a terminal using a
1151 filehandle that transparently encodes to UTF-8 you certainly do NOT want
1152 to UTF-8 encode your data first and have Perl encode it another time).
1153
1154 =item C<utf8> flag enabled
1155
1156 If the C<utf8>-flag is enabled, C<encode>/C<decode> will encode all
1157 characters using the corresponding UTF-8 multi-byte sequence, and will
1158 expect your input strings to be encoded as UTF-8, that is, no "character"
1159 of the input string must have any value > 255, as UTF-8 does not allow
1160 that.
1161
1162 The C<utf8> flag therefore switches between two modes: disabled means you
1163 will get a Unicode string in Perl, enabled means you get an UTF-8 encoded
1164 octet/binary string in Perl.
1165
1166 =item C<latin1> or C<ascii> flags enabled
1167
1168 With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters
1169 with ordinal values > 255 (> 127 with C<ascii>) and encode the remaining
1170 characters as specified by the C<utf8> flag.
1171
1172 If C<utf8> is disabled, then the result is also correctly encoded in those
1173 character sets (as both are proper subsets of Unicode, meaning that a
1174 Unicode string with all character values < 256 is the same thing as a
1175 ISO-8859-1 string, and a Unicode string with all character values < 128 is
1176 the same thing as an ASCII string in Perl).
1177
1178 If C<utf8> is enabled, you still get a correct UTF-8-encoded string,
1179 regardless of these flags, just some more characters will be escaped using
1180 C<\uXXXX> then before.
1181
1182 Note that ISO-8859-1-I<encoded> strings are not compatible with UTF-8
1183 encoding, while ASCII-encoded strings are. That is because the ISO-8859-1
1184 encoding is NOT a subset of UTF-8 (despite the ISO-8859-1 I<codeset> being
1185 a subset of Unicode), while ASCII is.
1186
1187 Surprisingly, C<decode> will ignore these flags and so treat all input
1188 values as governed by the C<utf8> flag. If it is disabled, this allows you
1189 to decode ISO-8859-1- and ASCII-encoded strings, as both strict subsets of
1190 Unicode. If it is enabled, you can correctly decode UTF-8 encoded strings.
1191
1192 So neither C<latin1> nor C<ascii> are incompatible with the C<utf8> flag -
1193 they only govern when the JSON output engine escapes a character or not.
1194
1195 The main use for C<latin1> is to relatively efficiently store binary data
1196 as JSON, at the expense of breaking compatibility with most JSON decoders.
1197
1198 The main use for C<ascii> is to force the output to not contain characters
1199 with values > 127, which means you can interpret the resulting string
1200 as UTF-8, ISO-8859-1, ASCII, KOI8-R or most about any character set and
1201 8-bit-encoding, and still get the same data structure back. This is useful
1202 when your channel for JSON transfer is not 8-bit clean or the encoding
1203 might be mangled in between (e.g. in mail), and works because ASCII is a
1204 proper subset of most 8-bit and multibyte encodings in use in the world.
1205
1206 =back
1207
1208
1209 =head2 JSON and ECMAscript
1210
1211 JSON syntax is based on how literals are represented in javascript (the
1212 not-standardised predecessor of ECMAscript) which is presumably why it is
1213 called "JavaScript Object Notation".
1214
1215 However, JSON is not a subset (and also not a superset of course) of
1216 ECMAscript (the standard) or javascript (whatever browsers actually
1217 implement).
1218
1219 If you want to use javascript's C<eval> function to "parse" JSON, you
1220 might run into parse errors for valid JSON texts, or the resulting data
1221 structure might not be queryable:
1222
1223 One of the problems is that U+2028 and U+2029 are valid characters inside
1224 JSON strings, but are not allowed in ECMAscript string literals, so the
1225 following Perl fragment will not output something that can be guaranteed
1226 to be parsable by javascript's C<eval>:
1227
1228 use JSON::XS;
1229
1230 print encode_json [chr 0x2028];
1231
1232 The right fix for this is to use a proper JSON parser in your javascript
1233 programs, and not rely on C<eval> (see for example Douglas Crockford's
1234 F<json2.js> parser).
1235
1236 If this is not an option, you can, as a stop-gap measure, simply encode to
1237 ASCII-only JSON:
1238
1239 use JSON::XS;
1240
1241 print JSON::XS->new->ascii->encode ([chr 0x2028]);
1242
1243 Note that this will enlarge the resulting JSON text quite a bit if you
1244 have many non-ASCII characters. You might be tempted to run some regexes
1245 to only escape U+2028 and U+2029, e.g.:
1246
1247 # DO NOT USE THIS!
1248 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1249 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1250 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1251 print $json;
1252
1253 Note that I<this is a bad idea>: the above only works for U+2028 and
1254 U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1255 javascript implementations, however, have issues with other characters as
1256 well - using C<eval> naively simply I<will> cause problems.
1257
1258 Another problem is that some javascript implementations reserve
1259 some property names for their own purposes (which probably makes
1260 them non-ECMAscript-compliant). For example, Iceweasel reserves the
1261 C<__proto__> property name for its own purposes.
1262
1263 If that is a problem, you could parse try to filter the resulting JSON
1264 output for these property strings, e.g.:
1265
1266 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1267
1268 This works because C<__proto__> is not valid outside of strings, so every
1269 occurence of C<"__proto__"\s*:> must be a string used as property name.
1270
1271 If you know of other incompatibilities, please let me know.
1272
1273
1274 =head2 JSON and YAML
1275
1276 You often hear that JSON is a subset of YAML. This is, however, a mass
1277 hysteria(*) and very far from the truth (as of the time of this writing),
1278 so let me state it clearly: I<in general, there is no way to configure
1279 JSON::XS to output a data structure as valid YAML> that works in all
1280 cases.
1281
1282 If you really must use JSON::XS to generate YAML, you should use this
1283 algorithm (subject to change in future versions):
1284
1285 my $to_yaml = JSON::XS->new->utf8->space_after (1);
1286 my $yaml = $to_yaml->encode ($ref) . "\n";
1287
1288 This will I<usually> generate JSON texts that also parse as valid
1289 YAML. Please note that YAML has hardcoded limits on (simple) object key
1290 lengths that JSON doesn't have and also has different and incompatible
1291 unicode character escape syntax, so you should make sure that your hash
1292 keys are noticeably shorter than the 1024 "stream characters" YAML allows
1293 and that you do not have characters with codepoint values outside the
1294 Unicode BMP (basic multilingual page). YAML also does not allow C<\/>
1295 sequences in strings (which JSON::XS does not I<currently> generate, but
1296 other JSON generators might).
1297
1298 There might be other incompatibilities that I am not aware of (or the YAML
1299 specification has been changed yet again - it does so quite often). In
1300 general you should not try to generate YAML with a JSON generator or vice
1301 versa, or try to parse JSON with a YAML parser or vice versa: chances are
1302 high that you will run into severe interoperability problems when you
1303 least expect it.
1304
1305 =over 4
1306
1307 =item (*)
1308
1309 I have been pressured multiple times by Brian Ingerson (one of the
1310 authors of the YAML specification) to remove this paragraph, despite him
1311 acknowledging that the actual incompatibilities exist. As I was personally
1312 bitten by this "JSON is YAML" lie, I refused and said I will continue to
1313 educate people about these issues, so others do not run into the same
1314 problem again and again. After this, Brian called me a (quote)I<complete
1315 and worthless idiot>(unquote).
1316
1317 In my opinion, instead of pressuring and insulting people who actually
1318 clarify issues with YAML and the wrong statements of some of its
1319 proponents, I would kindly suggest reading the JSON spec (which is not
1320 that difficult or long) and finally make YAML compatible to it, and
1321 educating users about the changes, instead of spreading lies about the
1322 real compatibility for many I<years> and trying to silence people who
1323 point out that it isn't true.
1324
1325 Addendum/2009: the YAML 1.2 spec is still incompatible with JSON, even
1326 though the incompatibilities have been documented (and are known to Brian)
1327 for many years and the spec makes explicit claims that YAML is a superset
1328 of JSON. It would be so easy to fix, but apparently, bullying people and
1329 corrupting userdata is so much easier.
1330
1331 =back
1332
1333
1334 =head2 SPEED
1335
1336 It seems that JSON::XS is surprisingly fast, as shown in the following
1337 tables. They have been generated with the help of the C<eg/bench> program
1338 in the JSON::XS distribution, to make it easy to compare on your own
1339 system.
1340
1341 First comes a comparison between various modules using
1342 a very short single-line JSON string (also available at
1343 L<http://dist.schmorp.de/misc/json/short.json>).
1344
1345 {"method": "handleMessage", "params": ["user1",
1346 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1347 1, 0]}
1348
1349 It shows the number of encodes/decodes per second (JSON::XS uses
1350 the functional interface, while JSON::XS/2 uses the OO interface
1351 with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1352 shrink. JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ
1353 uses the from_json method). Higher is better:
1354
1355 module | encode | decode |
1356 --------------|------------|------------|
1357 JSON::DWIW/DS | 86302.551 | 102300.098 |
1358 JSON::DWIW/FJ | 86302.551 | 75983.768 |
1359 JSON::PP | 15827.562 | 6638.658 |
1360 JSON::Syck | 63358.066 | 47662.545 |
1361 JSON::XS | 511500.488 | 511500.488 |
1362 JSON::XS/2 | 291271.111 | 388361.481 |
1363 JSON::XS/3 | 361577.931 | 361577.931 |
1364 Storable | 66788.280 | 265462.278 |
1365 --------------+------------+------------+
1366
1367 That is, JSON::XS is almost six times faster than JSON::DWIW on encoding,
1368 about five times faster on decoding, and over thirty to seventy times
1369 faster than JSON's pure perl implementation. It also compares favourably
1370 to Storable for small amounts of data.
1371
1372 Using a longer test string (roughly 18KB, generated from Yahoo! Locals
1373 search API (L<http://dist.schmorp.de/misc/json/long.json>).
1374
1375 module | encode | decode |
1376 --------------|------------|------------|
1377 JSON::DWIW/DS | 1647.927 | 2673.916 |
1378 JSON::DWIW/FJ | 1630.249 | 2596.128 |
1379 JSON::PP | 400.640 | 62.311 |
1380 JSON::Syck | 1481.040 | 1524.869 |
1381 JSON::XS | 20661.596 | 9541.183 |
1382 JSON::XS/2 | 10683.403 | 9416.938 |
1383 JSON::XS/3 | 20661.596 | 9400.054 |
1384 Storable | 19765.806 | 10000.725 |
1385 --------------+------------+------------+
1386
1387 Again, JSON::XS leads by far (except for Storable which non-surprisingly
1388 decodes a bit faster).
1389
1390 On large strings containing lots of high Unicode characters, some modules
1391 (such as JSON::PC) seem to decode faster than JSON::XS, but the result
1392 will be broken due to missing (or wrong) Unicode handling. Others refuse
1393 to decode or encode properly, so it was impossible to prepare a fair
1394 comparison table for that case.
1395
1396
1397 =head1 SECURITY CONSIDERATIONS
1398
1399 When you are using JSON in a protocol, talking to untrusted potentially
1400 hostile creatures requires relatively few measures.
1401
1402 First of all, your JSON decoder should be secure, that is, should not have
1403 any buffer overflows. Obviously, this module should ensure that and I am
1404 trying hard on making that true, but you never know.
1405
1406 Second, you need to avoid resource-starving attacks. That means you should
1407 limit the size of JSON texts you accept, or make sure then when your
1408 resources run out, that's just fine (e.g. by using a separate process that
1409 can crash safely). The size of a JSON text in octets or characters is
1410 usually a good indication of the size of the resources required to decode
1411 it into a Perl structure. While JSON::XS can check the size of the JSON
1412 text, it might be too late when you already have it in memory, so you
1413 might want to check the size before you accept the string.
1414
1415 Third, JSON::XS recurses using the C stack when decoding objects and
1416 arrays. The C stack is a limited resource: for instance, on my amd64
1417 machine with 8MB of stack size I can decode around 180k nested arrays but
1418 only 14k nested JSON objects (due to perl itself recursing deeply on croak
1419 to free the temporary). If that is exceeded, the program crashes. To be
1420 conservative, the default nesting limit is set to 512. If your process
1421 has a smaller stack, you should adjust this setting accordingly with the
1422 C<max_depth> method.
1423
1424 Something else could bomb you, too, that I forgot to think of. In that
1425 case, you get to keep the pieces. I am always open for hints, though...
1426
1427 Also keep in mind that JSON::XS might leak contents of your Perl data
1428 structures in its error messages, so when you serialise sensitive
1429 information you might want to make sure that exceptions thrown by JSON::XS
1430 will not end up in front of untrusted eyes.
1431
1432 If you are using JSON::XS to return packets to consumption
1433 by JavaScript scripts in a browser you should have a look at
1434 L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to
1435 see whether you are vulnerable to some common attack vectors (which really
1436 are browser design bugs, but it is still you who will have to deal with
1437 it, as major browser developers care only for features, not about getting
1438 security right).
1439
1440
1441 =head1 THREADS
1442
1443 This module is I<not> guaranteed to be thread safe and there are no
1444 plans to change this until Perl gets thread support (as opposed to the
1445 horribly slow so-called "threads" which are simply slow and bloated
1446 process simulations - use fork, it's I<much> faster, cheaper, better).
1447
1448 (It might actually work, but you have been warned).
1449
1450
1451 =head1 THE PERILS OF SETLOCALE
1452
1453 Sometimes people avoid the Perl locale support and directly call the
1454 system's setlocale function with C<LC_ALL>.
1455
1456 This breaks both perl and modules such as JSON::XS, as stringification of
1457 numbers no longer works correcly (e.g. C<$x = 0.1; print "$x"+1> might
1458 print C<1>, and JSON::XS might output illegal JSON as JSON::XS relies on
1459 perl to stringify numbers).
1460
1461 The solution is simple: don't call C<setlocale>, or use it for only those
1462 categories you need, such as C<LC_MESSAGES> or C<LC_CTYPE>.
1463
1464 If you need C<LC_NUMERIC>, you should enable it only around the code that
1465 actually needs it (avoiding stringification of numbers), and restore it
1466 afterwards.
1467
1468
1469 =head1 BUGS
1470
1471 While the goal of this module is to be correct, that unfortunately does
1472 not mean it's bug-free, only that I think its design is bug-free. If you
1473 keep reporting bugs they will be fixed swiftly, though.
1474
1475 Please refrain from using rt.cpan.org or any other bug reporting
1476 service. I put the contact address into my modules for a reason.
1477
1478 =cut
1479
1480 our $true = do { bless \(my $dummy = 1), "JSON::XS::Boolean" };
1481 our $false = do { bless \(my $dummy = 0), "JSON::XS::Boolean" };
1482
1483 sub true() { $true }
1484 sub false() { $false }
1485
1486 sub is_bool($) {
1487 UNIVERSAL::isa $_[0], "JSON::XS::Boolean"
1488 # or UNIVERSAL::isa $_[0], "JSON::Literal"
1489 }
1490
1491 XSLoader::load "JSON::XS", $VERSION;
1492
1493 package JSON::XS::Boolean;
1494
1495 use overload
1496 "0+" => sub { ${$_[0]} },
1497 "++" => sub { $_[0] = ${$_[0]} + 1 },
1498 "--" => sub { $_[0] = ${$_[0]} - 1 },
1499 fallback => 1;
1500
1501 1;
1502
1503 =head1 SEE ALSO
1504
1505 The F<json_xs> command line utility for quick experiments.
1506
1507 =head1 AUTHOR
1508
1509 Marc Lehmann <schmorp@schmorp.de>
1510 http://home.schmorp.de/
1511
1512 =cut
1513