ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.100 by root, Sun Mar 30 09:27:16 2008 UTC vs.
Revision 1.123 by root, Sat Aug 8 10:06:02 2009 UTC

1=head1 NAME 1=head1 NAME
2 2
3JSON::XS - JSON serialising/deserialising, done correctly and fast
4
3=encoding utf-8 5=encoding utf-8
4
5JSON::XS - JSON serialising/deserialising, done correctly and fast
6 6
7JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ 7JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ
8 (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html) 8 (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html)
9 9
10=head1 SYNOPSIS 10=head1 SYNOPSIS
… …
37primary goal is to be I<correct> and its secondary goal is to be 37primary goal is to be I<correct> and its secondary goal is to be
38I<fast>. To reach the latter goal it was written in C. 38I<fast>. To reach the latter goal it was written in C.
39 39
40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overriden) with no overhead due to emulation (by inheritign constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need and doesn't
46require a C compiler when that is a problem. 46require a C compiler when that is a problem.
47 47
… …
49to write yet another JSON module? While it seems there are many JSON 49to write yet another JSON module? While it seems there are many JSON
50modules, none of them correctly handle all corner cases, and in most cases 50modules, none of them correctly handle all corner cases, and in most cases
51their maintainers are unresponsive, gone missing, or not listening to bug 51their maintainers are unresponsive, gone missing, or not listening to bug
52reports for other reasons. 52reports for other reasons.
53 53
54See COMPARISON, below, for a comparison to some other JSON modules.
55
56See MAPPING, below, on how JSON::XS maps perl values to JSON values and 54See MAPPING, below, on how JSON::XS maps perl values to JSON values and
57vice versa. 55vice versa.
58 56
59=head2 FEATURES 57=head2 FEATURES
60 58
… …
65This module knows how to handle Unicode, documents how and when it does 63This module knows how to handle Unicode, documents how and when it does
66so, and even documents what "correct" means. 64so, and even documents what "correct" means.
67 65
68=item * round-trip integrity 66=item * round-trip integrity
69 67
70When you serialise a perl data structure using only datatypes supported 68When you serialise a perl data structure using only data types supported
71by JSON, the deserialised data structure is identical on the Perl level. 69by JSON, the deserialised data structure is identical on the Perl level.
72(e.g. the string "2.0" doesn't suddenly become "2" just because it looks 70(e.g. the string "2.0" doesn't suddenly become "2" just because it looks
73like a number). There minor I<are> exceptions to this, read the MAPPING 71like a number). There minor I<are> exceptions to this, read the MAPPING
74section below to learn about those. 72section below to learn about those.
75 73
… …
84Compared to other JSON modules and other serialisers such as Storable, 82Compared to other JSON modules and other serialisers such as Storable,
85this module usually compares favourably in terms of speed, too. 83this module usually compares favourably in terms of speed, too.
86 84
87=item * simple to use 85=item * simple to use
88 86
89This module has both a simple functional interface as well as an objetc 87This module has both a simple functional interface as well as an object
90oriented interface interface. 88oriented interface interface.
91 89
92=item * reasonably versatile output formats 90=item * reasonably versatile output formats
93 91
94You can choose between the most compact guaranteed-single-line format 92You can choose between the most compact guaranteed-single-line format
95possible (nice for simple line-based protocols), a pure-ascii format 93possible (nice for simple line-based protocols), a pure-ASCII format
96(for when your transport is not 8-bit clean, still supports the whole 94(for when your transport is not 8-bit clean, still supports the whole
97Unicode range), or a pretty-printed format (for when you want to read that 95Unicode range), or a pretty-printed format (for when you want to read that
98stuff). Or you can combine those features in whatever way you like. 96stuff). Or you can combine those features in whatever way you like.
99 97
100=back 98=back
101 99
102=cut 100=cut
103 101
104package JSON::XS; 102package JSON::XS;
105 103
106use strict; 104use common::sense;
107 105
108our $VERSION = '2.2'; 106our $VERSION = '2.25';
109our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
110 108
111our @EXPORT = qw(encode_json decode_json to_json from_json); 109our @EXPORT = qw(encode_json decode_json to_json from_json);
112 110
113sub to_json($) { 111sub to_json($) {
… …
137 135
138This function call is functionally identical to: 136This function call is functionally identical to:
139 137
140 $json_text = JSON::XS->new->utf8->encode ($perl_scalar) 138 $json_text = JSON::XS->new->utf8->encode ($perl_scalar)
141 139
142except being faster. 140Except being faster.
143 141
144=item $perl_scalar = decode_json $json_text 142=item $perl_scalar = decode_json $json_text
145 143
146The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 144The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries
147to parse that as an UTF-8 encoded JSON text, returning the resulting 145to parse that as an UTF-8 encoded JSON text, returning the resulting
… …
149 147
150This function call is functionally identical to: 148This function call is functionally identical to:
151 149
152 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 150 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
153 151
154except being faster. 152Except being faster.
155 153
156=item $is_boolean = JSON::XS::is_bool $scalar 154=item $is_boolean = JSON::XS::is_bool $scalar
157 155
158Returns true if the passed scalar represents either JSON::XS::true or 156Returns true if the passed scalar represents either JSON::XS::true or
159JSON::XS::false, two constants that act like C<1> and C<0>, respectively 157JSON::XS::false, two constants that act like C<1> and C<0>, respectively
… …
197 195
198If you didn't know about that flag, just the better, pretend it doesn't 196If you didn't know about that flag, just the better, pretend it doesn't
199exist. 197exist.
200 198
201=item 4. A "Unicode String" is simply a string where each character can be 199=item 4. A "Unicode String" is simply a string where each character can be
202validly interpreted as a Unicode codepoint. 200validly interpreted as a Unicode code point.
203 201
204If you have UTF-8 encoded data, it is no longer a Unicode string, but a 202If you have UTF-8 encoded data, it is no longer a Unicode string, but a
205Unicode string encoded in UTF-8, giving you a binary string. 203Unicode string encoded in UTF-8, giving you a binary string.
206 204
207=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string. 205=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string.
… …
442the same JSON text (given the same overall settings). If it is disabled, 440the same JSON text (given the same overall settings). If it is disabled,
443the same hash might be encoded differently even if contains the same data, 441the same hash might be encoded differently even if contains the same data,
444as key-value pairs have no inherent ordering in Perl. 442as key-value pairs have no inherent ordering in Perl.
445 443
446This setting has no effect when decoding JSON texts. 444This setting has no effect when decoding JSON texts.
445
446This setting has currently no effect on tied hashes.
447 447
448=item $json = $json->allow_nonref ([$enable]) 448=item $json = $json->allow_nonref ([$enable])
449 449
450=item $enabled = $json->get_allow_nonref 450=item $enabled = $json->get_allow_nonref
451 451
… …
628=item $json = $json->max_depth ([$maximum_nesting_depth]) 628=item $json = $json->max_depth ([$maximum_nesting_depth])
629 629
630=item $max_depth = $json->get_max_depth 630=item $max_depth = $json->get_max_depth
631 631
632Sets the maximum nesting level (default C<512>) accepted while encoding 632Sets the maximum nesting level (default C<512>) accepted while encoding
633or decoding. If the JSON text or Perl data structure has an equal or 633or decoding. If a higher nesting level is detected in JSON text or a Perl
634higher nesting level then this limit, then the encoder and decoder will 634data structure, then the encoder and decoder will stop and croak at that
635stop and croak at that point. 635point.
636 636
637Nesting level is defined by number of hash- or arrayrefs that the encoder 637Nesting level is defined by number of hash- or arrayrefs that the encoder
638needs to traverse to reach a given point or the number of C<{> or C<[> 638needs to traverse to reach a given point or the number of C<{> or C<[>
639characters without their matching closing parenthesis crossed to reach a 639characters without their matching closing parenthesis crossed to reach a
640given character in a string. 640given character in a string.
641 641
642Setting the maximum depth to one disallows any nesting, so that ensures 642Setting the maximum depth to one disallows any nesting, so that ensures
643that the object is only a single hash/object or array. 643that the object is only a single hash/object or array.
644 644
645The argument to C<max_depth> will be rounded up to the next highest power
646of two. If no argument is given, the highest possible setting will be 645If no argument is given, the highest possible setting will be used, which
647used, which is rarely useful. 646is rarely useful.
647
648Note that nesting is implemented by recursion in C. The default value has
649been chosen to be as large as typical operating systems allow without
650crashing.
648 651
649See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 652See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
650 653
651=item $json = $json->max_size ([$maximum_string_size]) 654=item $json = $json->max_size ([$maximum_string_size])
652 655
653=item $max_size = $json->get_max_size 656=item $max_size = $json->get_max_size
654 657
655Set the maximum length a JSON text may have (in bytes) where decoding is 658Set the maximum length a JSON text may have (in bytes) where decoding is
656being attempted. The default is C<0>, meaning no limit. When C<decode> 659being attempted. The default is C<0>, meaning no limit. When C<decode>
657is called on a string longer then this number of characters it will not 660is called on a string that is longer then this many bytes, it will not
658attempt to decode the string but throw an exception. This setting has no 661attempt to decode the string but throw an exception. This setting has no
659effect on C<encode> (yet). 662effect on C<encode> (yet).
660 663
661The argument to C<max_size> will be rounded up to the next B<highest> 664If no argument is given, the limit check will be deactivated (same as when
662power of two (so may be more than requested). If no argument is given, the 665C<0> is specified).
663limit check will be deactivated (same as when C<0> is specified).
664 666
665See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 667See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
666 668
667=item $json_text = $json->encode ($perl_scalar) 669=item $json_text = $json->encode ($perl_scalar)
668 670
… …
699=back 701=back
700 702
701 703
702=head1 INCREMENTAL PARSING 704=head1 INCREMENTAL PARSING
703 705
704[This section and the API it details is still EXPERIMENTAL]
705
706In some cases, there is the need for incremental parsing of JSON 706In some cases, there is the need for incremental parsing of JSON
707texts. While this module always has to keep both JSON text and resulting 707texts. While this module always has to keep both JSON text and resulting
708Perl data structure in memory at one time, it does allow you to parse a 708Perl data structure in memory at one time, it does allow you to parse a
709JSON stream incrementally. It does so by accumulating text until it has 709JSON stream incrementally. It does so by accumulating text until it has
710a full JSON object, which it then can decode. This process is similar to 710a full JSON object, which it then can decode. This process is similar to
711using C<decode_prefix> to see if a full JSON object is available, but is 711using C<decode_prefix> to see if a full JSON object is available, but
712much more efficient (JSON::XS will only attempt to parse the JSON text 712is much more efficient (and can be implemented with a minimum of method
713calls).
714
715JSON::XS will only attempt to parse the JSON text once it is sure it
713once it is sure it has enough text to get a decisive result, using a very 716has enough text to get a decisive result, using a very simple but
714simple but truly incremental parser). 717truly incremental parser. This means that it sometimes won't stop as
718early as the full parser, for example, it doesn't detect parenthese
719mismatches. The only thing it guarantees is that it starts decoding as
720soon as a syntactically valid JSON text has been seen. This means you need
721to set resource limits (e.g. C<max_size>) to ensure the parser will stop
722parsing in the presence if syntax errors.
715 723
716The following two methods deal with this. 724The following methods implement this incremental parser.
717 725
718=over 4 726=over 4
719 727
720=item [void, scalar or list context] = $json->incr_parse ([$string]) 728=item [void, scalar or list context] = $json->incr_parse ([$string])
721 729
… …
759JSON object or b) parsing multiple JSON objects separated by non-JSON text 767JSON object or b) parsing multiple JSON objects separated by non-JSON text
760(such as commas). 768(such as commas).
761 769
762=item $json->incr_skip 770=item $json->incr_skip
763 771
764This will reset the state of the incremental parser and will remove the 772This will reset the state of the incremental parser and will remove
765parsed text from the input buffer. This is useful after C<incr_parse> 773the parsed text from the input buffer so far. This is useful after
766died, in which case the input buffer and incremental parser state is left 774C<incr_parse> died, in which case the input buffer and incremental parser
767unchanged, to skip the text parsed so far and to reset the parse state. 775state is left unchanged, to skip the text parsed so far and to reset the
776parse state.
777
778The difference to C<incr_reset> is that only text until the parse error
779occured is removed.
780
781=item $json->incr_reset
782
783This completely resets the incremental parser, that is, after this call,
784it will be as if the parser had never parsed anything.
785
786This is useful if you want to repeatedly parse JSON objects and want to
787ignore any trailing data, which means you have to reset the parser after
788each successful decode.
768 789
769=back 790=back
770 791
771=head2 LIMITATIONS 792=head2 LIMITATIONS
772 793
… …
1013Other unblessed references are generally not allowed and will cause an 1034Other unblessed references are generally not allowed and will cause an
1014exception to be thrown, except for references to the integers C<0> and 1035exception to be thrown, except for references to the integers C<0> and
1015C<1>, which get turned into C<false> and C<true> atoms in JSON. You can 1036C<1>, which get turned into C<false> and C<true> atoms in JSON. You can
1016also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability. 1037also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
1017 1038
1018 encode_json [\0,JSON::XS::true] # yields [false,true] 1039 encode_json [\0, JSON::XS::true] # yields [false,true]
1019 1040
1020=item JSON::XS::true, JSON::XS::false 1041=item JSON::XS::true, JSON::XS::false
1021 1042
1022These special values become JSON true and JSON false values, 1043These special values become JSON true and JSON false values,
1023respectively. You can also use C<\1> and C<\0> directly if you want. 1044respectively. You can also use C<\1> and C<\0> directly if you want.
… …
1163when your channel for JSON transfer is not 8-bit clean or the encoding 1184when your channel for JSON transfer is not 8-bit clean or the encoding
1164might be mangled in between (e.g. in mail), and works because ASCII is a 1185might be mangled in between (e.g. in mail), and works because ASCII is a
1165proper subset of most 8-bit and multibyte encodings in use in the world. 1186proper subset of most 8-bit and multibyte encodings in use in the world.
1166 1187
1167=back 1188=back
1189
1190
1191=head2 JSON and ECMAscript
1192
1193JSON syntax is based on how literals are represented in javascript (the
1194not-standardised predecessor of ECMAscript) which is presumably why it is
1195called "JavaScript Object Notation".
1196
1197However, JSON is not a subset (and also not a superset of course) of
1198ECMAscript (the standard) or javascript (whatever browsers actually
1199implement).
1200
1201If you want to use javascript's C<eval> function to "parse" JSON, you
1202might run into parse errors for valid JSON texts, or the resulting data
1203structure might not be queryable:
1204
1205One of the problems is that U+2028 and U+2029 are valid characters inside
1206JSON strings, but are not allowed in ECMAscript string literals, so the
1207following Perl fragment will not output something that can be guaranteed
1208to be parsable by javascript's C<eval>:
1209
1210 use JSON::XS;
1211
1212 print encode_json [chr 0x2028];
1213
1214The right fix for this is to use a proper JSON parser in your javascript
1215programs, and not rely on C<eval> (see for example Douglas Crockford's
1216F<json2.js> parser).
1217
1218If this is not an option, you can, as a stop-gap measure, simply encode to
1219ASCII-only JSON:
1220
1221 use JSON::XS;
1222
1223 print JSON::XS->new->ascii->encode ([chr 0x2028]);
1224
1225Note that this will enlarge the resulting JSON text quite a bit if you
1226have many non-ASCII characters. You might be tempted to run some regexes
1227to only escape U+2028 and U+2029, e.g.:
1228
1229 # DO NOT USE THIS!
1230 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1231 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1232 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1233 print $json;
1234
1235Note that I<this is a bad idea>: the above only works for U+2028 and
1236U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1237javascript implementations, however, have issues with other characters as
1238well - using C<eval> naively simply I<will> cause problems.
1239
1240Another problem is that some javascript implementations reserve
1241some property names for their own purposes (which probably makes
1242them non-ECMAscript-compliant). For example, Iceweasel reserves the
1243C<__proto__> property name for it's own purposes.
1244
1245If that is a problem, you could parse try to filter the resulting JSON
1246output for these property strings, e.g.:
1247
1248 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1249
1250This works because C<__proto__> is not valid outside of strings, so every
1251occurence of C<"__proto__"\s*:> must be a string used as property name.
1252
1253If you know of other incompatibilities, please let me know.
1168 1254
1169 1255
1170=head2 JSON and YAML 1256=head2 JSON and YAML
1171 1257
1172You often hear that JSON is a subset of YAML. This is, however, a mass 1258You often hear that JSON is a subset of YAML. This is, however, a mass
… …
1340 1426
1341 1427
1342=head1 BUGS 1428=head1 BUGS
1343 1429
1344While the goal of this module is to be correct, that unfortunately does 1430While the goal of this module is to be correct, that unfortunately does
1345not mean it's bug-free, only that I think its design is bug-free. It is 1431not mean it's bug-free, only that I think its design is bug-free. If you
1346still relatively early in its development. If you keep reporting bugs they 1432keep reporting bugs they will be fixed swiftly, though.
1347will be fixed swiftly, though.
1348 1433
1349Please refrain from using rt.cpan.org or any other bug reporting 1434Please refrain from using rt.cpan.org or any other bug reporting
1350service. I put the contact address into my modules for a reason. 1435service. I put the contact address into my modules for a reason.
1351 1436
1352=cut 1437=cut

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines