ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.104 by root, Thu May 8 15:33:06 2008 UTC vs.
Revision 1.123 by root, Sat Aug 8 10:06:02 2009 UTC

… …
37primary goal is to be I<correct> and its secondary goal is to be 37primary goal is to be I<correct> and its secondary goal is to be
38I<fast>. To reach the latter goal it was written in C. 38I<fast>. To reach the latter goal it was written in C.
39 39
40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overriden) with no overhead due to emulation (by inheritign constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need and doesn't
46require a C compiler when that is a problem. 46require a C compiler when that is a problem.
47 47
… …
49to write yet another JSON module? While it seems there are many JSON 49to write yet another JSON module? While it seems there are many JSON
50modules, none of them correctly handle all corner cases, and in most cases 50modules, none of them correctly handle all corner cases, and in most cases
51their maintainers are unresponsive, gone missing, or not listening to bug 51their maintainers are unresponsive, gone missing, or not listening to bug
52reports for other reasons. 52reports for other reasons.
53 53
54See COMPARISON, below, for a comparison to some other JSON modules.
55
56See MAPPING, below, on how JSON::XS maps perl values to JSON values and 54See MAPPING, below, on how JSON::XS maps perl values to JSON values and
57vice versa. 55vice versa.
58 56
59=head2 FEATURES 57=head2 FEATURES
60 58
… …
65This module knows how to handle Unicode, documents how and when it does 63This module knows how to handle Unicode, documents how and when it does
66so, and even documents what "correct" means. 64so, and even documents what "correct" means.
67 65
68=item * round-trip integrity 66=item * round-trip integrity
69 67
70When you serialise a perl data structure using only datatypes supported 68When you serialise a perl data structure using only data types supported
71by JSON, the deserialised data structure is identical on the Perl level. 69by JSON, the deserialised data structure is identical on the Perl level.
72(e.g. the string "2.0" doesn't suddenly become "2" just because it looks 70(e.g. the string "2.0" doesn't suddenly become "2" just because it looks
73like a number). There minor I<are> exceptions to this, read the MAPPING 71like a number). There minor I<are> exceptions to this, read the MAPPING
74section below to learn about those. 72section below to learn about those.
75 73
… …
84Compared to other JSON modules and other serialisers such as Storable, 82Compared to other JSON modules and other serialisers such as Storable,
85this module usually compares favourably in terms of speed, too. 83this module usually compares favourably in terms of speed, too.
86 84
87=item * simple to use 85=item * simple to use
88 86
89This module has both a simple functional interface as well as an objetc 87This module has both a simple functional interface as well as an object
90oriented interface interface. 88oriented interface interface.
91 89
92=item * reasonably versatile output formats 90=item * reasonably versatile output formats
93 91
94You can choose between the most compact guaranteed-single-line format 92You can choose between the most compact guaranteed-single-line format
95possible (nice for simple line-based protocols), a pure-ascii format 93possible (nice for simple line-based protocols), a pure-ASCII format
96(for when your transport is not 8-bit clean, still supports the whole 94(for when your transport is not 8-bit clean, still supports the whole
97Unicode range), or a pretty-printed format (for when you want to read that 95Unicode range), or a pretty-printed format (for when you want to read that
98stuff). Or you can combine those features in whatever way you like. 96stuff). Or you can combine those features in whatever way you like.
99 97
100=back 98=back
101 99
102=cut 100=cut
103 101
104package JSON::XS; 102package JSON::XS;
105 103
106use strict; 104use common::sense;
107 105
108our $VERSION = '2.2'; 106our $VERSION = '2.25';
109our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
110 108
111our @EXPORT = qw(encode_json decode_json to_json from_json); 109our @EXPORT = qw(encode_json decode_json to_json from_json);
112 110
113sub to_json($) { 111sub to_json($) {
… …
137 135
138This function call is functionally identical to: 136This function call is functionally identical to:
139 137
140 $json_text = JSON::XS->new->utf8->encode ($perl_scalar) 138 $json_text = JSON::XS->new->utf8->encode ($perl_scalar)
141 139
142except being faster. 140Except being faster.
143 141
144=item $perl_scalar = decode_json $json_text 142=item $perl_scalar = decode_json $json_text
145 143
146The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 144The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries
147to parse that as an UTF-8 encoded JSON text, returning the resulting 145to parse that as an UTF-8 encoded JSON text, returning the resulting
… …
149 147
150This function call is functionally identical to: 148This function call is functionally identical to:
151 149
152 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 150 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
153 151
154except being faster. 152Except being faster.
155 153
156=item $is_boolean = JSON::XS::is_bool $scalar 154=item $is_boolean = JSON::XS::is_bool $scalar
157 155
158Returns true if the passed scalar represents either JSON::XS::true or 156Returns true if the passed scalar represents either JSON::XS::true or
159JSON::XS::false, two constants that act like C<1> and C<0>, respectively 157JSON::XS::false, two constants that act like C<1> and C<0>, respectively
… …
197 195
198If you didn't know about that flag, just the better, pretend it doesn't 196If you didn't know about that flag, just the better, pretend it doesn't
199exist. 197exist.
200 198
201=item 4. A "Unicode String" is simply a string where each character can be 199=item 4. A "Unicode String" is simply a string where each character can be
202validly interpreted as a Unicode codepoint. 200validly interpreted as a Unicode code point.
203 201
204If you have UTF-8 encoded data, it is no longer a Unicode string, but a 202If you have UTF-8 encoded data, it is no longer a Unicode string, but a
205Unicode string encoded in UTF-8, giving you a binary string. 203Unicode string encoded in UTF-8, giving you a binary string.
206 204
207=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string. 205=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string.
… …
442the same JSON text (given the same overall settings). If it is disabled, 440the same JSON text (given the same overall settings). If it is disabled,
443the same hash might be encoded differently even if contains the same data, 441the same hash might be encoded differently even if contains the same data,
444as key-value pairs have no inherent ordering in Perl. 442as key-value pairs have no inherent ordering in Perl.
445 443
446This setting has no effect when decoding JSON texts. 444This setting has no effect when decoding JSON texts.
445
446This setting has currently no effect on tied hashes.
447 447
448=item $json = $json->allow_nonref ([$enable]) 448=item $json = $json->allow_nonref ([$enable])
449 449
450=item $enabled = $json->get_allow_nonref 450=item $enabled = $json->get_allow_nonref
451 451
… …
701=back 701=back
702 702
703 703
704=head1 INCREMENTAL PARSING 704=head1 INCREMENTAL PARSING
705 705
706[This section and the API it details is still EXPERIMENTAL]
707
708In some cases, there is the need for incremental parsing of JSON 706In some cases, there is the need for incremental parsing of JSON
709texts. While this module always has to keep both JSON text and resulting 707texts. While this module always has to keep both JSON text and resulting
710Perl data structure in memory at one time, it does allow you to parse a 708Perl data structure in memory at one time, it does allow you to parse a
711JSON stream incrementally. It does so by accumulating text until it has 709JSON stream incrementally. It does so by accumulating text until it has
712a full JSON object, which it then can decode. This process is similar to 710a full JSON object, which it then can decode. This process is similar to
713using C<decode_prefix> to see if a full JSON object is available, but is 711using C<decode_prefix> to see if a full JSON object is available, but
714much more efficient (JSON::XS will only attempt to parse the JSON text 712is much more efficient (and can be implemented with a minimum of method
713calls).
714
715JSON::XS will only attempt to parse the JSON text once it is sure it
715once it is sure it has enough text to get a decisive result, using a very 716has enough text to get a decisive result, using a very simple but
716simple but truly incremental parser). 717truly incremental parser. This means that it sometimes won't stop as
718early as the full parser, for example, it doesn't detect parenthese
719mismatches. The only thing it guarantees is that it starts decoding as
720soon as a syntactically valid JSON text has been seen. This means you need
721to set resource limits (e.g. C<max_size>) to ensure the parser will stop
722parsing in the presence if syntax errors.
717 723
718The following two methods deal with this. 724The following methods implement this incremental parser.
719 725
720=over 4 726=over 4
721 727
722=item [void, scalar or list context] = $json->incr_parse ([$string]) 728=item [void, scalar or list context] = $json->incr_parse ([$string])
723 729
… …
761JSON object or b) parsing multiple JSON objects separated by non-JSON text 767JSON object or b) parsing multiple JSON objects separated by non-JSON text
762(such as commas). 768(such as commas).
763 769
764=item $json->incr_skip 770=item $json->incr_skip
765 771
766This will reset the state of the incremental parser and will remove the 772This will reset the state of the incremental parser and will remove
767parsed text from the input buffer. This is useful after C<incr_parse> 773the parsed text from the input buffer so far. This is useful after
768died, in which case the input buffer and incremental parser state is left 774C<incr_parse> died, in which case the input buffer and incremental parser
769unchanged, to skip the text parsed so far and to reset the parse state. 775state is left unchanged, to skip the text parsed so far and to reset the
776parse state.
777
778The difference to C<incr_reset> is that only text until the parse error
779occured is removed.
780
781=item $json->incr_reset
782
783This completely resets the incremental parser, that is, after this call,
784it will be as if the parser had never parsed anything.
785
786This is useful if you want to repeatedly parse JSON objects and want to
787ignore any trailing data, which means you have to reset the parser after
788each successful decode.
770 789
771=back 790=back
772 791
773=head2 LIMITATIONS 792=head2 LIMITATIONS
774 793
… …
1165when your channel for JSON transfer is not 8-bit clean or the encoding 1184when your channel for JSON transfer is not 8-bit clean or the encoding
1166might be mangled in between (e.g. in mail), and works because ASCII is a 1185might be mangled in between (e.g. in mail), and works because ASCII is a
1167proper subset of most 8-bit and multibyte encodings in use in the world. 1186proper subset of most 8-bit and multibyte encodings in use in the world.
1168 1187
1169=back 1188=back
1189
1190
1191=head2 JSON and ECMAscript
1192
1193JSON syntax is based on how literals are represented in javascript (the
1194not-standardised predecessor of ECMAscript) which is presumably why it is
1195called "JavaScript Object Notation".
1196
1197However, JSON is not a subset (and also not a superset of course) of
1198ECMAscript (the standard) or javascript (whatever browsers actually
1199implement).
1200
1201If you want to use javascript's C<eval> function to "parse" JSON, you
1202might run into parse errors for valid JSON texts, or the resulting data
1203structure might not be queryable:
1204
1205One of the problems is that U+2028 and U+2029 are valid characters inside
1206JSON strings, but are not allowed in ECMAscript string literals, so the
1207following Perl fragment will not output something that can be guaranteed
1208to be parsable by javascript's C<eval>:
1209
1210 use JSON::XS;
1211
1212 print encode_json [chr 0x2028];
1213
1214The right fix for this is to use a proper JSON parser in your javascript
1215programs, and not rely on C<eval> (see for example Douglas Crockford's
1216F<json2.js> parser).
1217
1218If this is not an option, you can, as a stop-gap measure, simply encode to
1219ASCII-only JSON:
1220
1221 use JSON::XS;
1222
1223 print JSON::XS->new->ascii->encode ([chr 0x2028]);
1224
1225Note that this will enlarge the resulting JSON text quite a bit if you
1226have many non-ASCII characters. You might be tempted to run some regexes
1227to only escape U+2028 and U+2029, e.g.:
1228
1229 # DO NOT USE THIS!
1230 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1231 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1232 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1233 print $json;
1234
1235Note that I<this is a bad idea>: the above only works for U+2028 and
1236U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1237javascript implementations, however, have issues with other characters as
1238well - using C<eval> naively simply I<will> cause problems.
1239
1240Another problem is that some javascript implementations reserve
1241some property names for their own purposes (which probably makes
1242them non-ECMAscript-compliant). For example, Iceweasel reserves the
1243C<__proto__> property name for it's own purposes.
1244
1245If that is a problem, you could parse try to filter the resulting JSON
1246output for these property strings, e.g.:
1247
1248 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1249
1250This works because C<__proto__> is not valid outside of strings, so every
1251occurence of C<"__proto__"\s*:> must be a string used as property name.
1252
1253If you know of other incompatibilities, please let me know.
1170 1254
1171 1255
1172=head2 JSON and YAML 1256=head2 JSON and YAML
1173 1257
1174You often hear that JSON is a subset of YAML. This is, however, a mass 1258You often hear that JSON is a subset of YAML. This is, however, a mass

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines