ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.104 by root, Thu May 8 15:33:06 2008 UTC vs.
Revision 1.115 by root, Tue Feb 17 23:29:38 2009 UTC

… …
37primary goal is to be I<correct> and its secondary goal is to be 37primary goal is to be I<correct> and its secondary goal is to be
38I<fast>. To reach the latter goal it was written in C. 38I<fast>. To reach the latter goal it was written in C.
39 39
40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overriden) with no overhead due to emulation (by inheritign constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need and doesn't
46require a C compiler when that is a problem. 46require a C compiler when that is a problem.
47 47
… …
49to write yet another JSON module? While it seems there are many JSON 49to write yet another JSON module? While it seems there are many JSON
50modules, none of them correctly handle all corner cases, and in most cases 50modules, none of them correctly handle all corner cases, and in most cases
51their maintainers are unresponsive, gone missing, or not listening to bug 51their maintainers are unresponsive, gone missing, or not listening to bug
52reports for other reasons. 52reports for other reasons.
53 53
54See COMPARISON, below, for a comparison to some other JSON modules.
55
56See MAPPING, below, on how JSON::XS maps perl values to JSON values and 54See MAPPING, below, on how JSON::XS maps perl values to JSON values and
57vice versa. 55vice versa.
58 56
59=head2 FEATURES 57=head2 FEATURES
60 58
… …
65This module knows how to handle Unicode, documents how and when it does 63This module knows how to handle Unicode, documents how and when it does
66so, and even documents what "correct" means. 64so, and even documents what "correct" means.
67 65
68=item * round-trip integrity 66=item * round-trip integrity
69 67
70When you serialise a perl data structure using only datatypes supported 68When you serialise a perl data structure using only data types supported
71by JSON, the deserialised data structure is identical on the Perl level. 69by JSON, the deserialised data structure is identical on the Perl level.
72(e.g. the string "2.0" doesn't suddenly become "2" just because it looks 70(e.g. the string "2.0" doesn't suddenly become "2" just because it looks
73like a number). There minor I<are> exceptions to this, read the MAPPING 71like a number). There minor I<are> exceptions to this, read the MAPPING
74section below to learn about those. 72section below to learn about those.
75 73
… …
84Compared to other JSON modules and other serialisers such as Storable, 82Compared to other JSON modules and other serialisers such as Storable,
85this module usually compares favourably in terms of speed, too. 83this module usually compares favourably in terms of speed, too.
86 84
87=item * simple to use 85=item * simple to use
88 86
89This module has both a simple functional interface as well as an objetc 87This module has both a simple functional interface as well as an object
90oriented interface interface. 88oriented interface interface.
91 89
92=item * reasonably versatile output formats 90=item * reasonably versatile output formats
93 91
94You can choose between the most compact guaranteed-single-line format 92You can choose between the most compact guaranteed-single-line format
95possible (nice for simple line-based protocols), a pure-ascii format 93possible (nice for simple line-based protocols), a pure-ASCII format
96(for when your transport is not 8-bit clean, still supports the whole 94(for when your transport is not 8-bit clean, still supports the whole
97Unicode range), or a pretty-printed format (for when you want to read that 95Unicode range), or a pretty-printed format (for when you want to read that
98stuff). Or you can combine those features in whatever way you like. 96stuff). Or you can combine those features in whatever way you like.
99 97
100=back 98=back
101 99
102=cut 100=cut
103 101
104package JSON::XS; 102package JSON::XS;
105 103
104no warnings;
106use strict; 105use strict;
107 106
108our $VERSION = '2.2'; 107our $VERSION = '2.232';
109our @ISA = qw(Exporter); 108our @ISA = qw(Exporter);
110 109
111our @EXPORT = qw(encode_json decode_json to_json from_json); 110our @EXPORT = qw(encode_json decode_json to_json from_json);
112 111
113sub to_json($) { 112sub to_json($) {
… …
137 136
138This function call is functionally identical to: 137This function call is functionally identical to:
139 138
140 $json_text = JSON::XS->new->utf8->encode ($perl_scalar) 139 $json_text = JSON::XS->new->utf8->encode ($perl_scalar)
141 140
142except being faster. 141Except being faster.
143 142
144=item $perl_scalar = decode_json $json_text 143=item $perl_scalar = decode_json $json_text
145 144
146The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 145The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries
147to parse that as an UTF-8 encoded JSON text, returning the resulting 146to parse that as an UTF-8 encoded JSON text, returning the resulting
… …
149 148
150This function call is functionally identical to: 149This function call is functionally identical to:
151 150
152 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 151 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
153 152
154except being faster. 153Except being faster.
155 154
156=item $is_boolean = JSON::XS::is_bool $scalar 155=item $is_boolean = JSON::XS::is_bool $scalar
157 156
158Returns true if the passed scalar represents either JSON::XS::true or 157Returns true if the passed scalar represents either JSON::XS::true or
159JSON::XS::false, two constants that act like C<1> and C<0>, respectively 158JSON::XS::false, two constants that act like C<1> and C<0>, respectively
… …
197 196
198If you didn't know about that flag, just the better, pretend it doesn't 197If you didn't know about that flag, just the better, pretend it doesn't
199exist. 198exist.
200 199
201=item 4. A "Unicode String" is simply a string where each character can be 200=item 4. A "Unicode String" is simply a string where each character can be
202validly interpreted as a Unicode codepoint. 201validly interpreted as a Unicode code point.
203 202
204If you have UTF-8 encoded data, it is no longer a Unicode string, but a 203If you have UTF-8 encoded data, it is no longer a Unicode string, but a
205Unicode string encoded in UTF-8, giving you a binary string. 204Unicode string encoded in UTF-8, giving you a binary string.
206 205
207=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string. 206=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string.
… …
701=back 700=back
702 701
703 702
704=head1 INCREMENTAL PARSING 703=head1 INCREMENTAL PARSING
705 704
706[This section and the API it details is still EXPERIMENTAL]
707
708In some cases, there is the need for incremental parsing of JSON 705In some cases, there is the need for incremental parsing of JSON
709texts. While this module always has to keep both JSON text and resulting 706texts. While this module always has to keep both JSON text and resulting
710Perl data structure in memory at one time, it does allow you to parse a 707Perl data structure in memory at one time, it does allow you to parse a
711JSON stream incrementally. It does so by accumulating text until it has 708JSON stream incrementally. It does so by accumulating text until it has
712a full JSON object, which it then can decode. This process is similar to 709a full JSON object, which it then can decode. This process is similar to
713using C<decode_prefix> to see if a full JSON object is available, but is 710using C<decode_prefix> to see if a full JSON object is available, but
714much more efficient (JSON::XS will only attempt to parse the JSON text 711is much more efficient (and can be implemented with a minimum of method
712calls).
713
714JSON::XS will only attempt to parse the JSON text once it is sure it
715once it is sure it has enough text to get a decisive result, using a very 715has enough text to get a decisive result, using a very simple but
716simple but truly incremental parser). 716truly incremental parser. This means that it sometimes won't stop as
717early as the full parser, for example, it doesn't detect parenthese
718mismatches. The only thing it guarantees is that it starts decoding as
719soon as a syntactically valid JSON text has been seen. This means you need
720to set resource limits (e.g. C<max_size>) to ensure the parser will stop
721parsing in the presence if syntax errors.
717 722
718The following two methods deal with this. 723The following methods implement this incremental parser.
719 724
720=over 4 725=over 4
721 726
722=item [void, scalar or list context] = $json->incr_parse ([$string]) 727=item [void, scalar or list context] = $json->incr_parse ([$string])
723 728
… …
761JSON object or b) parsing multiple JSON objects separated by non-JSON text 766JSON object or b) parsing multiple JSON objects separated by non-JSON text
762(such as commas). 767(such as commas).
763 768
764=item $json->incr_skip 769=item $json->incr_skip
765 770
766This will reset the state of the incremental parser and will remove the 771This will reset the state of the incremental parser and will remove
767parsed text from the input buffer. This is useful after C<incr_parse> 772the parsed text from the input buffer so far. This is useful after
768died, in which case the input buffer and incremental parser state is left 773C<incr_parse> died, in which case the input buffer and incremental parser
769unchanged, to skip the text parsed so far and to reset the parse state. 774state is left unchanged, to skip the text parsed so far and to reset the
775parse state.
776
777The difference to C<incr_reset> is that only text until the parse error
778occured is removed.
779
780=item $json->incr_reset
781
782This completely resets the incremental parser, that is, after this call,
783it will be as if the parser had never parsed anything.
784
785This is useful if you want to repeatedly parse JSON objects and want to
786ignore any trailing data, which means you have to reset the parser after
787each successful decode.
770 788
771=back 789=back
772 790
773=head2 LIMITATIONS 791=head2 LIMITATIONS
774 792
… …
1165when your channel for JSON transfer is not 8-bit clean or the encoding 1183when your channel for JSON transfer is not 8-bit clean or the encoding
1166might be mangled in between (e.g. in mail), and works because ASCII is a 1184might be mangled in between (e.g. in mail), and works because ASCII is a
1167proper subset of most 8-bit and multibyte encodings in use in the world. 1185proper subset of most 8-bit and multibyte encodings in use in the world.
1168 1186
1169=back 1187=back
1188
1189
1190=head2 JSON and ECMAscript
1191
1192JSON syntax is based on how literals are represented in javascript (the
1193not-standardised predecessor of ECMAscript) which is presumably why it is
1194called "JavaScript Object Notation".
1195
1196However, JSON is not a subset (and also not a superset of course) of
1197ECMAscript (the standard) or javascript (whatever browsers actually
1198implement).
1199
1200If you want to use javascript's C<eval> function to "parse" JSON, you
1201might run into parse errors for valid JSON texts, or the resulting data
1202structure might not be queryable:
1203
1204One of the problems is that U+2028 and U+2029 are valid characters inside
1205JSON strings, but are not allowed in ECMAscript string literals, so the
1206following Perl fragment will not output something that can be guaranteed
1207to be parsable by javascript's C<eval>:
1208
1209 use JSON::XS;
1210
1211 print encode_json [chr 0x2028];
1212
1213The right fix for this is to use a proper JSON parser in your javascript
1214programs, and not rely on C<eval>.
1215
1216If this is not an option, you can, as a stop-gap measure, simply encode to
1217ASCII-only JSON:
1218
1219 use JSON::XS;
1220
1221 print JSON::XS->new->ascii->encode ([chr 0x2028]);
1222
1223And if you are concerned about the size of the resulting JSON text, you
1224can run some regexes to only escape U+2028 and U+2029:
1225
1226 use JSON::XS;
1227
1228 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1229 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1230 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1231 print $json;
1232
1233This works because U+2028/U+2029 are not allowed outside of strings and
1234are not used for syntax, so replacing them unconditionally just works.
1235
1236Note, however, that fixing the broken JSON parser is better than working
1237around it in every other generator. The above regexes should work well in
1238other languages, as long as they operate on UTF-8. It is equally valid to
1239replace all occurences of U+2028/2029 directly by their \\u-escaped forms
1240in unicode texts, so they can simply be used to fix any parsers relying on
1241C<eval> by first applying the regexes on the encoded texts.
1242
1243Another problem is that some javascript implementations reserve
1244some property names for their own purposes (which probably makes
1245them non-ECMAscript-compliant). For example, Iceweasel reserves the
1246C<__proto__> property name for it's own purposes.
1247
1248If that is a problem, you could parse try to filter the resulting JSON
1249output for these property strings, e.g.:
1250
1251 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1252
1253This works because C<__proto__> is not valid outside of strings, so every
1254occurence of C<"__proto__"\s*:> must be a string used as property name.
1255
1256If you know of other incompatibilities, please let me know.
1170 1257
1171 1258
1172=head2 JSON and YAML 1259=head2 JSON and YAML
1173 1260
1174You often hear that JSON is a subset of YAML. This is, however, a mass 1261You often hear that JSON is a subset of YAML. This is, however, a mass

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines