ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.102 by root, Sat Apr 19 03:39:27 2008 UTC vs.
Revision 1.117 by root, Wed Feb 18 00:08:28 2009 UTC

… …
37primary goal is to be I<correct> and its secondary goal is to be 37primary goal is to be I<correct> and its secondary goal is to be
38I<fast>. To reach the latter goal it was written in C. 38I<fast>. To reach the latter goal it was written in C.
39 39
40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overriden) with no overhead due to emulation (by inheritign constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need and doesn't
46require a C compiler when that is a problem. 46require a C compiler when that is a problem.
47 47
… …
49to write yet another JSON module? While it seems there are many JSON 49to write yet another JSON module? While it seems there are many JSON
50modules, none of them correctly handle all corner cases, and in most cases 50modules, none of them correctly handle all corner cases, and in most cases
51their maintainers are unresponsive, gone missing, or not listening to bug 51their maintainers are unresponsive, gone missing, or not listening to bug
52reports for other reasons. 52reports for other reasons.
53 53
54See COMPARISON, below, for a comparison to some other JSON modules.
55
56See MAPPING, below, on how JSON::XS maps perl values to JSON values and 54See MAPPING, below, on how JSON::XS maps perl values to JSON values and
57vice versa. 55vice versa.
58 56
59=head2 FEATURES 57=head2 FEATURES
60 58
… …
65This module knows how to handle Unicode, documents how and when it does 63This module knows how to handle Unicode, documents how and when it does
66so, and even documents what "correct" means. 64so, and even documents what "correct" means.
67 65
68=item * round-trip integrity 66=item * round-trip integrity
69 67
70When you serialise a perl data structure using only datatypes supported 68When you serialise a perl data structure using only data types supported
71by JSON, the deserialised data structure is identical on the Perl level. 69by JSON, the deserialised data structure is identical on the Perl level.
72(e.g. the string "2.0" doesn't suddenly become "2" just because it looks 70(e.g. the string "2.0" doesn't suddenly become "2" just because it looks
73like a number). There minor I<are> exceptions to this, read the MAPPING 71like a number). There minor I<are> exceptions to this, read the MAPPING
74section below to learn about those. 72section below to learn about those.
75 73
… …
84Compared to other JSON modules and other serialisers such as Storable, 82Compared to other JSON modules and other serialisers such as Storable,
85this module usually compares favourably in terms of speed, too. 83this module usually compares favourably in terms of speed, too.
86 84
87=item * simple to use 85=item * simple to use
88 86
89This module has both a simple functional interface as well as an objetc 87This module has both a simple functional interface as well as an object
90oriented interface interface. 88oriented interface interface.
91 89
92=item * reasonably versatile output formats 90=item * reasonably versatile output formats
93 91
94You can choose between the most compact guaranteed-single-line format 92You can choose between the most compact guaranteed-single-line format
95possible (nice for simple line-based protocols), a pure-ascii format 93possible (nice for simple line-based protocols), a pure-ASCII format
96(for when your transport is not 8-bit clean, still supports the whole 94(for when your transport is not 8-bit clean, still supports the whole
97Unicode range), or a pretty-printed format (for when you want to read that 95Unicode range), or a pretty-printed format (for when you want to read that
98stuff). Or you can combine those features in whatever way you like. 96stuff). Or you can combine those features in whatever way you like.
99 97
100=back 98=back
101 99
102=cut 100=cut
103 101
104package JSON::XS; 102package JSON::XS;
105 103
104no warnings;
106use strict; 105use strict;
107 106
108our $VERSION = '2.2'; 107our $VERSION = '2.232';
109our @ISA = qw(Exporter); 108our @ISA = qw(Exporter);
110 109
111our @EXPORT = qw(encode_json decode_json to_json from_json); 110our @EXPORT = qw(encode_json decode_json to_json from_json);
112 111
113sub to_json($) { 112sub to_json($) {
… …
137 136
138This function call is functionally identical to: 137This function call is functionally identical to:
139 138
140 $json_text = JSON::XS->new->utf8->encode ($perl_scalar) 139 $json_text = JSON::XS->new->utf8->encode ($perl_scalar)
141 140
142except being faster. 141Except being faster.
143 142
144=item $perl_scalar = decode_json $json_text 143=item $perl_scalar = decode_json $json_text
145 144
146The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 145The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries
147to parse that as an UTF-8 encoded JSON text, returning the resulting 146to parse that as an UTF-8 encoded JSON text, returning the resulting
… …
149 148
150This function call is functionally identical to: 149This function call is functionally identical to:
151 150
152 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 151 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
153 152
154except being faster. 153Except being faster.
155 154
156=item $is_boolean = JSON::XS::is_bool $scalar 155=item $is_boolean = JSON::XS::is_bool $scalar
157 156
158Returns true if the passed scalar represents either JSON::XS::true or 157Returns true if the passed scalar represents either JSON::XS::true or
159JSON::XS::false, two constants that act like C<1> and C<0>, respectively 158JSON::XS::false, two constants that act like C<1> and C<0>, respectively
… …
197 196
198If you didn't know about that flag, just the better, pretend it doesn't 197If you didn't know about that flag, just the better, pretend it doesn't
199exist. 198exist.
200 199
201=item 4. A "Unicode String" is simply a string where each character can be 200=item 4. A "Unicode String" is simply a string where each character can be
202validly interpreted as a Unicode codepoint. 201validly interpreted as a Unicode code point.
203 202
204If you have UTF-8 encoded data, it is no longer a Unicode string, but a 203If you have UTF-8 encoded data, it is no longer a Unicode string, but a
205Unicode string encoded in UTF-8, giving you a binary string. 204Unicode string encoded in UTF-8, giving you a binary string.
206 205
207=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string. 206=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string.
… …
701=back 700=back
702 701
703 702
704=head1 INCREMENTAL PARSING 703=head1 INCREMENTAL PARSING
705 704
706[This section and the API it details is still EXPERIMENTAL]
707
708In some cases, there is the need for incremental parsing of JSON 705In some cases, there is the need for incremental parsing of JSON
709texts. While this module always has to keep both JSON text and resulting 706texts. While this module always has to keep both JSON text and resulting
710Perl data structure in memory at one time, it does allow you to parse a 707Perl data structure in memory at one time, it does allow you to parse a
711JSON stream incrementally. It does so by accumulating text until it has 708JSON stream incrementally. It does so by accumulating text until it has
712a full JSON object, which it then can decode. This process is similar to 709a full JSON object, which it then can decode. This process is similar to
713using C<decode_prefix> to see if a full JSON object is available, but is 710using C<decode_prefix> to see if a full JSON object is available, but
714much more efficient (JSON::XS will only attempt to parse the JSON text 711is much more efficient (and can be implemented with a minimum of method
712calls).
713
714JSON::XS will only attempt to parse the JSON text once it is sure it
715once it is sure it has enough text to get a decisive result, using a very 715has enough text to get a decisive result, using a very simple but
716simple but truly incremental parser). 716truly incremental parser. This means that it sometimes won't stop as
717early as the full parser, for example, it doesn't detect parenthese
718mismatches. The only thing it guarantees is that it starts decoding as
719soon as a syntactically valid JSON text has been seen. This means you need
720to set resource limits (e.g. C<max_size>) to ensure the parser will stop
721parsing in the presence if syntax errors.
717 722
718The following two methods deal with this. 723The following methods implement this incremental parser.
719 724
720=over 4 725=over 4
721 726
722=item [void, scalar or list context] = $json->incr_parse ([$string]) 727=item [void, scalar or list context] = $json->incr_parse ([$string])
723 728
… …
761JSON object or b) parsing multiple JSON objects separated by non-JSON text 766JSON object or b) parsing multiple JSON objects separated by non-JSON text
762(such as commas). 767(such as commas).
763 768
764=item $json->incr_skip 769=item $json->incr_skip
765 770
766This will reset the state of the incremental parser and will remove the 771This will reset the state of the incremental parser and will remove
767parsed text from the input buffer. This is useful after C<incr_parse> 772the parsed text from the input buffer so far. This is useful after
768died, in which case the input buffer and incremental parser state is left 773C<incr_parse> died, in which case the input buffer and incremental parser
769unchanged, to skip the text parsed so far and to reset the parse state. 774state is left unchanged, to skip the text parsed so far and to reset the
775parse state.
776
777The difference to C<incr_reset> is that only text until the parse error
778occured is removed.
779
780=item $json->incr_reset
781
782This completely resets the incremental parser, that is, after this call,
783it will be as if the parser had never parsed anything.
784
785This is useful if you want to repeatedly parse JSON objects and want to
786ignore any trailing data, which means you have to reset the parser after
787each successful decode.
770 788
771=back 789=back
772 790
773=head2 LIMITATIONS 791=head2 LIMITATIONS
774 792
… …
1015Other unblessed references are generally not allowed and will cause an 1033Other unblessed references are generally not allowed and will cause an
1016exception to be thrown, except for references to the integers C<0> and 1034exception to be thrown, except for references to the integers C<0> and
1017C<1>, which get turned into C<false> and C<true> atoms in JSON. You can 1035C<1>, which get turned into C<false> and C<true> atoms in JSON. You can
1018also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability. 1036also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
1019 1037
1020 encode_json [\0,JSON::XS::true] # yields [false,true] 1038 encode_json [\0, JSON::XS::true] # yields [false,true]
1021 1039
1022=item JSON::XS::true, JSON::XS::false 1040=item JSON::XS::true, JSON::XS::false
1023 1041
1024These special values become JSON true and JSON false values, 1042These special values become JSON true and JSON false values,
1025respectively. You can also use C<\1> and C<\0> directly if you want. 1043respectively. You can also use C<\1> and C<\0> directly if you want.
… …
1165when your channel for JSON transfer is not 8-bit clean or the encoding 1183when your channel for JSON transfer is not 8-bit clean or the encoding
1166might be mangled in between (e.g. in mail), and works because ASCII is a 1184might be mangled in between (e.g. in mail), and works because ASCII is a
1167proper subset of most 8-bit and multibyte encodings in use in the world. 1185proper subset of most 8-bit and multibyte encodings in use in the world.
1168 1186
1169=back 1187=back
1188
1189
1190=head2 JSON and ECMAscript
1191
1192JSON syntax is based on how literals are represented in javascript (the
1193not-standardised predecessor of ECMAscript) which is presumably why it is
1194called "JavaScript Object Notation".
1195
1196However, JSON is not a subset (and also not a superset of course) of
1197ECMAscript (the standard) or javascript (whatever browsers actually
1198implement).
1199
1200If you want to use javascript's C<eval> function to "parse" JSON, you
1201might run into parse errors for valid JSON texts, or the resulting data
1202structure might not be queryable:
1203
1204One of the problems is that U+2028 and U+2029 are valid characters inside
1205JSON strings, but are not allowed in ECMAscript string literals, so the
1206following Perl fragment will not output something that can be guaranteed
1207to be parsable by javascript's C<eval>:
1208
1209 use JSON::XS;
1210
1211 print encode_json [chr 0x2028];
1212
1213The right fix for this is to use a proper JSON parser in your javascript
1214programs, and not rely on C<eval> (see for example Douglas Crockford's
1215F<json2.js> parser).
1216
1217If this is not an option, you can, as a stop-gap measure, simply encode to
1218ASCII-only JSON:
1219
1220 use JSON::XS;
1221
1222 print JSON::XS->new->ascii->encode ([chr 0x2028]);
1223
1224Note that this will enlarge the resulting JSON text quite a bit if you
1225have many non-ASCII characters. You might be tempted to run some regexes
1226to only escape U+2028 and U+2029, e.g.:
1227
1228 # DO NOT USE THIS!
1229 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1230 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1231 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1232 print $json;
1233
1234Note that I<this is a bad idea>: the above only works for U+2028 and
1235U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1236javascript implementations, however, have issues with other characters as
1237well - using C<eval> naively simply I<will> cause problems.
1238
1239Another problem is that some javascript implementations reserve
1240some property names for their own purposes (which probably makes
1241them non-ECMAscript-compliant). For example, Iceweasel reserves the
1242C<__proto__> property name for it's own purposes.
1243
1244If that is a problem, you could parse try to filter the resulting JSON
1245output for these property strings, e.g.:
1246
1247 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1248
1249This works because C<__proto__> is not valid outside of strings, so every
1250occurence of C<"__proto__"\s*:> must be a string used as property name.
1251
1252If you know of other incompatibilities, please let me know.
1170 1253
1171 1254
1172=head2 JSON and YAML 1255=head2 JSON and YAML
1173 1256
1174You often hear that JSON is a subset of YAML. This is, however, a mass 1257You often hear that JSON is a subset of YAML. This is, however, a mass
… …
1342 1425
1343 1426
1344=head1 BUGS 1427=head1 BUGS
1345 1428
1346While the goal of this module is to be correct, that unfortunately does 1429While the goal of this module is to be correct, that unfortunately does
1347not mean it's bug-free, only that I think its design is bug-free. It is 1430not mean it's bug-free, only that I think its design is bug-free. If you
1348still relatively early in its development. If you keep reporting bugs they 1431keep reporting bugs they will be fixed swiftly, though.
1349will be fixed swiftly, though.
1350 1432
1351Please refrain from using rt.cpan.org or any other bug reporting 1433Please refrain from using rt.cpan.org or any other bug reporting
1352service. I put the contact address into my modules for a reason. 1434service. I put the contact address into my modules for a reason.
1353 1435
1354=cut 1436=cut

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines