ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/README
(Generate patch)

Comparing JSON-XS/README (file contents):
Revision 1.29 by root, Thu Feb 19 01:13:46 2009 UTC vs.
Revision 1.37 by root, Thu May 23 09:32:02 2013 UTC

20 $perl_scalar = $coder->decode ($unicode_json_text); 20 $perl_scalar = $coder->decode ($unicode_json_text);
21 21
22 # Note that JSON version 2.0 and above will automatically use JSON::XS 22 # Note that JSON version 2.0 and above will automatically use JSON::XS
23 # if available, at virtually no speed overhead either, so you should 23 # if available, at virtually no speed overhead either, so you should
24 # be able to just: 24 # be able to just:
25 25
26 use JSON; 26 use JSON;
27 27
28 # and do the same things, except that you have a pure-perl fallback now. 28 # and do the same things, except that you have a pure-perl fallback now.
29 29
30DESCRIPTION 30DESCRIPTION
31 This module converts Perl data structures to JSON and vice versa. Its 31 This module converts Perl data structures to JSON and vice versa. Its
56 does so, and even documents what "correct" means. 56 does so, and even documents what "correct" means.
57 57
58 * round-trip integrity 58 * round-trip integrity
59 59
60 When you serialise a perl data structure using only data types 60 When you serialise a perl data structure using only data types
61 supported by JSON, the deserialised data structure is identical on 61 supported by JSON and Perl, the deserialised data structure is
62 the Perl level. (e.g. the string "2.0" doesn't suddenly become "2" 62 identical on the Perl level. (e.g. the string "2.0" doesn't suddenly
63 just because it looks like a number). There minor *are* exceptions 63 become "2" just because it looks like a number). There *are* minor
64 to this, read the MAPPING section below to learn about those. 64 exceptions to this, read the MAPPING section below to learn about
65 those.
65 66
66 * strict checking of JSON correctness 67 * strict checking of JSON correctness
67 68
68 There is no guessing, no generating of illegal JSON texts by 69 There is no guessing, no generating of illegal JSON texts by
69 default, and only JSON is accepted as input by default (the latter 70 default, and only JSON is accepted as input by default (the latter
368 output JSON objects by sorting their keys. This is adding a 369 output JSON objects by sorting their keys. This is adding a
369 comparatively high overhead. 370 comparatively high overhead.
370 371
371 If $enable is false, then the "encode" method will output key-value 372 If $enable is false, then the "encode" method will output key-value
372 pairs in the order Perl stores them (which will likely change 373 pairs in the order Perl stores them (which will likely change
373 between runs of the same script). 374 between runs of the same script, and can change even within the same
375 run from 5.18 onwards).
374 376
375 This option is useful if you want the same data structure to be 377 This option is useful if you want the same data structure to be
376 encoded as the same JSON text (given the same overall settings). If 378 encoded as the same JSON text (given the same overall settings). If
377 it is disabled, the same hash might be encoded differently even if 379 it is disabled, the same hash might be encoded differently even if
378 contains the same data, as key-value pairs have no inherent ordering 380 contains the same data, as key-value pairs have no inherent ordering
379 in Perl. 381 in Perl.
380 382
381 This setting has no effect when decoding JSON texts. 383 This setting has no effect when decoding JSON texts.
384
385 This setting has currently no effect on tied hashes.
382 386
383 $json = $json->allow_nonref ([$enable]) 387 $json = $json->allow_nonref ([$enable])
384 $enabled = $json->get_allow_nonref 388 $enabled = $json->get_allow_nonref
385 If $enable is true (or missing), then the "encode" method can 389 If $enable is true (or missing), then the "encode" method can
386 convert a non-reference into its corresponding string, number or 390 convert a non-reference into its corresponding string, number or
633 calls). 637 calls).
634 638
635 JSON::XS will only attempt to parse the JSON text once it is sure it has 639 JSON::XS will only attempt to parse the JSON text once it is sure it has
636 enough text to get a decisive result, using a very simple but truly 640 enough text to get a decisive result, using a very simple but truly
637 incremental parser. This means that it sometimes won't stop as early as 641 incremental parser. This means that it sometimes won't stop as early as
638 the full parser, for example, it doesn't detect parenthese mismatches. 642 the full parser, for example, it doesn't detect mismatched parentheses.
639 The only thing it guarantees is that it starts decoding as soon as a 643 The only thing it guarantees is that it starts decoding as soon as a
640 syntactically valid JSON text has been seen. This means you need to set 644 syntactically valid JSON text has been seen. This means you need to set
641 resource limits (e.g. "max_size") to ensure the parser will stop parsing 645 resource limits (e.g. "max_size") to ensure the parser will stop parsing
642 in the presence if syntax errors. 646 in the presence if syntax errors.
643 647
667 otherwise. For this to work, there must be no separators between the 671 otherwise. For this to work, there must be no separators between the
668 JSON objects or arrays, instead they must be concatenated 672 JSON objects or arrays, instead they must be concatenated
669 back-to-back. If an error occurs, an exception will be raised as in 673 back-to-back. If an error occurs, an exception will be raised as in
670 the scalar context case. Note that in this case, any 674 the scalar context case. Note that in this case, any
671 previously-parsed JSON texts will be lost. 675 previously-parsed JSON texts will be lost.
676
677 Example: Parse some JSON arrays/objects in a given string and return
678 them.
679
680 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
672 681
673 $lvalue_string = $json->incr_text 682 $lvalue_string = $json->incr_text
674 This method returns the currently stored JSON fragment as an lvalue, 683 This method returns the currently stored JSON fragment as an lvalue,
675 that is, you can manipulate it. This *only* works when a preceding 684 that is, you can manipulate it. This *only* works when a preceding
676 call to "incr_parse" in *scalar context* successfully returned an 685 call to "incr_parse" in *scalar context* successfully returned an
891 Numbers containing a fractional or exponential part will always be 900 Numbers containing a fractional or exponential part will always be
892 represented as numeric (floating point) values, possibly at a loss 901 represented as numeric (floating point) values, possibly at a loss
893 of precision (in which case you might lose perfect roundtripping 902 of precision (in which case you might lose perfect roundtripping
894 ability, but the JSON number will still be re-encoded as a JSON 903 ability, but the JSON number will still be re-encoded as a JSON
895 number). 904 number).
905
906 Note that precision is not accuracy - binary floating point values
907 cannot represent most decimal fractions exactly, and when converting
908 from and to floating point, JSON::XS only guarantees precision up to
909 but not including the leats significant bit.
896 910
897 true, false 911 true, false
898 These JSON atoms become "JSON::XS::true" and "JSON::XS::false", 912 These JSON atoms become "JSON::XS::true" and "JSON::XS::false",
899 respectively. They are overloaded to act almost exactly like the 913 respectively. They are overloaded to act almost exactly like the
900 numbers 1 and 0. You can check whether a scalar is a JSON boolean by 914 numbers 1 and 0. You can check whether a scalar is a JSON boolean by
977 991
978 You can not currently force the type in other, less obscure, ways. 992 You can not currently force the type in other, less obscure, ways.
979 Tell me if you need this capability (but don't forget to explain why 993 Tell me if you need this capability (but don't forget to explain why
980 it's needed :). 994 it's needed :).
981 995
996 Note that numerical precision has the same meaning as under Perl (so
997 binary to decimal conversion follows the same rules as in Perl,
998 which can differ to other languages). Also, your perl interpreter
999 might expose extensions to the floating point numbers of your
1000 platform, such as infinities or NaN's - these cannot be represented
1001 in JSON, and it is an error to pass those in.
1002
982ENCODING/CODESET FLAG NOTES 1003ENCODING/CODESET FLAG NOTES
983 The interested reader might have seen a number of flags that signify 1004 The interested reader might have seen a number of flags that signify
984 encodings or codesets - "utf8", "latin1" and "ascii". There seems to be 1005 encodings or codesets - "utf8", "latin1" and "ascii". There seems to be
985 some confusion on what these do, so here is a short comparison: 1006 some confusion on what these do, so here is a short comparison:
986 1007
1123 characters as well - using "eval" naively simply *will* cause problems. 1144 characters as well - using "eval" naively simply *will* cause problems.
1124 1145
1125 Another problem is that some javascript implementations reserve some 1146 Another problem is that some javascript implementations reserve some
1126 property names for their own purposes (which probably makes them 1147 property names for their own purposes (which probably makes them
1127 non-ECMAscript-compliant). For example, Iceweasel reserves the 1148 non-ECMAscript-compliant). For example, Iceweasel reserves the
1128 "__proto__" property name for it's own purposes. 1149 "__proto__" property name for its own purposes.
1129 1150
1130 If that is a problem, you could parse try to filter the resulting JSON 1151 If that is a problem, you could parse try to filter the resulting JSON
1131 output for these property strings, e.g.: 1152 output for these property strings, e.g.:
1132 1153
1133 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g; 1154 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1151 my $yaml = $to_yaml->encode ($ref) . "\n"; 1172 my $yaml = $to_yaml->encode ($ref) . "\n";
1152 1173
1153 This will *usually* generate JSON texts that also parse as valid YAML. 1174 This will *usually* generate JSON texts that also parse as valid YAML.
1154 Please note that YAML has hardcoded limits on (simple) object key 1175 Please note that YAML has hardcoded limits on (simple) object key
1155 lengths that JSON doesn't have and also has different and incompatible 1176 lengths that JSON doesn't have and also has different and incompatible
1156 unicode handling, so you should make sure that your hash keys are 1177 unicode character escape syntax, so you should make sure that your hash
1157 noticeably shorter than the 1024 "stream characters" YAML allows and 1178 keys are noticeably shorter than the 1024 "stream characters" YAML
1158 that you do not have characters with codepoint values outside the 1179 allows and that you do not have characters with codepoint values outside
1159 Unicode BMP (basic multilingual page). YAML also does not allow "\/" 1180 the Unicode BMP (basic multilingual page). YAML also does not allow "\/"
1160 sequences in strings (which JSON::XS does not *currently* generate, but 1181 sequences in strings (which JSON::XS does not *currently* generate, but
1161 other JSON generators might). 1182 other JSON generators might).
1162 1183
1163 There might be other incompatibilities that I am not aware of (or the 1184 There might be other incompatibilities that I am not aware of (or the
1164 YAML specification has been changed yet again - it does so quite often). 1185 YAML specification has been changed yet again - it does so quite often).
1181 (which is not that difficult or long) and finally make YAML 1202 (which is not that difficult or long) and finally make YAML
1182 compatible to it, and educating users about the changes, instead of 1203 compatible to it, and educating users about the changes, instead of
1183 spreading lies about the real compatibility for many *years* and 1204 spreading lies about the real compatibility for many *years* and
1184 trying to silence people who point out that it isn't true. 1205 trying to silence people who point out that it isn't true.
1185 1206
1207 Addendum/2009: the YAML 1.2 spec is still incompatible with JSON,
1208 even though the incompatibilities have been documented (and are
1209 known to Brian) for many years and the spec makes explicit claims
1210 that YAML is a superset of JSON. It would be so easy to fix, but
1211 apparently, bullying people and corrupting userdata is so much
1212 easier.
1213
1186 SPEED 1214 SPEED
1187 It seems that JSON::XS is surprisingly fast, as shown in the following 1215 It seems that JSON::XS is surprisingly fast, as shown in the following
1188 tables. They have been generated with the help of the "eg/bench" program 1216 tables. They have been generated with the help of the "eg/bench" program
1189 in the JSON::XS distribution, to make it easy to compare on your own 1217 in the JSON::XS distribution, to make it easy to compare on your own
1190 system. 1218 system.
1193 single-line JSON string (also available at 1221 single-line JSON string (also available at
1194 <http://dist.schmorp.de/misc/json/short.json>). 1222 <http://dist.schmorp.de/misc/json/short.json>).
1195 1223
1196 {"method": "handleMessage", "params": ["user1", 1224 {"method": "handleMessage", "params": ["user1",
1197 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7, 1225 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1198 true, false]} 1226 1, 0]}
1199 1227
1200 It shows the number of encodes/decodes per second (JSON::XS uses the 1228 It shows the number of encodes/decodes per second (JSON::XS uses the
1201 functional interface, while JSON::XS/2 uses the OO interface with 1229 functional interface, while JSON::XS/2 uses the OO interface with
1202 pretty-printing and hashkey sorting enabled, JSON::XS/3 enables shrink). 1230 pretty-printing and hashkey sorting enabled, JSON::XS/3 enables shrink.
1203 Higher is better: 1231 JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ uses
1232 the from_json method). Higher is better:
1204 1233
1205 module | encode | decode | 1234 module | encode | decode |
1206 -----------|------------|------------| 1235 --------------|------------|------------|
1207 JSON 1.x | 4990.842 | 4088.813 | 1236 JSON::DWIW/DS | 86302.551 | 102300.098 |
1208 JSON::DWIW | 51653.990 | 71575.154 | 1237 JSON::DWIW/FJ | 86302.551 | 75983.768 |
1209 JSON::PC | 65948.176 | 74631.744 | 1238 JSON::PP | 15827.562 | 6638.658 |
1210 JSON::PP | 8931.652 | 3817.168 | 1239 JSON::Syck | 63358.066 | 47662.545 |
1211 JSON::Syck | 24877.248 | 27776.848 | 1240 JSON::XS | 511500.488 | 511500.488 |
1212 JSON::XS | 388361.481 | 227951.304 | 1241 JSON::XS/2 | 291271.111 | 388361.481 |
1213 JSON::XS/2 | 227951.304 | 218453.333 | 1242 JSON::XS/3 | 361577.931 | 361577.931 |
1214 JSON::XS/3 | 338250.323 | 218453.333 | 1243 Storable | 66788.280 | 265462.278 |
1215 Storable | 16500.016 | 135300.129 |
1216 -----------+------------+------------+ 1244 --------------+------------+------------+
1217 1245
1218 That is, JSON::XS is about five times faster than JSON::DWIW on 1246 That is, JSON::XS is almost six times faster than JSON::DWIW on
1219 encoding, about three times faster on decoding, and over forty times 1247 encoding, about five times faster on decoding, and over thirty to
1220 faster than JSON, even with pretty-printing and key sorting. It also 1248 seventy times faster than JSON's pure perl implementation. It also
1221 compares favourably to Storable for small amounts of data. 1249 compares favourably to Storable for small amounts of data.
1222 1250
1223 Using a longer test string (roughly 18KB, generated from Yahoo! Locals 1251 Using a longer test string (roughly 18KB, generated from Yahoo! Locals
1224 search API (<http://dist.schmorp.de/misc/json/long.json>). 1252 search API (<http://dist.schmorp.de/misc/json/long.json>).
1225 1253
1226 module | encode | decode | 1254 module | encode | decode |
1227 -----------|------------|------------| 1255 --------------|------------|------------|
1228 JSON 1.x | 55.260 | 34.971 | 1256 JSON::DWIW/DS | 1647.927 | 2673.916 |
1229 JSON::DWIW | 825.228 | 1082.513 | 1257 JSON::DWIW/FJ | 1630.249 | 2596.128 |
1230 JSON::PC | 3571.444 | 2394.829 |
1231 JSON::PP | 210.987 | 32.574 | 1258 JSON::PP | 400.640 | 62.311 |
1232 JSON::Syck | 552.551 | 787.544 | 1259 JSON::Syck | 1481.040 | 1524.869 |
1233 JSON::XS | 5780.463 | 4854.519 | 1260 JSON::XS | 20661.596 | 9541.183 |
1234 JSON::XS/2 | 3869.998 | 4798.975 | 1261 JSON::XS/2 | 10683.403 | 9416.938 |
1235 JSON::XS/3 | 5862.880 | 4798.975 | 1262 JSON::XS/3 | 20661.596 | 9400.054 |
1236 Storable | 4445.002 | 5235.027 | 1263 Storable | 19765.806 | 10000.725 |
1237 -----------+------------+------------+ 1264 --------------+------------+------------+
1238 1265
1239 Again, JSON::XS leads by far (except for Storable which non-surprisingly 1266 Again, JSON::XS leads by far (except for Storable which non-surprisingly
1240 decodes faster). 1267 decodes a bit faster).
1241 1268
1242 On large strings containing lots of high Unicode characters, some 1269 On large strings containing lots of high Unicode characters, some
1243 modules (such as JSON::PC) seem to decode faster than JSON::XS, but the 1270 modules (such as JSON::PC) seem to decode faster than JSON::XS, but the
1244 result will be broken due to missing (or wrong) Unicode handling. Others 1271 result will be broken due to missing (or wrong) Unicode handling. Others
1245 refuse to decode or encode properly, so it was impossible to prepare a 1272 refuse to decode or encode properly, so it was impossible to prepare a
1280 information you might want to make sure that exceptions thrown by 1307 information you might want to make sure that exceptions thrown by
1281 JSON::XS will not end up in front of untrusted eyes. 1308 JSON::XS will not end up in front of untrusted eyes.
1282 1309
1283 If you are using JSON::XS to return packets to consumption by JavaScript 1310 If you are using JSON::XS to return packets to consumption by JavaScript
1284 scripts in a browser you should have a look at 1311 scripts in a browser you should have a look at
1285 <http://jpsykes.com/47/practical-csrf-and-json-security> to see whether 1312 <http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/>
1286 you are vulnerable to some common attack vectors (which really are 1313 to see whether you are vulnerable to some common attack vectors (which
1287 browser design bugs, but it is still you who will have to deal with it, 1314 really are browser design bugs, but it is still you who will have to
1288 as major browser developers care only for features, not about getting 1315 deal with it, as major browser developers care only for features, not
1289 security right). 1316 about getting security right).
1290 1317
1291THREADS 1318THREADS
1292 This module is *not* guaranteed to be thread safe and there are no plans 1319 This module is *not* guaranteed to be thread safe and there are no plans
1293 to change this until Perl gets thread support (as opposed to the 1320 to change this until Perl gets thread support (as opposed to the
1294 horribly slow so-called "threads" which are simply slow and bloated 1321 horribly slow so-called "threads" which are simply slow and bloated
1295 process simulations - use fork, it's *much* faster, cheaper, better). 1322 process simulations - use fork, it's *much* faster, cheaper, better).
1296 1323
1297 (It might actually work, but you have been warned). 1324 (It might actually work, but you have been warned).
1298 1325
1326THE PERILS OF SETLOCALE
1327 Sometimes people avoid the Perl locale support and directly call the
1328 system's setlocale function with "LC_ALL".
1329
1330 This breaks both perl and modules such as JSON::XS, as stringification
1331 of numbers no longer works correcly (e.g. "$x = 0.1; print "$x"+1" might
1332 print 1, and JSON::XS might output illegal JSON as JSON::XS relies on
1333 perl to stringify numbers).
1334
1335 The solution is simple: don't call "setlocale", or use it for only those
1336 categories you need, such as "LC_MESSAGES" or "LC_CTYPE".
1337
1338 If you need "LC_NUMERIC", you should enable it only around the code that
1339 actually needs it (avoiding stringification of numbers), and restore it
1340 afterwards.
1341
1299BUGS 1342BUGS
1300 While the goal of this module is to be correct, that unfortunately does 1343 While the goal of this module is to be correct, that unfortunately does
1301 not mean it's bug-free, only that I think its design is bug-free. If you 1344 not mean it's bug-free, only that I think its design is bug-free. If you
1302 keep reporting bugs they will be fixed swiftly, though. 1345 keep reporting bugs they will be fixed swiftly, though.
1303 1346

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines