ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.116 by root, Tue Feb 17 23:41:20 2009 UTC vs.
Revision 1.129 by root, Tue Jan 19 01:02:19 2010 UTC

99 99
100=cut 100=cut
101 101
102package JSON::XS; 102package JSON::XS;
103 103
104no warnings; 104use common::sense;
105use strict;
106 105
107our $VERSION = '2.232'; 106our $VERSION = '2.28';
108our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
109 108
110our @EXPORT = qw(encode_json decode_json to_json from_json); 109our @EXPORT = qw(encode_json decode_json to_json from_json);
111 110
112sub to_json($) { 111sub to_json($) {
441the same JSON text (given the same overall settings). If it is disabled, 440the same JSON text (given the same overall settings). If it is disabled,
442the same hash might be encoded differently even if contains the same data, 441the same hash might be encoded differently even if contains the same data,
443as key-value pairs have no inherent ordering in Perl. 442as key-value pairs have no inherent ordering in Perl.
444 443
445This setting has no effect when decoding JSON texts. 444This setting has no effect when decoding JSON texts.
445
446This setting has currently no effect on tied hashes.
446 447
447=item $json = $json->allow_nonref ([$enable]) 448=item $json = $json->allow_nonref ([$enable])
448 449
449=item $enabled = $json->get_allow_nonref 450=item $enabled = $json->get_allow_nonref
450 451
1209 use JSON::XS; 1210 use JSON::XS;
1210 1211
1211 print encode_json [chr 0x2028]; 1212 print encode_json [chr 0x2028];
1212 1213
1213The right fix for this is to use a proper JSON parser in your javascript 1214The right fix for this is to use a proper JSON parser in your javascript
1214programs, and not rely on C<eval>. 1215programs, and not rely on C<eval> (see for example Douglas Crockford's
1216F<json2.js> parser).
1215 1217
1216If this is not an option, you can, as a stop-gap measure, simply encode to 1218If this is not an option, you can, as a stop-gap measure, simply encode to
1217ASCII-only JSON: 1219ASCII-only JSON:
1218 1220
1219 use JSON::XS; 1221 use JSON::XS;
1220 1222
1221 print JSON::XS->new->ascii->encode ([chr 0x2028]); 1223 print JSON::XS->new->ascii->encode ([chr 0x2028]);
1222 1224
1223And if you are concerned about the size of the resulting JSON text, you 1225Note that this will enlarge the resulting JSON text quite a bit if you
1224can run some regexes to only escape U+2028 and U+2029: 1226have many non-ASCII characters. You might be tempted to run some regexes
1227to only escape U+2028 and U+2029, e.g.:
1225 1228
1226 use JSON::XS; 1229 # DO NOT USE THIS!
1227
1228 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]); 1230 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1229 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028 1231 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1230 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029 1232 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1231 print $json; 1233 print $json;
1232 1234
1233This works because U+2028/U+2029 are not allowed outside of strings and 1235Note that I<this is a bad idea>: the above only works for U+2028 and
1234are not used for syntax, so replacing them unconditionally just works.
1235
1236Note, however, that fixing the broken JSON parser is better than working
1237around it in every other generator. The above regexes should work well in
1238other languages, as long as they operate on UTF-8. It is equally valid to
1239replace all occurences of U+2028/2029 directly by their \\u-escaped forms
1240in unicode texts, so they can simply be used to fix any parsers relying on
1241C<eval> by first applying the regexes on the encoded texts.
1242
1243Note also that the above only works for U+2028 and U+2029 and thus
1244only for fully ECMAscript-compliant parsers. Many existing javascript 1236U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1245implementations misparse other characters as well. Best rely on a good 1237javascript implementations, however, have issues with other characters as
1246JSON parser, such as Douglas Crockfords F<json2.js>, which escapes the 1238well - using C<eval> naively simply I<will> cause problems.
1247above and many more problematic characters properly before passing them
1248into C<eval>.
1249 1239
1250Another problem is that some javascript implementations reserve 1240Another problem is that some javascript implementations reserve
1251some property names for their own purposes (which probably makes 1241some property names for their own purposes (which probably makes
1252them non-ECMAscript-compliant). For example, Iceweasel reserves the 1242them non-ECMAscript-compliant). For example, Iceweasel reserves the
1253C<__proto__> property name for it's own purposes. 1243C<__proto__> property name for it's own purposes.
1278 my $yaml = $to_yaml->encode ($ref) . "\n"; 1268 my $yaml = $to_yaml->encode ($ref) . "\n";
1279 1269
1280This will I<usually> generate JSON texts that also parse as valid 1270This will I<usually> generate JSON texts that also parse as valid
1281YAML. Please note that YAML has hardcoded limits on (simple) object key 1271YAML. Please note that YAML has hardcoded limits on (simple) object key
1282lengths that JSON doesn't have and also has different and incompatible 1272lengths that JSON doesn't have and also has different and incompatible
1283unicode handling, so you should make sure that your hash keys are 1273unicode character escape syntax, so you should make sure that your hash
1284noticeably shorter than the 1024 "stream characters" YAML allows and that 1274keys are noticeably shorter than the 1024 "stream characters" YAML allows
1285you do not have characters with codepoint values outside the Unicode BMP 1275and that you do not have characters with codepoint values outside the
1286(basic multilingual page). YAML also does not allow C<\/> sequences in 1276Unicode BMP (basic multilingual page). YAML also does not allow C<\/>
1287strings (which JSON::XS does not I<currently> generate, but other JSON 1277sequences in strings (which JSON::XS does not I<currently> generate, but
1288generators might). 1278other JSON generators might).
1289 1279
1290There might be other incompatibilities that I am not aware of (or the YAML 1280There might be other incompatibilities that I am not aware of (or the YAML
1291specification has been changed yet again - it does so quite often). In 1281specification has been changed yet again - it does so quite often). In
1292general you should not try to generate YAML with a JSON generator or vice 1282general you should not try to generate YAML with a JSON generator or vice
1293versa, or try to parse JSON with a YAML parser or vice versa: chances are 1283versa, or try to parse JSON with a YAML parser or vice versa: chances are
1312that difficult or long) and finally make YAML compatible to it, and 1302that difficult or long) and finally make YAML compatible to it, and
1313educating users about the changes, instead of spreading lies about the 1303educating users about the changes, instead of spreading lies about the
1314real compatibility for many I<years> and trying to silence people who 1304real compatibility for many I<years> and trying to silence people who
1315point out that it isn't true. 1305point out that it isn't true.
1316 1306
1307Addendum/2009: the YAML 1.2 spec is still incomaptible with JSON, even
1308though the incompatibilities have been documented (and are known to
1309Brian) for many years and the spec makes explicit claims that YAML is a
1310superset of JSON. It would be so easy to fix, but apparently, bullying and
1311corrupting userdata is so much easier.
1312
1317=back 1313=back
1318 1314
1319 1315
1320=head2 SPEED 1316=head2 SPEED
1321 1317
1328a very short single-line JSON string (also available at 1324a very short single-line JSON string (also available at
1329L<http://dist.schmorp.de/misc/json/short.json>). 1325L<http://dist.schmorp.de/misc/json/short.json>).
1330 1326
1331 {"method": "handleMessage", "params": ["user1", 1327 {"method": "handleMessage", "params": ["user1",
1332 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7, 1328 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1333 true, false]} 1329 1, 0]}
1334 1330
1335It shows the number of encodes/decodes per second (JSON::XS uses 1331It shows the number of encodes/decodes per second (JSON::XS uses
1336the functional interface, while JSON::XS/2 uses the OO interface 1332the functional interface, while JSON::XS/2 uses the OO interface
1337with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables 1333with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1338shrink). Higher is better: 1334shrink. JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ
1335uses the from_json method). Higher is better:
1339 1336
1340 module | encode | decode | 1337 module | encode | decode |
1341 -----------|------------|------------| 1338 --------------|------------|------------|
1342 JSON 1.x | 4990.842 | 4088.813 | 1339 JSON::DWIW/DS | 86302.551 | 102300.098 |
1343 JSON::DWIW | 51653.990 | 71575.154 | 1340 JSON::DWIW/FJ | 86302.551 | 75983.768 |
1344 JSON::PC | 65948.176 | 74631.744 | 1341 JSON::PP | 15827.562 | 6638.658 |
1345 JSON::PP | 8931.652 | 3817.168 | 1342 JSON::Syck | 63358.066 | 47662.545 |
1346 JSON::Syck | 24877.248 | 27776.848 | 1343 JSON::XS | 511500.488 | 511500.488 |
1347 JSON::XS | 388361.481 | 227951.304 | 1344 JSON::XS/2 | 291271.111 | 388361.481 |
1348 JSON::XS/2 | 227951.304 | 218453.333 | 1345 JSON::XS/3 | 361577.931 | 361577.931 |
1349 JSON::XS/3 | 338250.323 | 218453.333 | 1346 Storable | 66788.280 | 265462.278 |
1350 Storable | 16500.016 | 135300.129 |
1351 -----------+------------+------------+ 1347 --------------+------------+------------+
1352 1348
1353That is, JSON::XS is about five times faster than JSON::DWIW on encoding, 1349That is, JSON::XS is almost six times faster than JSON::DWIW on encoding,
1354about three times faster on decoding, and over forty times faster 1350about five times faster on decoding, and over thirty to seventy times
1355than JSON, even with pretty-printing and key sorting. It also compares 1351faster than JSON's pure perl implementation. It also compares favourably
1356favourably to Storable for small amounts of data. 1352to Storable for small amounts of data.
1357 1353
1358Using a longer test string (roughly 18KB, generated from Yahoo! Locals 1354Using a longer test string (roughly 18KB, generated from Yahoo! Locals
1359search API (L<http://dist.schmorp.de/misc/json/long.json>). 1355search API (L<http://dist.schmorp.de/misc/json/long.json>).
1360 1356
1361 module | encode | decode | 1357 module | encode | decode |
1362 -----------|------------|------------| 1358 --------------|------------|------------|
1363 JSON 1.x | 55.260 | 34.971 | 1359 JSON::DWIW/DS | 1647.927 | 2673.916 |
1364 JSON::DWIW | 825.228 | 1082.513 | 1360 JSON::DWIW/FJ | 1630.249 | 2596.128 |
1365 JSON::PC | 3571.444 | 2394.829 |
1366 JSON::PP | 210.987 | 32.574 | 1361 JSON::PP | 400.640 | 62.311 |
1367 JSON::Syck | 552.551 | 787.544 | 1362 JSON::Syck | 1481.040 | 1524.869 |
1368 JSON::XS | 5780.463 | 4854.519 | 1363 JSON::XS | 20661.596 | 9541.183 |
1369 JSON::XS/2 | 3869.998 | 4798.975 | 1364 JSON::XS/2 | 10683.403 | 9416.938 |
1370 JSON::XS/3 | 5862.880 | 4798.975 | 1365 JSON::XS/3 | 20661.596 | 9400.054 |
1371 Storable | 4445.002 | 5235.027 | 1366 Storable | 19765.806 | 10000.725 |
1372 -----------+------------+------------+ 1367 --------------+------------+------------+
1373 1368
1374Again, JSON::XS leads by far (except for Storable which non-surprisingly 1369Again, JSON::XS leads by far (except for Storable which non-surprisingly
1375decodes faster). 1370decodes a bit faster).
1376 1371
1377On large strings containing lots of high Unicode characters, some modules 1372On large strings containing lots of high Unicode characters, some modules
1378(such as JSON::PC) seem to decode faster than JSON::XS, but the result 1373(such as JSON::PC) seem to decode faster than JSON::XS, but the result
1379will be broken due to missing (or wrong) Unicode handling. Others refuse 1374will be broken due to missing (or wrong) Unicode handling. Others refuse
1380to decode or encode properly, so it was impossible to prepare a fair 1375to decode or encode properly, so it was impossible to prepare a fair
1416information you might want to make sure that exceptions thrown by JSON::XS 1411information you might want to make sure that exceptions thrown by JSON::XS
1417will not end up in front of untrusted eyes. 1412will not end up in front of untrusted eyes.
1418 1413
1419If you are using JSON::XS to return packets to consumption 1414If you are using JSON::XS to return packets to consumption
1420by JavaScript scripts in a browser you should have a look at 1415by JavaScript scripts in a browser you should have a look at
1421L<http://jpsykes.com/47/practical-csrf-and-json-security> to see whether 1416L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to
1422you are vulnerable to some common attack vectors (which really are browser 1417see whether you are vulnerable to some common attack vectors (which really
1423design bugs, but it is still you who will have to deal with it, as major 1418are browser design bugs, but it is still you who will have to deal with
1424browser developers care only for features, not about getting security 1419it, as major browser developers care only for features, not about getting
1425right). 1420security right).
1426 1421
1427 1422
1428=head1 THREADS 1423=head1 THREADS
1429 1424
1430This module is I<not> guaranteed to be thread safe and there are no 1425This module is I<not> guaranteed to be thread safe and there are no

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines