ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.115 by root, Tue Feb 17 23:29:38 2009 UTC vs.
Revision 1.130 by root, Thu Mar 11 17:36:09 2010 UTC

… …
99 99
100=cut 100=cut
101 101
102package JSON::XS; 102package JSON::XS;
103 103
104no warnings; 104use common::sense;
105use strict;
106 105
107our $VERSION = '2.232'; 106our $VERSION = '2.28';
108our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
109 108
110our @EXPORT = qw(encode_json decode_json to_json from_json); 109our @EXPORT = qw(encode_json decode_json to_json from_json);
111 110
112sub to_json($) { 111sub to_json($) {
… …
441the same JSON text (given the same overall settings). If it is disabled, 440the same JSON text (given the same overall settings). If it is disabled,
442the same hash might be encoded differently even if contains the same data, 441the same hash might be encoded differently even if contains the same data,
443as key-value pairs have no inherent ordering in Perl. 442as key-value pairs have no inherent ordering in Perl.
444 443
445This setting has no effect when decoding JSON texts. 444This setting has no effect when decoding JSON texts.
445
446This setting has currently no effect on tied hashes.
446 447
447=item $json = $json->allow_nonref ([$enable]) 448=item $json = $json->allow_nonref ([$enable])
448 449
449=item $enabled = $json->get_allow_nonref 450=item $enabled = $json->get_allow_nonref
450 451
… …
750objects or arrays, instead they must be concatenated back-to-back. If 751objects or arrays, instead they must be concatenated back-to-back. If
751an error occurs, an exception will be raised as in the scalar context 752an error occurs, an exception will be raised as in the scalar context
752case. Note that in this case, any previously-parsed JSON texts will be 753case. Note that in this case, any previously-parsed JSON texts will be
753lost. 754lost.
754 755
756Example: Parse some JSON arrays/objects in a given string and return
757them.
758
759 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
760
755=item $lvalue_string = $json->incr_text 761=item $lvalue_string = $json->incr_text
756 762
757This method returns the currently stored JSON fragment as an lvalue, that 763This method returns the currently stored JSON fragment as an lvalue, that
758is, you can manipulate it. This I<only> works when a preceding call to 764is, you can manipulate it. This I<only> works when a preceding call to
759C<incr_parse> in I<scalar context> successfully returned an object. Under 765C<incr_parse> in I<scalar context> successfully returned an object. Under
… …
1209 use JSON::XS; 1215 use JSON::XS;
1210 1216
1211 print encode_json [chr 0x2028]; 1217 print encode_json [chr 0x2028];
1212 1218
1213The right fix for this is to use a proper JSON parser in your javascript 1219The right fix for this is to use a proper JSON parser in your javascript
1214programs, and not rely on C<eval>. 1220programs, and not rely on C<eval> (see for example Douglas Crockford's
1221F<json2.js> parser).
1215 1222
1216If this is not an option, you can, as a stop-gap measure, simply encode to 1223If this is not an option, you can, as a stop-gap measure, simply encode to
1217ASCII-only JSON: 1224ASCII-only JSON:
1218 1225
1219 use JSON::XS; 1226 use JSON::XS;
1220 1227
1221 print JSON::XS->new->ascii->encode ([chr 0x2028]); 1228 print JSON::XS->new->ascii->encode ([chr 0x2028]);
1222 1229
1223And if you are concerned about the size of the resulting JSON text, you 1230Note that this will enlarge the resulting JSON text quite a bit if you
1224can run some regexes to only escape U+2028 and U+2029: 1231have many non-ASCII characters. You might be tempted to run some regexes
1232to only escape U+2028 and U+2029, e.g.:
1225 1233
1226 use JSON::XS; 1234 # DO NOT USE THIS!
1227
1228 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]); 1235 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1229 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028 1236 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1230 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029 1237 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1231 print $json; 1238 print $json;
1232 1239
1233This works because U+2028/U+2029 are not allowed outside of strings and 1240Note that I<this is a bad idea>: the above only works for U+2028 and
1234are not used for syntax, so replacing them unconditionally just works. 1241U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1235 1242javascript implementations, however, have issues with other characters as
1236Note, however, that fixing the broken JSON parser is better than working 1243well - using C<eval> naively simply I<will> cause problems.
1237around it in every other generator. The above regexes should work well in
1238other languages, as long as they operate on UTF-8. It is equally valid to
1239replace all occurences of U+2028/2029 directly by their \\u-escaped forms
1240in unicode texts, so they can simply be used to fix any parsers relying on
1241C<eval> by first applying the regexes on the encoded texts.
1242 1244
1243Another problem is that some javascript implementations reserve 1245Another problem is that some javascript implementations reserve
1244some property names for their own purposes (which probably makes 1246some property names for their own purposes (which probably makes
1245them non-ECMAscript-compliant). For example, Iceweasel reserves the 1247them non-ECMAscript-compliant). For example, Iceweasel reserves the
1246C<__proto__> property name for it's own purposes. 1248C<__proto__> property name for it's own purposes.
… …
1271 my $yaml = $to_yaml->encode ($ref) . "\n"; 1273 my $yaml = $to_yaml->encode ($ref) . "\n";
1272 1274
1273This will I<usually> generate JSON texts that also parse as valid 1275This will I<usually> generate JSON texts that also parse as valid
1274YAML. Please note that YAML has hardcoded limits on (simple) object key 1276YAML. Please note that YAML has hardcoded limits on (simple) object key
1275lengths that JSON doesn't have and also has different and incompatible 1277lengths that JSON doesn't have and also has different and incompatible
1276unicode handling, so you should make sure that your hash keys are 1278unicode character escape syntax, so you should make sure that your hash
1277noticeably shorter than the 1024 "stream characters" YAML allows and that 1279keys are noticeably shorter than the 1024 "stream characters" YAML allows
1278you do not have characters with codepoint values outside the Unicode BMP 1280and that you do not have characters with codepoint values outside the
1279(basic multilingual page). YAML also does not allow C<\/> sequences in 1281Unicode BMP (basic multilingual page). YAML also does not allow C<\/>
1280strings (which JSON::XS does not I<currently> generate, but other JSON 1282sequences in strings (which JSON::XS does not I<currently> generate, but
1281generators might). 1283other JSON generators might).
1282 1284
1283There might be other incompatibilities that I am not aware of (or the YAML 1285There might be other incompatibilities that I am not aware of (or the YAML
1284specification has been changed yet again - it does so quite often). In 1286specification has been changed yet again - it does so quite often). In
1285general you should not try to generate YAML with a JSON generator or vice 1287general you should not try to generate YAML with a JSON generator or vice
1286versa, or try to parse JSON with a YAML parser or vice versa: chances are 1288versa, or try to parse JSON with a YAML parser or vice versa: chances are
… …
1305that difficult or long) and finally make YAML compatible to it, and 1307that difficult or long) and finally make YAML compatible to it, and
1306educating users about the changes, instead of spreading lies about the 1308educating users about the changes, instead of spreading lies about the
1307real compatibility for many I<years> and trying to silence people who 1309real compatibility for many I<years> and trying to silence people who
1308point out that it isn't true. 1310point out that it isn't true.
1309 1311
1312Addendum/2009: the YAML 1.2 spec is still incomaptible with JSON, even
1313though the incompatibilities have been documented (and are known to
1314Brian) for many years and the spec makes explicit claims that YAML is a
1315superset of JSON. It would be so easy to fix, but apparently, bullying and
1316corrupting userdata is so much easier.
1317
1310=back 1318=back
1311 1319
1312 1320
1313=head2 SPEED 1321=head2 SPEED
1314 1322
… …
1321a very short single-line JSON string (also available at 1329a very short single-line JSON string (also available at
1322L<http://dist.schmorp.de/misc/json/short.json>). 1330L<http://dist.schmorp.de/misc/json/short.json>).
1323 1331
1324 {"method": "handleMessage", "params": ["user1", 1332 {"method": "handleMessage", "params": ["user1",
1325 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7, 1333 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1326 true, false]} 1334 1, 0]}
1327 1335
1328It shows the number of encodes/decodes per second (JSON::XS uses 1336It shows the number of encodes/decodes per second (JSON::XS uses
1329the functional interface, while JSON::XS/2 uses the OO interface 1337the functional interface, while JSON::XS/2 uses the OO interface
1330with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables 1338with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1331shrink). Higher is better: 1339shrink. JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ
1340uses the from_json method). Higher is better:
1332 1341
1333 module | encode | decode | 1342 module | encode | decode |
1334 -----------|------------|------------| 1343 --------------|------------|------------|
1335 JSON 1.x | 4990.842 | 4088.813 | 1344 JSON::DWIW/DS | 86302.551 | 102300.098 |
1336 JSON::DWIW | 51653.990 | 71575.154 | 1345 JSON::DWIW/FJ | 86302.551 | 75983.768 |
1337 JSON::PC | 65948.176 | 74631.744 | 1346 JSON::PP | 15827.562 | 6638.658 |
1338 JSON::PP | 8931.652 | 3817.168 | 1347 JSON::Syck | 63358.066 | 47662.545 |
1339 JSON::Syck | 24877.248 | 27776.848 | 1348 JSON::XS | 511500.488 | 511500.488 |
1340 JSON::XS | 388361.481 | 227951.304 | 1349 JSON::XS/2 | 291271.111 | 388361.481 |
1341 JSON::XS/2 | 227951.304 | 218453.333 | 1350 JSON::XS/3 | 361577.931 | 361577.931 |
1342 JSON::XS/3 | 338250.323 | 218453.333 | 1351 Storable | 66788.280 | 265462.278 |
1343 Storable | 16500.016 | 135300.129 |
1344 -----------+------------+------------+ 1352 --------------+------------+------------+
1345 1353
1346That is, JSON::XS is about five times faster than JSON::DWIW on encoding, 1354That is, JSON::XS is almost six times faster than JSON::DWIW on encoding,
1347about three times faster on decoding, and over forty times faster 1355about five times faster on decoding, and over thirty to seventy times
1348than JSON, even with pretty-printing and key sorting. It also compares 1356faster than JSON's pure perl implementation. It also compares favourably
1349favourably to Storable for small amounts of data. 1357to Storable for small amounts of data.
1350 1358
1351Using a longer test string (roughly 18KB, generated from Yahoo! Locals 1359Using a longer test string (roughly 18KB, generated from Yahoo! Locals
1352search API (L<http://dist.schmorp.de/misc/json/long.json>). 1360search API (L<http://dist.schmorp.de/misc/json/long.json>).
1353 1361
1354 module | encode | decode | 1362 module | encode | decode |
1355 -----------|------------|------------| 1363 --------------|------------|------------|
1356 JSON 1.x | 55.260 | 34.971 | 1364 JSON::DWIW/DS | 1647.927 | 2673.916 |
1357 JSON::DWIW | 825.228 | 1082.513 | 1365 JSON::DWIW/FJ | 1630.249 | 2596.128 |
1358 JSON::PC | 3571.444 | 2394.829 |
1359 JSON::PP | 210.987 | 32.574 | 1366 JSON::PP | 400.640 | 62.311 |
1360 JSON::Syck | 552.551 | 787.544 | 1367 JSON::Syck | 1481.040 | 1524.869 |
1361 JSON::XS | 5780.463 | 4854.519 | 1368 JSON::XS | 20661.596 | 9541.183 |
1362 JSON::XS/2 | 3869.998 | 4798.975 | 1369 JSON::XS/2 | 10683.403 | 9416.938 |
1363 JSON::XS/3 | 5862.880 | 4798.975 | 1370 JSON::XS/3 | 20661.596 | 9400.054 |
1364 Storable | 4445.002 | 5235.027 | 1371 Storable | 19765.806 | 10000.725 |
1365 -----------+------------+------------+ 1372 --------------+------------+------------+
1366 1373
1367Again, JSON::XS leads by far (except for Storable which non-surprisingly 1374Again, JSON::XS leads by far (except for Storable which non-surprisingly
1368decodes faster). 1375decodes a bit faster).
1369 1376
1370On large strings containing lots of high Unicode characters, some modules 1377On large strings containing lots of high Unicode characters, some modules
1371(such as JSON::PC) seem to decode faster than JSON::XS, but the result 1378(such as JSON::PC) seem to decode faster than JSON::XS, but the result
1372will be broken due to missing (or wrong) Unicode handling. Others refuse 1379will be broken due to missing (or wrong) Unicode handling. Others refuse
1373to decode or encode properly, so it was impossible to prepare a fair 1380to decode or encode properly, so it was impossible to prepare a fair
… …
1409information you might want to make sure that exceptions thrown by JSON::XS 1416information you might want to make sure that exceptions thrown by JSON::XS
1410will not end up in front of untrusted eyes. 1417will not end up in front of untrusted eyes.
1411 1418
1412If you are using JSON::XS to return packets to consumption 1419If you are using JSON::XS to return packets to consumption
1413by JavaScript scripts in a browser you should have a look at 1420by JavaScript scripts in a browser you should have a look at
1414L<http://jpsykes.com/47/practical-csrf-and-json-security> to see whether 1421L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to
1415you are vulnerable to some common attack vectors (which really are browser 1422see whether you are vulnerable to some common attack vectors (which really
1416design bugs, but it is still you who will have to deal with it, as major 1423are browser design bugs, but it is still you who will have to deal with
1417browser developers care only for features, not about getting security 1424it, as major browser developers care only for features, not about getting
1418right). 1425security right).
1419 1426
1420 1427
1421=head1 THREADS 1428=head1 THREADS
1422 1429
1423This module is I<not> guaranteed to be thread safe and there are no 1430This module is I<not> guaranteed to be thread safe and there are no

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines