ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.114 by root, Wed Jan 21 05:34:08 2009 UTC vs.
Revision 1.130 by root, Thu Mar 11 17:36:09 2010 UTC

… …
99 99
100=cut 100=cut
101 101
102package JSON::XS; 102package JSON::XS;
103 103
104no warnings; 104use common::sense;
105use strict;
106 105
107our $VERSION = '2.231'; 106our $VERSION = '2.28';
108our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
109 108
110our @EXPORT = qw(encode_json decode_json to_json from_json); 109our @EXPORT = qw(encode_json decode_json to_json from_json);
111 110
112sub to_json($) { 111sub to_json($) {
… …
441the same JSON text (given the same overall settings). If it is disabled, 440the same JSON text (given the same overall settings). If it is disabled,
442the same hash might be encoded differently even if contains the same data, 441the same hash might be encoded differently even if contains the same data,
443as key-value pairs have no inherent ordering in Perl. 442as key-value pairs have no inherent ordering in Perl.
444 443
445This setting has no effect when decoding JSON texts. 444This setting has no effect when decoding JSON texts.
445
446This setting has currently no effect on tied hashes.
446 447
447=item $json = $json->allow_nonref ([$enable]) 448=item $json = $json->allow_nonref ([$enable])
448 449
449=item $enabled = $json->get_allow_nonref 450=item $enabled = $json->get_allow_nonref
450 451
… …
750objects or arrays, instead they must be concatenated back-to-back. If 751objects or arrays, instead they must be concatenated back-to-back. If
751an error occurs, an exception will be raised as in the scalar context 752an error occurs, an exception will be raised as in the scalar context
752case. Note that in this case, any previously-parsed JSON texts will be 753case. Note that in this case, any previously-parsed JSON texts will be
753lost. 754lost.
754 755
756Example: Parse some JSON arrays/objects in a given string and return
757them.
758
759 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
760
755=item $lvalue_string = $json->incr_text 761=item $lvalue_string = $json->incr_text
756 762
757This method returns the currently stored JSON fragment as an lvalue, that 763This method returns the currently stored JSON fragment as an lvalue, that
758is, you can manipulate it. This I<only> works when a preceding call to 764is, you can manipulate it. This I<only> works when a preceding call to
759C<incr_parse> in I<scalar context> successfully returned an object. Under 765C<incr_parse> in I<scalar context> successfully returned an object. Under
… …
1185proper subset of most 8-bit and multibyte encodings in use in the world. 1191proper subset of most 8-bit and multibyte encodings in use in the world.
1186 1192
1187=back 1193=back
1188 1194
1189 1195
1196=head2 JSON and ECMAscript
1197
1198JSON syntax is based on how literals are represented in javascript (the
1199not-standardised predecessor of ECMAscript) which is presumably why it is
1200called "JavaScript Object Notation".
1201
1202However, JSON is not a subset (and also not a superset of course) of
1203ECMAscript (the standard) or javascript (whatever browsers actually
1204implement).
1205
1206If you want to use javascript's C<eval> function to "parse" JSON, you
1207might run into parse errors for valid JSON texts, or the resulting data
1208structure might not be queryable:
1209
1210One of the problems is that U+2028 and U+2029 are valid characters inside
1211JSON strings, but are not allowed in ECMAscript string literals, so the
1212following Perl fragment will not output something that can be guaranteed
1213to be parsable by javascript's C<eval>:
1214
1215 use JSON::XS;
1216
1217 print encode_json [chr 0x2028];
1218
1219The right fix for this is to use a proper JSON parser in your javascript
1220programs, and not rely on C<eval> (see for example Douglas Crockford's
1221F<json2.js> parser).
1222
1223If this is not an option, you can, as a stop-gap measure, simply encode to
1224ASCII-only JSON:
1225
1226 use JSON::XS;
1227
1228 print JSON::XS->new->ascii->encode ([chr 0x2028]);
1229
1230Note that this will enlarge the resulting JSON text quite a bit if you
1231have many non-ASCII characters. You might be tempted to run some regexes
1232to only escape U+2028 and U+2029, e.g.:
1233
1234 # DO NOT USE THIS!
1235 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1236 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1237 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1238 print $json;
1239
1240Note that I<this is a bad idea>: the above only works for U+2028 and
1241U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1242javascript implementations, however, have issues with other characters as
1243well - using C<eval> naively simply I<will> cause problems.
1244
1245Another problem is that some javascript implementations reserve
1246some property names for their own purposes (which probably makes
1247them non-ECMAscript-compliant). For example, Iceweasel reserves the
1248C<__proto__> property name for it's own purposes.
1249
1250If that is a problem, you could parse try to filter the resulting JSON
1251output for these property strings, e.g.:
1252
1253 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1254
1255This works because C<__proto__> is not valid outside of strings, so every
1256occurence of C<"__proto__"\s*:> must be a string used as property name.
1257
1258If you know of other incompatibilities, please let me know.
1259
1260
1190=head2 JSON and YAML 1261=head2 JSON and YAML
1191 1262
1192You often hear that JSON is a subset of YAML. This is, however, a mass 1263You often hear that JSON is a subset of YAML. This is, however, a mass
1193hysteria(*) and very far from the truth (as of the time of this writing), 1264hysteria(*) and very far from the truth (as of the time of this writing),
1194so let me state it clearly: I<in general, there is no way to configure 1265so let me state it clearly: I<in general, there is no way to configure
… …
1202 my $yaml = $to_yaml->encode ($ref) . "\n"; 1273 my $yaml = $to_yaml->encode ($ref) . "\n";
1203 1274
1204This will I<usually> generate JSON texts that also parse as valid 1275This will I<usually> generate JSON texts that also parse as valid
1205YAML. Please note that YAML has hardcoded limits on (simple) object key 1276YAML. Please note that YAML has hardcoded limits on (simple) object key
1206lengths that JSON doesn't have and also has different and incompatible 1277lengths that JSON doesn't have and also has different and incompatible
1207unicode handling, so you should make sure that your hash keys are 1278unicode character escape syntax, so you should make sure that your hash
1208noticeably shorter than the 1024 "stream characters" YAML allows and that 1279keys are noticeably shorter than the 1024 "stream characters" YAML allows
1209you do not have characters with codepoint values outside the Unicode BMP 1280and that you do not have characters with codepoint values outside the
1210(basic multilingual page). YAML also does not allow C<\/> sequences in 1281Unicode BMP (basic multilingual page). YAML also does not allow C<\/>
1211strings (which JSON::XS does not I<currently> generate, but other JSON 1282sequences in strings (which JSON::XS does not I<currently> generate, but
1212generators might). 1283other JSON generators might).
1213 1284
1214There might be other incompatibilities that I am not aware of (or the YAML 1285There might be other incompatibilities that I am not aware of (or the YAML
1215specification has been changed yet again - it does so quite often). In 1286specification has been changed yet again - it does so quite often). In
1216general you should not try to generate YAML with a JSON generator or vice 1287general you should not try to generate YAML with a JSON generator or vice
1217versa, or try to parse JSON with a YAML parser or vice versa: chances are 1288versa, or try to parse JSON with a YAML parser or vice versa: chances are
… …
1236that difficult or long) and finally make YAML compatible to it, and 1307that difficult or long) and finally make YAML compatible to it, and
1237educating users about the changes, instead of spreading lies about the 1308educating users about the changes, instead of spreading lies about the
1238real compatibility for many I<years> and trying to silence people who 1309real compatibility for many I<years> and trying to silence people who
1239point out that it isn't true. 1310point out that it isn't true.
1240 1311
1312Addendum/2009: the YAML 1.2 spec is still incomaptible with JSON, even
1313though the incompatibilities have been documented (and are known to
1314Brian) for many years and the spec makes explicit claims that YAML is a
1315superset of JSON. It would be so easy to fix, but apparently, bullying and
1316corrupting userdata is so much easier.
1317
1241=back 1318=back
1242 1319
1243 1320
1244=head2 SPEED 1321=head2 SPEED
1245 1322
… …
1252a very short single-line JSON string (also available at 1329a very short single-line JSON string (also available at
1253L<http://dist.schmorp.de/misc/json/short.json>). 1330L<http://dist.schmorp.de/misc/json/short.json>).
1254 1331
1255 {"method": "handleMessage", "params": ["user1", 1332 {"method": "handleMessage", "params": ["user1",
1256 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7, 1333 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1257 true, false]} 1334 1, 0]}
1258 1335
1259It shows the number of encodes/decodes per second (JSON::XS uses 1336It shows the number of encodes/decodes per second (JSON::XS uses
1260the functional interface, while JSON::XS/2 uses the OO interface 1337the functional interface, while JSON::XS/2 uses the OO interface
1261with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables 1338with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1262shrink). Higher is better: 1339shrink. JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ
1340uses the from_json method). Higher is better:
1263 1341
1264 module | encode | decode | 1342 module | encode | decode |
1265 -----------|------------|------------| 1343 --------------|------------|------------|
1266 JSON 1.x | 4990.842 | 4088.813 | 1344 JSON::DWIW/DS | 86302.551 | 102300.098 |
1267 JSON::DWIW | 51653.990 | 71575.154 | 1345 JSON::DWIW/FJ | 86302.551 | 75983.768 |
1268 JSON::PC | 65948.176 | 74631.744 | 1346 JSON::PP | 15827.562 | 6638.658 |
1269 JSON::PP | 8931.652 | 3817.168 | 1347 JSON::Syck | 63358.066 | 47662.545 |
1270 JSON::Syck | 24877.248 | 27776.848 | 1348 JSON::XS | 511500.488 | 511500.488 |
1271 JSON::XS | 388361.481 | 227951.304 | 1349 JSON::XS/2 | 291271.111 | 388361.481 |
1272 JSON::XS/2 | 227951.304 | 218453.333 | 1350 JSON::XS/3 | 361577.931 | 361577.931 |
1273 JSON::XS/3 | 338250.323 | 218453.333 | 1351 Storable | 66788.280 | 265462.278 |
1274 Storable | 16500.016 | 135300.129 |
1275 -----------+------------+------------+ 1352 --------------+------------+------------+
1276 1353
1277That is, JSON::XS is about five times faster than JSON::DWIW on encoding, 1354That is, JSON::XS is almost six times faster than JSON::DWIW on encoding,
1278about three times faster on decoding, and over forty times faster 1355about five times faster on decoding, and over thirty to seventy times
1279than JSON, even with pretty-printing and key sorting. It also compares 1356faster than JSON's pure perl implementation. It also compares favourably
1280favourably to Storable for small amounts of data. 1357to Storable for small amounts of data.
1281 1358
1282Using a longer test string (roughly 18KB, generated from Yahoo! Locals 1359Using a longer test string (roughly 18KB, generated from Yahoo! Locals
1283search API (L<http://dist.schmorp.de/misc/json/long.json>). 1360search API (L<http://dist.schmorp.de/misc/json/long.json>).
1284 1361
1285 module | encode | decode | 1362 module | encode | decode |
1286 -----------|------------|------------| 1363 --------------|------------|------------|
1287 JSON 1.x | 55.260 | 34.971 | 1364 JSON::DWIW/DS | 1647.927 | 2673.916 |
1288 JSON::DWIW | 825.228 | 1082.513 | 1365 JSON::DWIW/FJ | 1630.249 | 2596.128 |
1289 JSON::PC | 3571.444 | 2394.829 |
1290 JSON::PP | 210.987 | 32.574 | 1366 JSON::PP | 400.640 | 62.311 |
1291 JSON::Syck | 552.551 | 787.544 | 1367 JSON::Syck | 1481.040 | 1524.869 |
1292 JSON::XS | 5780.463 | 4854.519 | 1368 JSON::XS | 20661.596 | 9541.183 |
1293 JSON::XS/2 | 3869.998 | 4798.975 | 1369 JSON::XS/2 | 10683.403 | 9416.938 |
1294 JSON::XS/3 | 5862.880 | 4798.975 | 1370 JSON::XS/3 | 20661.596 | 9400.054 |
1295 Storable | 4445.002 | 5235.027 | 1371 Storable | 19765.806 | 10000.725 |
1296 -----------+------------+------------+ 1372 --------------+------------+------------+
1297 1373
1298Again, JSON::XS leads by far (except for Storable which non-surprisingly 1374Again, JSON::XS leads by far (except for Storable which non-surprisingly
1299decodes faster). 1375decodes a bit faster).
1300 1376
1301On large strings containing lots of high Unicode characters, some modules 1377On large strings containing lots of high Unicode characters, some modules
1302(such as JSON::PC) seem to decode faster than JSON::XS, but the result 1378(such as JSON::PC) seem to decode faster than JSON::XS, but the result
1303will be broken due to missing (or wrong) Unicode handling. Others refuse 1379will be broken due to missing (or wrong) Unicode handling. Others refuse
1304to decode or encode properly, so it was impossible to prepare a fair 1380to decode or encode properly, so it was impossible to prepare a fair
… …
1340information you might want to make sure that exceptions thrown by JSON::XS 1416information you might want to make sure that exceptions thrown by JSON::XS
1341will not end up in front of untrusted eyes. 1417will not end up in front of untrusted eyes.
1342 1418
1343If you are using JSON::XS to return packets to consumption 1419If you are using JSON::XS to return packets to consumption
1344by JavaScript scripts in a browser you should have a look at 1420by JavaScript scripts in a browser you should have a look at
1345L<http://jpsykes.com/47/practical-csrf-and-json-security> to see whether 1421L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to
1346you are vulnerable to some common attack vectors (which really are browser 1422see whether you are vulnerable to some common attack vectors (which really
1347design bugs, but it is still you who will have to deal with it, as major 1423are browser design bugs, but it is still you who will have to deal with
1348browser developers care only for features, not about getting security 1424it, as major browser developers care only for features, not about getting
1349right). 1425security right).
1350 1426
1351 1427
1352=head1 THREADS 1428=head1 THREADS
1353 1429
1354This module is I<not> guaranteed to be thread safe and there are no 1430This module is I<not> guaranteed to be thread safe and there are no

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines