[ViewVC] Diff of: cvs/JSON-XS/XS.pm

Comparing JSON-XS/XS.pm (file contents):
Revision 1.115 by root, Tue Feb 17 23:29:38 2009 UTC vs.
Revision 1.134 by root, Mon Feb 21 15:38:06 2011 UTC

…		…
64	so, and even documents what "correct" means.	64	so, and even documents what "correct" means.
65		65
66	=item * round-trip integrity	66	=item * round-trip integrity
67		67
68	When you serialise a perl data structure using only data types supported	68	When you serialise a perl data structure using only data types supported
69	by JSON, the deserialised data structure is identical on the Perl level.	69	by JSON and Perl, the deserialised data structure is identical on the Perl
70	(e.g. the string "2.0" doesn't suddenly become "2" just because it looks	70	level. (e.g. the string "2.0" doesn't suddenly become "2" just because
71	like a number). There minor I<are> exceptions to this, read the MAPPING	71	it looks like a number). There I<are> minor exceptions to this, read the
72	section below to learn about those.	72	MAPPING section below to learn about those.
73		73
74	=item * strict checking of JSON correctness	74	=item * strict checking of JSON correctness
75		75
76	There is no guessing, no generating of illegal JSON texts by default,	76	There is no guessing, no generating of illegal JSON texts by default,
77	and only JSON is accepted as input by default (the latter is a security	77	and only JSON is accepted as input by default (the latter is a security
…		…
99		99
100	=cut	100	=cut
101		101
102	package JSON::XS;	102	package JSON::XS;
103		103
104	no warnings;	104	use common::sense;
105	use strict;
106		105
107	our $VERSION = '2.232';	106	our $VERSION = '2.3';
108	our @ISA = qw(Exporter);	107	our @ISA = qw(Exporter);
109		108
110	our @EXPORT = qw(encode_json decode_json to_json from_json);	109	our @EXPORT = qw(encode_json decode_json to_json from_json);
111		110
112	sub to_json($) {	111	sub to_json($) {
…		…
441	the same JSON text (given the same overall settings). If it is disabled,	440	the same JSON text (given the same overall settings). If it is disabled,
442	the same hash might be encoded differently even if contains the same data,	441	the same hash might be encoded differently even if contains the same data,
443	as key-value pairs have no inherent ordering in Perl.	442	as key-value pairs have no inherent ordering in Perl.
444		443
445	This setting has no effect when decoding JSON texts.	444	This setting has no effect when decoding JSON texts.
		445
		446	This setting has currently no effect on tied hashes.
446		447
447	=item $json = $json->allow_nonref ([$enable])	448	=item $json = $json->allow_nonref ([$enable])
448		449
449	=item $enabled = $json->get_allow_nonref	450	=item $enabled = $json->get_allow_nonref
450		451
…		…
712	calls).	713	calls).
713		714
714	JSON::XS will only attempt to parse the JSON text once it is sure it	715	JSON::XS will only attempt to parse the JSON text once it is sure it
715	has enough text to get a decisive result, using a very simple but	716	has enough text to get a decisive result, using a very simple but
716	truly incremental parser. This means that it sometimes won't stop as	717	truly incremental parser. This means that it sometimes won't stop as
717	early as the full parser, for example, it doesn't detect parenthese	718	early as the full parser, for example, it doesn't detect mismatched
718	mismatches. The only thing it guarantees is that it starts decoding as	719	parentheses. The only thing it guarantees is that it starts decoding as
719	soon as a syntactically valid JSON text has been seen. This means you need	720	soon as a syntactically valid JSON text has been seen. This means you need
720	to set resource limits (e.g. C<max_size>) to ensure the parser will stop	721	to set resource limits (e.g. C<max_size>) to ensure the parser will stop
721	parsing in the presence if syntax errors.	722	parsing in the presence if syntax errors.
722		723
723	The following methods implement this incremental parser.	724	The following methods implement this incremental parser.
…		…
749	otherwise. For this to work, there must be no separators between the JSON	750	otherwise. For this to work, there must be no separators between the JSON
750	objects or arrays, instead they must be concatenated back-to-back. If	751	objects or arrays, instead they must be concatenated back-to-back. If
751	an error occurs, an exception will be raised as in the scalar context	752	an error occurs, an exception will be raised as in the scalar context
752	case. Note that in this case, any previously-parsed JSON texts will be	753	case. Note that in this case, any previously-parsed JSON texts will be
753	lost.	754	lost.
		755
		756	Example: Parse some JSON arrays/objects in a given string and return
		757	them.
		758
		759	my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
754		760
755	=item $lvalue_string = $json->incr_text	761	=item $lvalue_string = $json->incr_text
756		762
757	This method returns the currently stored JSON fragment as an lvalue, that	763	This method returns the currently stored JSON fragment as an lvalue, that
758	is, you can manipulate it. This I<only> works when a preceding call to	764	is, you can manipulate it. This I<only> works when a preceding call to
…		…
988	Numbers containing a fractional or exponential part will always be	994	Numbers containing a fractional or exponential part will always be
989	represented as numeric (floating point) values, possibly at a loss of	995	represented as numeric (floating point) values, possibly at a loss of
990	precision (in which case you might lose perfect roundtripping ability, but	996	precision (in which case you might lose perfect roundtripping ability, but
991	the JSON number will still be re-encoded as a JSON number).	997	the JSON number will still be re-encoded as a JSON number).
992		998
		999	Note that precision is not accuracy - binary floating point values cannot
		1000	represent most decimal fractions exactly, and when converting from and to
		1001	floating point, JSON::XS only guarantees precision up to but not including
		1002	the leats significant bit.
		1003
993	=item true, false	1004	=item true, false
994		1005
995	These JSON atoms become C<JSON::XS::true> and C<JSON::XS::false>,	1006	These JSON atoms become C<JSON::XS::true> and C<JSON::XS::false>,
996	respectively. They are overloaded to act almost exactly like the numbers	1007	respectively. They are overloaded to act almost exactly like the numbers
997	C<1> and C<0>. You can check whether a scalar is a JSON boolean by using	1008	C<1> and C<0>. You can check whether a scalar is a JSON boolean by using
…		…
1084		1095
1085	You can not currently force the type in other, less obscure, ways. Tell me	1096	You can not currently force the type in other, less obscure, ways. Tell me
1086	if you need this capability (but don't forget to explain why it's needed	1097	if you need this capability (but don't forget to explain why it's needed
1087	:).	1098	:).
1088		1099
		1100	Note that numerical precision has the same meaning as under Perl (so
		1101	binary to decimal conversion follows the same rules as in Perl, which
		1102	can differ to other languages). Also, your perl interpreter might expose
		1103	extensions to the floating point numbers of your platform, such as
		1104	infinities or NaN's - these cannot be represented in JSON, and it is an
		1105	error to pass those in.
		1106
1089	=back	1107	=back
1090		1108
1091		1109
1092	=head1 ENCODING/CODESET FLAG NOTES	1110	=head1 ENCODING/CODESET FLAG NOTES
1093		1111
…		…
1209	use JSON::XS;	1227	use JSON::XS;
1210		1228
1211	print encode_json [chr 0x2028];	1229	print encode_json [chr 0x2028];
1212		1230
1213	The right fix for this is to use a proper JSON parser in your javascript	1231	The right fix for this is to use a proper JSON parser in your javascript
1214	programs, and not rely on C<eval>.	1232	programs, and not rely on C<eval> (see for example Douglas Crockford's
		1233	F<json2.js> parser).
1215		1234
1216	If this is not an option, you can, as a stop-gap measure, simply encode to	1235	If this is not an option, you can, as a stop-gap measure, simply encode to
1217	ASCII-only JSON:	1236	ASCII-only JSON:
1218		1237
1219	use JSON::XS;	1238	use JSON::XS;
1220		1239
1221	print JSON::XS->new->ascii->encode ([chr 0x2028]);	1240	print JSON::XS->new->ascii->encode ([chr 0x2028]);
1222		1241
1223	And if you are concerned about the size of the resulting JSON text, you	1242	Note that this will enlarge the resulting JSON text quite a bit if you
1224	can run some regexes to only escape U+2028 and U+2029:	1243	have many non-ASCII characters. You might be tempted to run some regexes
		1244	to only escape U+2028 and U+2029, e.g.:
1225		1245
1226	use JSON::XS;	1246	# DO NOT USE THIS!
1227
1228	my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);	1247	my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1229	$json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028	1248	$json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1230	$json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029	1249	$json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1231	print $json;	1250	print $json;
1232		1251
1233	This works because U+2028/U+2029 are not allowed outside of strings and	1252	Note that I<this is a bad idea>: the above only works for U+2028 and
1234	are not used for syntax, so replacing them unconditionally just works.	1253	U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1235		1254	javascript implementations, however, have issues with other characters as
1236	Note, however, that fixing the broken JSON parser is better than working	1255	well - using C<eval> naively simply I<will> cause problems.
1237	around it in every other generator. The above regexes should work well in
1238	other languages, as long as they operate on UTF-8. It is equally valid to
1239	replace all occurences of U+2028/2029 directly by their \\u-escaped forms
1240	in unicode texts, so they can simply be used to fix any parsers relying on
1241	C<eval> by first applying the regexes on the encoded texts.
1242		1256
1243	Another problem is that some javascript implementations reserve	1257	Another problem is that some javascript implementations reserve
1244	some property names for their own purposes (which probably makes	1258	some property names for their own purposes (which probably makes
1245	them non-ECMAscript-compliant). For example, Iceweasel reserves the	1259	them non-ECMAscript-compliant). For example, Iceweasel reserves the
1246	C<__proto__> property name for it's own purposes.	1260	C<__proto__> property name for its own purposes.
1247		1261
1248	If that is a problem, you could parse try to filter the resulting JSON	1262	If that is a problem, you could parse try to filter the resulting JSON
1249	output for these property strings, e.g.:	1263	output for these property strings, e.g.:
1250		1264
1251	$json =~ s/"__proto__"\s*:/"__proto__renamed":/g;	1265	$json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
…		…
1271	my $yaml = $to_yaml->encode ($ref) . "\n";	1285	my $yaml = $to_yaml->encode ($ref) . "\n";
1272		1286
1273	This will I<usually> generate JSON texts that also parse as valid	1287	This will I<usually> generate JSON texts that also parse as valid
1274	YAML. Please note that YAML has hardcoded limits on (simple) object key	1288	YAML. Please note that YAML has hardcoded limits on (simple) object key
1275	lengths that JSON doesn't have and also has different and incompatible	1289	lengths that JSON doesn't have and also has different and incompatible
1276	unicode handling, so you should make sure that your hash keys are	1290	unicode character escape syntax, so you should make sure that your hash
1277	noticeably shorter than the 1024 "stream characters" YAML allows and that	1291	keys are noticeably shorter than the 1024 "stream characters" YAML allows
1278	you do not have characters with codepoint values outside the Unicode BMP	1292	and that you do not have characters with codepoint values outside the
1279	(basic multilingual page). YAML also does not allow C<\/> sequences in	1293	Unicode BMP (basic multilingual page). YAML also does not allow C<\/>
1280	strings (which JSON::XS does not I<currently> generate, but other JSON	1294	sequences in strings (which JSON::XS does not I<currently> generate, but
1281	generators might).	1295	other JSON generators might).
1282		1296
1283	There might be other incompatibilities that I am not aware of (or the YAML	1297	There might be other incompatibilities that I am not aware of (or the YAML
1284	specification has been changed yet again - it does so quite often). In	1298	specification has been changed yet again - it does so quite often). In
1285	general you should not try to generate YAML with a JSON generator or vice	1299	general you should not try to generate YAML with a JSON generator or vice
1286	versa, or try to parse JSON with a YAML parser or vice versa: chances are	1300	versa, or try to parse JSON with a YAML parser or vice versa: chances are
…		…
1305	that difficult or long) and finally make YAML compatible to it, and	1319	that difficult or long) and finally make YAML compatible to it, and
1306	educating users about the changes, instead of spreading lies about the	1320	educating users about the changes, instead of spreading lies about the
1307	real compatibility for many I<years> and trying to silence people who	1321	real compatibility for many I<years> and trying to silence people who
1308	point out that it isn't true.	1322	point out that it isn't true.
1309		1323
		1324	Addendum/2009: the YAML 1.2 spec is still incomaptible with JSON, even
		1325	though the incompatibilities have been documented (and are known to
		1326	Brian) for many years and the spec makes explicit claims that YAML is a
		1327	superset of JSON. It would be so easy to fix, but apparently, bullying and
		1328	corrupting userdata is so much easier.
		1329
1310	=back	1330	=back
1311		1331
1312		1332
1313	=head2 SPEED	1333	=head2 SPEED
1314		1334
…		…
1321	a very short single-line JSON string (also available at	1341	a very short single-line JSON string (also available at
1322	L<http://dist.schmorp.de/misc/json/short.json>).	1342	L<http://dist.schmorp.de/misc/json/short.json>).
1323		1343
1324	{"method": "handleMessage", "params": ["user1",	1344	{"method": "handleMessage", "params": ["user1",
1325	"we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,	1345	"we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1326	true, false]}	1346	1, 0]}
1327		1347
1328	It shows the number of encodes/decodes per second (JSON::XS uses	1348	It shows the number of encodes/decodes per second (JSON::XS uses
1329	the functional interface, while JSON::XS/2 uses the OO interface	1349	the functional interface, while JSON::XS/2 uses the OO interface
1330	with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables	1350	with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1331	shrink). Higher is better:	1351	shrink. JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ
		1352	uses the from_json method). Higher is better:
1332		1353
1333	module \| encode \| decode \|	1354	module \| encode \| decode \|
1334	-----------\|------------\|------------\|	1355	--------------\|------------\|------------\|
1335	JSON 1.x \| 4990.842 \| 4088.813 \|	1356	JSON::DWIW/DS \| 86302.551 \| 102300.098 \|
1336	JSON::DWIW \| 51653.990 \| 71575.154 \|	1357	JSON::DWIW/FJ \| 86302.551 \| 75983.768 \|
1337	JSON::PC \| 65948.176 \| 74631.744 \|	1358	JSON::PP \| 15827.562 \| 6638.658 \|
1338	JSON::PP \| 8931.652 \| 3817.168 \|	1359	JSON::Syck \| 63358.066 \| 47662.545 \|
1339	JSON::Syck \| 24877.248 \| 27776.848 \|	1360	JSON::XS \| 511500.488 \| 511500.488 \|
1340	JSON::XS \| 388361.481 \| 227951.304 \|	1361	JSON::XS/2 \| 291271.111 \| 388361.481 \|
1341	JSON::XS/2 \| 227951.304 \| 218453.333 \|	1362	JSON::XS/3 \| 361577.931 \| 361577.931 \|
1342	JSON::XS/3 \| 338250.323 \| 218453.333 \|	1363	Storable \| 66788.280 \| 265462.278 \|
1343	Storable \| 16500.016 \| 135300.129 \|
1344	-----------+------------+------------+	1364	--------------+------------+------------+
1345		1365
1346	That is, JSON::XS is about five times faster than JSON::DWIW on encoding,	1366	That is, JSON::XS is almost six times faster than JSON::DWIW on encoding,
1347	about three times faster on decoding, and over forty times faster	1367	about five times faster on decoding, and over thirty to seventy times
1348	than JSON, even with pretty-printing and key sorting. It also compares	1368	faster than JSON's pure perl implementation. It also compares favourably
1349	favourably to Storable for small amounts of data.	1369	to Storable for small amounts of data.
1350		1370
1351	Using a longer test string (roughly 18KB, generated from Yahoo! Locals	1371	Using a longer test string (roughly 18KB, generated from Yahoo! Locals
1352	search API (L<http://dist.schmorp.de/misc/json/long.json>).	1372	search API (L<http://dist.schmorp.de/misc/json/long.json>).
1353		1373
1354	module \| encode \| decode \|	1374	module \| encode \| decode \|
1355	-----------\|------------\|------------\|	1375	--------------\|------------\|------------\|
1356	JSON 1.x \| 55.260 \| 34.971 \|	1376	JSON::DWIW/DS \| 1647.927 \| 2673.916 \|
1357	JSON::DWIW \| 825.228 \| 1082.513 \|	1377	JSON::DWIW/FJ \| 1630.249 \| 2596.128 \|
1358	JSON::PC \| 3571.444 \| 2394.829 \|
1359	JSON::PP \| 210.987 \| 32.574 \|	1378	JSON::PP \| 400.640 \| 62.311 \|
1360	JSON::Syck \| 552.551 \| 787.544 \|	1379	JSON::Syck \| 1481.040 \| 1524.869 \|
1361	JSON::XS \| 5780.463 \| 4854.519 \|	1380	JSON::XS \| 20661.596 \| 9541.183 \|
1362	JSON::XS/2 \| 3869.998 \| 4798.975 \|	1381	JSON::XS/2 \| 10683.403 \| 9416.938 \|
1363	JSON::XS/3 \| 5862.880 \| 4798.975 \|	1382	JSON::XS/3 \| 20661.596 \| 9400.054 \|
1364	Storable \| 4445.002 \| 5235.027 \|	1383	Storable \| 19765.806 \| 10000.725 \|
1365	-----------+------------+------------+	1384	--------------+------------+------------+
1366		1385
1367	Again, JSON::XS leads by far (except for Storable which non-surprisingly	1386	Again, JSON::XS leads by far (except for Storable which non-surprisingly
1368	decodes faster).	1387	decodes a bit faster).
1369		1388
1370	On large strings containing lots of high Unicode characters, some modules	1389	On large strings containing lots of high Unicode characters, some modules
1371	(such as JSON::PC) seem to decode faster than JSON::XS, but the result	1390	(such as JSON::PC) seem to decode faster than JSON::XS, but the result
1372	will be broken due to missing (or wrong) Unicode handling. Others refuse	1391	will be broken due to missing (or wrong) Unicode handling. Others refuse
1373	to decode or encode properly, so it was impossible to prepare a fair	1392	to decode or encode properly, so it was impossible to prepare a fair
…		…
1409	information you might want to make sure that exceptions thrown by JSON::XS	1428	information you might want to make sure that exceptions thrown by JSON::XS
1410	will not end up in front of untrusted eyes.	1429	will not end up in front of untrusted eyes.
1411		1430
1412	If you are using JSON::XS to return packets to consumption	1431	If you are using JSON::XS to return packets to consumption
1413	by JavaScript scripts in a browser you should have a look at	1432	by JavaScript scripts in a browser you should have a look at
1414	L<http://jpsykes.com/47/practical-csrf-and-json-security> to see whether	1433	L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to
1415	you are vulnerable to some common attack vectors (which really are browser	1434	see whether you are vulnerable to some common attack vectors (which really
1416	design bugs, but it is still you who will have to deal with it, as major	1435	are browser design bugs, but it is still you who will have to deal with
1417	browser developers care only for features, not about getting security	1436	it, as major browser developers care only for features, not about getting
1418	right).	1437	security right).
1419		1438
1420		1439
1421	=head1 THREADS	1440	=head1 THREADS
1422		1441
1423	This module is I<not> guaranteed to be thread safe and there are no	1442	This module is I<not> guaranteed to be thread safe and there are no

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing JSON-XS/XS.pm (file contents): Revision 1.115 by root, Tue Feb 17 23:29:38 2009 UTC vs. Revision 1.134 by root, Mon Feb 21 15:38:06 2011 UTC

Diff Legend

Comparing JSON-XS/XS.pm (file contents):
Revision 1.115 by root, Tue Feb 17 23:29:38 2009 UTC vs.
Revision 1.134 by root, Mon Feb 21 15:38:06 2011 UTC