[ViewVC] Diff of: cvs/JSON-XS/XS.pm

Comparing JSON-XS/XS.pm (file contents):
Revision 1.105 by root, Sat May 24 21:55:43 2008 UTC vs.
Revision 1.116 by root, Tue Feb 17 23:41:20 2009 UTC

…		…
49	to write yet another JSON module? While it seems there are many JSON	49	to write yet another JSON module? While it seems there are many JSON
50	modules, none of them correctly handle all corner cases, and in most cases	50	modules, none of them correctly handle all corner cases, and in most cases
51	their maintainers are unresponsive, gone missing, or not listening to bug	51	their maintainers are unresponsive, gone missing, or not listening to bug
52	reports for other reasons.	52	reports for other reasons.
53		53
54	See COMPARISON, below, for a comparison to some other JSON modules.
55
56	See MAPPING, below, on how JSON::XS maps perl values to JSON values and	54	See MAPPING, below, on how JSON::XS maps perl values to JSON values and
57	vice versa.	55	vice versa.
58		56
59	=head2 FEATURES	57	=head2 FEATURES
60		58
…		…
101		99
102	=cut	100	=cut
103		101
104	package JSON::XS;	102	package JSON::XS;
105		103
		104	no warnings;
106	use strict;	105	use strict;
107		106
108	our $VERSION = '2.2';	107	our $VERSION = '2.232';
109	our @ISA = qw(Exporter);	108	our @ISA = qw(Exporter);
110		109
111	our @EXPORT = qw(encode_json decode_json to_json from_json);	110	our @EXPORT = qw(encode_json decode_json to_json from_json);
112		111
113	sub to_json($) {	112	sub to_json($) {
…		…
706	In some cases, there is the need for incremental parsing of JSON	705	In some cases, there is the need for incremental parsing of JSON
707	texts. While this module always has to keep both JSON text and resulting	706	texts. While this module always has to keep both JSON text and resulting
708	Perl data structure in memory at one time, it does allow you to parse a	707	Perl data structure in memory at one time, it does allow you to parse a
709	JSON stream incrementally. It does so by accumulating text until it has	708	JSON stream incrementally. It does so by accumulating text until it has
710	a full JSON object, which it then can decode. This process is similar to	709	a full JSON object, which it then can decode. This process is similar to
711	using C<decode_prefix> to see if a full JSON object is available, but is	710	using C<decode_prefix> to see if a full JSON object is available, but
712	much more efficient (JSON::XS will only attempt to parse the JSON text	711	is much more efficient (and can be implemented with a minimum of method
		712	calls).
		713
		714	JSON::XS will only attempt to parse the JSON text once it is sure it
713	once it is sure it has enough text to get a decisive result, using a very	715	has enough text to get a decisive result, using a very simple but
714	simple but truly incremental parser).	716	truly incremental parser. This means that it sometimes won't stop as
		717	early as the full parser, for example, it doesn't detect parenthese
		718	mismatches. The only thing it guarantees is that it starts decoding as
		719	soon as a syntactically valid JSON text has been seen. This means you need
		720	to set resource limits (e.g. C<max_size>) to ensure the parser will stop
		721	parsing in the presence if syntax errors.
715		722
716	The following two methods deal with this.	723	The following methods implement this incremental parser.
717		724
718	=over 4	725	=over 4
719		726
720	=item [void, scalar or list context] = $json->incr_parse ([$string])	727	=item [void, scalar or list context] = $json->incr_parse ([$string])
721		728
…		…
759	JSON object or b) parsing multiple JSON objects separated by non-JSON text	766	JSON object or b) parsing multiple JSON objects separated by non-JSON text
760	(such as commas).	767	(such as commas).
761		768
762	=item $json->incr_skip	769	=item $json->incr_skip
763		770
764	This will reset the state of the incremental parser and will remove the	771	This will reset the state of the incremental parser and will remove
765	parsed text from the input buffer. This is useful after C<incr_parse>	772	the parsed text from the input buffer so far. This is useful after
766	died, in which case the input buffer and incremental parser state is left	773	C<incr_parse> died, in which case the input buffer and incremental parser
767	unchanged, to skip the text parsed so far and to reset the parse state.	774	state is left unchanged, to skip the text parsed so far and to reset the
		775	parse state.
		776
		777	The difference to C<incr_reset> is that only text until the parse error
		778	occured is removed.
		779
		780	=item $json->incr_reset
		781
		782	This completely resets the incremental parser, that is, after this call,
		783	it will be as if the parser had never parsed anything.
		784
		785	This is useful if you want to repeatedly parse JSON objects and want to
		786	ignore any trailing data, which means you have to reset the parser after
		787	each successful decode.
768		788
769	=back	789	=back
770		790
771	=head2 LIMITATIONS	791	=head2 LIMITATIONS
772		792
…		…
1163	when your channel for JSON transfer is not 8-bit clean or the encoding	1183	when your channel for JSON transfer is not 8-bit clean or the encoding
1164	might be mangled in between (e.g. in mail), and works because ASCII is a	1184	might be mangled in between (e.g. in mail), and works because ASCII is a
1165	proper subset of most 8-bit and multibyte encodings in use in the world.	1185	proper subset of most 8-bit and multibyte encodings in use in the world.
1166		1186
1167	=back	1187	=back
		1188
		1189
		1190	=head2 JSON and ECMAscript
		1191
		1192	JSON syntax is based on how literals are represented in javascript (the
		1193	not-standardised predecessor of ECMAscript) which is presumably why it is
		1194	called "JavaScript Object Notation".
		1195
		1196	However, JSON is not a subset (and also not a superset of course) of
		1197	ECMAscript (the standard) or javascript (whatever browsers actually
		1198	implement).
		1199
		1200	If you want to use javascript's C<eval> function to "parse" JSON, you
		1201	might run into parse errors for valid JSON texts, or the resulting data
		1202	structure might not be queryable:
		1203
		1204	One of the problems is that U+2028 and U+2029 are valid characters inside
		1205	JSON strings, but are not allowed in ECMAscript string literals, so the
		1206	following Perl fragment will not output something that can be guaranteed
		1207	to be parsable by javascript's C<eval>:
		1208
		1209	use JSON::XS;
		1210
		1211	print encode_json [chr 0x2028];
		1212
		1213	The right fix for this is to use a proper JSON parser in your javascript
		1214	programs, and not rely on C<eval>.
		1215
		1216	If this is not an option, you can, as a stop-gap measure, simply encode to
		1217	ASCII-only JSON:
		1218
		1219	use JSON::XS;
		1220
		1221	print JSON::XS->new->ascii->encode ([chr 0x2028]);
		1222
		1223	And if you are concerned about the size of the resulting JSON text, you
		1224	can run some regexes to only escape U+2028 and U+2029:
		1225
		1226	use JSON::XS;
		1227
		1228	my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
		1229	$json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
		1230	$json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
		1231	print $json;
		1232
		1233	This works because U+2028/U+2029 are not allowed outside of strings and
		1234	are not used for syntax, so replacing them unconditionally just works.
		1235
		1236	Note, however, that fixing the broken JSON parser is better than working
		1237	around it in every other generator. The above regexes should work well in
		1238	other languages, as long as they operate on UTF-8. It is equally valid to
		1239	replace all occurences of U+2028/2029 directly by their \\u-escaped forms
		1240	in unicode texts, so they can simply be used to fix any parsers relying on
		1241	C<eval> by first applying the regexes on the encoded texts.
		1242
		1243	Note also that the above only works for U+2028 and U+2029 and thus
		1244	only for fully ECMAscript-compliant parsers. Many existing javascript
		1245	implementations misparse other characters as well. Best rely on a good
		1246	JSON parser, such as Douglas Crockfords F<json2.js>, which escapes the
		1247	above and many more problematic characters properly before passing them
		1248	into C<eval>.
		1249
		1250	Another problem is that some javascript implementations reserve
		1251	some property names for their own purposes (which probably makes
		1252	them non-ECMAscript-compliant). For example, Iceweasel reserves the
		1253	C<__proto__> property name for it's own purposes.
		1254
		1255	If that is a problem, you could parse try to filter the resulting JSON
		1256	output for these property strings, e.g.:
		1257
		1258	$json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
		1259
		1260	This works because C<__proto__> is not valid outside of strings, so every
		1261	occurence of C<"__proto__"\s*:> must be a string used as property name.
		1262
		1263	If you know of other incompatibilities, please let me know.
1168		1264
1169		1265
1170	=head2 JSON and YAML	1266	=head2 JSON and YAML
1171		1267
1172	You often hear that JSON is a subset of YAML. This is, however, a mass	1268	You often hear that JSON is a subset of YAML. This is, however, a mass

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing JSON-XS/XS.pm (file contents): Revision 1.105 by root, Sat May 24 21:55:43 2008 UTC vs. Revision 1.116 by root, Tue Feb 17 23:41:20 2009 UTC

Diff Legend

Comparing JSON-XS/XS.pm (file contents):
Revision 1.105 by root, Sat May 24 21:55:43 2008 UTC vs.
Revision 1.116 by root, Tue Feb 17 23:41:20 2009 UTC