[ViewVC] Diff of: cvs/JSON-XS/XS.pm

Comparing JSON-XS/XS.pm (file contents):
Revision 1.21 by root, Sun Mar 25 02:32:40 2007 UTC vs.
Revision 1.41 by root, Mon Jun 11 03:45:26 2007 UTC

…		…
4		4
5	=head1 SYNOPSIS	5	=head1 SYNOPSIS
6		6
7	use JSON::XS;	7	use JSON::XS;
8		8
9	# exported functions, croak on error	9	# exported functions, they croak on error
		10	# and expect/generate UTF-8
10		11
11	$utf8_encoded_json_text = to_json $perl_hash_or_arrayref;	12	$utf8_encoded_json_text = to_json $perl_hash_or_arrayref;
12	$perl_hash_or_arrayref = from_json $utf8_encoded_json_text;	13	$perl_hash_or_arrayref = from_json $utf8_encoded_json_text;
13		14
14	# objToJson and jsonToObj are exported for JSON	15	# objToJson and jsonToObj aliases to to_json and from_json
		16	# are exported for compatibility to the JSON module,
15	# compatibility, but should not be used in new code.	17	# but should not be used in new code.
16		18
17	# oo-interface	19	# OO-interface
18		20
19	$coder = JSON::XS->new->ascii->pretty->allow_nonref;	21	$coder = JSON::XS->new->ascii->pretty->allow_nonref;
20	$pretty_printed_unencoded = $coder->encode ($perl_scalar);	22	$pretty_printed_unencoded = $coder->encode ($perl_scalar);
21	$perl_scalar = $coder->decode ($unicode_json_text);	23	$perl_scalar = $coder->decode ($unicode_json_text);
22		24
…		…
84	package JSON::XS;	86	package JSON::XS;
85		87
86	use strict;	88	use strict;
87		89
88	BEGIN {	90	BEGIN {
89	our $VERSION = '0.8';	91	our $VERSION = '1.24';
90	our @ISA = qw(Exporter);	92	our @ISA = qw(Exporter);
91		93
92	our @EXPORT = qw(to_json from_json objToJson jsonToObj);	94	our @EXPORT = qw(to_json from_json objToJson jsonToObj);
93	require Exporter;	95	require Exporter;
94		96
…		…
126	$perl_scalar = JSON::XS->new->utf8->decode ($json_text)	128	$perl_scalar = JSON::XS->new->utf8->decode ($json_text)
127		129
128	except being faster.	130	except being faster.
129		131
130	=back	132	=back
		133
131		134
132	=head1 OBJECT-ORIENTED INTERFACE	135	=head1 OBJECT-ORIENTED INTERFACE
133		136
134	The object oriented interface lets you configure your own encoding or	137	The object oriented interface lets you configure your own encoding or
135	decoding style, within the limits of supported formats.	138	decoding style, within the limits of supported formats.
…		…
151		154
152	If C<$enable> is true (or missing), then the C<encode> method will not	155	If C<$enable> is true (or missing), then the C<encode> method will not
153	generate characters outside the code range C<0..127> (which is ASCII). Any	156	generate characters outside the code range C<0..127> (which is ASCII). Any
154	unicode characters outside that range will be escaped using either a	157	unicode characters outside that range will be escaped using either a
155	single \uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence,	158	single \uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence,
156	as per RFC4627.	159	as per RFC4627. The resulting encoded JSON text can be treated as a native
		160	unicode string, an ascii-encoded, latin1-encoded or UTF-8 encoded string,
		161	or any other superset of ASCII.
157		162
158	If C<$enable> is false, then the C<encode> method will not escape Unicode	163	If C<$enable> is false, then the C<encode> method will not escape Unicode
159	characters unless required by the JSON syntax. This results in a faster	164	characters unless required by the JSON syntax or other flags. This results
160	and more compact format.	165	in a faster and more compact format.
		166
		167	The main use for this flag is to produce JSON texts that can be
		168	transmitted over a 7-bit channel, as the encoded JSON texts will not
		169	contain any 8 bit characters.
161		170
162	JSON::XS->new->ascii (1)->encode ([chr 0x10401])	171	JSON::XS->new->ascii (1)->encode ([chr 0x10401])
163	=> ["\ud801\udc01"]	172	=> ["\ud801\udc01"]
		173
		174	=item $json = $json->latin1 ([$enable])
		175
		176	If C<$enable> is true (or missing), then the C<encode> method will encode
		177	the resulting JSON text as latin1 (or iso-8859-1), escaping any characters
		178	outside the code range C<0..255>. The resulting string can be treated as a
		179	latin1-encoded JSON text or a native unicode string. The C<decode> method
		180	will not be affected in any way by this flag, as C<decode> by default
		181	expects unicode, which is a strict superset of latin1.
		182
		183	If C<$enable> is false, then the C<encode> method will not escape Unicode
		184	characters unless required by the JSON syntax or other flags.
		185
		186	The main use for this flag is efficiently encoding binary data as JSON
		187	text, as most octets will not be escaped, resulting in a smaller encoded
		188	size. The disadvantage is that the resulting JSON text is encoded
		189	in latin1 (and must correctly be treated as such when storing and
		190	transfering), a rare encoding for JSON. It is therefore most useful when
		191	you want to store data structures known to contain binary data efficiently
		192	in files or databases, not when talking to other JSON encoders/decoders.
		193
		194	JSON::XS->new->latin1->encode (["\x{89}\x{abc}"]
		195	=> ["\x{89}\\u0abc"] # (perl syntax, U+abc escaped, U+89 not)
164		196
165	=item $json = $json->utf8 ([$enable])	197	=item $json = $json->utf8 ([$enable])
166		198
167	If C<$enable> is true (or missing), then the C<encode> method will encode	199	If C<$enable> is true (or missing), then the C<encode> method will encode
168	the JSON result into UTF-8, as required by many protocols, while the	200	the JSON result into UTF-8, as required by many protocols, while the
…		…
281	=> "Hello, World!"	313	=> "Hello, World!"
282		314
283	=item $json = $json->shrink ([$enable])	315	=item $json = $json->shrink ([$enable])
284		316
285	Perl usually over-allocates memory a bit when allocating space for	317	Perl usually over-allocates memory a bit when allocating space for
286	strings. This flag optionally resizes strings generated by either	318	strings. This flag optionally resizes strings generated by either
287	C<encode> or C<decode> to their minimum size possible. This can save	319	C<encode> or C<decode> to their minimum size possible. This can save
288	memory when your JSON texts are either very very long or you have many	320	memory when your JSON texts are either very very long or you have many
289	short strings. It will also try to downgrade any strings to octet-form	321	short strings. It will also try to downgrade any strings to octet-form
290	if possible: perl stores strings internally either in an encoding called	322	if possible: perl stores strings internally either in an encoding called
291	UTF-X or in octet-form. The latter cannot store everything but uses less	323	UTF-X or in octet-form. The latter cannot store everything but uses less
292	space in general.	324	space in general (and some buggy Perl or C code might even rely on that
		325	internal representation being used).
293		326
		327	The actual definition of what shrink does might change in future versions,
		328	but it will always try to save space at the expense of time.
		329
294	If C<$enable> is true (or missing), the string returned by C<encode> will be shrunk-to-fit,	330	If C<$enable> is true (or missing), the string returned by C<encode> will
295	while all strings generated by C<decode> will also be shrunk-to-fit.	331	be shrunk-to-fit, while all strings generated by C<decode> will also be
		332	shrunk-to-fit.
296		333
297	If C<$enable> is false, then the normal perl allocation algorithms are used.	334	If C<$enable> is false, then the normal perl allocation algorithms are used.
298	If you work with your data, then this is likely to be faster.	335	If you work with your data, then this is likely to be faster.
299		336
300	In the future, this setting might control other things, such as converting	337	In the future, this setting might control other things, such as converting
301	strings that look like integers or floats into integers or floats	338	strings that look like integers or floats into integers or floats
302	internally (there is no difference on the Perl level), saving space.	339	internally (there is no difference on the Perl level), saving space.
		340
		341	=item $json = $json->max_depth ([$maximum_nesting_depth])
		342
		343	Sets the maximum nesting level (default C<512>) accepted while encoding
		344	or decoding. If the JSON text or Perl data structure has an equal or
		345	higher nesting level then this limit, then the encoder and decoder will
		346	stop and croak at that point.
		347
		348	Nesting level is defined by number of hash- or arrayrefs that the encoder
		349	needs to traverse to reach a given point or the number of C<{> or C<[>
		350	characters without their matching closing parenthesis crossed to reach a
		351	given character in a string.
		352
		353	Setting the maximum depth to one disallows any nesting, so that ensures
		354	that the object is only a single hash/object or array.
		355
		356	The argument to C<max_depth> will be rounded up to the next nearest power
		357	of two.
		358
		359	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
303		360
304	=item $json_text = $json->encode ($perl_scalar)	361	=item $json_text = $json->encode ($perl_scalar)
305		362
306	Converts the given Perl data structure (a simple scalar or a reference	363	Converts the given Perl data structure (a simple scalar or a reference
307	to a hash or array) to its JSON representation. Simple scalars will be	364	to a hash or array) to its JSON representation. Simple scalars will be
…		…
317		374
318	JSON numbers and strings become simple Perl scalars. JSON arrays become	375	JSON numbers and strings become simple Perl scalars. JSON arrays become
319	Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes	376	Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
320	C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.	377	C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.
321		378
		379	=item ($perl_scalar, $characters) = $json->decode_prefix ($json_text)
		380
		381	This works like the C<decode> method, but instead of raising an exception
		382	when there is trailing garbage after the first JSON object, it will
		383	silently stop parsing there and return the number of characters consumed
		384	so far.
		385
		386	This is useful if your JSON texts are not delimited by an outer protocol
		387	(which is not the brightest thing to do in the first place) and you need
		388	to know where the JSON text ends.
		389
		390	JSON::XS->new->decode_prefix ("[1] the tail")
		391	=> ([], 3)
		392
322	=back	393	=back
		394
323		395
324	=head1 MAPPING	396	=head1 MAPPING
325		397
326	This section describes how JSON::XS maps Perl values to JSON values and	398	This section describes how JSON::XS maps Perl values to JSON values and
327	vice versa. These mappings are designed to "do the right thing" in most	399	vice versa. These mappings are designed to "do the right thing" in most
…		…
329	(what you put in comes out as something equivalent).	401	(what you put in comes out as something equivalent).
330		402
331	For the more enlightened: note that in the following descriptions,	403	For the more enlightened: note that in the following descriptions,
332	lowercase I<perl> refers to the Perl interpreter, while uppcercase I<Perl>	404	lowercase I<perl> refers to the Perl interpreter, while uppcercase I<Perl>
333	refers to the abstract Perl language itself.	405	refers to the abstract Perl language itself.
		406
334		407
335	=head2 JSON -> PERL	408	=head2 JSON -> PERL
336		409
337	=over 4	410	=over 4
338		411
…		…
370		443
371	A JSON null atom becomes C<undef> in Perl.	444	A JSON null atom becomes C<undef> in Perl.
372		445
373	=back	446	=back
374		447
		448
375	=head2 PERL -> JSON	449	=head2 PERL -> JSON
376		450
377	The mapping from Perl to JSON is slightly more difficult, as Perl is a	451	The mapping from Perl to JSON is slightly more difficult, as Perl is a
378	truly typeless language, so we can only guess which JSON type is meant by	452	truly typeless language, so we can only guess which JSON type is meant by
379	a Perl value.	453	a Perl value.
…		…
381	=over 4	455	=over 4
382		456
383	=item hash references	457	=item hash references
384		458
385	Perl hash references become JSON objects. As there is no inherent ordering	459	Perl hash references become JSON objects. As there is no inherent ordering
386	in hash keys, they will usually be encoded in a pseudo-random order that	460	in hash keys (or JSON objects), they will usually be encoded in a
387	can change between runs of the same program but stays generally the same	461	pseudo-random order that can change between runs of the same program but
388	within a single run of a program. JSON::XS can optionally sort the hash	462	stays generally the same within a single run of a program. JSON::XS can
389	keys (determined by the I<canonical> flag), so the same datastructure	463	optionally sort the hash keys (determined by the I<canonical> flag), so
390	will serialise to the same JSON text (given same settings and version of	464	the same datastructure will serialise to the same JSON text (given same
391	JSON::XS), but this incurs a runtime overhead.	465	settings and version of JSON::XS), but this incurs a runtime overhead
		466	and is only rarely useful, e.g. when you want to compare some JSON text
		467	against another for equality.
392		468
393	=item array references	469	=item array references
394		470
395	Perl array references become JSON arrays.	471	Perl array references become JSON arrays.
		472
		473	=item other references
		474
		475	Other unblessed references are generally not allowed and will cause an
		476	exception to be thrown, except for references to the integers C<0> and
		477	C<1>, which get turned into C<false> and C<true> atoms in JSON. You can
		478	also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
		479
		480	to_json [\0,JSON::XS::true] # yields [false,true]
396		481
397	=item blessed objects	482	=item blessed objects
398		483
399	Blessed objects are not allowed. JSON::XS currently tries to encode their	484	Blessed objects are not allowed. JSON::XS currently tries to encode their
400	underlying representation (hash- or arrayref), but this behaviour might	485	underlying representation (hash- or arrayref), but this behaviour might
…		…
433	$x *= 1; # same thing, the choise is yours.	518	$x *= 1; # same thing, the choise is yours.
434		519
435	You can not currently output JSON booleans or force the type in other,	520	You can not currently output JSON booleans or force the type in other,
436	less obscure, ways. Tell me if you need this capability.	521	less obscure, ways. Tell me if you need this capability.
437		522
438	=item circular data structures
439
440	Those will be encoded until memory or stackspace runs out.
441
442	=back	523	=back
		524
443		525
444	=head1 COMPARISON	526	=head1 COMPARISON
445		527
446	As already mentioned, this module was created because none of the existing	528	As already mentioned, this module was created because none of the existing
447	JSON modules could be made to work correctly. First I will describe the	529	JSON modules could be made to work correctly. First I will describe the
…		…
525		607
526	Does not check input for validity.	608	Does not check input for validity.
527		609
528	=back	610	=back
529		611
		612
		613	=head2 JSON and YAML
		614
		615	You often hear that JSON is a subset (or a close subset) of YAML. This is,
		616	however, a mass hysteria and very far from the truth. In general, there is
		617	no way to configure JSON::XS to output a data structure as valid YAML.
		618
		619	If you really must use JSON::XS to generate YAML, you should use this
		620	algorithm (subject to change in future versions):
		621
		622	my $to_yaml = JSON::XS->new->utf8->space_after (1);
		623	my $yaml = $to_yaml->encode ($ref) . "\n";
		624
		625	This will usually generate JSON texts that also parse as valid
		626	YAML. Please note that YAML has hardcoded limits on (simple) object key
		627	lengths that JSON doesn't have, so you should make sure that your hash
		628	keys are noticably shorter than the 1024 characters YAML allows.
		629
		630	There might be other incompatibilities that I am not aware of. In general
		631	you should not try to generate YAML with a JSON generator or vice versa,
		632	or try to parse JSON with a YAML parser or vice versa: chances are high
		633	that you will run into severe interoperability problems.
		634
		635
530	=head2 SPEED	636	=head2 SPEED
531		637
532	It seems that JSON::XS is surprisingly fast, as shown in the following	638	It seems that JSON::XS is surprisingly fast, as shown in the following
533	tables. They have been generated with the help of the C<eg/bench> program	639	tables. They have been generated with the help of the C<eg/bench> program
534	in the JSON::XS distribution, to make it easy to compare on your own	640	in the JSON::XS distribution, to make it easy to compare on your own
535	system.	641	system.
536		642
537	First comes a comparison between various modules using a very short JSON	643	First comes a comparison between various modules using a very short
538	string:	644	single-line JSON string:
539		645
540	{"method": "handleMessage", "params": ["user1", "we were just talking"], "id": null}	646	{"method": "handleMessage", "params": ["user1", "we were just talking"], \
		647	"id": null, "array":[1,11,234,-5,1e5,1e7, true, false]}
541		648
542	It shows the number of encodes/decodes per second (JSON::XS uses the	649	It shows the number of encodes/decodes per second (JSON::XS uses
543	functional interface, while JSON::XS/2 uses the OO interface with	650	the functional interface, while JSON::XS/2 uses the OO interface
544	pretty-printing and hashkey sorting enabled). Higher is better:	651	with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
		652	shrink). Higher is better:
545		653
546	module \| encode \| decode \|	654	module \| encode \| decode \|
547	-----------\|------------\|------------\|	655	-----------\|------------\|------------\|
548	JSON \| 11488.516 \| 7823.035 \|	656	JSON \| 7645.468 \| 4208.613 \|
549	JSON::DWIW \| 94708.054 \| 129094.260 \|	657	JSON::DWIW \| 40721.398 \| 77101.176 \|
550	JSON::PC \| 63884.157 \| 128528.212 \|	658	JSON::PC \| 65948.176 \| 78251.940 \|
551	JSON::Syck \| 34898.677 \| 42096.911 \|	659	JSON::Syck \| 22844.793 \| 26479.192 \|
552	JSON::XS \| 654027.064 \| 396423.669 \|	660	JSON::XS \| 388361.481 \| 199728.762 \|
553	JSON::XS/2 \| 371564.190 \| 371725.613 \|	661	JSON::XS/2 \| 218453.333 \| 192399.266 \|
		662	JSON::XS/3 \| 338250.323 \| 192399.266 \|
		663	Storable \| 15779.925 \| 14169.946 \|
554	-----------+------------+------------+	664	-----------+------------+------------+
555		665
556	That is, JSON::XS is more than six times faster than JSON::DWIW on	666	That is, JSON::XS is about five times faster than JSON::DWIW on encoding,
557	encoding, more than three times faster on decoding, and about thirty times	667	about three times faster on decoding, and over fourty times faster
558	faster than JSON, even with pretty-printing and key sorting.	668	than JSON, even with pretty-printing and key sorting. It also compares
		669	favourably to Storable for small amounts of data.
559		670
560	Using a longer test string (roughly 18KB, generated from Yahoo! Locals	671	Using a longer test string (roughly 18KB, generated from Yahoo! Locals
561	search API (http://nanoref.com/yahooapis/mgPdGg):	672	search API (http://nanoref.com/yahooapis/mgPdGg):
562		673
563	module \| encode \| decode \|	674	module \| encode \| decode \|
564	-----------\|------------\|------------\|	675	-----------\|------------\|------------\|
565	JSON \| 273.023 \| 44.674 \|	676	JSON \| 254.685 \| 37.665 \|
566	JSON::DWIW \| 1089.383 \| 1145.704 \|	677	JSON::DWIW \| 843.343 \| 1049.731 \|
567	JSON::PC \| 3097.419 \| 2393.921 \|	678	JSON::PC \| 3602.116 \| 2307.352 \|
568	JSON::Syck \| 514.060 \| 843.053 \|	679	JSON::Syck \| 505.107 \| 787.899 \|
569	JSON::XS \| 6479.668 \| 3636.364 \|	680	JSON::XS \| 5747.196 \| 3690.220 \|
570	JSON::XS/2 \| 3774.221 \| 3599.124 \|	681	JSON::XS/2 \| 3968.121 \| 3676.634 \|
		682	JSON::XS/3 \| 6105.246 \| 3662.508 \|
		683	Storable \| 4417.337 \| 5285.161 \|
571	-----------+------------+------------+	684	-----------+------------+------------+
572		685
573	Again, JSON::XS leads by far.	686	Again, JSON::XS leads by far (except for Storable which non-surprisingly
		687	decodes faster).
574		688
575	On large strings containing lots of high unicode characters, some modules	689	On large strings containing lots of high unicode characters, some modules
576	(such as JSON::PC) seem to decode faster than JSON::XS, but the result	690	(such as JSON::PC) seem to decode faster than JSON::XS, but the result
577	will be broken due to missing (or wrong) unicode handling. Others refuse	691	will be broken due to missing (or wrong) unicode handling. Others refuse
578	to decode or encode properly, so it was impossible to prepare a fair	692	to decode or encode properly, so it was impossible to prepare a fair
579	comparison table for that case.	693	comparison table for that case.
580		694
581	=head1 RESOURCE LIMITS
582		695
583	JSON::XS does not impose any limits on the size of JSON texts or Perl	696	=head1 SECURITY CONSIDERATIONS
584	values they represent - if your machine can handle it, JSON::XS will	697
585	encode or decode it. Future versions might optionally impose structure	698	When you are using JSON in a protocol, talking to untrusted potentially
586	depth and memory use resource limits.	699	hostile creatures requires relatively few measures.
		700
		701	First of all, your JSON decoder should be secure, that is, should not have
		702	any buffer overflows. Obviously, this module should ensure that and I am
		703	trying hard on making that true, but you never know.
		704
		705	Second, you need to avoid resource-starving attacks. That means you should
		706	limit the size of JSON texts you accept, or make sure then when your
		707	resources run out, thats just fine (e.g. by using a separate process that
		708	can crash safely). The size of a JSON text in octets or characters is
		709	usually a good indication of the size of the resources required to decode
		710	it into a Perl structure.
		711
		712	Third, JSON::XS recurses using the C stack when decoding objects and
		713	arrays. The C stack is a limited resource: for instance, on my amd64
		714	machine with 8MB of stack size I can decode around 180k nested arrays but
		715	only 14k nested JSON objects (due to perl itself recursing deeply on croak
		716	to free the temporary). If that is exceeded, the program crashes. to be
		717	conservative, the default nesting limit is set to 512. If your process
		718	has a smaller stack, you should adjust this setting accordingly with the
		719	C<max_depth> method.
		720
		721	And last but least, something else could bomb you that I forgot to think
		722	of. In that case, you get to keep the pieces. I am always open for hints,
		723	though...
		724
587		725
588	=head1 BUGS	726	=head1 BUGS
589		727
590	While the goal of this module is to be correct, that unfortunately does	728	While the goal of this module is to be correct, that unfortunately does
591	not mean its bug-free, only that I think its design is bug-free. It is	729	not mean its bug-free, only that I think its design is bug-free. It is
592	still very young and not well-tested. If you keep reporting bugs they will	730	still relatively early in its development. If you keep reporting bugs they
593	be fixed swiftly, though.	731	will be fixed swiftly, though.
594		732
595	=cut	733	=cut
		734
		735	sub true() { \1 }
		736	sub false() { \0 }
596		737
597	1;	738	1;
598		739
599	=head1 AUTHOR	740	=head1 AUTHOR
600		741

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing JSON-XS/XS.pm (file contents): Revision 1.21 by root, Sun Mar 25 02:32:40 2007 UTC vs. Revision 1.41 by root, Mon Jun 11 03:45:26 2007 UTC

Diff Legend

Comparing JSON-XS/XS.pm (file contents):
Revision 1.21 by root, Sun Mar 25 02:32:40 2007 UTC vs.
Revision 1.41 by root, Mon Jun 11 03:45:26 2007 UTC