[ViewVC] Diff of: cvs/JSON-XS/XS.pm

Comparing JSON-XS/XS.pm (file contents):
Revision 1.20 by root, Sun Mar 25 00:47:42 2007 UTC vs.
Revision 1.37 by root, Wed Jun 6 14:52:49 2007 UTC

…		…
4		4
5	=head1 SYNOPSIS	5	=head1 SYNOPSIS
6		6
7	use JSON::XS;	7	use JSON::XS;
8		8
9	# exported functions, croak on error	9	# exported functions, they croak on error
		10	# and expect/generate UTF-8
10		11
11	$utf8_encoded_json_text = to_json $perl_hash_or_arrayref;	12	$utf8_encoded_json_text = to_json $perl_hash_or_arrayref;
12	$perl_hash_or_arrayref = from_json $utf8_encoded_json_text;	13	$perl_hash_or_arrayref = from_json $utf8_encoded_json_text;
13		14
		15	# objToJson and jsonToObj aliases to to_json and from_json
		16	# are exported for compatibility to the JSON module,
		17	# but should not be used in new code.
		18
14	# oo-interface	19	# OO-interface
15		20
16	$coder = JSON::XS->new->ascii->pretty->allow_nonref;	21	$coder = JSON::XS->new->ascii->pretty->allow_nonref;
17	$pretty_printed_unencoded = $coder->encode ($perl_scalar);	22	$pretty_printed_unencoded = $coder->encode ($perl_scalar);
18	$perl_scalar = $coder->decode ($unicode_json_text);	23	$perl_scalar = $coder->decode ($unicode_json_text);
19		24
…		…
36		41
37	=head2 FEATURES	42	=head2 FEATURES
38		43
39	=over 4	44	=over 4
40		45
41	=item * correct handling of unicode issues	46	=item * correct unicode handling
42		47
43	This module knows how to handle Unicode, and even documents how and when	48	This module knows how to handle Unicode, and even documents how and when
44	it does so.	49	it does so.
45		50
46	=item * round-trip integrity	51	=item * round-trip integrity
47		52
48	When you serialise a perl data structure using only datatypes supported	53	When you serialise a perl data structure using only datatypes supported
49	by JSON, the deserialised data structure is identical on the Perl level.	54	by JSON, the deserialised data structure is identical on the Perl level.
50	(e.g. the string "2.0" doesn't suddenly become "2").	55	(e.g. the string "2.0" doesn't suddenly become "2" just because it looks
		56	like a number).
51		57
52	=item * strict checking of JSON correctness	58	=item * strict checking of JSON correctness
53		59
54	There is no guessing, no generating of illegal JSON texts by default,	60	There is no guessing, no generating of illegal JSON texts by default,
55	and only JSON is accepted as input by default (the latter is a security	61	and only JSON is accepted as input by default (the latter is a security
…		…
66	interface.	72	interface.
67		73
68	=item * reasonably versatile output formats	74	=item * reasonably versatile output formats
69		75
70	You can choose between the most compact guarenteed single-line format	76	You can choose between the most compact guarenteed single-line format
71	possible (nice for simple line-based protocols), a pure-ascii format (for	77	possible (nice for simple line-based protocols), a pure-ascii format
72	when your transport is not 8-bit clean), or a pretty-printed format (for	78	(for when your transport is not 8-bit clean, still supports the whole
73	when you want to read that stuff). Or you can combine those features in	79	unicode range), or a pretty-printed format (for when you want to read that
74	whatever way you like.	80	stuff). Or you can combine those features in whatever way you like.
75		81
76	=back	82	=back
77		83
78	=cut	84	=cut
79		85
80	package JSON::XS;	86	package JSON::XS;
81		87
82	use strict;	88	use strict;
83		89
84	BEGIN {	90	BEGIN {
85	our $VERSION = '0.7';	91	our $VERSION = '1.22';
86	our @ISA = qw(Exporter);	92	our @ISA = qw(Exporter);
87		93
88	our @EXPORT = qw(to_json from_json);	94	our @EXPORT = qw(to_json from_json objToJson jsonToObj);
89	require Exporter;	95	require Exporter;
90		96
91	require XSLoader;	97	require XSLoader;
92	XSLoader::load JSON::XS::, $VERSION;	98	XSLoader::load JSON::XS::, $VERSION;
93	}	99	}
…		…
122	$perl_scalar = JSON::XS->new->utf8->decode ($json_text)	128	$perl_scalar = JSON::XS->new->utf8->decode ($json_text)
123		129
124	except being faster.	130	except being faster.
125		131
126	=back	132	=back
		133
127		134
128	=head1 OBJECT-ORIENTED INTERFACE	135	=head1 OBJECT-ORIENTED INTERFACE
129		136
130	The object oriented interface lets you configure your own encoding or	137	The object oriented interface lets you configure your own encoding or
131	decoding style, within the limits of supported formats.	138	decoding style, within the limits of supported formats.
…		…
147		154
148	If C<$enable> is true (or missing), then the C<encode> method will not	155	If C<$enable> is true (or missing), then the C<encode> method will not
149	generate characters outside the code range C<0..127> (which is ASCII). Any	156	generate characters outside the code range C<0..127> (which is ASCII). Any
150	unicode characters outside that range will be escaped using either a	157	unicode characters outside that range will be escaped using either a
151	single \uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence,	158	single \uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence,
152	as per RFC4627.	159	as per RFC4627. The resulting encoded JSON text can be treated as a native
		160	unicode string, an ascii-encoded, latin1-encoded or UTF-8 encoded string,
		161	or any other superset of ASCII.
153		162
154	If C<$enable> is false, then the C<encode> method will not escape Unicode	163	If C<$enable> is false, then the C<encode> method will not escape Unicode
155	characters unless required by the JSON syntax. This results in a faster	164	characters unless required by the JSON syntax or other flags. This results
156	and more compact format.	165	in a faster and more compact format.
		166
		167	The main use for this flag is to produce JSON texts that can be
		168	transmitted over a 7-bit channel, as the encoded JSON texts will not
		169	contain any 8 bit characters.
157		170
158	JSON::XS->new->ascii (1)->encode ([chr 0x10401])	171	JSON::XS->new->ascii (1)->encode ([chr 0x10401])
159	=> ["\ud801\udc01"]	172	=> ["\ud801\udc01"]
		173
		174	=item $json = $json->latin1 ([$enable])
		175
		176	If C<$enable> is true (or missing), then the C<encode> method will encode
		177	the resulting JSON text as latin1 (or iso-8859-1), escaping any characters
		178	outside the code range C<0..255>. The resulting string can be treated as a
		179	latin1-encoded JSON text or a native unicode string. The C<decode> method
		180	will not be affected in any way by this flag, as C<decode> by default
		181	expects unicode, which is a strict superset of latin1.
		182
		183	If C<$enable> is false, then the C<encode> method will not escape Unicode
		184	characters unless required by the JSON syntax or other flags.
		185
		186	The main use for this flag is efficiently encoding binary data as JSON
		187	text, as most octets will not be escaped, resulting in a smaller encoded
		188	size. The disadvantage is that the resulting JSON text is encoded
		189	in latin1 (and must correctly be treated as such when storing and
		190	transfering), a rare encoding for JSON. It is therefore most useful when
		191	you want to store data structures known to contain binary data efficiently
		192	in files or databases, not when talking to other JSON encoders/decoders.
		193
		194	JSON::XS->new->latin1->encode (["\x{89}\x{abc}"]
		195	=> ["\x{89}\\u0abc"] # (perl syntax, U+abc escaped, U+89 not)
160		196
161	=item $json = $json->utf8 ([$enable])	197	=item $json = $json->utf8 ([$enable])
162		198
163	If C<$enable> is true (or missing), then the C<encode> method will encode	199	If C<$enable> is true (or missing), then the C<encode> method will encode
164	the JSON result into UTF-8, as required by many protocols, while the	200	the JSON result into UTF-8, as required by many protocols, while the
…		…
277	=> "Hello, World!"	313	=> "Hello, World!"
278		314
279	=item $json = $json->shrink ([$enable])	315	=item $json = $json->shrink ([$enable])
280		316
281	Perl usually over-allocates memory a bit when allocating space for	317	Perl usually over-allocates memory a bit when allocating space for
282	strings. This flag optionally resizes strings generated by either	318	strings. This flag optionally resizes strings generated by either
283	C<encode> or C<decode> to their minimum size possible. This can save	319	C<encode> or C<decode> to their minimum size possible. This can save
284	memory when your JSON texts are either very very long or you have many	320	memory when your JSON texts are either very very long or you have many
285	short strings. It will also try to downgrade any strings to octet-form	321	short strings. It will also try to downgrade any strings to octet-form
286	if possible: perl stores strings internally either in an encoding called	322	if possible: perl stores strings internally either in an encoding called
287	UTF-X or in octet-form. The latter cannot store everything but uses less	323	UTF-X or in octet-form. The latter cannot store everything but uses less
288	space in general.	324	space in general (and some buggy Perl or C code might even rely on that
		325	internal representation being used).
289		326
		327	The actual definition of what shrink does might change in future versions,
		328	but it will always try to save space at the expense of time.
		329
290	If C<$enable> is true (or missing), the string returned by C<encode> will be shrunk-to-fit,	330	If C<$enable> is true (or missing), the string returned by C<encode> will
291	while all strings generated by C<decode> will also be shrunk-to-fit.	331	be shrunk-to-fit, while all strings generated by C<decode> will also be
		332	shrunk-to-fit.
292		333
293	If C<$enable> is false, then the normal perl allocation algorithms are used.	334	If C<$enable> is false, then the normal perl allocation algorithms are used.
294	If you work with your data, then this is likely to be faster.	335	If you work with your data, then this is likely to be faster.
295		336
296	In the future, this setting might control other things, such as converting	337	In the future, this setting might control other things, such as converting
297	strings that look like integers or floats into integers or floats	338	strings that look like integers or floats into integers or floats
298	internally (there is no difference on the Perl level), saving space.	339	internally (there is no difference on the Perl level), saving space.
		340
		341	=item $json = $json->max_depth ([$maximum_nesting_depth])
		342
		343	Sets the maximum nesting level (default C<512>) accepted while encoding
		344	or decoding. If the JSON text or Perl data structure has an equal or
		345	higher nesting level then this limit, then the encoder and decoder will
		346	stop and croak at that point.
		347
		348	Nesting level is defined by number of hash- or arrayrefs that the encoder
		349	needs to traverse to reach a given point or the number of C<{> or C<[>
		350	characters without their matching closing parenthesis crossed to reach a
		351	given character in a string.
		352
		353	Setting the maximum depth to one disallows any nesting, so that ensures
		354	that the object is only a single hash/object or array.
		355
		356	The argument to C<max_depth> will be rounded up to the next nearest power
		357	of two.
		358
		359	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
299		360
300	=item $json_text = $json->encode ($perl_scalar)	361	=item $json_text = $json->encode ($perl_scalar)
301		362
302	Converts the given Perl data structure (a simple scalar or a reference	363	Converts the given Perl data structure (a simple scalar or a reference
303	to a hash or array) to its JSON representation. Simple scalars will be	364	to a hash or array) to its JSON representation. Simple scalars will be
…		…
313		374
314	JSON numbers and strings become simple Perl scalars. JSON arrays become	375	JSON numbers and strings become simple Perl scalars. JSON arrays become
315	Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes	376	Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
316	C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.	377	C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.
317		378
		379	=item ($perl_scalar, $characters) = $json->decode_prefix ($json_text)
		380
		381	This works like the C<decode> method, but instead of raising an exception
		382	when there is trailing garbage after the first JSON object, it will
		383	silently stop parsing there and return the number of characters consumed
		384	so far.
		385
		386	This is useful if your JSON texts are not delimited by an outer protocol
		387	(which is not the brightest thing to do in the first place) and you need
		388	to know where the JSON text ends.
		389
		390	JSON::XS->new->decode_prefix ("[1] the tail")
		391	=> ([], 3)
		392
318	=back	393	=back
		394
319		395
320	=head1 MAPPING	396	=head1 MAPPING
321		397
322	This section describes how JSON::XS maps Perl values to JSON values and	398	This section describes how JSON::XS maps Perl values to JSON values and
323	vice versa. These mappings are designed to "do the right thing" in most	399	vice versa. These mappings are designed to "do the right thing" in most
…		…
377	=over 4	453	=over 4
378		454
379	=item hash references	455	=item hash references
380		456
381	Perl hash references become JSON objects. As there is no inherent ordering	457	Perl hash references become JSON objects. As there is no inherent ordering
382	in hash keys, they will usually be encoded in a pseudo-random order that	458	in hash keys (or JSON objects), they will usually be encoded in a
383	can change between runs of the same program but stays generally the same	459	pseudo-random order that can change between runs of the same program but
384	within a single run of a program. JSON::XS can optionally sort the hash	460	stays generally the same within a single run of a program. JSON::XS can
385	keys (determined by the I<canonical> flag), so the same datastructure	461	optionally sort the hash keys (determined by the I<canonical> flag), so
386	will serialise to the same JSON text (given same settings and version of	462	the same datastructure will serialise to the same JSON text (given same
387	JSON::XS), but this incurs a runtime overhead.	463	settings and version of JSON::XS), but this incurs a runtime overhead
		464	and is only rarely useful, e.g. when you want to compare some JSON text
		465	against another for equality.
388		466
389	=item array references	467	=item array references
390		468
391	Perl array references become JSON arrays.	469	Perl array references become JSON arrays.
		470
		471	=item other references
		472
		473	Other unblessed references are generally not allowed and will cause an
		474	exception to be thrown, except for references to the integers C<0> and
		475	C<1>, which get turned into C<false> and C<true> atoms in JSON. You can
		476	also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
		477
		478	to_json [\0,JSON::XS::true] # yields [false,true]
392		479
393	=item blessed objects	480	=item blessed objects
394		481
395	Blessed objects are not allowed. JSON::XS currently tries to encode their	482	Blessed objects are not allowed. JSON::XS currently tries to encode their
396	underlying representation (hash- or arrayref), but this behaviour might	483	underlying representation (hash- or arrayref), but this behaviour might
…		…
429	$x *= 1; # same thing, the choise is yours.	516	$x *= 1; # same thing, the choise is yours.
430		517
431	You can not currently output JSON booleans or force the type in other,	518	You can not currently output JSON booleans or force the type in other,
432	less obscure, ways. Tell me if you need this capability.	519	less obscure, ways. Tell me if you need this capability.
433		520
434	=item circular data structures
435
436	Those will be encoded until memory or stackspace runs out.
437
438	=back	521	=back
		522
439		523
440	=head1 COMPARISON	524	=head1 COMPARISON
441		525
442	As already mentioned, this module was created because none of the existing	526	As already mentioned, this module was created because none of the existing
443	JSON modules could be made to work correctly. First I will describe the	527	JSON modules could be made to work correctly. First I will describe the
…		…
528	It seems that JSON::XS is surprisingly fast, as shown in the following	612	It seems that JSON::XS is surprisingly fast, as shown in the following
529	tables. They have been generated with the help of the C<eg/bench> program	613	tables. They have been generated with the help of the C<eg/bench> program
530	in the JSON::XS distribution, to make it easy to compare on your own	614	in the JSON::XS distribution, to make it easy to compare on your own
531	system.	615	system.
532		616
533	First comes a comparison between various modules using a very short JSON	617	First comes a comparison between various modules using a very short
534	string:	618	single-line JSON string:
535		619
536	{"method": "handleMessage", "params": ["user1", "we were just talking"], "id": null}	620	{"method": "handleMessage", "params": ["user1", "we were just talking"], \
		621	"id": null, [1,11,234,-5,1e5,1e7, true, false]}
537		622
538	It shows the number of encodes/decodes per second (JSON::XS uses the	623	It shows the number of encodes/decodes per second (JSON::XS uses the
539	functional interface, while JSON::XS/2 uses the OO interface with	624	functional interface, while JSON::XS/2 uses the OO interface with
540	pretty-printing and hashkey sorting enabled). Higher is better:	625	pretty-printing and hashkey sorting enabled). Higher is better:
541		626
		627	-----------+------------+------------+
542	module \| encode \| decode \|	628	module \| encode \| decode \|
543	-----------\|------------\|------------\|	629	-----------\|------------\|------------\|
544	JSON \| 11488.516 \| 7823.035 \|	630	JSON \| 10597.029 \| 5740.903 \|
545	JSON::DWIW \| 94708.054 \| 129094.260 \|	631	JSON::DWIW \| 78251.940 \| 98457.840 \|
546	JSON::PC \| 63884.157 \| 128528.212 \|	632	JSON::PC \| 70611.178 \| 92794.336 \|
547	JSON::Syck \| 34898.677 \| 42096.911 \|	633	JSON::Syck \| 28767.517 \| 38199.490 \|
548	JSON::XS \| 654027.064 \| 396423.669 \|	634	JSON::XS \| 419430.400 \| 265462.278 \|
549	JSON::XS/2 \| 371564.190 \| 371725.613 \|	635	JSON::XS/2 \| 279620.267 \| 265462.278 \|
		636	JSON::XS/3 \| 388361.481 \| 265462.278 \|
		637	Storable \| 16294.887 \| 16844.594 \|
550	-----------+------------+------------+	638	-----------+------------+------------+
551		639
552	That is, JSON::XS is more than six times faster than JSON::DWIW on	640	That is, JSON::XS is about five times faster than JSON::DWIW on encoding,
553	encoding, more than three times faster on decoding, and about thirty times	641	about three times faster on decoding, and about fourty times faster
554	faster than JSON, even with pretty-printing and key sorting.	642	than JSON, even with pretty-printing and key sorting. It also compares
		643	favourably to Storable for small amounts of data.
555		644
556	Using a longer test string (roughly 18KB, generated from Yahoo! Locals	645	Using a longer test string (roughly 18KB, generated from Yahoo! Locals
557	search API (http://nanoref.com/yahooapis/mgPdGg):	646	search API (http://nanoref.com/yahooapis/mgPdGg):
558		647
559	module \| encode \| decode \|	648	module \| encode \| decode \|
560	-----------\|------------\|------------\|	649	-----------\|------------\|------------\|
561	JSON \| 273.023 \| 44.674 \|	650	JSON \| 254.685 \| 37.665 \|
562	JSON::DWIW \| 1089.383 \| 1145.704 \|	651	JSON::DWIW \| 1014.244 \| 1087.678 \|
563	JSON::PC \| 3097.419 \| 2393.921 \|	652	JSON::PC \| 3602.116 \| 2307.352 \|
564	JSON::Syck \| 514.060 \| 843.053 \|	653	JSON::Syck \| 558.035 \| 776.263 \|
565	JSON::XS \| 6479.668 \| 3636.364 \|	654	JSON::XS \| 5747.196 \| 3543.684 \|
566	JSON::XS/2 \| 3774.221 \| 3599.124 \|	655	JSON::XS/2 \| 3968.121 \| 3589.170 \|
		656	JSON::XS/3 \| 6105.246 \| 3561.134 \|
		657	Storable \| 4456.337 \| 5320.020 \|
567	-----------+------------+------------+	658	-----------+------------+------------+
568		659
569	Again, JSON::XS leads by far.	660	Again, JSON::XS leads by far.
570		661
571	On large strings containing lots of high unicode characters, some modules	662	On large strings containing lots of high unicode characters, some modules
572	(such as JSON::PC) seem to decode faster than JSON::XS, but the result	663	(such as JSON::PC) seem to decode faster than JSON::XS, but the result
573	will be broken due to missing (or wrong) unicode handling. Others refuse	664	will be broken due to missing (or wrong) unicode handling. Others refuse
574	to decode or encode properly, so it was impossible to prepare a fair	665	to decode or encode properly, so it was impossible to prepare a fair
575	comparison table for that case.	666	comparison table for that case.
576		667
577	=head1 RESOURCE LIMITS
578		668
579	JSON::XS does not impose any limits on the size of JSON texts or Perl	669	=head1 SECURITY CONSIDERATIONS
580	values they represent - if your machine can handle it, JSON::XS will	670
581	encode or decode it. Future versions might optionally impose structure	671	When you are using JSON in a protocol, talking to untrusted potentially
582	depth and memory use resource limits.	672	hostile creatures requires relatively few measures.
		673
		674	First of all, your JSON decoder should be secure, that is, should not have
		675	any buffer overflows. Obviously, this module should ensure that and I am
		676	trying hard on making that true, but you never know.
		677
		678	Second, you need to avoid resource-starving attacks. That means you should
		679	limit the size of JSON texts you accept, or make sure then when your
		680	resources run out, thats just fine (e.g. by using a separate process that
		681	can crash safely). The size of a JSON text in octets or characters is
		682	usually a good indication of the size of the resources required to decode
		683	it into a Perl structure.
		684
		685	Third, JSON::XS recurses using the C stack when decoding objects and
		686	arrays. The C stack is a limited resource: for instance, on my amd64
		687	machine with 8MB of stack size I can decode around 180k nested arrays but
		688	only 14k nested JSON objects (due to perl itself recursing deeply on croak
		689	to free the temporary). If that is exceeded, the program crashes. to be
		690	conservative, the default nesting limit is set to 512. If your process
		691	has a smaller stack, you should adjust this setting accordingly with the
		692	C<max_depth> method.
		693
		694	And last but least, something else could bomb you that I forgot to think
		695	of. In that case, you get to keep the pieces. I am always open for hints,
		696	though...
		697
583		698
584	=head1 BUGS	699	=head1 BUGS
585		700
586	While the goal of this module is to be correct, that unfortunately does	701	While the goal of this module is to be correct, that unfortunately does
587	not mean its bug-free, only that I think its design is bug-free. It is	702	not mean its bug-free, only that I think its design is bug-free. It is
588	still very young and not well-tested. If you keep reporting bugs they will	703	still relatively early in its development. If you keep reporting bugs they
589	be fixed swiftly, though.	704	will be fixed swiftly, though.
590		705
591	=cut	706	=cut
		707
		708	sub true() { \1 }
		709	sub false() { \0 }
592		710
593	1;	711	1;
594		712
595	=head1 AUTHOR	713	=head1 AUTHOR
596		714

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing JSON-XS/XS.pm (file contents): Revision 1.20 by root, Sun Mar 25 00:47:42 2007 UTC vs. Revision 1.37 by root, Wed Jun 6 14:52:49 2007 UTC

Diff Legend

Comparing JSON-XS/XS.pm (file contents):
Revision 1.20 by root, Sun Mar 25 00:47:42 2007 UTC vs.
Revision 1.37 by root, Wed Jun 6 14:52:49 2007 UTC