[ViewVC] Diff of: cvs/JSON-XS/XS.pm

Comparing JSON-XS/XS.pm (file contents):
Revision 1.44 by root, Mon Jun 25 04:08:17 2007 UTC vs.
Revision 1.63 by root, Thu Oct 11 23:07:43 2007 UTC

…		…
		1	=encoding utf-8
		2
1	=head1 NAME	3	=head1 NAME
2		4
3	JSON::XS - JSON serialising/deserialising, done correctly and fast	5	JSON::XS - JSON serialising/deserialising, done correctly and fast
		6
		7	JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ
		8	(http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html)
4		9
5	=head1 SYNOPSIS	10	=head1 SYNOPSIS
6		11
7	use JSON::XS;	12	use JSON::XS;
8		13
9	# exported functions, they croak on error	14	# exported functions, they croak on error
10	# and expect/generate UTF-8	15	# and expect/generate UTF-8
11		16
12	$utf8_encoded_json_text = to_json $perl_hash_or_arrayref;	17	$utf8_encoded_json_text = to_json $perl_hash_or_arrayref;
13	$perl_hash_or_arrayref = from_json $utf8_encoded_json_text;	18	$perl_hash_or_arrayref = from_json $utf8_encoded_json_text;
14
15	# objToJson and jsonToObj aliases to to_json and from_json
16	# are exported for compatibility to the JSON module,
17	# but should not be used in new code.
18		19
19	# OO-interface	20	# OO-interface
20		21
21	$coder = JSON::XS->new->ascii->pretty->allow_nonref;	22	$coder = JSON::XS->new->ascii->pretty->allow_nonref;
22	$pretty_printed_unencoded = $coder->encode ($perl_scalar);	23	$pretty_printed_unencoded = $coder->encode ($perl_scalar);
85		86
86	package JSON::XS;	87	package JSON::XS;
87		88
88	use strict;	89	use strict;
89		90
90	our $VERSION = '1.4';	91	our $VERSION = '1.5';
91	our @ISA = qw(Exporter);	92	our @ISA = qw(Exporter);
92		93
93	our @EXPORT = qw(to_json from_json objToJson jsonToObj);	94	our @EXPORT = qw(to_json from_json);
94		95
95	use Exporter;	96	use Exporter;
96	use XSLoader;	97	use XSLoader;
97		98
98	=head1 FUNCTIONAL INTERFACE	99	=head1 FUNCTIONAL INTERFACE
…		…
102		103
103	=over 4	104	=over 4
104		105
105	=item $json_text = to_json $perl_scalar	106	=item $json_text = to_json $perl_scalar
106		107
107	Converts the given Perl data structure (a simple scalar or a reference to	108	Converts the given Perl data structure to a UTF-8 encoded, binary string
108	a hash or array) to a UTF-8 encoded, binary string (that is, the string contains	109	(that is, the string contains octets only). Croaks on error.
109	octets only). Croaks on error.
110		110
111	This function call is functionally identical to:	111	This function call is functionally identical to:
112		112
113	$json_text = JSON::XS->new->utf8->encode ($perl_scalar)	113	$json_text = JSON::XS->new->utf8->encode ($perl_scalar)
114		114
115	except being faster.	115	except being faster.
116		116
117	=item $perl_scalar = from_json $json_text	117	=item $perl_scalar = from_json $json_text
118		118
119	The opposite of C<to_json>: expects an UTF-8 (binary) string and tries to	119	The opposite of C<to_json>: expects an UTF-8 (binary) string and tries
120	parse that as an UTF-8 encoded JSON text, returning the resulting simple	120	to parse that as an UTF-8 encoded JSON text, returning the resulting
121	scalar or reference. Croaks on error.	121	reference. Croaks on error.
122		122
123	This function call is functionally identical to:	123	This function call is functionally identical to:
124		124
125	$perl_scalar = JSON::XS->new->utf8->decode ($json_text)	125	$perl_scalar = JSON::XS->new->utf8->decode ($json_text)
126		126
…		…
134		134
135	See MAPPING, below, for more information on how JSON values are mapped to	135	See MAPPING, below, for more information on how JSON values are mapped to
136	Perl.	136	Perl.
137		137
138	=back	138	=back
		139
		140
		141	=head1 A FEW NOTES ON UNICODE AND PERL
		142
		143	Since this often leads to confusion, here are a few very clear words on
		144	how Unicode works in Perl, modulo bugs.
		145
		146	=over 4
		147
		148	=item 1. Perl strings can store characters with ordinal values > 255.
		149
		150	This enables you to store unicode characters as single characters in a
		151	Perl string - very natural.
		152
		153	=item 2. Perl does I<not> associate an encoding with your strings.
		154
		155	Unless you force it to, e.g. when matching it against a regex, or printing
		156	the scalar to a file, in which case Perl either interprets your string as
		157	locale-encoded text, octets/binary, or as Unicode, depending on various
		158	settings. In no case is an encoding stored together with your data, it is
		159	I<use> that decides encoding, not any magical metadata.
		160
		161	=item 3. The internal utf-8 flag has no meaning with regards to the
		162	encoding of your string.
		163
		164	Just ignore that flag unless you debug a Perl bug, a module written in
		165	XS or want to dive into the internals of perl. Otherwise it will only
		166	confuse you, as, despite the name, it says nothing about how your string
		167	is encoded. You can have unicode strings with that flag set, with that
		168	flag clear, and you can have binary data with that flag set and that flag
		169	clear. Other possibilities exist, too.
		170
		171	If you didn't know about that flag, just the better, pretend it doesn't
		172	exist.
		173
		174	=item 4. A "Unicode String" is simply a string where each character can be
		175	validly interpreted as a Unicode codepoint.
		176
		177	If you have UTF-8 encoded data, it is no longer a Unicode string, but a
		178	Unicode string encoded in UTF-8, giving you a binary string.
		179
		180	=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string.
		181
		182	Its a fact. Learn to live with it.
		183
		184	=back
		185
		186	I hope this helps :)
139		187
140		188
141	=head1 OBJECT-ORIENTED INTERFACE	189	=head1 OBJECT-ORIENTED INTERFACE
142		190
143	The object oriented interface lets you configure your own encoding or	191	The object oriented interface lets you configure your own encoding or
…		…
282		330
283	Example, space_before and indent disabled, space_after enabled:	331	Example, space_before and indent disabled, space_after enabled:
284		332
285	{"key": "value"}	333	{"key": "value"}
286		334
		335	=item $json = $json->relaxed ([$enable])
		336
		337	If C<$enable> is true (or missing), then C<decode> will accept some
		338	extensions to normal JSON syntax (see below). C<encode> will not be
		339	affected in anyway. I<Be aware that this option makes you accept invalid
		340	JSON texts as if they were valid!>. I suggest only to use this option to
		341	parse application-specific files written by humans (configuration files,
		342	resource files etc.)
		343
		344	If C<$enable> is false (the default), then C<decode> will only accept
		345	valid JSON texts.
		346
		347	Currently accepted extensions are:
		348
		349	=over 4
		350
		351	=item * list items can have an end-comma
		352
		353	JSON I<separates> array elements and key-value pairs with commas. This
		354	can be annoying if you write JSON texts manually and want to be able to
		355	quickly append elements, so this extension accepts comma at the end of
		356	such items not just between them:
		357
		358	[
		359	1,
		360	2, <- this comma not normally allowed
		361	]
		362	{
		363	"k1": "v1",
		364	"k2": "v2", <- this comma not normally allowed
		365	}
		366
		367	=item * shell-style '#'-comments
		368
		369	Whenever JSON allows whitespace, shell-style comments are additionally
		370	allowed. They are terminated by the first carriage-return or line-feed
		371	character, after which more white-space and comments are allowed.
		372
		373	[
		374	1, # this comment not allowed in JSON
		375	# neither this one...
		376	]
		377
		378	=back
		379
287	=item $json = $json->canonical ([$enable])	380	=item $json = $json->canonical ([$enable])
288		381
289	If C<$enable> is true (or missing), then the C<encode> method will output JSON objects	382	If C<$enable> is true (or missing), then the C<encode> method will output JSON objects
290	by sorting their keys. This is adding a comparatively high overhead.	383	by sorting their keys. This is adding a comparatively high overhead.
291		384
…		…
341		434
342	The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON>	435	The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON>
343	returns other blessed objects, those will be handled in the same	436	returns other blessed objects, those will be handled in the same
344	way. C<TO_JSON> must take care of not causing an endless recursion cycle	437	way. C<TO_JSON> must take care of not causing an endless recursion cycle
345	(== crash) in this case. The name of C<TO_JSON> was chosen because other	438	(== crash) in this case. The name of C<TO_JSON> was chosen because other
346	methods called by the Perl core (== not the user of the object) are	439	methods called by the Perl core (== not by the user of the object) are
347	usually in upper case letters and to avoid collisions with the C<to_json>	440	usually in upper case letters and to avoid collisions with the C<to_json>
348	function.	441	function.
349		442
		443	This setting does not yet influence C<decode> in any way, but in the
		444	future, global hooks might get installed that influence C<decode> and are
		445	enabled by this setting.
		446
350	If C<$enable> is false, then the C<allow_blessed> setting will decide what	447	If C<$enable> is false, then the C<allow_blessed> setting will decide what
351	to do when a blessed object is found.	448	to do when a blessed object is found.
		449
		450	=item $json = $json->filter_json_object ([$coderef->($hashref)])
		451
		452	When C<$coderef> is specified, it will be called from C<decode> each
		453	time it decodes a JSON object. The only argument is a reference to the
		454	newly-created hash. If the code references returns a single scalar (which
		455	need not be a reference), this value (i.e. a copy of that scalar to avoid
		456	aliasing) is inserted into the deserialised data structure. If it returns
		457	an empty list (NOTE: I<not> C<undef>, which is a valid scalar), the
		458	original deserialised hash will be inserted. This setting can slow down
		459	decoding considerably.
		460
		461	When C<$coderef> is omitted or undefined, any existing callback will
		462	be removed and C<decode> will not change the deserialised hash in any
		463	way.
		464
		465	Example, convert all JSON objects into the integer 5:
		466
		467	my $js = JSON::XS->new->filter_json_object (sub { 5 });
		468	# returns [5]
		469	$js->decode ('[{}]')
		470	# throw an exception because allow_nonref is not enabled
		471	# so a lone 5 is not allowed.
		472	$js->decode ('{"a":1, "b":2}');
		473
		474	=item $json = $json->filter_json_single_key_object ($key [=> $coderef->($value)])
		475
		476	Works remotely similar to C<filter_json_object>, but is only called for
		477	JSON objects having a single key named C<$key>.
		478
		479	This C<$coderef> is called before the one specified via
		480	C<filter_json_object>, if any. It gets passed the single value in the JSON
		481	object. If it returns a single value, it will be inserted into the data
		482	structure. If it returns nothing (not even C<undef> but the empty list),
		483	the callback from C<filter_json_object> will be called next, as if no
		484	single-key callback were specified.
		485
		486	If C<$coderef> is omitted or undefined, the corresponding callback will be
		487	disabled. There can only ever be one callback for a given key.
		488
		489	As this callback gets called less often then the C<filter_json_object>
		490	one, decoding speed will not usually suffer as much. Therefore, single-key
		491	objects make excellent targets to serialise Perl objects into, especially
		492	as single-key JSON objects are as close to the type-tagged value concept
		493	as JSON gets (its basically an ID/VALUE tuple). Of course, JSON does not
		494	support this in any way, so you need to make sure your data never looks
		495	like a serialised Perl hash.
		496
		497	Typical names for the single object key are C<__class_whatever__>, or
		498	C<$__dollars_are_rarely_used__$> or C<}ugly_brace_placement>, or even
		499	things like C<__class_md5sum(classname)__>, to reduce the risk of clashing
		500	with real hashes.
		501
		502	Example, decode JSON objects of the form C<< { "__widget__" => <id> } >>
		503	into the corresponding C<< $WIDGET{<id>} >> object:
		504
		505	# return whatever is in $WIDGET{5}:
		506	JSON::XS
		507	->new
		508	->filter_json_single_key_object (__widget__ => sub {
		509	$WIDGET{ $_[0] }
		510	})
		511	->decode ('{"__widget__": 5')
		512
		513	# this can be used with a TO_JSON method in some "widget" class
		514	# for serialisation to json:
		515	sub WidgetBase::TO_JSON {
		516	my ($self) = @_;
		517
		518	unless ($self->{id}) {
		519	$self->{id} = ..get..some..id..;
		520	$WIDGET{$self->{id}} = $self;
		521	}
		522
		523	{ __widget__ => $self->{id} }
		524	}
352		525
353	=item $json = $json->shrink ([$enable])	526	=item $json = $json->shrink ([$enable])
354		527
355	Perl usually over-allocates memory a bit when allocating space for	528	Perl usually over-allocates memory a bit when allocating space for
356	strings. This flag optionally resizes strings generated by either	529	strings. This flag optionally resizes strings generated by either
…		…
389	given character in a string.	562	given character in a string.
390		563
391	Setting the maximum depth to one disallows any nesting, so that ensures	564	Setting the maximum depth to one disallows any nesting, so that ensures
392	that the object is only a single hash/object or array.	565	that the object is only a single hash/object or array.
393		566
394	The argument to C<max_depth> will be rounded up to the next nearest power	567	The argument to C<max_depth> will be rounded up to the next highest power
395	of two.	568	of two. If no argument is given, the highest possible setting will be
		569	used, which is rarely useful.
		570
		571	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
		572
		573	=item $json = $json->max_size ([$maximum_string_size])
		574
		575	Set the maximum length a JSON text may have (in bytes) where decoding is
		576	being attempted. The default is C<0>, meaning no limit. When C<decode>
		577	is called on a string longer then this number of characters it will not
		578	attempt to decode the string but throw an exception. This setting has no
		579	effect on C<encode> (yet).
		580
		581	The argument to C<max_size> will be rounded up to the next B<highest>
		582	power of two (so may be more than requested). If no argument is given, the
		583	limit check will be deactivated (same as when C<0> is specified).
396		584
397	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.	585	See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
398		586
399	=item $json_text = $json->encode ($perl_scalar)	587	=item $json_text = $json->encode ($perl_scalar)
400		588
…		…
462	are represented by the same codepoints in the Perl string, so no manual	650	are represented by the same codepoints in the Perl string, so no manual
463	decoding is necessary.	651	decoding is necessary.
464		652
465	=item number	653	=item number
466		654
467	A JSON number becomes either an integer or numeric (floating point)	655	A JSON number becomes either an integer, numeric (floating point) or
468	scalar in perl, depending on its range and any fractional parts. On the	656	string scalar in perl, depending on its range and any fractional parts. On
469	Perl level, there is no difference between those as Perl handles all the	657	the Perl level, there is no difference between those as Perl handles all
470	conversion details, but an integer may take slightly less memory and might	658	the conversion details, but an integer may take slightly less memory and
471	represent more values exactly than (floating point) numbers.	659	might represent more values exactly than (floating point) numbers.
		660
		661	If the number consists of digits only, JSON::XS will try to represent
		662	it as an integer value. If that fails, it will try to represent it as
		663	a numeric (floating point) value if that is possible without loss of
		664	precision. Otherwise it will preserve the number as a string value.
		665
		666	Numbers containing a fractional or exponential part will always be
		667	represented as numeric (floating point) values, possibly at a loss of
		668	precision.
		669
		670	This might create round-tripping problems as numbers might become strings,
		671	but as Perl is typeless there is no other way to do it.
472		672
473	=item true, false	673	=item true, false
474		674
475	These JSON atoms become C<JSON::XS::true> and C<JSON::XS::false>,	675	These JSON atoms become C<JSON::XS::true> and C<JSON::XS::false>,
476	respectively. They are overloaded to act almost exactly like the numbers	676	respectively. They are overloaded to act almost exactly like the numbers
…		…
518	to_json [\0,JSON::XS::true] # yields [false,true]	718	to_json [\0,JSON::XS::true] # yields [false,true]
519		719
520	=item JSON::XS::true, JSON::XS::false	720	=item JSON::XS::true, JSON::XS::false
521		721
522	These special values become JSON true and JSON false values,	722	These special values become JSON true and JSON false values,
523	respectively. You cna alos use C<\1> and C<\0> directly if you want.	723	respectively. You can also use C<\1> and C<\0> directly if you want.
524		724
525	=item blessed objects	725	=item blessed objects
526		726
527	Blessed objects are not allowed. JSON::XS currently tries to encode their	727	Blessed objects are not allowed. JSON::XS currently tries to encode their
528	underlying representation (hash- or arrayref), but this behaviour might	728	underlying representation (hash- or arrayref), but this behaviour might
…		…
692	It shows the number of encodes/decodes per second (JSON::XS uses	892	It shows the number of encodes/decodes per second (JSON::XS uses
693	the functional interface, while JSON::XS/2 uses the OO interface	893	the functional interface, while JSON::XS/2 uses the OO interface
694	with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables	894	with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
695	shrink). Higher is better:	895	shrink). Higher is better:
696		896
		897	Storable \| 15779.925 \| 14169.946 \|
		898	-----------+------------+------------+
697	module \| encode \| decode \|	899	module \| encode \| decode \|
698	-----------\|------------\|------------\|	900	-----------\|------------\|------------\|
699	JSON \| 7645.468 \| 4208.613 \|	901	JSON \| 4990.842 \| 4088.813 \|
700	JSON::DWIW \| 40721.398 \| 77101.176 \|	902	JSON::DWIW \| 51653.990 \| 71575.154 \|
701	JSON::PC \| 65948.176 \| 78251.940 \|	903	JSON::PC \| 65948.176 \| 74631.744 \|
702	JSON::Syck \| 22844.793 \| 26479.192 \|	904	JSON::PP \| 8931.652 \| 3817.168 \|
		905	JSON::Syck \| 24877.248 \| 27776.848 \|
703	JSON::XS \| 388361.481 \| 199728.762 \|	906	JSON::XS \| 388361.481 \| 227951.304 \|
704	JSON::XS/2 \| 218453.333 \| 192399.266 \|	907	JSON::XS/2 \| 227951.304 \| 218453.333 \|
705	JSON::XS/3 \| 338250.323 \| 192399.266 \|	908	JSON::XS/3 \| 338250.323 \| 218453.333 \|
706	Storable \| 15779.925 \| 14169.946 \|	909	Storable \| 16500.016 \| 135300.129 \|
707	-----------+------------+------------+	910	-----------+------------+------------+
708		911
709	That is, JSON::XS is about five times faster than JSON::DWIW on encoding,	912	That is, JSON::XS is about five times faster than JSON::DWIW on encoding,
710	about three times faster on decoding, and over fourty times faster	913	about three times faster on decoding, and over fourty times faster
711	than JSON, even with pretty-printing and key sorting. It also compares	914	than JSON, even with pretty-printing and key sorting. It also compares
…		…
714	Using a longer test string (roughly 18KB, generated from Yahoo! Locals	917	Using a longer test string (roughly 18KB, generated from Yahoo! Locals
715	search API (http://nanoref.com/yahooapis/mgPdGg):	918	search API (http://nanoref.com/yahooapis/mgPdGg):
716		919
717	module \| encode \| decode \|	920	module \| encode \| decode \|
718	-----------\|------------\|------------\|	921	-----------\|------------\|------------\|
719	JSON \| 254.685 \| 37.665 \|	922	JSON \| 55.260 \| 34.971 \|
720	JSON::DWIW \| 843.343 \| 1049.731 \|	923	JSON::DWIW \| 825.228 \| 1082.513 \|
721	JSON::PC \| 3602.116 \| 2307.352 \|	924	JSON::PC \| 3571.444 \| 2394.829 \|
		925	JSON::PP \| 210.987 \| 32.574 \|
722	JSON::Syck \| 505.107 \| 787.899 \|	926	JSON::Syck \| 552.551 \| 787.544 \|
723	JSON::XS \| 5747.196 \| 3690.220 \|	927	JSON::XS \| 5780.463 \| 4854.519 \|
724	JSON::XS/2 \| 3968.121 \| 3676.634 \|	928	JSON::XS/2 \| 3869.998 \| 4798.975 \|
725	JSON::XS/3 \| 6105.246 \| 3662.508 \|	929	JSON::XS/3 \| 5862.880 \| 4798.975 \|
726	Storable \| 4417.337 \| 5285.161 \|	930	Storable \| 4445.002 \| 5235.027 \|
727	-----------+------------+------------+	931	-----------+------------+------------+
728		932
729	Again, JSON::XS leads by far (except for Storable which non-surprisingly	933	Again, JSON::XS leads by far (except for Storable which non-surprisingly
730	decodes faster).	934	decodes faster).
731		935
…		…
748	Second, you need to avoid resource-starving attacks. That means you should	952	Second, you need to avoid resource-starving attacks. That means you should
749	limit the size of JSON texts you accept, or make sure then when your	953	limit the size of JSON texts you accept, or make sure then when your
750	resources run out, thats just fine (e.g. by using a separate process that	954	resources run out, thats just fine (e.g. by using a separate process that
751	can crash safely). The size of a JSON text in octets or characters is	955	can crash safely). The size of a JSON text in octets or characters is
752	usually a good indication of the size of the resources required to decode	956	usually a good indication of the size of the resources required to decode
753	it into a Perl structure.	957	it into a Perl structure. While JSON::XS can check the size of the JSON
		958	text, it might be too late when you already have it in memory, so you
		959	might want to check the size before you accept the string.
754		960
755	Third, JSON::XS recurses using the C stack when decoding objects and	961	Third, JSON::XS recurses using the C stack when decoding objects and
756	arrays. The C stack is a limited resource: for instance, on my amd64	962	arrays. The C stack is a limited resource: for instance, on my amd64
757	machine with 8MB of stack size I can decode around 180k nested arrays but	963	machine with 8MB of stack size I can decode around 180k nested arrays but
758	only 14k nested JSON objects (due to perl itself recursing deeply on croak	964	only 14k nested JSON objects (due to perl itself recursing deeply on croak

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing JSON-XS/XS.pm (file contents): Revision 1.44 by root, Mon Jun 25 04:08:17 2007 UTC vs. Revision 1.63 by root, Thu Oct 11 23:07:43 2007 UTC

Diff Legend

Comparing JSON-XS/XS.pm (file contents):
Revision 1.44 by root, Mon Jun 25 04:08:17 2007 UTC vs.
Revision 1.63 by root, Thu Oct 11 23:07:43 2007 UTC