[ViewVC] Diff of: cvs/JSON-XS/XS.pm

Comparing JSON-XS/XS.pm (file contents):
Revision 1.1 by root, Thu Mar 22 16:40:16 2007 UTC vs.
Revision 1.10 by root, Fri Mar 23 17:40:29 2007 UTC

…		…
6		6
7	use JSON::XS;	7	use JSON::XS;
8		8
9	=head1 DESCRIPTION	9	=head1 DESCRIPTION
10		10
		11	This module converts Perl data structures to JSON and vice versa. Its
		12	primary goal is to be I<correct> and its secondary goal is to be
		13	I<fast>. To reach the latter goal it was written in C.
		14
		15	As this is the n-th-something JSON module on CPAN, what was the reason
		16	to write yet another JSON module? While it seems there are many JSON
		17	modules, none of them correctly handle all corner cases, and in most cases
		18	their maintainers are unresponsive, gone missing, or not listening to bug
		19	reports for other reasons.
		20
		21	See COMPARISON, below, for a comparison to some other JSON modules.
		22
		23	See MAPPING, below, on how JSON::XS maps perl values to JSON values and
		24	vice versa.
		25
		26	=head2 FEATURES
		27
11	=over 4	28	=over 4
12		29
		30	=item * correct handling of unicode issues
		31
		32	This module knows how to handle Unicode, and even documents how and when
		33	it does so.
		34
		35	=item * round-trip integrity
		36
		37	When you serialise a perl data structure using only datatypes supported
		38	by JSON, the deserialised data structure is identical on the Perl level.
		39	(e.g. the string "2.0" doesn't suddenly become "2").
		40
		41	=item * strict checking of JSON correctness
		42
		43	There is no guessing, no generating of illegal JSON strings by default,
		44	and only JSON is accepted as input by default (the latter is a security
		45	feature).
		46
		47	=item * fast
		48
		49	Compared to other JSON modules, this module compares favourably in terms
		50	of speed, too.
		51
		52	=item * simple to use
		53
		54	This module has both a simple functional interface as well as an OO
		55	interface.
		56
		57	=item * reasonably versatile output formats
		58
		59	You can choose between the most compact guarenteed single-line format
		60	possible (nice for simple line-based protocols), a pure-ascii format (for
		61	when your transport is not 8-bit clean), or a pretty-printed format (for
		62	when you want to read that stuff). Or you can combine those features in
		63	whatever way you like.
		64
		65	=back
		66
13	=cut	67	=cut
14		68
15	package JSON::XS;	69	package JSON::XS;
16		70
17	BEGIN {	71	BEGIN {
18	$VERSION = '0.1';	72	$VERSION = '0.3';
19	@ISA = qw(Exporter);	73	@ISA = qw(Exporter);
20		74
		75	@EXPORT = qw(to_json from_json);
21	require Exporter;	76	require Exporter;
22		77
23	require XSLoader;	78	require XSLoader;
24	XSLoader::load JSON::XS::, $VERSION;	79	XSLoader::load JSON::XS::, $VERSION;
25	}	80	}
26		81
27	=item	82	=head1 FUNCTIONAL INTERFACE
		83
		84	The following convinience methods are provided by this module. They are
		85	exported by default:
		86
		87	=over 4
		88
		89	=item $json_string = to_json $perl_scalar
		90
		91	Converts the given Perl data structure (a simple scalar or a reference to
		92	a hash or array) to a UTF-8 encoded, binary string (that is, the string contains
		93	octets only). Croaks on error.
		94
		95	This function call is functionally identical to C<< JSON::XS->new->utf8->encode ($perl_scalar) >>.
		96
		97	=item $perl_scalar = from_json $json_string
		98
		99	The opposite of C<to_json>: expects an UTF-8 (binary) string and tries to
		100	parse that as an UTF-8 encoded JSON string, returning the resulting simple
		101	scalar or reference. Croaks on error.
		102
		103	This function call is functionally identical to C<< JSON::XS->new->utf8->decode ($json_string) >>.
		104
		105	=back
		106
		107	=head1 OBJECT-ORIENTED INTERFACE
		108
		109	The object oriented interface lets you configure your own encoding or
		110	decoding style, within the limits of supported formats.
		111
		112	=over 4
		113
		114	=item $json = new JSON::XS
		115
		116	Creates a new JSON::XS object that can be used to de/encode JSON
		117	strings. All boolean flags described below are by default I<disabled>.
		118
		119	The mutators for flags all return the JSON object again and thus calls can
		120	be chained:
		121
		122	my $json = JSON::XS->new->utf8(1)->space_after(1)->encode ({a => [1,2]})
		123	=> {"a": [1, 2]}
		124
		125	=item $json = $json->ascii ([$enable])
		126
		127	If C<$enable> is true (or missing), then the C<encode> method will
		128	not generate characters outside the code range C<0..127>. Any unicode
		129	characters outside that range will be escaped using either a single
		130	\uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence, as per
		131	RFC4627.
		132
		133	If C<$enable> is false, then the C<encode> method will not escape Unicode
		134	characters unless necessary.
		135
		136	JSON::XS->new->ascii (1)->encode (chr 0x10401)
		137	=> \ud801\udc01
		138
		139	=item $json = $json->utf8 ([$enable])
		140
		141	If C<$enable> is true (or missing), then the C<encode> method will encode
		142	the JSON string into UTF-8, as required by many protocols, while the
		143	C<decode> method expects to be handled an UTF-8-encoded string. Please
		144	note that UTF-8-encoded strings do not contain any characters outside the
		145	range C<0..255>, they are thus useful for bytewise/binary I/O.
		146
		147	If C<$enable> is false, then the C<encode> method will return the JSON
		148	string as a (non-encoded) unicode string, while C<decode> expects thus a
		149	unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs
		150	to be done yourself, e.g. using the Encode module.
		151
		152	=item $json = $json->pretty ([$enable])
		153
		154	This enables (or disables) all of the C<indent>, C<space_before> and
		155	C<space_after> (and in the future possibly more) flags in one call to
		156	generate the most readable (or most compact) form possible.
		157
		158	my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
		159	=>
		160	{
		161	"a" : [
		162	1,
		163	2
		164	]
		165	}
		166
		167	=item $json = $json->indent ([$enable])
		168
		169	If C<$enable> is true (or missing), then the C<encode> method will use a multiline
		170	format as output, putting every array member or object/hash key-value pair
		171	into its own line, identing them properly.
		172
		173	If C<$enable> is false, no newlines or indenting will be produced, and the
		174	resulting JSON strings is guarenteed not to contain any C<newlines>.
		175
		176	This setting has no effect when decoding JSON strings.
		177
		178	=item $json = $json->space_before ([$enable])
		179
		180	If C<$enable> is true (or missing), then the C<encode> method will add an extra
		181	optional space before the C<:> separating keys from values in JSON objects.
		182
		183	If C<$enable> is false, then the C<encode> method will not add any extra
		184	space at those places.
		185
		186	This setting has no effect when decoding JSON strings. You will also most
		187	likely combine this setting with C<space_after>.
		188
		189	=item $json = $json->space_after ([$enable])
		190
		191	If C<$enable> is true (or missing), then the C<encode> method will add an extra
		192	optional space after the C<:> separating keys from values in JSON objects
		193	and extra whitespace after the C<,> separating key-value pairs and array
		194	members.
		195
		196	If C<$enable> is false, then the C<encode> method will not add any extra
		197	space at those places.
		198
		199	This setting has no effect when decoding JSON strings.
		200
		201	=item $json = $json->canonical ([$enable])
		202
		203	If C<$enable> is true (or missing), then the C<encode> method will output JSON objects
		204	by sorting their keys. This is adding a comparatively high overhead.
		205
		206	If C<$enable> is false, then the C<encode> method will output key-value
		207	pairs in the order Perl stores them (which will likely change between runs
		208	of the same script).
		209
		210	This option is useful if you want the same data structure to be encoded as
		211	the same JSON string (given the same overall settings). If it is disabled,
		212	the same hash migh be encoded differently even if contains the same data,
		213	as key-value pairs have no inherent ordering in Perl.
		214
		215	This setting has no effect when decoding JSON strings.
		216
		217	=item $json = $json->allow_nonref ([$enable])
		218
		219	If C<$enable> is true (or missing), then the C<encode> method can convert a
		220	non-reference into its corresponding string, number or null JSON value,
		221	which is an extension to RFC4627. Likewise, C<decode> will accept those JSON
		222	values instead of croaking.
		223
		224	If C<$enable> is false, then the C<encode> method will croak if it isn't
		225	passed an arrayref or hashref, as JSON strings must either be an object
		226	or array. Likewise, C<decode> will croak if given something that is not a
		227	JSON object or array.
		228
		229	=item $json = $json->shrink ([$enable])
		230
		231	Perl usually over-allocates memory a bit when allocating space for
		232	strings. This flag optionally resizes strings generated by either
		233	C<encode> or C<decode> to their minimum size possible. This can save
		234	memory when your JSON strings are either very very long or you have many
		235	short strings. It will also try to downgrade any strings to octet-form
		236	if possible: perl stores strings internally either in an encoding called
		237	UTF-X or in octet-form. The latter cannot store everything but uses less
		238	space in general.
		239
		240	If C<$enable> is true (or missing), the string returned by C<encode> will be shrunk-to-fit,
		241	while all strings generated by C<decode> will also be shrunk-to-fit.
		242
		243	If C<$enable> is false, then the normal perl allocation algorithms are used.
		244	If you work with your data, then this is likely to be faster.
		245
		246	In the future, this setting might control other things, such as converting
		247	strings that look like integers or floats into integers or floats
		248	internally (there is no difference on the Perl level), saving space.
		249
		250	=item $json_string = $json->encode ($perl_scalar)
		251
		252	Converts the given Perl data structure (a simple scalar or a reference
		253	to a hash or array) to its JSON representation. Simple scalars will be
		254	converted into JSON string or number sequences, while references to arrays
		255	become JSON arrays and references to hashes become JSON objects. Undefined
		256	Perl values (e.g. C<undef>) become JSON C<null> values. Neither C<true>
		257	nor C<false> values will be generated.
		258
		259	=item $perl_scalar = $json->decode ($json_string)
		260
		261	The opposite of C<encode>: expects a JSON string and tries to parse it,
		262	returning the resulting simple scalar or reference. Croaks on error.
		263
		264	JSON numbers and strings become simple Perl scalars. JSON arrays become
		265	Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
		266	C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.
		267
		268	=back
		269
		270	=head1 MAPPING
		271
		272	This section describes how JSON::XS maps Perl values to JSON values and
		273	vice versa. These mappings are designed to "do the right thing" in most
		274	circumstances automatically, preserving round-tripping characteristics
		275	(what you put in comes out as something equivalent).
		276
		277	For the more enlightened: note that in the following descriptions,
		278	lowercase I<perl> refers to the Perl interpreter, while uppcercase I<Perl>
		279	refers to the abstract Perl language itself.
		280
		281	=head2 JSON -> PERL
		282
		283	=over 4
		284
		285	=item object
		286
		287	A JSON object becomes a reference to a hash in Perl. No ordering of object
		288	keys is preserved.
		289
		290	=item array
		291
		292	A JSON array becomes a reference to an array in Perl.
		293
		294	=item string
		295
		296	A JSON string becomes a string scalar in Perl - Unicode codepoints in JSON
		297	are represented by the same codepoints in the Perl string, so no manual
		298	decoding is necessary.
		299
		300	=item number
		301
		302	A JSON number becomes either an integer or numeric (floating point)
		303	scalar in perl, depending on its range and any fractional parts. On the
		304	Perl level, there is no difference between those as Perl handles all the
		305	conversion details, but an integer may take slightly less memory and might
		306	represent more values exactly than (floating point) numbers.
		307
		308	=item true, false
		309
		310	These JSON atoms become C<0>, C<1>, respectively. Information is lost in
		311	this process. Future versions might represent those values differently,
		312	but they will be guarenteed to act like these integers would normally in
		313	Perl.
		314
		315	=item null
		316
		317	A JSON null atom becomes C<undef> in Perl.
		318
		319	=back
		320
		321	=head2 PERL -> JSON
		322
		323	The mapping from Perl to JSON is slightly more difficult, as Perl is a
		324	truly typeless language, so we can only guess which JSON type is meant by
		325	a Perl value.
		326
		327	=over 4
		328
		329	=item hash references
		330
		331	Perl hash references become JSON objects. As there is no inherent ordering
		332	in hash keys, they will usually be encoded in a pseudo-random order that
		333	can change between runs of the same program but stays generally the same
		334	within the single run of a program. JSON::XS can optionally sort the hash
		335	keys (determined by the I<canonical> flag), so the same datastructure
		336	will serialise to the same JSON text (given same settings and version of
		337	JSON::XS), but this incurs a runtime overhead.
		338
		339	=item array references
		340
		341	Perl array references become JSON arrays.
		342
		343	=item blessed objects
		344
		345	Blessed objects are not allowed. JSON::XS currently tries to encode their
		346	underlying representation (hash- or arrayref), but this behaviour might
		347	change in future versions.
		348
		349	=item simple scalars
		350
		351	Simple Perl scalars (any scalar that is not a reference) are the most
		352	difficult objects to encode: JSON::XS will encode undefined scalars as
		353	JSON null value, scalars that have last been used in a string context
		354	before encoding as JSON strings and anything else as number value:
		355
		356	# dump as number
		357	to_json [2] # yields [2]
		358	to_json [-3.0e17] # yields [-3e+17]
		359	my $value = 5; to_json [$value] # yields [5]
		360
		361	# used as string, so dump as string
		362	print $value;
		363	to_json [$value] # yields ["5"]
		364
		365	# undef becomes null
		366	to_json [undef] # yields [null]
		367
		368	You can force the type to be a string by stringifying it:
		369
		370	my $x = 3.1; # some variable containing a number
		371	"$x"; # stringified
		372	$x .= ""; # another, more awkward way to stringify
		373	print $x; # perl does it for you, too, quite often
		374
		375	You can force the type to be a number by numifying it:
		376
		377	my $x = "3"; # some variable containing a string
		378	$x += 0; # numify it, ensuring it will be dumped as a number
		379	$x *= 1; # same thing, the choise is yours.
		380
		381	You can not currently output JSON booleans or force the type in other,
		382	less obscure, ways. Tell me if you need this capability.
		383
		384	=back
		385
		386	=head1 COMPARISON
		387
		388	As already mentioned, this module was created because none of the existing
		389	JSON modules could be made to work correctly. First I will describe the
		390	problems (or pleasures) I encountered with various existing JSON modules,
		391	followed by some benchmark values. JSON::XS was designed not to suffer
		392	from any of these problems or limitations.
		393
		394	=over 4
		395
		396	=item JSON 1.07
		397
		398	Slow (but very portable, as it is written in pure Perl).
		399
		400	Undocumented/buggy Unicode handling (how JSON handles unicode values is
		401	undocumented. One can get far by feeding it unicode strings and doing
		402	en-/decoding oneself, but unicode escapes are not working properly).
		403
		404	No roundtripping (strings get clobbered if they look like numbers, e.g.
		405	the string C<2.0> will encode to C<2.0> instead of C<"2.0">, and that will
		406	decode into the number 2.
		407
		408	=item JSON::PC 0.01
		409
		410	Very fast.
		411
		412	Undocumented/buggy Unicode handling.
		413
		414	No roundtripping.
		415
		416	Has problems handling many Perl values (e.g. regex results and other magic
		417	values will make it croak).
		418
		419	Does not even generate valid JSON (C<{1,2}> gets converted to C<{1:2}>
		420	which is not a valid JSON string.
		421
		422	Unmaintained (maintainer unresponsive for many months, bugs are not
		423	getting fixed).
		424
		425	=item JSON::Syck 0.21
		426
		427	Very buggy (often crashes).
		428
		429	Very inflexible (no human-readable format supported, format pretty much
		430	undocumented. I need at least a format for easy reading by humans and a
		431	single-line compact format for use in a protocol, and preferably a way to
		432	generate ASCII-only JSON strings).
		433
		434	Completely broken (and confusingly documented) Unicode handling (unicode
		435	escapes are not working properly, you need to set ImplicitUnicode to
		436	I<different> values on en- and decoding to get symmetric behaviour).
		437
		438	No roundtripping (simple cases work, but this depends on wether the scalar
		439	value was used in a numeric context or not).
		440
		441	Dumping hashes may skip hash values depending on iterator state.
		442
		443	Unmaintained (maintainer unresponsive for many months, bugs are not
		444	getting fixed).
		445
		446	Does not check input for validity (i.e. will accept non-JSON input and
		447	return "something" instead of raising an exception. This is a security
		448	issue: imagine two banks transfering money between each other using
		449	JSON. One bank might parse a given non-JSON request and deduct money,
		450	while the other might reject the transaction with a syntax error. While a
		451	good protocol will at least recover, that is extra unnecessary work and
		452	the transaction will still not succeed).
		453
		454	=item JSON::DWIW 0.04
		455
		456	Very fast. Very natural. Very nice.
		457
		458	Undocumented unicode handling (but the best of the pack. Unicode escapes
		459	still don't get parsed properly).
		460
		461	Very inflexible.
		462
		463	No roundtripping.
		464
		465	Does not generate valid JSON (key strings are often unquoted, empty keys
		466	result in nothing being output)
		467
		468	Does not check input for validity.
		469
		470	=back
		471
		472	=head2 SPEED
		473
		474	It seems that JSON::XS is surprisingly fast, as shown in the following
		475	tables. They have been generated with the help of the C<eg/bench> program
		476	in the JSON::XS distribution, to make it easy to compare on your own
		477	system.
		478
		479	First is a comparison between various modules using a very simple JSON
		480	string, showing the number of encodes/decodes per second (JSON::XS is
		481	the functional interface, while JSON::XS/2 is the OO interface with
		482	pretty-printing and hashkey sorting enabled).
		483
		484	module \| encode \| decode \|
		485	-----------\|------------\|------------\|
		486	JSON \| 14006 \| 6820 \|
		487	JSON::DWIW \| 200937 \| 120386 \|
		488	JSON::PC \| 85065 \| 129366 \|
		489	JSON::Syck \| 59898 \| 44232 \|
		490	JSON::XS \| 1171478 \| 342435 \|
		491	JSON::XS/2 \| 730760 \| 328714 \|
		492	-----------+------------+------------+
		493
		494	That is, JSON::XS is 6 times faster than than JSON::DWIW and about 80
		495	times faster than JSON, even with pretty-printing and key sorting.
		496
		497	Using a longer test string (roughly 8KB, generated from Yahoo! Locals
		498	search API (http://nanoref.com/yahooapis/mgPdGg):
		499
		500	module \| encode \| decode \|
		501	-----------\|------------\|------------\|
		502	JSON \| 673 \| 38 \|
		503	JSON::DWIW \| 5271 \| 770 \|
		504	JSON::PC \| 9901 \| 2491 \|
		505	JSON::Syck \| 2360 \| 786 \|
		506	JSON::XS \| 37398 \| 3202 \|
		507	JSON::XS/2 \| 13765 \| 3153 \|
		508	-----------+------------+------------+
		509
		510	Again, JSON::XS leads by far in the encoding case, while still beating
		511	every other module in the decoding case.
		512
		513	Last example is an almost 8MB large hash with many large binary values
		514	(PNG files), resulting in a lot of escaping:
		515
		516	=head1 BUGS
		517
		518	While the goal of this module is to be correct, that unfortunately does
		519	not mean its bug-free, only that I think its design is bug-free. It is
		520	still very young and not well-tested. If you keep reporting bugs they will
		521	be fixed swiftly, though.
28		522
29	=cut	523	=cut
30		524
31	use JSON::DWIW;
32	use Benchmark;
33
34	use utf8;
35	#my $json = '{"ü":1,"a":[1,{"3":4},2],"b":5,"üü":2}';
36	my $json = '{"test":9555555555555555555,"hu" : -1e+5, "arr" : [ 1,2,3,4,5]}';
37
38	my $js = JSON::XS->new;
39	warn $js->indent (0);
40	warn $js->canonical (0);
41	warn $js->ascii (0);
42	warn $js->space_after (0);
43	use Data::Dumper;
44	warn Dumper $js->decode ($json);
45	warn Dumper $js->encode ($js->decode ($json));
46	#my $x = {"üü" => 2, "ü" => 1, "a" => [1,{3,4},2], b => 5};
47
48	#my $js2 = JSON::DWIW->new;
49	#
50	#timethese 200000, {
51	# a => sub { $js->encode ($x) },
52	# b => sub { $js2->to_json ($x) },
53	#};
54
55	1;	525	1;
56
57	=back
58		526
59	=head1 AUTHOR	527	=head1 AUTHOR
60		528
61	Marc Lehmann <schmorp@schmorp.de>	529	Marc Lehmann <schmorp@schmorp.de>
62	http://home.schmorp.de/	530	http://home.schmorp.de/

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing JSON-XS/XS.pm (file contents): Revision 1.1 by root, Thu Mar 22 16:40:16 2007 UTC vs. Revision 1.10 by root, Fri Mar 23 17:40:29 2007 UTC

Diff Legend

Comparing JSON-XS/XS.pm (file contents):
Revision 1.1 by root, Thu Mar 22 16:40:16 2007 UTC vs.
Revision 1.10 by root, Fri Mar 23 17:40:29 2007 UTC