[ViewVC] Diff of: cvs/JSON-XS/XS.pm

Comparing JSON-XS/XS.pm (file contents):
Revision 1.1 by root, Thu Mar 22 16:40:16 2007 UTC vs.
Revision 1.12 by root, Fri Mar 23 18:33:50 2007 UTC

…		…
4		4
5	=head1 SYNOPSIS	5	=head1 SYNOPSIS
6		6
7	use JSON::XS;	7	use JSON::XS;
8		8
		9	# exported functions, croak on error
		10
		11	$utf8_encoded_json_text = to_json $perl_hash_or_arrayref;
		12	$perl_hash_or_arrayref = from_json $utf8_encoded_json_text;
		13
		14	# oo-interface
		15
		16	$coder = JSON::XS->new->ascii->pretty->allow_nonref;
		17	$pretty_printed_unencoded = $coder->encode ($perl_scalar);
		18	$perl_scalar = $coder->decode ($unicode_json_text);
		19
9	=head1 DESCRIPTION	20	=head1 DESCRIPTION
10		21
		22	This module converts Perl data structures to JSON and vice versa. Its
		23	primary goal is to be I<correct> and its secondary goal is to be
		24	I<fast>. To reach the latter goal it was written in C.
		25
		26	As this is the n-th-something JSON module on CPAN, what was the reason
		27	to write yet another JSON module? While it seems there are many JSON
		28	modules, none of them correctly handle all corner cases, and in most cases
		29	their maintainers are unresponsive, gone missing, or not listening to bug
		30	reports for other reasons.
		31
		32	See COMPARISON, below, for a comparison to some other JSON modules.
		33
		34	See MAPPING, below, on how JSON::XS maps perl values to JSON values and
		35	vice versa.
		36
		37	=head2 FEATURES
		38
11	=over 4	39	=over 4
12		40
		41	=item * correct handling of unicode issues
		42
		43	This module knows how to handle Unicode, and even documents how and when
		44	it does so.
		45
		46	=item * round-trip integrity
		47
		48	When you serialise a perl data structure using only datatypes supported
		49	by JSON, the deserialised data structure is identical on the Perl level.
		50	(e.g. the string "2.0" doesn't suddenly become "2").
		51
		52	=item * strict checking of JSON correctness
		53
		54	There is no guessing, no generating of illegal JSON strings by default,
		55	and only JSON is accepted as input by default (the latter is a security
		56	feature).
		57
		58	=item * fast
		59
		60	Compared to other JSON modules, this module compares favourably in terms
		61	of speed, too.
		62
		63	=item * simple to use
		64
		65	This module has both a simple functional interface as well as an OO
		66	interface.
		67
		68	=item * reasonably versatile output formats
		69
		70	You can choose between the most compact guarenteed single-line format
		71	possible (nice for simple line-based protocols), a pure-ascii format (for
		72	when your transport is not 8-bit clean), or a pretty-printed format (for
		73	when you want to read that stuff). Or you can combine those features in
		74	whatever way you like.
		75
		76	=back
		77
13	=cut	78	=cut
14		79
15	package JSON::XS;	80	package JSON::XS;
16		81
17	BEGIN {	82	BEGIN {
18	$VERSION = '0.1';	83	$VERSION = '0.3';
19	@ISA = qw(Exporter);	84	@ISA = qw(Exporter);
20		85
		86	@EXPORT = qw(to_json from_json);
21	require Exporter;	87	require Exporter;
22		88
23	require XSLoader;	89	require XSLoader;
24	XSLoader::load JSON::XS::, $VERSION;	90	XSLoader::load JSON::XS::, $VERSION;
25	}	91	}
26		92
27	=item	93	=head1 FUNCTIONAL INTERFACE
		94
		95	The following convinience methods are provided by this module. They are
		96	exported by default:
		97
		98	=over 4
		99
		100	=item $json_string = to_json $perl_scalar
		101
		102	Converts the given Perl data structure (a simple scalar or a reference to
		103	a hash or array) to a UTF-8 encoded, binary string (that is, the string contains
		104	octets only). Croaks on error.
		105
		106	This function call is functionally identical to C<< JSON::XS->new->utf8->encode ($perl_scalar) >>.
		107
		108	=item $perl_scalar = from_json $json_string
		109
		110	The opposite of C<to_json>: expects an UTF-8 (binary) string and tries to
		111	parse that as an UTF-8 encoded JSON string, returning the resulting simple
		112	scalar or reference. Croaks on error.
		113
		114	This function call is functionally identical to C<< JSON::XS->new->utf8->decode ($json_string) >>.
		115
		116	=back
		117
		118	=head1 OBJECT-ORIENTED INTERFACE
		119
		120	The object oriented interface lets you configure your own encoding or
		121	decoding style, within the limits of supported formats.
		122
		123	=over 4
		124
		125	=item $json = new JSON::XS
		126
		127	Creates a new JSON::XS object that can be used to de/encode JSON
		128	strings. All boolean flags described below are by default I<disabled>.
		129
		130	The mutators for flags all return the JSON object again and thus calls can
		131	be chained:
		132
		133	my $json = JSON::XS->new->utf8(1)->space_after(1)->encode ({a => [1,2]})
		134	=> {"a": [1, 2]}
		135
		136	=item $json = $json->ascii ([$enable])
		137
		138	If C<$enable> is true (or missing), then the C<encode> method will
		139	not generate characters outside the code range C<0..127>. Any unicode
		140	characters outside that range will be escaped using either a single
		141	\uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence, as per
		142	RFC4627.
		143
		144	If C<$enable> is false, then the C<encode> method will not escape Unicode
		145	characters unless necessary.
		146
		147	JSON::XS->new->ascii (1)->encode (chr 0x10401)
		148	=> \ud801\udc01
		149
		150	=item $json = $json->utf8 ([$enable])
		151
		152	If C<$enable> is true (or missing), then the C<encode> method will encode
		153	the JSON string into UTF-8, as required by many protocols, while the
		154	C<decode> method expects to be handled an UTF-8-encoded string. Please
		155	note that UTF-8-encoded strings do not contain any characters outside the
		156	range C<0..255>, they are thus useful for bytewise/binary I/O.
		157
		158	If C<$enable> is false, then the C<encode> method will return the JSON
		159	string as a (non-encoded) unicode string, while C<decode> expects thus a
		160	unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs
		161	to be done yourself, e.g. using the Encode module.
		162
		163	Example, output UTF-16-encoded JSON:
		164
		165	=item $json = $json->pretty ([$enable])
		166
		167	This enables (or disables) all of the C<indent>, C<space_before> and
		168	C<space_after> (and in the future possibly more) flags in one call to
		169	generate the most readable (or most compact) form possible.
		170
		171	Example, pretty-print some simple structure:
		172
		173	my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
		174	=>
		175	{
		176	"a" : [
		177	1,
		178	2
		179	]
		180	}
		181
		182	=item $json = $json->indent ([$enable])
		183
		184	If C<$enable> is true (or missing), then the C<encode> method will use a multiline
		185	format as output, putting every array member or object/hash key-value pair
		186	into its own line, identing them properly.
		187
		188	If C<$enable> is false, no newlines or indenting will be produced, and the
		189	resulting JSON strings is guarenteed not to contain any C<newlines>.
		190
		191	This setting has no effect when decoding JSON strings.
		192
		193	=item $json = $json->space_before ([$enable])
		194
		195	If C<$enable> is true (or missing), then the C<encode> method will add an extra
		196	optional space before the C<:> separating keys from values in JSON objects.
		197
		198	If C<$enable> is false, then the C<encode> method will not add any extra
		199	space at those places.
		200
		201	This setting has no effect when decoding JSON strings. You will also most
		202	likely combine this setting with C<space_after>.
		203
		204	Example, space_before enabled, space_after and indent disabled:
		205
		206	{"key" :"value"}
		207
		208	=item $json = $json->space_after ([$enable])
		209
		210	If C<$enable> is true (or missing), then the C<encode> method will add an extra
		211	optional space after the C<:> separating keys from values in JSON objects
		212	and extra whitespace after the C<,> separating key-value pairs and array
		213	members.
		214
		215	If C<$enable> is false, then the C<encode> method will not add any extra
		216	space at those places.
		217
		218	This setting has no effect when decoding JSON strings.
		219
		220	Example, space_before and indent disabled, space_after enabled:
		221
		222	{"key": "value"}
		223
		224	=item $json = $json->canonical ([$enable])
		225
		226	If C<$enable> is true (or missing), then the C<encode> method will output JSON objects
		227	by sorting their keys. This is adding a comparatively high overhead.
		228
		229	If C<$enable> is false, then the C<encode> method will output key-value
		230	pairs in the order Perl stores them (which will likely change between runs
		231	of the same script).
		232
		233	This option is useful if you want the same data structure to be encoded as
		234	the same JSON string (given the same overall settings). If it is disabled,
		235	the same hash migh be encoded differently even if contains the same data,
		236	as key-value pairs have no inherent ordering in Perl.
		237
		238	This setting has no effect when decoding JSON strings.
		239
		240	=item $json = $json->allow_nonref ([$enable])
		241
		242	If C<$enable> is true (or missing), then the C<encode> method can convert a
		243	non-reference into its corresponding string, number or null JSON value,
		244	which is an extension to RFC4627. Likewise, C<decode> will accept those JSON
		245	values instead of croaking.
		246
		247	If C<$enable> is false, then the C<encode> method will croak if it isn't
		248	passed an arrayref or hashref, as JSON strings must either be an object
		249	or array. Likewise, C<decode> will croak if given something that is not a
		250	JSON object or array.
		251
		252	Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>,
		253	resulting in an invalid JSON text:
		254
		255	JSON::XS->new->allow_nonref->encode ("Hello, World!")
		256	=> "Hello, World!"
		257
		258	=item $json = $json->shrink ([$enable])
		259
		260	Perl usually over-allocates memory a bit when allocating space for
		261	strings. This flag optionally resizes strings generated by either
		262	C<encode> or C<decode> to their minimum size possible. This can save
		263	memory when your JSON strings are either very very long or you have many
		264	short strings. It will also try to downgrade any strings to octet-form
		265	if possible: perl stores strings internally either in an encoding called
		266	UTF-X or in octet-form. The latter cannot store everything but uses less
		267	space in general.
		268
		269	If C<$enable> is true (or missing), the string returned by C<encode> will be shrunk-to-fit,
		270	while all strings generated by C<decode> will also be shrunk-to-fit.
		271
		272	If C<$enable> is false, then the normal perl allocation algorithms are used.
		273	If you work with your data, then this is likely to be faster.
		274
		275	In the future, this setting might control other things, such as converting
		276	strings that look like integers or floats into integers or floats
		277	internally (there is no difference on the Perl level), saving space.
		278
		279	=item $json_string = $json->encode ($perl_scalar)
		280
		281	Converts the given Perl data structure (a simple scalar or a reference
		282	to a hash or array) to its JSON representation. Simple scalars will be
		283	converted into JSON string or number sequences, while references to arrays
		284	become JSON arrays and references to hashes become JSON objects. Undefined
		285	Perl values (e.g. C<undef>) become JSON C<null> values. Neither C<true>
		286	nor C<false> values will be generated.
		287
		288	=item $perl_scalar = $json->decode ($json_string)
		289
		290	The opposite of C<encode>: expects a JSON string and tries to parse it,
		291	returning the resulting simple scalar or reference. Croaks on error.
		292
		293	JSON numbers and strings become simple Perl scalars. JSON arrays become
		294	Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
		295	C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.
		296
		297	=back
		298
		299	=head1 MAPPING
		300
		301	This section describes how JSON::XS maps Perl values to JSON values and
		302	vice versa. These mappings are designed to "do the right thing" in most
		303	circumstances automatically, preserving round-tripping characteristics
		304	(what you put in comes out as something equivalent).
		305
		306	For the more enlightened: note that in the following descriptions,
		307	lowercase I<perl> refers to the Perl interpreter, while uppcercase I<Perl>
		308	refers to the abstract Perl language itself.
		309
		310	=head2 JSON -> PERL
		311
		312	=over 4
		313
		314	=item object
		315
		316	A JSON object becomes a reference to a hash in Perl. No ordering of object
		317	keys is preserved.
		318
		319	=item array
		320
		321	A JSON array becomes a reference to an array in Perl.
		322
		323	=item string
		324
		325	A JSON string becomes a string scalar in Perl - Unicode codepoints in JSON
		326	are represented by the same codepoints in the Perl string, so no manual
		327	decoding is necessary.
		328
		329	=item number
		330
		331	A JSON number becomes either an integer or numeric (floating point)
		332	scalar in perl, depending on its range and any fractional parts. On the
		333	Perl level, there is no difference between those as Perl handles all the
		334	conversion details, but an integer may take slightly less memory and might
		335	represent more values exactly than (floating point) numbers.
		336
		337	=item true, false
		338
		339	These JSON atoms become C<0>, C<1>, respectively. Information is lost in
		340	this process. Future versions might represent those values differently,
		341	but they will be guarenteed to act like these integers would normally in
		342	Perl.
		343
		344	=item null
		345
		346	A JSON null atom becomes C<undef> in Perl.
		347
		348	=back
		349
		350	=head2 PERL -> JSON
		351
		352	The mapping from Perl to JSON is slightly more difficult, as Perl is a
		353	truly typeless language, so we can only guess which JSON type is meant by
		354	a Perl value.
		355
		356	=over 4
		357
		358	=item hash references
		359
		360	Perl hash references become JSON objects. As there is no inherent ordering
		361	in hash keys, they will usually be encoded in a pseudo-random order that
		362	can change between runs of the same program but stays generally the same
		363	within the single run of a program. JSON::XS can optionally sort the hash
		364	keys (determined by the I<canonical> flag), so the same datastructure
		365	will serialise to the same JSON text (given same settings and version of
		366	JSON::XS), but this incurs a runtime overhead.
		367
		368	=item array references
		369
		370	Perl array references become JSON arrays.
		371
		372	=item blessed objects
		373
		374	Blessed objects are not allowed. JSON::XS currently tries to encode their
		375	underlying representation (hash- or arrayref), but this behaviour might
		376	change in future versions.
		377
		378	=item simple scalars
		379
		380	Simple Perl scalars (any scalar that is not a reference) are the most
		381	difficult objects to encode: JSON::XS will encode undefined scalars as
		382	JSON null value, scalars that have last been used in a string context
		383	before encoding as JSON strings and anything else as number value:
		384
		385	# dump as number
		386	to_json [2] # yields [2]
		387	to_json [-3.0e17] # yields [-3e+17]
		388	my $value = 5; to_json [$value] # yields [5]
		389
		390	# used as string, so dump as string
		391	print $value;
		392	to_json [$value] # yields ["5"]
		393
		394	# undef becomes null
		395	to_json [undef] # yields [null]
		396
		397	You can force the type to be a string by stringifying it:
		398
		399	my $x = 3.1; # some variable containing a number
		400	"$x"; # stringified
		401	$x .= ""; # another, more awkward way to stringify
		402	print $x; # perl does it for you, too, quite often
		403
		404	You can force the type to be a number by numifying it:
		405
		406	my $x = "3"; # some variable containing a string
		407	$x += 0; # numify it, ensuring it will be dumped as a number
		408	$x *= 1; # same thing, the choise is yours.
		409
		410	You can not currently output JSON booleans or force the type in other,
		411	less obscure, ways. Tell me if you need this capability.
		412
		413	=item circular data structures
		414
		415	Those will be encoded until memory or stackspace runs out.
		416
		417	=back
		418
		419	=head1 COMPARISON
		420
		421	As already mentioned, this module was created because none of the existing
		422	JSON modules could be made to work correctly. First I will describe the
		423	problems (or pleasures) I encountered with various existing JSON modules,
		424	followed by some benchmark values. JSON::XS was designed not to suffer
		425	from any of these problems or limitations.
		426
		427	=over 4
		428
		429	=item JSON 1.07
		430
		431	Slow (but very portable, as it is written in pure Perl).
		432
		433	Undocumented/buggy Unicode handling (how JSON handles unicode values is
		434	undocumented. One can get far by feeding it unicode strings and doing
		435	en-/decoding oneself, but unicode escapes are not working properly).
		436
		437	No roundtripping (strings get clobbered if they look like numbers, e.g.
		438	the string C<2.0> will encode to C<2.0> instead of C<"2.0">, and that will
		439	decode into the number 2.
		440
		441	=item JSON::PC 0.01
		442
		443	Very fast.
		444
		445	Undocumented/buggy Unicode handling.
		446
		447	No roundtripping.
		448
		449	Has problems handling many Perl values (e.g. regex results and other magic
		450	values will make it croak).
		451
		452	Does not even generate valid JSON (C<{1,2}> gets converted to C<{1:2}>
		453	which is not a valid JSON string.
		454
		455	Unmaintained (maintainer unresponsive for many months, bugs are not
		456	getting fixed).
		457
		458	=item JSON::Syck 0.21
		459
		460	Very buggy (often crashes).
		461
		462	Very inflexible (no human-readable format supported, format pretty much
		463	undocumented. I need at least a format for easy reading by humans and a
		464	single-line compact format for use in a protocol, and preferably a way to
		465	generate ASCII-only JSON strings).
		466
		467	Completely broken (and confusingly documented) Unicode handling (unicode
		468	escapes are not working properly, you need to set ImplicitUnicode to
		469	I<different> values on en- and decoding to get symmetric behaviour).
		470
		471	No roundtripping (simple cases work, but this depends on wether the scalar
		472	value was used in a numeric context or not).
		473
		474	Dumping hashes may skip hash values depending on iterator state.
		475
		476	Unmaintained (maintainer unresponsive for many months, bugs are not
		477	getting fixed).
		478
		479	Does not check input for validity (i.e. will accept non-JSON input and
		480	return "something" instead of raising an exception. This is a security
		481	issue: imagine two banks transfering money between each other using
		482	JSON. One bank might parse a given non-JSON request and deduct money,
		483	while the other might reject the transaction with a syntax error. While a
		484	good protocol will at least recover, that is extra unnecessary work and
		485	the transaction will still not succeed).
		486
		487	=item JSON::DWIW 0.04
		488
		489	Very fast. Very natural. Very nice.
		490
		491	Undocumented unicode handling (but the best of the pack. Unicode escapes
		492	still don't get parsed properly).
		493
		494	Very inflexible.
		495
		496	No roundtripping.
		497
		498	Does not generate valid JSON (key strings are often unquoted, empty keys
		499	result in nothing being output)
		500
		501	Does not check input for validity.
		502
		503	=back
		504
		505	=head2 SPEED
		506
		507	It seems that JSON::XS is surprisingly fast, as shown in the following
		508	tables. They have been generated with the help of the C<eg/bench> program
		509	in the JSON::XS distribution, to make it easy to compare on your own
		510	system.
		511
		512	First is a comparison between various modules using a very simple JSON
		513	string, showing the number of encodes/decodes per second (JSON::XS is
		514	the functional interface, while JSON::XS/2 is the OO interface with
		515	pretty-printing and hashkey sorting enabled).
		516
		517	module \| encode \| decode \|
		518	-----------\|------------\|------------\|
		519	JSON \| 14006 \| 6820 \|
		520	JSON::DWIW \| 200937 \| 120386 \|
		521	JSON::PC \| 85065 \| 129366 \|
		522	JSON::Syck \| 59898 \| 44232 \|
		523	JSON::XS \| 1171478 \| 342435 \|
		524	JSON::XS/2 \| 730760 \| 328714 \|
		525	-----------+------------+------------+
		526
		527	That is, JSON::XS is 6 times faster than than JSON::DWIW and about 80
		528	times faster than JSON, even with pretty-printing and key sorting.
		529
		530	Using a longer test string (roughly 8KB, generated from Yahoo! Locals
		531	search API (http://nanoref.com/yahooapis/mgPdGg):
		532
		533	module \| encode \| decode \|
		534	-----------\|------------\|------------\|
		535	JSON \| 673 \| 38 \|
		536	JSON::DWIW \| 5271 \| 770 \|
		537	JSON::PC \| 9901 \| 2491 \|
		538	JSON::Syck \| 2360 \| 786 \|
		539	JSON::XS \| 37398 \| 3202 \|
		540	JSON::XS/2 \| 13765 \| 3153 \|
		541	-----------+------------+------------+
		542
		543	Again, JSON::XS leads by far in the encoding case, while still beating
		544	every other module in the decoding case.
		545
		546	=head1 RESOURCE LIMITS
		547
		548	JSON::XS does not impose any limits on the size of JSON texts or Perl
		549	values they represent - if your machine can handle it, JSON::XS will
		550	encode or decode it. Future versions might optionally impose structure
		551	depth and memory use resource limits.
		552
		553	=head1 BUGS
		554
		555	While the goal of this module is to be correct, that unfortunately does
		556	not mean its bug-free, only that I think its design is bug-free. It is
		557	still very young and not well-tested. If you keep reporting bugs they will
		558	be fixed swiftly, though.
28		559
29	=cut	560	=cut
30		561
31	use JSON::DWIW;
32	use Benchmark;
33
34	use utf8;
35	#my $json = '{"ü":1,"a":[1,{"3":4},2],"b":5,"üü":2}';
36	my $json = '{"test":9555555555555555555,"hu" : -1e+5, "arr" : [ 1,2,3,4,5]}';
37
38	my $js = JSON::XS->new;
39	warn $js->indent (0);
40	warn $js->canonical (0);
41	warn $js->ascii (0);
42	warn $js->space_after (0);
43	use Data::Dumper;
44	warn Dumper $js->decode ($json);
45	warn Dumper $js->encode ($js->decode ($json));
46	#my $x = {"üü" => 2, "ü" => 1, "a" => [1,{3,4},2], b => 5};
47
48	#my $js2 = JSON::DWIW->new;
49	#
50	#timethese 200000, {
51	# a => sub { $js->encode ($x) },
52	# b => sub { $js2->to_json ($x) },
53	#};
54
55	1;	562	1;
56
57	=back
58		563
59	=head1 AUTHOR	564	=head1 AUTHOR
60		565
61	Marc Lehmann <schmorp@schmorp.de>	566	Marc Lehmann <schmorp@schmorp.de>
62	http://home.schmorp.de/	567	http://home.schmorp.de/

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing JSON-XS/XS.pm (file contents): Revision 1.1 by root, Thu Mar 22 16:40:16 2007 UTC vs. Revision 1.12 by root, Fri Mar 23 18:33:50 2007 UTC

Diff Legend

Comparing JSON-XS/XS.pm (file contents):
Revision 1.1 by root, Thu Mar 22 16:40:16 2007 UTC vs.
Revision 1.12 by root, Fri Mar 23 18:33:50 2007 UTC