JSON-XS/XS.pm

=head1 NAME

JSON::XS - JSON serialising/deserialising, done correctly and fast

=head1 SYNOPSIS

 use JSON::XS;

=head1 DESCRIPTION

This module converts Perl data structures to JSON and vice versa. Its
primary goal is to be I<correct> and its secondary goal is to be
I<fast>. To reach the latter goal it was written in C.

As this is the n-th-something JSON module on CPAN, what was the reason
to write yet another JSON module? While it seems there are many JSON
modules, none of them correctly handle all corner cases, and in most cases
their maintainers are unresponsive, gone missing, or not listening to bug
reports for other reasons.

See COMPARISON, below, for a comparison to some other JSON modules.

=head2 FEATURES

=over 4

=item * correct handling of unicode issues

This module knows how to handle Unicode, and even documents how it does so.

=item * round-trip integrity

When you serialise a perl data structure using only datatypes supported
by JSON, the deserialised data structure is identical on the Perl level.
(e.g. the string "2.0" doesn't suddenly become "2").

=item * strict checking of JSON correctness

There is no guessing, no generating of illegal JSON strings by default,
and only JSON is accepted as input (the latter is a security feature).

=item * fast

compared to other JSON modules, this module compares favourably.

=item * simple to use

This module has both a simple functional interface as well as an OO
interface.

=item * reasonably versatile output formats

You can choose between the most compact format possible, a pure-ascii
format, or a pretty-printed format. Or you can combine those features in
whatever way you like.

=back

=cut

package JSON::XS;

BEGIN {
   $VERSION = '0.1';
   @ISA = qw(Exporter);

   @EXPORT = qw(to_json from_json);
   require Exporter;

   require XSLoader;
   XSLoader::load JSON::XS::, $VERSION;
}

=head1 FUNCTIONAL INTERFACE

The following convinience methods are provided by this module. They are
exported by default:

=over 4

=item $json_string = to_json $perl_scalar

Converts the given Perl data structure (a simple scalar or a reference to
a hash or array) to a UTF-8 encoded, binary string (that is, the string contains
octets only). Croaks on error.

This function call is functionally identical to C<< JSON::XS->new->utf8
(1)->encode ($perl_scalar) >>.

=item $perl_scalar = from_json $json_string

The opposite of C<to_json>: expects an UTF-8 (binary) string and tries to
parse that as an UTF-8 encoded JSON string, returning the resulting simple
scalar or reference. Croaks on error.

This function call is functionally identical to C<< JSON::XS->new->utf8
(1)->decode ($json_string) >>.

=back

=head1 OBJECT-ORIENTED INTERFACE

The object oriented interface lets you configure your own encoding or
decoding style, within the limits of supported formats.

=over 4

=item $json = new JSON::XS

Creates a new JSON::XS object that can be used to de/encode JSON
strings. All boolean flags described below are by default I<disabled>.

The mutators for flags all return the JSON object again and thus calls can
be chained:

   my $json = JSON::XS->new->utf8(1)->space_after(1)->encode ({a => [1,2]})
   => {"a": [1, 2]}

=item $json = $json->ascii ($enable)

If C<$enable> is true, then the C<encode> method will not generate
characters outside the code range C<0..127>. Any unicode characters
outside that range will be escaped using either a single \uXXXX (BMP
characters) or a double \uHHHH\uLLLLL escape sequence, as per RFC4627.

If C<$enable> is false, then the C<encode> method will not escape Unicode
characters unless necessary.

  JSON::XS->new->ascii (1)->encode (chr 0x10401)
  => \ud801\udc01

=item $json = $json->utf8 ($enable)

If C<$enable> is true, then the C<encode> method will encode the JSON
string into UTF-8, as required by many protocols, while the C<decode>
method expects to be handled an UTF-8-encoded string.  Please note that
UTF-8-encoded strings do not contain any characters outside the range
C<0..255>, they are thus useful for bytewise/binary I/O.

If C<$enable> is false, then the C<encode> method will return the JSON
string as a (non-encoded) unicode string, while C<decode> expects thus a
unicode string.  Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs
to be done yourself, e.g. using the Encode module.

=item $json = $json->pretty ($enable)

This enables (or disables) all of the C<indent>, C<space_before> and
C<space_after> (and in the future possibly more) flags in one call to
generate the most readable (or most compact) form possible.

   my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
   =>
   {
      "a" : [
         1,
         2
      ]
   }

=item $json = $json->indent ($enable)

If C<$enable> is true, then the C<encode> method will use a multiline
format as output, putting every array member or object/hash key-value pair
into its own line, identing them properly.

If C<$enable> is false, no newlines or indenting will be produced, and the
resulting JSON strings is guarenteed not to contain any C<newlines>.

This setting has no effect when decoding JSON strings.

=item $json = $json->space_before ($enable)

If C<$enable> is true, then the C<encode> method will add an extra
optional space before the C<:> separating keys from values in JSON objects.

If C<$enable> is false, then the C<encode> method will not add any extra
space at those places.

This setting has no effect when decoding JSON strings. You will also most
likely combine this setting with C<space_after>.

=item $json = $json->space_after ($enable)

If C<$enable> is true, then the C<encode> method will add an extra
optional space after the C<:> separating keys from values in JSON objects
and extra whitespace after the C<,> separating key-value pairs and array
members.

If C<$enable> is false, then the C<encode> method will not add any extra
space at those places.

This setting has no effect when decoding JSON strings.

=item $json = $json->canonical ($enable)

If C<$enable> is true, then the C<encode> method will output JSON objects
by sorting their keys. This is adding a comparatively high overhead.

If C<$enable> is false, then the C<encode> method will output key-value
pairs in the order Perl stores them (which will likely change between runs
of the same script).

This option is useful if you want the same data structure to be encoded as
the same JSON string (given the same overall settings). If it is disabled,
the same hash migh be encoded differently even if contains the same data,
as key-value pairs have no inherent ordering in Perl.

This setting has no effect when decoding JSON strings.

=item $json = $json->allow_nonref ($enable)

If C<$enable> is true, then the C<encode> method can convert a
non-reference into its corresponding string, number or null JSON value,
which is an extension to RFC4627. Likewise, C<decode> will accept those JSON
values instead of croaking.

If C<$enable> is false, then the C<encode> method will croak if it isn't
passed an arrayref or hashref, as JSON strings must either be an object
or array. Likewise, C<decode> will croak if given something that is not a
JSON object or array.

=item $json_string = $json->encode ($perl_scalar)

Converts the given Perl data structure (a simple scalar or a reference
to a hash or array) to its JSON representation. Simple scalars will be
converted into JSON string or number sequences, while references to arrays
become JSON arrays and references to hashes become JSON objects. Undefined
Perl values (e.g. C<undef>) become JSON C<null> values. Neither C<true>
nor C<false> values will be generated.

=item $perl_scalar = $json->decode ($json_string)

The opposite of C<encode>: expects a JSON string and tries to parse it,
returning the resulting simple scalar or reference. Croaks on error.

JSON numbers and strings become simple Perl scalars. JSON arrays become
Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.

=back

=head1 COMPARISON

As already mentioned, this module was created because none of the existing
JSON modules could be made to work correctly. First I will describe the
problems (or pleasures) I encountered with various existing JSON modules,
followed by some benchmark values.

=over 4

=item JSON

Slow (but very portable, as it is written in pure Perl).

Undocumented/buggy Unicode handling (how JSON handles unicode values is
undocumented. One can get far by feeding it unicode strings and doing
en-/decoding oneself, but unicode escapes are not working properly).

No roundtripping (strings get clobbered if they look like numbers, e.g.
the string C<2.0> will encode to C<2.0> instead of C<"2.0">, and that will
decode into the number 2.

=item JSON::PC

Very fast.

Very inflexible (no human-readable format supported, format pretty much
undocumented. I need at least a format for easy reading by humans and a
single-line compact format for use in a protocol, and preferably a way to
generate ASCII-only JSON strings).

Undocumented/buggy Unicode handling.

No roundtripping.

Has problems handling many Perl values.

Does not even generate valid JSON (C<{1,2}> gets converted to C<{1:2}>
which is not a valid JSON string.

Unmaintained (maintainer unresponsive for many months, bugs are not
getting fixed).

=item JSON::Syck

Very buggy (often crashes).

Very inflexible.

Completely broken (and confusingly documented) Unicode handling (unicode
escapes are not working properly, you need to set ImplicitUnicode to
I<different> values on en- and decoding to get symmetric behaviour).

No roundtripping (simple cases work, but this depends on wether the scalar
value was used in a numeric context or not).

Dumping hashes may skip hash values depending on iterator state.

Unmaintained (maintainer unresponsive for many months, bugs are not
getting fixed).

Does not check input for validity (i.e. will accept non-JSON input and
return "something" instead of raising an exception. This is a security
issue: imagine two banks transfering money between each other using
JSON. One bank might parse a given non-JSON request and deduct money,
while the other might reject the transaction with a syntax error. While a
good protocol will at least recover, that is extra unnecessary work and
the transaction will still not succeed).

=item JSON::DWIW

Very fast. Very natural. Very nice.

Undocumented unicode handling (but the best of the pack. Unicode escapes
still don't get parsed properly).

Very inflexible.

No roundtripping.

Does not check input for validity.

=back

=head2 SPEED

=cut

1;

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut

Revision:	1.3
Committed:	Thu Mar 22 18:10:29 2007 UTC (17 years, 2 months ago) by root
Branch:	MAIN
Changes since 1.2:	+113 -4 lines
Log Message:	rant
#	Content
1	=head1 NAME
2
3	JSON::XS - JSON serialising/deserialising, done correctly and fast
4
5	=head1 SYNOPSIS
6
7	use JSON::XS;
8
9	=head1 DESCRIPTION
10
11	This module converts Perl data structures to JSON and vice versa. Its
12	primary goal is to be I<correct> and its secondary goal is to be
13	I<fast>. To reach the latter goal it was written in C.
14
15	As this is the n-th-something JSON module on CPAN, what was the reason
16	to write yet another JSON module? While it seems there are many JSON
17	modules, none of them correctly handle all corner cases, and in most cases
18	their maintainers are unresponsive, gone missing, or not listening to bug
19	reports for other reasons.
20
21	See COMPARISON, below, for a comparison to some other JSON modules.
22
23	=head2 FEATURES
24
25	=over 4
26
27	=item * correct handling of unicode issues
28
29	This module knows how to handle Unicode, and even documents how it does so.
30
31	=item * round-trip integrity
32
33	When you serialise a perl data structure using only datatypes supported
34	by JSON, the deserialised data structure is identical on the Perl level.
35	(e.g. the string "2.0" doesn't suddenly become "2").
36
37	=item * strict checking of JSON correctness
38
39	There is no guessing, no generating of illegal JSON strings by default,
40	and only JSON is accepted as input (the latter is a security feature).
41
42	=item * fast
43
44	compared to other JSON modules, this module compares favourably.
45
46	=item * simple to use
47
48	This module has both a simple functional interface as well as an OO
49	interface.
50
51	=item * reasonably versatile output formats
52
53	You can choose between the most compact format possible, a pure-ascii
54	format, or a pretty-printed format. Or you can combine those features in
55	whatever way you like.
56
57	=back
58
59	=cut
60
61	package JSON::XS;
62
63	BEGIN {
64	$VERSION = '0.1';
65	@ISA = qw(Exporter);
66
67	@EXPORT = qw(to_json from_json);
68	require Exporter;
69
70	require XSLoader;
71	XSLoader::load JSON::XS::, $VERSION;
72	}
73
74	=head1 FUNCTIONAL INTERFACE
75
76	The following convinience methods are provided by this module. They are
77	exported by default:
78
79	=over 4
80
81	=item $json_string = to_json $perl_scalar
82
83	Converts the given Perl data structure (a simple scalar or a reference to
84	a hash or array) to a UTF-8 encoded, binary string (that is, the string contains
85	octets only). Croaks on error.
86
87	This function call is functionally identical to C<< JSON::XS->new->utf8
88	(1)->encode ($perl_scalar) >>.
89
90	=item $perl_scalar = from_json $json_string
91
92	The opposite of C<to_json>: expects an UTF-8 (binary) string and tries to
93	parse that as an UTF-8 encoded JSON string, returning the resulting simple
94	scalar or reference. Croaks on error.
95
96	This function call is functionally identical to C<< JSON::XS->new->utf8
97	(1)->decode ($json_string) >>.
98
99	=back
100
101	=head1 OBJECT-ORIENTED INTERFACE
102
103	The object oriented interface lets you configure your own encoding or
104	decoding style, within the limits of supported formats.
105
106	=over 4
107
108	=item $json = new JSON::XS
109
110	Creates a new JSON::XS object that can be used to de/encode JSON
111	strings. All boolean flags described below are by default I<disabled>.
112
113	The mutators for flags all return the JSON object again and thus calls can
114	be chained:
115
116	my $json = JSON::XS->new->utf8(1)->space_after(1)->encode ({a => [1,2]})
117	=> {"a": [1, 2]}
118
119	=item $json = $json->ascii ($enable)
120
121	If C<$enable> is true, then the C<encode> method will not generate
122	characters outside the code range C<0..127>. Any unicode characters
123	outside that range will be escaped using either a single \uXXXX (BMP
124	characters) or a double \uHHHH\uLLLLL escape sequence, as per RFC4627.
125
126	If C<$enable> is false, then the C<encode> method will not escape Unicode
127	characters unless necessary.
128
129	JSON::XS->new->ascii (1)->encode (chr 0x10401)
130	=> \ud801\udc01
131
132	=item $json = $json->utf8 ($enable)
133
134	If C<$enable> is true, then the C<encode> method will encode the JSON
135	string into UTF-8, as required by many protocols, while the C<decode>
136	method expects to be handled an UTF-8-encoded string. Please note that
137	UTF-8-encoded strings do not contain any characters outside the range
138	C<0..255>, they are thus useful for bytewise/binary I/O.
139
140	If C<$enable> is false, then the C<encode> method will return the JSON
141	string as a (non-encoded) unicode string, while C<decode> expects thus a
142	unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs
143	to be done yourself, e.g. using the Encode module.
144
145	=item $json = $json->pretty ($enable)
146
147	This enables (or disables) all of the C<indent>, C<space_before> and
148	C<space_after> (and in the future possibly more) flags in one call to
149	generate the most readable (or most compact) form possible.
150
151	my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
152	=>
153	{
154	"a" : [
155	1,
156	2
157	]
158	}
159
160	=item $json = $json->indent ($enable)
161
162	If C<$enable> is true, then the C<encode> method will use a multiline
163	format as output, putting every array member or object/hash key-value pair
164	into its own line, identing them properly.
165
166	If C<$enable> is false, no newlines or indenting will be produced, and the
167	resulting JSON strings is guarenteed not to contain any C<newlines>.
168
169	This setting has no effect when decoding JSON strings.
170
171	=item $json = $json->space_before ($enable)
172
173	If C<$enable> is true, then the C<encode> method will add an extra
174	optional space before the C<:> separating keys from values in JSON objects.
175
176	If C<$enable> is false, then the C<encode> method will not add any extra
177	space at those places.
178
179	This setting has no effect when decoding JSON strings. You will also most
180	likely combine this setting with C<space_after>.
181
182	=item $json = $json->space_after ($enable)
183
184	If C<$enable> is true, then the C<encode> method will add an extra
185	optional space after the C<:> separating keys from values in JSON objects
186	and extra whitespace after the C<,> separating key-value pairs and array
187	members.
188
189	If C<$enable> is false, then the C<encode> method will not add any extra
190	space at those places.
191
192	This setting has no effect when decoding JSON strings.
193
194	=item $json = $json->canonical ($enable)
195
196	If C<$enable> is true, then the C<encode> method will output JSON objects
197	by sorting their keys. This is adding a comparatively high overhead.
198
199	If C<$enable> is false, then the C<encode> method will output key-value
200	pairs in the order Perl stores them (which will likely change between runs
201	of the same script).
202
203	This option is useful if you want the same data structure to be encoded as
204	the same JSON string (given the same overall settings). If it is disabled,
205	the same hash migh be encoded differently even if contains the same data,
206	as key-value pairs have no inherent ordering in Perl.
207
208	This setting has no effect when decoding JSON strings.
209
210	=item $json = $json->allow_nonref ($enable)
211
212	If C<$enable> is true, then the C<encode> method can convert a
213	non-reference into its corresponding string, number or null JSON value,
214	which is an extension to RFC4627. Likewise, C<decode> will accept those JSON
215	values instead of croaking.
216
217	If C<$enable> is false, then the C<encode> method will croak if it isn't
218	passed an arrayref or hashref, as JSON strings must either be an object
219	or array. Likewise, C<decode> will croak if given something that is not a
220	JSON object or array.
221
222	=item $json_string = $json->encode ($perl_scalar)
223
224	Converts the given Perl data structure (a simple scalar or a reference
225	to a hash or array) to its JSON representation. Simple scalars will be
226	converted into JSON string or number sequences, while references to arrays
227	become JSON arrays and references to hashes become JSON objects. Undefined
228	Perl values (e.g. C<undef>) become JSON C<null> values. Neither C<true>
229	nor C<false> values will be generated.
230
231	=item $perl_scalar = $json->decode ($json_string)
232
233	The opposite of C<encode>: expects a JSON string and tries to parse it,
234	returning the resulting simple scalar or reference. Croaks on error.
235
236	JSON numbers and strings become simple Perl scalars. JSON arrays become
237	Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
238	C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.
239
240	=back
241
242	=head1 COMPARISON
243
244	As already mentioned, this module was created because none of the existing
245	JSON modules could be made to work correctly. First I will describe the
246	problems (or pleasures) I encountered with various existing JSON modules,
247	followed by some benchmark values.
248
249	=over 4
250
251	=item JSON
252
253	Slow (but very portable, as it is written in pure Perl).
254
255	Undocumented/buggy Unicode handling (how JSON handles unicode values is
256	undocumented. One can get far by feeding it unicode strings and doing
257	en-/decoding oneself, but unicode escapes are not working properly).
258
259	No roundtripping (strings get clobbered if they look like numbers, e.g.
260	the string C<2.0> will encode to C<2.0> instead of C<"2.0">, and that will
261	decode into the number 2.
262
263	=item JSON::PC
264
265	Very fast.
266
267	Very inflexible (no human-readable format supported, format pretty much
268	undocumented. I need at least a format for easy reading by humans and a
269	single-line compact format for use in a protocol, and preferably a way to
270	generate ASCII-only JSON strings).
271
272	Undocumented/buggy Unicode handling.
273
274	No roundtripping.
275
276	Has problems handling many Perl values.
277
278	Does not even generate valid JSON (C<{1,2}> gets converted to C<{1:2}>
279	which is not a valid JSON string.
280
281	Unmaintained (maintainer unresponsive for many months, bugs are not
282	getting fixed).
283
284	=item JSON::Syck
285
286	Very buggy (often crashes).
287
288	Very inflexible.
289
290	Completely broken (and confusingly documented) Unicode handling (unicode
291	escapes are not working properly, you need to set ImplicitUnicode to
292	I<different> values on en- and decoding to get symmetric behaviour).
293
294	No roundtripping (simple cases work, but this depends on wether the scalar
295	value was used in a numeric context or not).
296
297	Dumping hashes may skip hash values depending on iterator state.
298
299	Unmaintained (maintainer unresponsive for many months, bugs are not
300	getting fixed).
301
302	Does not check input for validity (i.e. will accept non-JSON input and
303	return "something" instead of raising an exception. This is a security
304	issue: imagine two banks transfering money between each other using
305	JSON. One bank might parse a given non-JSON request and deduct money,
306	while the other might reject the transaction with a syntax error. While a
307	good protocol will at least recover, that is extra unnecessary work and
308	the transaction will still not succeed).
309
310	=item JSON::DWIW
311
312	Very fast. Very natural. Very nice.
313
314	Undocumented unicode handling (but the best of the pack. Unicode escapes
315	still don't get parsed properly).
316
317	Very inflexible.
318
319	No roundtripping.
320
321	Does not check input for validity.
322
323	=back
324
325	=head2 SPEED
326
327	=cut
328
329	1;
330
331	=head1 AUTHOR
332
333	Marc Lehmann <schmorp@schmorp.de>
334	http://home.schmorp.de/
335
336	=cut
337