JSON-XS/XS.pm

=head1 NAME

JSON::XS - JSON serialising/deserialising, done correctly and fast

=head1 SYNOPSIS

 use JSON::XS;

=head1 DESCRIPTION

This module converts Perl data structures to JSON and vice versa. Its
primary goal is to be I<correct> and its secondary goal is to be
I<fast>. To reach the latter goal it was written in C.

As this is the n-th-something JSON module on CPAN, what was the reason
to write yet another JSON module? While it seems there are many JSON
modules, none of them correctly handle all corner cases, and in most cases
their maintainers are unresponsive, gone missing, or not listening to bug
reports for other reasons.

See COMPARISON, below, for a comparison to some other JSON modules.

=head2 FEATURES

=over 4

=item * correct handling of unicode issues

This module knows how to handle Unicode, and even documents how it does so.

=item * round-trip integrity

When you serialise a perl data structure using only datatypes supported
by JSON, the deserialised data structure is identical on the Perl level.
(e.g. the string "2.0" doesn't suddenly become "2").

=item * strict checking of JSON correctness

There is no guessing, no generating of illegal JSON strings by default,
and only JSON is accepted as input (the latter is a security feature).

=item * fast

compared to other JSON modules, this module compares favourably.

=item * simple to use

This module has both a simple functional interface as well as an OO
interface.

=item * reasonably versatile output formats

You can choose between the most compact format possible, a pure-ascii
format, or a pretty-printed format. Or you can combine those features in
whatever way you like.

=back

=cut

package JSON::XS;

BEGIN {
   $VERSION = '0.1';
   @ISA = qw(Exporter);

   @EXPORT = qw(to_json from_json);
   require Exporter;

   require XSLoader;
   XSLoader::load JSON::XS::, $VERSION;
}

=head1 FUNCTIONAL INTERFACE

The following convinience methods are provided by this module. They are
exported by default:

=over 4

=item $json_string = to_json $perl_scalar

Converts the given Perl data structure (a simple scalar or a reference to
a hash or array) to a UTF-8 encoded, binary string (that is, the string contains
octets only). Croaks on error.

This function call is functionally identical to C<< JSON::XS->new->utf8
(1)->encode ($perl_scalar) >>.

=item $perl_scalar = from_json $json_string

The opposite of C<to_json>: expects an UTF-8 (binary) string and tries to
parse that as an UTF-8 encoded JSON string, returning the resulting simple
scalar or reference. Croaks on error.

This function call is functionally identical to C<< JSON::XS->new->utf8
(1)->decode ($json_string) >>.

=back

=head1 OBJECT-ORIENTED INTERFACE

The object oriented interface lets you configure your own encoding or
decoding style, within the limits of supported formats.

=over 4

=item $json = new JSON::XS

Creates a new JSON::XS object that can be used to de/encode JSON
strings. All boolean flags described below are by default I<disabled>.

The mutators for flags all return the JSON object again and thus calls can
be chained:

   my $json = JSON::XS->new->utf8(1)->space_after(1)->encode ({a => [1,2]})
   => {"a": [1, 2]}

=item $json = $json->ascii ($enable)

If C<$enable> is true, then the C<encode> method will not generate
characters outside the code range C<0..127>. Any unicode characters
outside that range will be escaped using either a single \uXXXX (BMP
characters) or a double \uHHHH\uLLLLL escape sequence, as per RFC4627.

If C<$enable> is false, then the C<encode> method will not escape Unicode
characters unless necessary.

  JSON::XS->new->ascii (1)->encode (chr 0x10401)
  => \ud801\udc01

=item $json = $json->utf8 ($enable)

If C<$enable> is true, then the C<encode> method will encode the JSON
string into UTF-8, as required by many protocols, while the C<decode>
method expects to be handled an UTF-8-encoded string.  Please note that
UTF-8-encoded strings do not contain any characters outside the range
C<0..255>, they are thus useful for bytewise/binary I/O.

If C<$enable> is false, then the C<encode> method will return the JSON
string as a (non-encoded) unicode string, while C<decode> expects thus a
unicode string.  Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs
to be done yourself, e.g. using the Encode module.

=item $json = $json->pretty ($enable)

This enables (or disables) all of the C<indent>, C<space_before> and
C<space_after> (and in the future possibly more) flags in one call to
generate the most readable (or most compact) form possible.

   my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
   =>
   {
      "a" : [
         1,
         2
      ]
   }

=item $json = $json->indent ($enable)

If C<$enable> is true, then the C<encode> method will use a multiline
format as output, putting every array member or object/hash key-value pair
into its own line, identing them properly.

If C<$enable> is false, no newlines or indenting will be produced, and the
resulting JSON strings is guarenteed not to contain any C<newlines>.

This setting has no effect when decoding JSON strings.

=item $json = $json->space_before ($enable)

If C<$enable> is true, then the C<encode> method will add an extra
optional space before the C<:> separating keys from values in JSON objects.

If C<$enable> is false, then the C<encode> method will not add any extra
space at those places.

This setting has no effect when decoding JSON strings. You will also most
likely combine this setting with C<space_after>.

=item $json = $json->space_after ($enable)

If C<$enable> is true, then the C<encode> method will add an extra
optional space after the C<:> separating keys from values in JSON objects
and extra whitespace after the C<,> separating key-value pairs and array
members.

If C<$enable> is false, then the C<encode> method will not add any extra
space at those places.

This setting has no effect when decoding JSON strings.

=item $json = $json->canonical ($enable)

If C<$enable> is true, then the C<encode> method will output JSON objects
by sorting their keys. This is adding a comparatively high overhead.

If C<$enable> is false, then the C<encode> method will output key-value
pairs in the order Perl stores them (which will likely change between runs
of the same script).

This option is useful if you want the same data structure to be encoded as
the same JSON string (given the same overall settings). If it is disabled,
the same hash migh be encoded differently even if contains the same data,
as key-value pairs have no inherent ordering in Perl.

This setting has no effect when decoding JSON strings.

=item $json = $json->allow_nonref ($enable)

If C<$enable> is true, then the C<encode> method can convert a
non-reference into its corresponding string, number or null JSON value,
which is an extension to RFC4627. Likewise, C<decode> will accept those JSON
values instead of croaking.

If C<$enable> is false, then the C<encode> method will croak if it isn't
passed an arrayref or hashref, as JSON strings must either be an object
or array. Likewise, C<decode> will croak if given something that is not a
JSON object or array.

=item $json_string = $json->encode ($perl_scalar)

Converts the given Perl data structure (a simple scalar or a reference
to a hash or array) to its JSON representation. Simple scalars will be
converted into JSON string or number sequences, while references to arrays
become JSON arrays and references to hashes become JSON objects. Undefined
Perl values (e.g. C<undef>) become JSON C<null> values. Neither C<true>
nor C<false> values will be generated.

=item $perl_scalar = $json->decode ($json_string)

The opposite of C<encode>: expects a JSON string and tries to parse it,
returning the resulting simple scalar or reference. Croaks on error.

JSON numbers and strings become simple Perl scalars. JSON arrays become
Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.

=back

=head1 COMPARISON

As already mentioned, this module was created because none of the existing
JSON modules could be made to work correctly. First I will describe the
problems (or pleasures) I encountered with various existing JSON modules,
followed by some benchmark values.

=over 4

=item JSON

Slow (but very portable, as it is written in pure Perl).

Undocumented/buggy Unicode handling (how JSON handles unicode values is
undocumented. One can get far by feeding it unicode strings and doing
en-/decoding oneself, but unicode escapes are not working properly).

No roundtripping (strings get clobbered if they look like numbers, e.g.
the string C<2.0> will encode to C<2.0> instead of C<"2.0">, and that will
decode into the number 2.

=item JSON::PC

Very fast.

Very inflexible (no human-readable format supported, format pretty much
undocumented. I need at least a format for easy reading by humans and a
single-line compact format for use in a protocol, and preferably a way to
generate ASCII-only JSON strings).

Undocumented/buggy Unicode handling.

No roundtripping.

Has problems handling many Perl values.

Does not even generate valid JSON (C<{1,2}> gets converted to C<{1:2}>
which is not a valid JSON string.

Unmaintained (maintainer unresponsive for many months, bugs are not
getting fixed).

=item JSON::Syck

Very buggy (often crashes).

Very inflexible.

Completely broken (and confusingly documented) Unicode handling (unicode
escapes are not working properly, you need to set ImplicitUnicode to
I<different> values on en- and decoding to get symmetric behaviour).

No roundtripping (simple cases work, but this depends on wether the scalar
value was used in a numeric context or not).

Dumping hashes may skip hash values depending on iterator state.

Unmaintained (maintainer unresponsive for many months, bugs are not
getting fixed).

Does not check input for validity (i.e. will accept non-JSON input and
return "something" instead of raising an exception. This is a security
issue: imagine two banks transfering money between each other using
JSON. One bank might parse a given non-JSON request and deduct money,
while the other might reject the transaction with a syntax error. While a
good protocol will at least recover, that is extra unnecessary work and
the transaction will still not succeed).

=item JSON::DWIW

Very fast. Very natural. Very nice.

Undocumented unicode handling (but the best of the pack. Unicode escapes
still don't get parsed properly).

Very inflexible.

No roundtripping.

Does not check input for validity.

=back

=head2 SPEED

=cut

1;

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

=cut

Revision:	1.3
Committed:	Thu Mar 22 18:10:29 2007 UTC (17 years, 2 months ago) by root
Branch:	MAIN
Changes since 1.2:	+113 -4 lines
Log Message:	rant
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3			JSON::XS - JSON serialising/deserialising, done correctly and fast
4
5			=head1 SYNOPSIS
6
7			use JSON::XS;
8
9			=head1 DESCRIPTION
10
11	root	1.2	This module converts Perl data structures to JSON and vice versa. Its
12			primary goal is to be I<correct> and its secondary goal is to be
13			I<fast>. To reach the latter goal it was written in C.
14
15			As this is the n-th-something JSON module on CPAN, what was the reason
16			to write yet another JSON module? While it seems there are many JSON
17			modules, none of them correctly handle all corner cases, and in most cases
18			their maintainers are unresponsive, gone missing, or not listening to bug
19			reports for other reasons.
20
21			See COMPARISON, below, for a comparison to some other JSON modules.
22
23			=head2 FEATURES
24
25	root	1.1	=over 4
26
27	root	1.2	=item * correct handling of unicode issues
28
29			This module knows how to handle Unicode, and even documents how it does so.
30
31			=item * round-trip integrity
32
33			When you serialise a perl data structure using only datatypes supported
34			by JSON, the deserialised data structure is identical on the Perl level.
35			(e.g. the string "2.0" doesn't suddenly become "2").
36
37			=item * strict checking of JSON correctness
38
39			There is no guessing, no generating of illegal JSON strings by default,
40			and only JSON is accepted as input (the latter is a security feature).
41
42			=item * fast
43
44			compared to other JSON modules, this module compares favourably.
45
46			=item * simple to use
47
48			This module has both a simple functional interface as well as an OO
49			interface.
50
51			=item * reasonably versatile output formats
52
53			You can choose between the most compact format possible, a pure-ascii
54			format, or a pretty-printed format. Or you can combine those features in
55			whatever way you like.
56
57			=back
58
59	root	1.1	=cut
60
61			package JSON::XS;
62
63			BEGIN {
64			$VERSION = '0.1';
65			@ISA = qw(Exporter);
66
67	root	1.2	@EXPORT = qw(to_json from_json);
68	root	1.1	require Exporter;
69
70			require XSLoader;
71			XSLoader::load JSON::XS::, $VERSION;
72			}
73
74	root	1.2	=head1 FUNCTIONAL INTERFACE
75
76			The following convinience methods are provided by this module. They are
77			exported by default:
78
79			=over 4
80
81			=item $json_string = to_json $perl_scalar
82
83			Converts the given Perl data structure (a simple scalar or a reference to
84			a hash or array) to a UTF-8 encoded, binary string (that is, the string contains
85			octets only). Croaks on error.
86
87			This function call is functionally identical to C<< JSON::XS->new->utf8
88			(1)->encode ($perl_scalar) >>.
89
90			=item $perl_scalar = from_json $json_string
91
92			The opposite of C<to_json>: expects an UTF-8 (binary) string and tries to
93			parse that as an UTF-8 encoded JSON string, returning the resulting simple
94			scalar or reference. Croaks on error.
95
96			This function call is functionally identical to C<< JSON::XS->new->utf8
97			(1)->decode ($json_string) >>.
98
99			=back
100
101			=head1 OBJECT-ORIENTED INTERFACE
102
103			The object oriented interface lets you configure your own encoding or
104			decoding style, within the limits of supported formats.
105
106			=over 4
107
108			=item $json = new JSON::XS
109
110			Creates a new JSON::XS object that can be used to de/encode JSON
111			strings. All boolean flags described below are by default I<disabled>.
112	root	1.1
113	root	1.2	The mutators for flags all return the JSON object again and thus calls can
114			be chained:
115
116	root	1.3	my $json = JSON::XS->new->utf8(1)->space_after(1)->encode ({a => [1,2]})
117			=> {"a": [1, 2]}
118	root	1.2
119			=item $json = $json->ascii ($enable)
120
121			If C<$enable> is true, then the C<encode> method will not generate
122			characters outside the code range C<0..127>. Any unicode characters
123			outside that range will be escaped using either a single \uXXXX (BMP
124			characters) or a double \uHHHH\uLLLLL escape sequence, as per RFC4627.
125
126			If C<$enable> is false, then the C<encode> method will not escape Unicode
127			characters unless necessary.
128
129	root	1.3	JSON::XS->new->ascii (1)->encode (chr 0x10401)
130			=> \ud801\udc01
131
132	root	1.2	=item $json = $json->utf8 ($enable)
133
134			If C<$enable> is true, then the C<encode> method will encode the JSON
135			string into UTF-8, as required by many protocols, while the C<decode>
136			method expects to be handled an UTF-8-encoded string. Please note that
137			UTF-8-encoded strings do not contain any characters outside the range
138			C<0..255>, they are thus useful for bytewise/binary I/O.
139
140			If C<$enable> is false, then the C<encode> method will return the JSON
141			string as a (non-encoded) unicode string, while C<decode> expects thus a
142			unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs
143			to be done yourself, e.g. using the Encode module.
144
145	root	1.3	=item $json = $json->pretty ($enable)
146	root	1.2
147			This enables (or disables) all of the C<indent>, C<space_before> and
148	root	1.3	C<space_after> (and in the future possibly more) flags in one call to
149	root	1.2	generate the most readable (or most compact) form possible.
150
151	root	1.3	my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
152			=>
153			{
154			"a" : [
155			1,
156			2
157			]
158			}
159
160	root	1.2	=item $json = $json->indent ($enable)
161
162			If C<$enable> is true, then the C<encode> method will use a multiline
163			format as output, putting every array member or object/hash key-value pair
164			into its own line, identing them properly.
165
166			If C<$enable> is false, no newlines or indenting will be produced, and the
167			resulting JSON strings is guarenteed not to contain any C<newlines>.
168
169			This setting has no effect when decoding JSON strings.
170
171			=item $json = $json->space_before ($enable)
172
173			If C<$enable> is true, then the C<encode> method will add an extra
174			optional space before the C<:> separating keys from values in JSON objects.
175
176			If C<$enable> is false, then the C<encode> method will not add any extra
177			space at those places.
178
179			This setting has no effect when decoding JSON strings. You will also most
180			likely combine this setting with C<space_after>.
181
182			=item $json = $json->space_after ($enable)
183
184			If C<$enable> is true, then the C<encode> method will add an extra
185			optional space after the C<:> separating keys from values in JSON objects
186			and extra whitespace after the C<,> separating key-value pairs and array
187			members.
188
189			If C<$enable> is false, then the C<encode> method will not add any extra
190			space at those places.
191
192			This setting has no effect when decoding JSON strings.
193
194			=item $json = $json->canonical ($enable)
195
196			If C<$enable> is true, then the C<encode> method will output JSON objects
197			by sorting their keys. This is adding a comparatively high overhead.
198
199			If C<$enable> is false, then the C<encode> method will output key-value
200			pairs in the order Perl stores them (which will likely change between runs
201			of the same script).
202
203			This option is useful if you want the same data structure to be encoded as
204			the same JSON string (given the same overall settings). If it is disabled,
205			the same hash migh be encoded differently even if contains the same data,
206			as key-value pairs have no inherent ordering in Perl.
207
208			This setting has no effect when decoding JSON strings.
209
210	root	1.3	=item $json = $json->allow_nonref ($enable)
211
212			If C<$enable> is true, then the C<encode> method can convert a
213			non-reference into its corresponding string, number or null JSON value,
214			which is an extension to RFC4627. Likewise, C<decode> will accept those JSON
215			values instead of croaking.
216
217			If C<$enable> is false, then the C<encode> method will croak if it isn't
218			passed an arrayref or hashref, as JSON strings must either be an object
219			or array. Likewise, C<decode> will croak if given something that is not a
220			JSON object or array.
221
222	root	1.2	=item $json_string = $json->encode ($perl_scalar)
223
224			Converts the given Perl data structure (a simple scalar or a reference
225			to a hash or array) to its JSON representation. Simple scalars will be
226			converted into JSON string or number sequences, while references to arrays
227			become JSON arrays and references to hashes become JSON objects. Undefined
228			Perl values (e.g. C<undef>) become JSON C<null> values. Neither C<true>
229			nor C<false> values will be generated.
230	root	1.1
231	root	1.2	=item $perl_scalar = $json->decode ($json_string)
232	root	1.1
233	root	1.2	The opposite of C<encode>: expects a JSON string and tries to parse it,
234			returning the resulting simple scalar or reference. Croaks on error.
235	root	1.1
236	root	1.2	JSON numbers and strings become simple Perl scalars. JSON arrays become
237			Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
238			C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.
239	root	1.1
240			=back
241
242	root	1.3	=head1 COMPARISON
243
244			As already mentioned, this module was created because none of the existing
245			JSON modules could be made to work correctly. First I will describe the
246			problems (or pleasures) I encountered with various existing JSON modules,
247			followed by some benchmark values.
248
249			=over 4
250
251			=item JSON
252
253			Slow (but very portable, as it is written in pure Perl).
254
255			Undocumented/buggy Unicode handling (how JSON handles unicode values is
256			undocumented. One can get far by feeding it unicode strings and doing
257			en-/decoding oneself, but unicode escapes are not working properly).
258
259			No roundtripping (strings get clobbered if they look like numbers, e.g.
260			the string C<2.0> will encode to C<2.0> instead of C<"2.0">, and that will
261			decode into the number 2.
262
263			=item JSON::PC
264
265			Very fast.
266
267			Very inflexible (no human-readable format supported, format pretty much
268			undocumented. I need at least a format for easy reading by humans and a
269			single-line compact format for use in a protocol, and preferably a way to
270			generate ASCII-only JSON strings).
271
272			Undocumented/buggy Unicode handling.
273
274			No roundtripping.
275
276			Has problems handling many Perl values.
277
278			Does not even generate valid JSON (C<{1,2}> gets converted to C<{1:2}>
279			which is not a valid JSON string.
280
281			Unmaintained (maintainer unresponsive for many months, bugs are not
282			getting fixed).
283
284			=item JSON::Syck
285
286			Very buggy (often crashes).
287
288			Very inflexible.
289
290			Completely broken (and confusingly documented) Unicode handling (unicode
291			escapes are not working properly, you need to set ImplicitUnicode to
292			I<different> values on en- and decoding to get symmetric behaviour).
293
294			No roundtripping (simple cases work, but this depends on wether the scalar
295			value was used in a numeric context or not).
296
297			Dumping hashes may skip hash values depending on iterator state.
298
299			Unmaintained (maintainer unresponsive for many months, bugs are not
300			getting fixed).
301
302			Does not check input for validity (i.e. will accept non-JSON input and
303			return "something" instead of raising an exception. This is a security
304			issue: imagine two banks transfering money between each other using
305			JSON. One bank might parse a given non-JSON request and deduct money,
306			while the other might reject the transaction with a syntax error. While a
307			good protocol will at least recover, that is extra unnecessary work and
308			the transaction will still not succeed).
309
310			=item JSON::DWIW
311
312			Very fast. Very natural. Very nice.
313
314			Undocumented unicode handling (but the best of the pack. Unicode escapes
315			still don't get parsed properly).
316
317			Very inflexible.
318
319			No roundtripping.
320
321			Does not check input for validity.
322
323			=back
324
325			=head2 SPEED
326
327	root	1.2	=cut
328
329			1;
330
331	root	1.1	=head1 AUTHOR
332
333			Marc Lehmann <schmorp@schmorp.de>
334			http://home.schmorp.de/
335
336			=cut
337