[ViewVC] Annotation of: cvs/JSON-XS/README

NAME
    JSON::XS - JSON serialising/deserialising, done correctly and fast

SYNOPSIS
     use JSON::XS;

DESCRIPTION
    This module converts Perl data structures to JSON and vice versa. Its
    primary goal is to be *correct* and its secondary goal is to be *fast*.
    To reach the latter goal it was written in C.

    As this is the n-th-something JSON module on CPAN, what was the reason
    to write yet another JSON module? While it seems there are many JSON
    modules, none of them correctly handle all corner cases, and in most
    cases their maintainers are unresponsive, gone missing, or not listening
    to bug reports for other reasons.

    See COMPARISON, below, for a comparison to some other JSON modules.

  FEATURES
    * correct handling of unicode issues
        This module knows how to handle Unicode, and even documents how it
        does so.

    * round-trip integrity
        When you serialise a perl data structure using only datatypes
        supported by JSON, the deserialised data structure is identical on
        the Perl level. (e.g. the string "2.0" doesn't suddenly become "2").

    * strict checking of JSON correctness
        There is no guessing, no generating of illegal JSON strings by
        default, and only JSON is accepted as input (the latter is a
        security feature).

    * fast
        compared to other JSON modules, this module compares favourably.

    * simple to use
        This module has both a simple functional interface as well as an OO
        interface.

    * reasonably versatile output formats
        You can choose between the most compact format possible, a
        pure-ascii format, or a pretty-printed format. Or you can combine
        those features in whatever way you like.

FUNCTIONAL INTERFACE
    The following convinience methods are provided by this module. They are
    exported by default:

    $json_string = to_json $perl_scalar
        Converts the given Perl data structure (a simple scalar or a
        reference to a hash or array) to a UTF-8 encoded, binary string
        (that is, the string contains octets only). Croaks on error.

        This function call is functionally identical to "JSON::XS->new->utf8
        (1)->encode ($perl_scalar)".

    $perl_scalar = from_json $json_string
        The opposite of "to_json": expects an UTF-8 (binary) string and
        tries to parse that as an UTF-8 encoded JSON string, returning the
        resulting simple scalar or reference. Croaks on error.

        This function call is functionally identical to "JSON::XS->new->utf8
        (1)->decode ($json_string)".

OBJECT-ORIENTED INTERFACE
    The object oriented interface lets you configure your own encoding or
    decoding style, within the limits of supported formats.

    $json = new JSON::XS
        Creates a new JSON::XS object that can be used to de/encode JSON
        strings. All boolean flags described below are by default
        *disabled*.

        The mutators for flags all return the JSON object again and thus
        calls can be chained:

           my $json = JSON::XS->new->utf8(1)->space_after(1)->encode ({a => [1,2]})
           => {"a": [1, 2]}

    $json = $json->ascii ($enable)
        If $enable is true, then the "encode" method will not generate
        characters outside the code range 0..127. Any unicode characters
        outside that range will be escaped using either a single \uXXXX (BMP
        characters) or a double \uHHHH\uLLLLL escape sequence, as per
        RFC4627.

        If $enable is false, then the "encode" method will not escape
        Unicode characters unless necessary.

          JSON::XS->new->ascii (1)->encode (chr 0x10401)
          => \ud801\udc01

    $json = $json->utf8 ($enable)
        If $enable is true, then the "encode" method will encode the JSON
        string into UTF-8, as required by many protocols, while the "decode"
        method expects to be handled an UTF-8-encoded string. Please note
        that UTF-8-encoded strings do not contain any characters outside the
        range 0..255, they are thus useful for bytewise/binary I/O.

        If $enable is false, then the "encode" method will return the JSON
        string as a (non-encoded) unicode string, while "decode" expects
        thus a unicode string. Any decoding or encoding (e.g. to UTF-8 or
        UTF-16) needs to be done yourself, e.g. using the Encode module.

    $json = $json->pretty ($enable)
        This enables (or disables) all of the "indent", "space_before" and
        "space_after" (and in the future possibly more) flags in one call to
        generate the most readable (or most compact) form possible.

           my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
           =>
           {
              "a" : [
                 1,
                 2
              ]
           }

    $json = $json->indent ($enable)
        If $enable is true, then the "encode" method will use a multiline
        format as output, putting every array member or object/hash
        key-value pair into its own line, identing them properly.

        If $enable is false, no newlines or indenting will be produced, and
        the resulting JSON strings is guarenteed not to contain any
        "newlines".

        This setting has no effect when decoding JSON strings.

    $json = $json->space_before ($enable)
        If $enable is true, then the "encode" method will add an extra
        optional space before the ":" separating keys from values in JSON
        objects.

        If $enable is false, then the "encode" method will not add any extra
        space at those places.

        This setting has no effect when decoding JSON strings. You will also
        most likely combine this setting with "space_after".

    $json = $json->space_after ($enable)
        If $enable is true, then the "encode" method will add an extra
        optional space after the ":" separating keys from values in JSON
        objects and extra whitespace after the "," separating key-value
        pairs and array members.

        If $enable is false, then the "encode" method will not add any extra
        space at those places.

        This setting has no effect when decoding JSON strings.

    $json = $json->canonical ($enable)
        If $enable is true, then the "encode" method will output JSON
        objects by sorting their keys. This is adding a comparatively high
        overhead.

        If $enable is false, then the "encode" method will output key-value
        pairs in the order Perl stores them (which will likely change
        between runs of the same script).

        This option is useful if you want the same data structure to be
        encoded as the same JSON string (given the same overall settings).
        If it is disabled, the same hash migh be encoded differently even if
        contains the same data, as key-value pairs have no inherent ordering
        in Perl.

        This setting has no effect when decoding JSON strings.

    $json = $json->allow_nonref ($enable)
        If $enable is true, then the "encode" method can convert a
        non-reference into its corresponding string, number or null JSON
        value, which is an extension to RFC4627. Likewise, "decode" will
        accept those JSON values instead of croaking.

        If $enable is false, then the "encode" method will croak if it isn't
        passed an arrayref or hashref, as JSON strings must either be an
        object or array. Likewise, "decode" will croak if given something
        that is not a JSON object or array.

    $json_string = $json->encode ($perl_scalar)
        Converts the given Perl data structure (a simple scalar or a
        reference to a hash or array) to its JSON representation. Simple
        scalars will be converted into JSON string or number sequences,
        while references to arrays become JSON arrays and references to
        hashes become JSON objects. Undefined Perl values (e.g. "undef")
        become JSON "null" values. Neither "true" nor "false" values will be
        generated.

    $perl_scalar = $json->decode ($json_string)
        The opposite of "encode": expects a JSON string and tries to parse
        it, returning the resulting simple scalar or reference. Croaks on
        error.

        JSON numbers and strings become simple Perl scalars. JSON arrays
        become Perl arrayrefs and JSON objects become Perl hashrefs. "true"
        becomes 1, "false" becomes 0 and "null" becomes "undef".

COMPARISON
    As already mentioned, this module was created because none of the
    existing JSON modules could be made to work correctly. First I will
    describe the problems (or pleasures) I encountered with various existing
    JSON modules, followed by some benchmark values. JSON::XS was designed
    not to suffer from any of these problems or limitations.

    JSON 1.07
        Slow (but very portable, as it is written in pure Perl).

        Undocumented/buggy Unicode handling (how JSON handles unicode values
        is undocumented. One can get far by feeding it unicode strings and
        doing en-/decoding oneself, but unicode escapes are not working
        properly).

        No roundtripping (strings get clobbered if they look like numbers,
        e.g. the string 2.0 will encode to 2.0 instead of "2.0", and that
        will decode into the number 2.

    JSON::PC 0.01
        Very fast.

        Undocumented/buggy Unicode handling.

        No roundtripping.

        Has problems handling many Perl values (e.g. regex results and other
        magic values will make it croak).

        Does not even generate valid JSON ("{1,2}" gets converted to "{1:2}"
        which is not a valid JSON string.

        Unmaintained (maintainer unresponsive for many months, bugs are not
        getting fixed).

    JSON::Syck 0.21
        Very buggy (often crashes).

        Very inflexible (no human-readable format supported, format pretty
        much undocumented. I need at least a format for easy reading by
        humans and a single-line compact format for use in a protocol, and
        preferably a way to generate ASCII-only JSON strings).

        Completely broken (and confusingly documented) Unicode handling
        (unicode escapes are not working properly, you need to set
        ImplicitUnicode to *different* values on en- and decoding to get
        symmetric behaviour).

        No roundtripping (simple cases work, but this depends on wether the
        scalar value was used in a numeric context or not).

        Dumping hashes may skip hash values depending on iterator state.

        Unmaintained (maintainer unresponsive for many months, bugs are not
        getting fixed).

        Does not check input for validity (i.e. will accept non-JSON input
        and return "something" instead of raising an exception. This is a
        security issue: imagine two banks transfering money between each
        other using JSON. One bank might parse a given non-JSON request and
        deduct money, while the other might reject the transaction with a
        syntax error. While a good protocol will at least recover, that is
        extra unnecessary work and the transaction will still not succeed).

    JSON::DWIW 0.04
        Very fast. Very natural. Very nice.

        Undocumented unicode handling (but the best of the pack. Unicode
        escapes still don't get parsed properly).

        Very inflexible.

        No roundtripping.

        Does not generate valid JSON (key strings are often unquoted, empty
        keys result in nothing being output)

        Does not check input for validity.

  SPEED
    It seems that JSON::XS is surprisingly fast, as shown in the following
    tables. They have been generated with the help of the "eg/bench" program
    in the JSON::XS distribution, to make it easy to compare on your own
    system.

    First is a comparison between various modules using a very simple JSON
    string, showing the number of encodes/decodes per second (JSON::XS is
    the functional interface, while JSON::XS/2 is the OO interface with
    pretty-printing and hashkey sorting enabled).

       module     |     encode |     decode |
       -----------|------------|------------|
       JSON       |      14006 |       6820 |
       JSON::DWIW |     200937 |     120386 |
       JSON::PC   |      85065 |     129366 |
       JSON::Syck |      59898 |      44232 |
       JSON::XS   |    1171478 |     342435 |
       JSON::XS/2 |     730760 |     328714 |
       -----------+------------+------------+

    That is, JSON::XS is 6 times faster than than JSON::DWIW and about 80
    times faster than JSON, even with pretty-printing and key sorting.

    Using a longer test string (roughly 8KB, generated from Yahoo! Locals
    search API (http://nanoref.com/yahooapis/mgPdGg):

       module     |     encode |     decode |
       -----------|------------|------------|
       JSON       |        673 |         38 |
       JSON::DWIW |       5271 |        770 |
       JSON::PC   |       9901 |       2491 |
       JSON::Syck |       2360 |        786 |
       JSON::XS   |      37398 |       3202 |
       JSON::XS/2 |      13765 |       3153 |
       -----------+------------+------------+

    Again, JSON::XS leads by far in the encoding case, while still beating
    every other module in the decoding case.

    Last example is an almost 8MB large hash with many large binary values
    (PNG files), resulting in a lot of escaping:

BUGS
    While the goal of this module is to be correct, that unfortunately does
    not mean its bug-free, only that I think its design is bug-free. It is
    still very young and not well-tested. If you keep reporting bugs they
    will be fixed swiftly, though.

AUTHOR
     Marc Lehmann <schmorp@schmorp.de>
     http://home.schmorp.de/

Revision:	1.3
Committed:	Thu Mar 22 23:24:18 2007 UTC (17 years, 2 months ago) by root
Branch:	MAIN
CVS Tags:	rel-0_2
Changes since 1.2:	+4 -4 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	NAME
2	root	1.2	JSON::XS - JSON serialising/deserialising, done correctly and fast
3	root	1.1
4			SYNOPSIS
5	root	1.2	use JSON::XS;
6	root	1.1
7			DESCRIPTION
8	root	1.2	This module converts Perl data structures to JSON and vice versa. Its
9			primary goal is to be correct and its secondary goal is to be fast.
10			To reach the latter goal it was written in C.
11
12			As this is the n-th-something JSON module on CPAN, what was the reason
13			to write yet another JSON module? While it seems there are many JSON
14			modules, none of them correctly handle all corner cases, and in most
15			cases their maintainers are unresponsive, gone missing, or not listening
16			to bug reports for other reasons.
17
18			See COMPARISON, below, for a comparison to some other JSON modules.
19
20			FEATURES
21			* correct handling of unicode issues
22			This module knows how to handle Unicode, and even documents how it
23			does so.
24
25			* round-trip integrity
26			When you serialise a perl data structure using only datatypes
27			supported by JSON, the deserialised data structure is identical on
28			the Perl level. (e.g. the string "2.0" doesn't suddenly become "2").
29
30			* strict checking of JSON correctness
31			There is no guessing, no generating of illegal JSON strings by
32			default, and only JSON is accepted as input (the latter is a
33			security feature).
34
35			* fast
36			compared to other JSON modules, this module compares favourably.
37
38			* simple to use
39			This module has both a simple functional interface as well as an OO
40			interface.
41
42			* reasonably versatile output formats
43			You can choose between the most compact format possible, a
44			pure-ascii format, or a pretty-printed format. Or you can combine
45			those features in whatever way you like.
46
47			FUNCTIONAL INTERFACE
48			The following convinience methods are provided by this module. They are
49			exported by default:
50
51			$json_string = to_json $perl_scalar
52			Converts the given Perl data structure (a simple scalar or a
53			reference to a hash or array) to a UTF-8 encoded, binary string
54			(that is, the string contains octets only). Croaks on error.
55
56			This function call is functionally identical to "JSON::XS->new->utf8
57			(1)->encode ($perl_scalar)".
58
59			$perl_scalar = from_json $json_string
60			The opposite of "to_json": expects an UTF-8 (binary) string and
61			tries to parse that as an UTF-8 encoded JSON string, returning the
62			resulting simple scalar or reference. Croaks on error.
63
64			This function call is functionally identical to "JSON::XS->new->utf8
65			(1)->decode ($json_string)".
66
67			OBJECT-ORIENTED INTERFACE
68			The object oriented interface lets you configure your own encoding or
69			decoding style, within the limits of supported formats.
70
71			$json = new JSON::XS
72			Creates a new JSON::XS object that can be used to de/encode JSON
73			strings. All boolean flags described below are by default
74			disabled.
75
76			The mutators for flags all return the JSON object again and thus
77			calls can be chained:
78
79			my $json = JSON::XS->new->utf8(1)->space_after(1)->encode ({a => [1,2]})
80			=> {"a": [1, 2]}
81
82			$json = $json->ascii ($enable)
83			If $enable is true, then the "encode" method will not generate
84			characters outside the code range 0..127. Any unicode characters
85			outside that range will be escaped using either a single \uXXXX (BMP
86			characters) or a double \uHHHH\uLLLLL escape sequence, as per
87			RFC4627.
88
89			If $enable is false, then the "encode" method will not escape
90			Unicode characters unless necessary.
91
92			JSON::XS->new->ascii (1)->encode (chr 0x10401)
93			=> \ud801\udc01
94
95			$json = $json->utf8 ($enable)
96			If $enable is true, then the "encode" method will encode the JSON
97			string into UTF-8, as required by many protocols, while the "decode"
98			method expects to be handled an UTF-8-encoded string. Please note
99			that UTF-8-encoded strings do not contain any characters outside the
100			range 0..255, they are thus useful for bytewise/binary I/O.
101
102			If $enable is false, then the "encode" method will return the JSON
103			string as a (non-encoded) unicode string, while "decode" expects
104			thus a unicode string. Any decoding or encoding (e.g. to UTF-8 or
105			UTF-16) needs to be done yourself, e.g. using the Encode module.
106
107			$json = $json->pretty ($enable)
108			This enables (or disables) all of the "indent", "space_before" and
109			"space_after" (and in the future possibly more) flags in one call to
110			generate the most readable (or most compact) form possible.
111
112			my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
113			=>
114			{
115			"a" : [
116			1,
117			2
118			]
119			}
120
121			$json = $json->indent ($enable)
122			If $enable is true, then the "encode" method will use a multiline
123			format as output, putting every array member or object/hash
124			key-value pair into its own line, identing them properly.
125
126			If $enable is false, no newlines or indenting will be produced, and
127			the resulting JSON strings is guarenteed not to contain any
128			"newlines".
129
130			This setting has no effect when decoding JSON strings.
131
132			$json = $json->space_before ($enable)
133			If $enable is true, then the "encode" method will add an extra
134			optional space before the ":" separating keys from values in JSON
135			objects.
136
137			If $enable is false, then the "encode" method will not add any extra
138			space at those places.
139
140			This setting has no effect when decoding JSON strings. You will also
141			most likely combine this setting with "space_after".
142
143			$json = $json->space_after ($enable)
144			If $enable is true, then the "encode" method will add an extra
145			optional space after the ":" separating keys from values in JSON
146			objects and extra whitespace after the "," separating key-value
147			pairs and array members.
148
149			If $enable is false, then the "encode" method will not add any extra
150			space at those places.
151
152			This setting has no effect when decoding JSON strings.
153
154			$json = $json->canonical ($enable)
155			If $enable is true, then the "encode" method will output JSON
156			objects by sorting their keys. This is adding a comparatively high
157			overhead.
158
159			If $enable is false, then the "encode" method will output key-value
160			pairs in the order Perl stores them (which will likely change
161			between runs of the same script).
162
163			This option is useful if you want the same data structure to be
164			encoded as the same JSON string (given the same overall settings).
165			If it is disabled, the same hash migh be encoded differently even if
166			contains the same data, as key-value pairs have no inherent ordering
167			in Perl.
168
169			This setting has no effect when decoding JSON strings.
170
171			$json = $json->allow_nonref ($enable)
172			If $enable is true, then the "encode" method can convert a
173			non-reference into its corresponding string, number or null JSON
174			value, which is an extension to RFC4627. Likewise, "decode" will
175			accept those JSON values instead of croaking.
176
177			If $enable is false, then the "encode" method will croak if it isn't
178			passed an arrayref or hashref, as JSON strings must either be an
179			object or array. Likewise, "decode" will croak if given something
180			that is not a JSON object or array.
181
182			$json_string = $json->encode ($perl_scalar)
183			Converts the given Perl data structure (a simple scalar or a
184			reference to a hash or array) to its JSON representation. Simple
185			scalars will be converted into JSON string or number sequences,
186			while references to arrays become JSON arrays and references to
187			hashes become JSON objects. Undefined Perl values (e.g. "undef")
188			become JSON "null" values. Neither "true" nor "false" values will be
189			generated.
190
191			$perl_scalar = $json->decode ($json_string)
192			The opposite of "encode": expects a JSON string and tries to parse
193			it, returning the resulting simple scalar or reference. Croaks on
194			error.
195
196			JSON numbers and strings become simple Perl scalars. JSON arrays
197			become Perl arrayrefs and JSON objects become Perl hashrefs. "true"
198			becomes 1, "false" becomes 0 and "null" becomes "undef".
199
200			COMPARISON
201			As already mentioned, this module was created because none of the
202			existing JSON modules could be made to work correctly. First I will
203			describe the problems (or pleasures) I encountered with various existing
204			JSON modules, followed by some benchmark values. JSON::XS was designed
205			not to suffer from any of these problems or limitations.
206
207	root	1.3	JSON 1.07
208	root	1.2	Slow (but very portable, as it is written in pure Perl).
209
210			Undocumented/buggy Unicode handling (how JSON handles unicode values
211			is undocumented. One can get far by feeding it unicode strings and
212			doing en-/decoding oneself, but unicode escapes are not working
213			properly).
214
215			No roundtripping (strings get clobbered if they look like numbers,
216			e.g. the string 2.0 will encode to 2.0 instead of "2.0", and that
217			will decode into the number 2.
218
219	root	1.3	JSON::PC 0.01
220	root	1.2	Very fast.
221
222			Undocumented/buggy Unicode handling.
223
224			No roundtripping.
225
226			Has problems handling many Perl values (e.g. regex results and other
227			magic values will make it croak).
228
229			Does not even generate valid JSON ("{1,2}" gets converted to "{1:2}"
230			which is not a valid JSON string.
231
232			Unmaintained (maintainer unresponsive for many months, bugs are not
233			getting fixed).
234
235	root	1.3	JSON::Syck 0.21
236	root	1.2	Very buggy (often crashes).
237
238			Very inflexible (no human-readable format supported, format pretty
239			much undocumented. I need at least a format for easy reading by
240			humans and a single-line compact format for use in a protocol, and
241			preferably a way to generate ASCII-only JSON strings).
242
243			Completely broken (and confusingly documented) Unicode handling
244			(unicode escapes are not working properly, you need to set
245			ImplicitUnicode to different values on en- and decoding to get
246			symmetric behaviour).
247
248			No roundtripping (simple cases work, but this depends on wether the
249			scalar value was used in a numeric context or not).
250
251			Dumping hashes may skip hash values depending on iterator state.
252
253			Unmaintained (maintainer unresponsive for many months, bugs are not
254			getting fixed).
255
256			Does not check input for validity (i.e. will accept non-JSON input
257			and return "something" instead of raising an exception. This is a
258			security issue: imagine two banks transfering money between each
259			other using JSON. One bank might parse a given non-JSON request and
260			deduct money, while the other might reject the transaction with a
261			syntax error. While a good protocol will at least recover, that is
262			extra unnecessary work and the transaction will still not succeed).
263
264	root	1.3	JSON::DWIW 0.04
265	root	1.2	Very fast. Very natural. Very nice.
266
267			Undocumented unicode handling (but the best of the pack. Unicode
268			escapes still don't get parsed properly).
269
270			Very inflexible.
271
272			No roundtripping.
273
274			Does not generate valid JSON (key strings are often unquoted, empty
275			keys result in nothing being output)
276
277			Does not check input for validity.
278
279			SPEED
280			It seems that JSON::XS is surprisingly fast, as shown in the following
281			tables. They have been generated with the help of the "eg/bench" program
282			in the JSON::XS distribution, to make it easy to compare on your own
283			system.
284
285			First is a comparison between various modules using a very simple JSON
286			string, showing the number of encodes/decodes per second (JSON::XS is
287			the functional interface, while JSON::XS/2 is the OO interface with
288			pretty-printing and hashkey sorting enabled).
289
290			module \| encode \| decode \|
291			-----------\|------------\|------------\|
292			JSON \| 14006 \| 6820 \|
293			JSON::DWIW \| 200937 \| 120386 \|
294			JSON::PC \| 85065 \| 129366 \|
295			JSON::Syck \| 59898 \| 44232 \|
296			JSON::XS \| 1171478 \| 342435 \|
297			JSON::XS/2 \| 730760 \| 328714 \|
298			-----------+------------+------------+
299
300			That is, JSON::XS is 6 times faster than than JSON::DWIW and about 80
301			times faster than JSON, even with pretty-printing and key sorting.
302
303			Using a longer test string (roughly 8KB, generated from Yahoo! Locals
304			search API (http://nanoref.com/yahooapis/mgPdGg):
305
306			module \| encode \| decode \|
307			-----------\|------------\|------------\|
308			JSON \| 673 \| 38 \|
309			JSON::DWIW \| 5271 \| 770 \|
310			JSON::PC \| 9901 \| 2491 \|
311			JSON::Syck \| 2360 \| 786 \|
312			JSON::XS \| 37398 \| 3202 \|
313			JSON::XS/2 \| 13765 \| 3153 \|
314			-----------+------------+------------+
315
316			Again, JSON::XS leads by far in the encoding case, while still beating
317			every other module in the decoding case.
318
319			Last example is an almost 8MB large hash with many large binary values
320			(PNG files), resulting in a lot of escaping:
321
322			BUGS
323			While the goal of this module is to be correct, that unfortunately does
324			not mean its bug-free, only that I think its design is bug-free. It is
325			still very young and not well-tested. If you keep reporting bugs they
326			will be fixed swiftly, though.
327	root	1.1
328			AUTHOR
329			Marc Lehmann <schmorp@schmorp.de>
330			http://home.schmorp.de/
331