ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.94 by root, Tue Mar 25 07:46:15 2008 UTC vs.
Revision 1.109 by root, Sat Jul 19 04:21:32 2008 UTC

1=head1 NAME 1=head1 NAME
2 2
3JSON::XS - JSON serialising/deserialising, done correctly and fast
4
3=encoding utf-8 5=encoding utf-8
4
5JSON::XS - JSON serialising/deserialising, done correctly and fast
6 6
7JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ 7JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ
8 (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html) 8 (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html)
9 9
10=head1 SYNOPSIS 10=head1 SYNOPSIS
… …
37primary goal is to be I<correct> and its secondary goal is to be 37primary goal is to be I<correct> and its secondary goal is to be
38I<fast>. To reach the latter goal it was written in C. 38I<fast>. To reach the latter goal it was written in C.
39 39
40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overriden) with no overhead due to emulation (by inheritign constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need and doesn't
46require a C compiler when that is a problem. 46require a C compiler when that is a problem.
47 47
… …
65This module knows how to handle Unicode, documents how and when it does 65This module knows how to handle Unicode, documents how and when it does
66so, and even documents what "correct" means. 66so, and even documents what "correct" means.
67 67
68=item * round-trip integrity 68=item * round-trip integrity
69 69
70When you serialise a perl data structure using only datatypes supported 70When you serialise a perl data structure using only data types supported
71by JSON, the deserialised data structure is identical on the Perl level. 71by JSON, the deserialised data structure is identical on the Perl level.
72(e.g. the string "2.0" doesn't suddenly become "2" just because it looks 72(e.g. the string "2.0" doesn't suddenly become "2" just because it looks
73like a number). There minor I<are> exceptions to this, read the MAPPING 73like a number). There minor I<are> exceptions to this, read the MAPPING
74section below to learn about those. 74section below to learn about those.
75 75
… …
84Compared to other JSON modules and other serialisers such as Storable, 84Compared to other JSON modules and other serialisers such as Storable,
85this module usually compares favourably in terms of speed, too. 85this module usually compares favourably in terms of speed, too.
86 86
87=item * simple to use 87=item * simple to use
88 88
89This module has both a simple functional interface as well as an objetc 89This module has both a simple functional interface as well as an object
90oriented interface interface. 90oriented interface interface.
91 91
92=item * reasonably versatile output formats 92=item * reasonably versatile output formats
93 93
94You can choose between the most compact guaranteed-single-line format 94You can choose between the most compact guaranteed-single-line format
95possible (nice for simple line-based protocols), a pure-ascii format 95possible (nice for simple line-based protocols), a pure-ASCII format
96(for when your transport is not 8-bit clean, still supports the whole 96(for when your transport is not 8-bit clean, still supports the whole
97Unicode range), or a pretty-printed format (for when you want to read that 97Unicode range), or a pretty-printed format (for when you want to read that
98stuff). Or you can combine those features in whatever way you like. 98stuff). Or you can combine those features in whatever way you like.
99 99
100=back 100=back
… …
103 103
104package JSON::XS; 104package JSON::XS;
105 105
106use strict; 106use strict;
107 107
108our $VERSION = '2.1'; 108our $VERSION = '2.222';
109our @ISA = qw(Exporter); 109our @ISA = qw(Exporter);
110 110
111our @EXPORT = qw(encode_json decode_json to_json from_json); 111our @EXPORT = qw(encode_json decode_json to_json from_json);
112 112
113sub to_json($) { 113sub to_json($) {
… …
137 137
138This function call is functionally identical to: 138This function call is functionally identical to:
139 139
140 $json_text = JSON::XS->new->utf8->encode ($perl_scalar) 140 $json_text = JSON::XS->new->utf8->encode ($perl_scalar)
141 141
142except being faster. 142Except being faster.
143 143
144=item $perl_scalar = decode_json $json_text 144=item $perl_scalar = decode_json $json_text
145 145
146The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 146The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries
147to parse that as an UTF-8 encoded JSON text, returning the resulting 147to parse that as an UTF-8 encoded JSON text, returning the resulting
… …
149 149
150This function call is functionally identical to: 150This function call is functionally identical to:
151 151
152 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 152 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
153 153
154except being faster. 154Except being faster.
155 155
156=item $is_boolean = JSON::XS::is_bool $scalar 156=item $is_boolean = JSON::XS::is_bool $scalar
157 157
158Returns true if the passed scalar represents either JSON::XS::true or 158Returns true if the passed scalar represents either JSON::XS::true or
159JSON::XS::false, two constants that act like C<1> and C<0>, respectively 159JSON::XS::false, two constants that act like C<1> and C<0>, respectively
… …
197 197
198If you didn't know about that flag, just the better, pretend it doesn't 198If you didn't know about that flag, just the better, pretend it doesn't
199exist. 199exist.
200 200
201=item 4. A "Unicode String" is simply a string where each character can be 201=item 4. A "Unicode String" is simply a string where each character can be
202validly interpreted as a Unicode codepoint. 202validly interpreted as a Unicode code point.
203 203
204If you have UTF-8 encoded data, it is no longer a Unicode string, but a 204If you have UTF-8 encoded data, it is no longer a Unicode string, but a
205Unicode string encoded in UTF-8, giving you a binary string. 205Unicode string encoded in UTF-8, giving you a binary string.
206 206
207=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string. 207=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string.
… …
462Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>, 462Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>,
463resulting in an invalid JSON text: 463resulting in an invalid JSON text:
464 464
465 JSON::XS->new->allow_nonref->encode ("Hello, World!") 465 JSON::XS->new->allow_nonref->encode ("Hello, World!")
466 => "Hello, World!" 466 => "Hello, World!"
467
468=item $json = $json->allow_unknown ([$enable])
469
470=item $enabled = $json->get_allow_unknown
471
472If C<$enable> is true (or missing), then C<encode> will I<not> throw an
473exception when it encounters values it cannot represent in JSON (for
474example, filehandles) but instead will encode a JSON C<null> value. Note
475that blessed objects are not included here and are handled separately by
476c<allow_nonref>.
477
478If C<$enable> is false (the default), then C<encode> will throw an
479exception when it encounters anything it cannot encode as JSON.
480
481This option does not affect C<decode> in any way, and it is recommended to
482leave it off unless you know your communications partner.
467 483
468=item $json = $json->allow_blessed ([$enable]) 484=item $json = $json->allow_blessed ([$enable])
469 485
470=item $enabled = $json->get_allow_blessed 486=item $enabled = $json->get_allow_blessed
471 487
… …
612=item $json = $json->max_depth ([$maximum_nesting_depth]) 628=item $json = $json->max_depth ([$maximum_nesting_depth])
613 629
614=item $max_depth = $json->get_max_depth 630=item $max_depth = $json->get_max_depth
615 631
616Sets the maximum nesting level (default C<512>) accepted while encoding 632Sets the maximum nesting level (default C<512>) accepted while encoding
617or decoding. If the JSON text or Perl data structure has an equal or 633or decoding. If a higher nesting level is detected in JSON text or a Perl
618higher nesting level then this limit, then the encoder and decoder will 634data structure, then the encoder and decoder will stop and croak at that
619stop and croak at that point. 635point.
620 636
621Nesting level is defined by number of hash- or arrayrefs that the encoder 637Nesting level is defined by number of hash- or arrayrefs that the encoder
622needs to traverse to reach a given point or the number of C<{> or C<[> 638needs to traverse to reach a given point or the number of C<{> or C<[>
623characters without their matching closing parenthesis crossed to reach a 639characters without their matching closing parenthesis crossed to reach a
624given character in a string. 640given character in a string.
625 641
626Setting the maximum depth to one disallows any nesting, so that ensures 642Setting the maximum depth to one disallows any nesting, so that ensures
627that the object is only a single hash/object or array. 643that the object is only a single hash/object or array.
628 644
629The argument to C<max_depth> will be rounded up to the next highest power
630of two. If no argument is given, the highest possible setting will be 645If no argument is given, the highest possible setting will be used, which
631used, which is rarely useful. 646is rarely useful.
647
648Note that nesting is implemented by recursion in C. The default value has
649been chosen to be as large as typical operating systems allow without
650crashing.
632 651
633See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 652See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
634 653
635=item $json = $json->max_size ([$maximum_string_size]) 654=item $json = $json->max_size ([$maximum_string_size])
636 655
637=item $max_size = $json->get_max_size 656=item $max_size = $json->get_max_size
638 657
639Set the maximum length a JSON text may have (in bytes) where decoding is 658Set the maximum length a JSON text may have (in bytes) where decoding is
640being attempted. The default is C<0>, meaning no limit. When C<decode> 659being attempted. The default is C<0>, meaning no limit. When C<decode>
641is called on a string longer then this number of characters it will not 660is called on a string that is longer then this many bytes, it will not
642attempt to decode the string but throw an exception. This setting has no 661attempt to decode the string but throw an exception. This setting has no
643effect on C<encode> (yet). 662effect on C<encode> (yet).
644 663
645The argument to C<max_size> will be rounded up to the next B<highest> 664If no argument is given, the limit check will be deactivated (same as when
646power of two (so may be more than requested). If no argument is given, the 665C<0> is specified).
647limit check will be deactivated (same as when C<0> is specified).
648 666
649See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 667See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
650 668
651=item $json_text = $json->encode ($perl_scalar) 669=item $json_text = $json->encode ($perl_scalar)
652 670
… …
683=back 701=back
684 702
685 703
686=head1 INCREMENTAL PARSING 704=head1 INCREMENTAL PARSING
687 705
688[This section is still EXPERIMENTAL]
689
690In some cases, there is the need for incremental parsing of JSON 706In some cases, there is the need for incremental parsing of JSON
691texts. While this module always has to keep both JSON text and resulting 707texts. While this module always has to keep both JSON text and resulting
692Perl data structure in memory at one time, it does allow you to parse a 708Perl data structure in memory at one time, it does allow you to parse a
693JSON stream incrementally. It does so by accumulating text until it has 709JSON stream incrementally. It does so by accumulating text until it has
694a full JSON object, which it then can decode. This process is similar to 710a full JSON object, which it then can decode. This process is similar to
695using C<decode_prefix> to see if a full JSON object is available, but is 711using C<decode_prefix> to see if a full JSON object is available, but
696much more efficient (JSON::XS will only attempt to parse the JSON text 712is much more efficient (and can be implemented with a minimum of method
713calls).
714
715JSON::XS will only attempt to parse the JSON text once it is sure it
697once it is sure it has enough text to get a decisive result, using a very 716has enough text to get a decisive result, using a very simple but
698simple but truly incremental parser). 717truly incremental parser. This means that it sometimes won't stop as
718early as the full parser, for example, it doesn't detect parenthese
719mismatches. The only thing it guarantees is that it starts decoding as
720soon as a syntactically valid JSON text has been seen. This means you need
721to set resource limits (e.g. C<max_size>) to ensure the parser will stop
722parsing in the presence if syntax errors.
699 723
700The following two methods deal with this. 724The following methods implement this incremental parser.
701 725
702=over 4 726=over 4
703 727
704=item [void, scalar or list context] = $json->incr_parse ([$string]) 728=item [void, scalar or list context] = $json->incr_parse ([$string])
705 729
… …
714return without doing anything further. This can be used to add more text 738return without doing anything further. This can be used to add more text
715in as many chunks as you want. 739in as many chunks as you want.
716 740
717If the method is called in scalar context, then it will try to extract 741If the method is called in scalar context, then it will try to extract
718exactly I<one> JSON object. If that is successful, it will return this 742exactly I<one> JSON object. If that is successful, it will return this
719object, otherwise it will return C<undef>. This is the most common way of 743object, otherwise it will return C<undef>. If there is a parse error,
744this method will croak just as C<decode> would do (one can then use
745C<incr_skip> to skip the errornous part). This is the most common way of
720using the method. 746using the method.
721 747
722And finally, in list context, it will try to extract as many objects 748And finally, in list context, it will try to extract as many objects
723from the stream as it can find and return them, or the empty list 749from the stream as it can find and return them, or the empty list
724otherwise. For this to work, there must be no separators between the JSON 750otherwise. For this to work, there must be no separators between the JSON
725objects or arrays, instead they must be concatenated back-to-back. 751objects or arrays, instead they must be concatenated back-to-back. If
752an error occurs, an exception will be raised as in the scalar context
753case. Note that in this case, any previously-parsed JSON texts will be
754lost.
726 755
727=item $lvalue_string = $json->incr_text 756=item $lvalue_string = $json->incr_text
728 757
729This method returns the currently stored JSON fragment as an lvalue, that 758This method returns the currently stored JSON fragment as an lvalue, that
730is, you can manipulate it. This I<only> works when a preceding call to 759is, you can manipulate it. This I<only> works when a preceding call to
… …
735method before having parsed anything. 764method before having parsed anything.
736 765
737This function is useful in two cases: a) finding the trailing text after a 766This function is useful in two cases: a) finding the trailing text after a
738JSON object or b) parsing multiple JSON objects separated by non-JSON text 767JSON object or b) parsing multiple JSON objects separated by non-JSON text
739(such as commas). 768(such as commas).
769
770=item $json->incr_skip
771
772This will reset the state of the incremental parser and will remove the
773parsed text from the input buffer. This is useful after C<incr_parse>
774died, in which case the input buffer and incremental parser state is left
775unchanged, to skip the text parsed so far and to reset the parse state.
776
777=item $json->incr_reset
778
779This completely resets the incremental parser, that is, after this call,
780it will be as if the parser had never parsed anything.
781
782This is useful if you want ot repeatedly parse JSON objects and want to
783ignore any trailing data, which means you have to reset the parser after
784each successful decode.
740 785
741=back 786=back
742 787
743=head2 LIMITATIONS 788=head2 LIMITATIONS
744 789
… …
985Other unblessed references are generally not allowed and will cause an 1030Other unblessed references are generally not allowed and will cause an
986exception to be thrown, except for references to the integers C<0> and 1031exception to be thrown, except for references to the integers C<0> and
987C<1>, which get turned into C<false> and C<true> atoms in JSON. You can 1032C<1>, which get turned into C<false> and C<true> atoms in JSON. You can
988also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability. 1033also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
989 1034
990 encode_json [\0,JSON::XS::true] # yields [false,true] 1035 encode_json [\0, JSON::XS::true] # yields [false,true]
991 1036
992=item JSON::XS::true, JSON::XS::false 1037=item JSON::XS::true, JSON::XS::false
993 1038
994These special values become JSON true and JSON false values, 1039These special values become JSON true and JSON false values,
995respectively. You can also use C<\1> and C<\0> directly if you want. 1040respectively. You can also use C<\1> and C<\0> directly if you want.
… …
1133as UTF-8, ISO-8859-1, ASCII, KOI8-R or most about any character set and 1178as UTF-8, ISO-8859-1, ASCII, KOI8-R or most about any character set and
11348-bit-encoding, and still get the same data structure back. This is useful 11798-bit-encoding, and still get the same data structure back. This is useful
1135when your channel for JSON transfer is not 8-bit clean or the encoding 1180when your channel for JSON transfer is not 8-bit clean or the encoding
1136might be mangled in between (e.g. in mail), and works because ASCII is a 1181might be mangled in between (e.g. in mail), and works because ASCII is a
1137proper subset of most 8-bit and multibyte encodings in use in the world. 1182proper subset of most 8-bit and multibyte encodings in use in the world.
1138
1139=back
1140
1141
1142=head1 COMPARISON
1143
1144As already mentioned, this module was created because none of the existing
1145JSON modules could be made to work correctly. First I will describe the
1146problems (or pleasures) I encountered with various existing JSON modules,
1147followed by some benchmark values. JSON::XS was designed not to suffer
1148from any of these problems or limitations.
1149
1150=over 4
1151
1152=item JSON 2.xx
1153
1154A marvellous piece of engineering, this module either uses JSON::XS
1155directly when available (so will be 100% compatible with it, including
1156speed), or it uses JSON::PP, which is basically JSON::XS translated to
1157Pure Perl, which should be 100% compatible with JSON::XS, just a bit
1158slower.
1159
1160You cannot really lose by using this module, especially as it tries very
1161hard to work even with ancient Perl versions, while JSON::XS does not.
1162
1163=item JSON 1.07
1164
1165Slow (but very portable, as it is written in pure Perl).
1166
1167Undocumented/buggy Unicode handling (how JSON handles Unicode values is
1168undocumented. One can get far by feeding it Unicode strings and doing
1169en-/decoding oneself, but Unicode escapes are not working properly).
1170
1171No round-tripping (strings get clobbered if they look like numbers, e.g.
1172the string C<2.0> will encode to C<2.0> instead of C<"2.0">, and that will
1173decode into the number 2.
1174
1175=item JSON::PC 0.01
1176
1177Very fast.
1178
1179Undocumented/buggy Unicode handling.
1180
1181No round-tripping.
1182
1183Has problems handling many Perl values (e.g. regex results and other magic
1184values will make it croak).
1185
1186Does not even generate valid JSON (C<{1,2}> gets converted to C<{1:2}>
1187which is not a valid JSON text.
1188
1189Unmaintained (maintainer unresponsive for many months, bugs are not
1190getting fixed).
1191
1192=item JSON::Syck 0.21
1193
1194Very buggy (often crashes).
1195
1196Very inflexible (no human-readable format supported, format pretty much
1197undocumented. I need at least a format for easy reading by humans and a
1198single-line compact format for use in a protocol, and preferably a way to
1199generate ASCII-only JSON texts).
1200
1201Completely broken (and confusingly documented) Unicode handling (Unicode
1202escapes are not working properly, you need to set ImplicitUnicode to
1203I<different> values on en- and decoding to get symmetric behaviour).
1204
1205No round-tripping (simple cases work, but this depends on whether the scalar
1206value was used in a numeric context or not).
1207
1208Dumping hashes may skip hash values depending on iterator state.
1209
1210Unmaintained (maintainer unresponsive for many months, bugs are not
1211getting fixed).
1212
1213Does not check input for validity (i.e. will accept non-JSON input and
1214return "something" instead of raising an exception. This is a security
1215issue: imagine two banks transferring money between each other using
1216JSON. One bank might parse a given non-JSON request and deduct money,
1217while the other might reject the transaction with a syntax error. While a
1218good protocol will at least recover, that is extra unnecessary work and
1219the transaction will still not succeed).
1220
1221=item JSON::DWIW 0.04
1222
1223Very fast. Very natural. Very nice.
1224
1225Undocumented Unicode handling (but the best of the pack. Unicode escapes
1226still don't get parsed properly).
1227
1228Very inflexible.
1229
1230No round-tripping.
1231
1232Does not generate valid JSON texts (key strings are often unquoted, empty keys
1233result in nothing being output)
1234
1235Does not check input for validity.
1236 1183
1237=back 1184=back
1238 1185
1239 1186
1240=head2 JSON and YAML 1187=head2 JSON and YAML
… …
1300 1247
1301First comes a comparison between various modules using 1248First comes a comparison between various modules using
1302a very short single-line JSON string (also available at 1249a very short single-line JSON string (also available at
1303L<http://dist.schmorp.de/misc/json/short.json>). 1250L<http://dist.schmorp.de/misc/json/short.json>).
1304 1251
1305 {"method": "handleMessage", "params": ["user1", "we were just talking"], \ 1252 {"method": "handleMessage", "params": ["user1",
1306 "id": null, "array":[1,11,234,-5,1e5,1e7, true, false]} 1253 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1254 true, false]}
1307 1255
1308It shows the number of encodes/decodes per second (JSON::XS uses 1256It shows the number of encodes/decodes per second (JSON::XS uses
1309the functional interface, while JSON::XS/2 uses the OO interface 1257the functional interface, while JSON::XS/2 uses the OO interface
1310with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables 1258with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1311shrink). Higher is better: 1259shrink). Higher is better:
… …
1409 1357
1410 1358
1411=head1 BUGS 1359=head1 BUGS
1412 1360
1413While the goal of this module is to be correct, that unfortunately does 1361While the goal of this module is to be correct, that unfortunately does
1414not mean it's bug-free, only that I think its design is bug-free. It is 1362not mean it's bug-free, only that I think its design is bug-free. If you
1415still relatively early in its development. If you keep reporting bugs they 1363keep reporting bugs they will be fixed swiftly, though.
1416will be fixed swiftly, though.
1417 1364
1418Please refrain from using rt.cpan.org or any other bug reporting 1365Please refrain from using rt.cpan.org or any other bug reporting
1419service. I put the contact address into my modules for a reason. 1366service. I put the contact address into my modules for a reason.
1420 1367
1421=cut 1368=cut

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines