ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.92 by root, Fri Mar 21 21:47:43 2008 UTC vs.
Revision 1.127 by root, Sun Jan 10 10:07:13 2010 UTC

1=head1 NAME 1=head1 NAME
2 2
3JSON::XS - JSON serialising/deserialising, done correctly and fast
4
3=encoding utf-8 5=encoding utf-8
4
5JSON::XS - JSON serialising/deserialising, done correctly and fast
6 6
7JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ 7JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ
8 (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html) 8 (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html)
9 9
10=head1 SYNOPSIS 10=head1 SYNOPSIS
37primary goal is to be I<correct> and its secondary goal is to be 37primary goal is to be I<correct> and its secondary goal is to be
38I<fast>. To reach the latter goal it was written in C. 38I<fast>. To reach the latter goal it was written in C.
39 39
40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overriden) with no overhead due to emulation (by inheritign constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need and doesn't
46require a C compiler when that is a problem. 46require a C compiler when that is a problem.
47 47
49to write yet another JSON module? While it seems there are many JSON 49to write yet another JSON module? While it seems there are many JSON
50modules, none of them correctly handle all corner cases, and in most cases 50modules, none of them correctly handle all corner cases, and in most cases
51their maintainers are unresponsive, gone missing, or not listening to bug 51their maintainers are unresponsive, gone missing, or not listening to bug
52reports for other reasons. 52reports for other reasons.
53 53
54See COMPARISON, below, for a comparison to some other JSON modules.
55
56See MAPPING, below, on how JSON::XS maps perl values to JSON values and 54See MAPPING, below, on how JSON::XS maps perl values to JSON values and
57vice versa. 55vice versa.
58 56
59=head2 FEATURES 57=head2 FEATURES
60 58
65This module knows how to handle Unicode, documents how and when it does 63This module knows how to handle Unicode, documents how and when it does
66so, and even documents what "correct" means. 64so, and even documents what "correct" means.
67 65
68=item * round-trip integrity 66=item * round-trip integrity
69 67
70When you serialise a perl data structure using only datatypes supported 68When you serialise a perl data structure using only data types supported
71by JSON, the deserialised data structure is identical on the Perl level. 69by JSON, the deserialised data structure is identical on the Perl level.
72(e.g. the string "2.0" doesn't suddenly become "2" just because it looks 70(e.g. the string "2.0" doesn't suddenly become "2" just because it looks
73like a number). There minor I<are> exceptions to this, read the MAPPING 71like a number). There minor I<are> exceptions to this, read the MAPPING
74section below to learn about those. 72section below to learn about those.
75 73
84Compared to other JSON modules and other serialisers such as Storable, 82Compared to other JSON modules and other serialisers such as Storable,
85this module usually compares favourably in terms of speed, too. 83this module usually compares favourably in terms of speed, too.
86 84
87=item * simple to use 85=item * simple to use
88 86
89This module has both a simple functional interface as well as an objetc 87This module has both a simple functional interface as well as an object
90oriented interface interface. 88oriented interface interface.
91 89
92=item * reasonably versatile output formats 90=item * reasonably versatile output formats
93 91
94You can choose between the most compact guaranteed-single-line format 92You can choose between the most compact guaranteed-single-line format
95possible (nice for simple line-based protocols), a pure-ascii format 93possible (nice for simple line-based protocols), a pure-ASCII format
96(for when your transport is not 8-bit clean, still supports the whole 94(for when your transport is not 8-bit clean, still supports the whole
97Unicode range), or a pretty-printed format (for when you want to read that 95Unicode range), or a pretty-printed format (for when you want to read that
98stuff). Or you can combine those features in whatever way you like. 96stuff). Or you can combine those features in whatever way you like.
99 97
100=back 98=back
101 99
102=cut 100=cut
103 101
104package JSON::XS; 102package JSON::XS;
105 103
106use strict; 104use common::sense;
107 105
108our $VERSION = '2.1'; 106our $VERSION = '2.27';
109our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
110 108
111our @EXPORT = qw(encode_json decode_json to_json from_json); 109our @EXPORT = qw(encode_json decode_json to_json from_json);
112 110
113sub to_json($) { 111sub to_json($) {
137 135
138This function call is functionally identical to: 136This function call is functionally identical to:
139 137
140 $json_text = JSON::XS->new->utf8->encode ($perl_scalar) 138 $json_text = JSON::XS->new->utf8->encode ($perl_scalar)
141 139
142except being faster. 140Except being faster.
143 141
144=item $perl_scalar = decode_json $json_text 142=item $perl_scalar = decode_json $json_text
145 143
146The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 144The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries
147to parse that as an UTF-8 encoded JSON text, returning the resulting 145to parse that as an UTF-8 encoded JSON text, returning the resulting
149 147
150This function call is functionally identical to: 148This function call is functionally identical to:
151 149
152 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 150 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
153 151
154except being faster. 152Except being faster.
155 153
156=item $is_boolean = JSON::XS::is_bool $scalar 154=item $is_boolean = JSON::XS::is_bool $scalar
157 155
158Returns true if the passed scalar represents either JSON::XS::true or 156Returns true if the passed scalar represents either JSON::XS::true or
159JSON::XS::false, two constants that act like C<1> and C<0>, respectively 157JSON::XS::false, two constants that act like C<1> and C<0>, respectively
197 195
198If you didn't know about that flag, just the better, pretend it doesn't 196If you didn't know about that flag, just the better, pretend it doesn't
199exist. 197exist.
200 198
201=item 4. A "Unicode String" is simply a string where each character can be 199=item 4. A "Unicode String" is simply a string where each character can be
202validly interpreted as a Unicode codepoint. 200validly interpreted as a Unicode code point.
203 201
204If you have UTF-8 encoded data, it is no longer a Unicode string, but a 202If you have UTF-8 encoded data, it is no longer a Unicode string, but a
205Unicode string encoded in UTF-8, giving you a binary string. 203Unicode string encoded in UTF-8, giving you a binary string.
206 204
207=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string. 205=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string.
443the same hash might be encoded differently even if contains the same data, 441the same hash might be encoded differently even if contains the same data,
444as key-value pairs have no inherent ordering in Perl. 442as key-value pairs have no inherent ordering in Perl.
445 443
446This setting has no effect when decoding JSON texts. 444This setting has no effect when decoding JSON texts.
447 445
446This setting has currently no effect on tied hashes.
447
448=item $json = $json->allow_nonref ([$enable]) 448=item $json = $json->allow_nonref ([$enable])
449 449
450=item $enabled = $json->get_allow_nonref 450=item $enabled = $json->get_allow_nonref
451 451
452If C<$enable> is true (or missing), then the C<encode> method can convert a 452If C<$enable> is true (or missing), then the C<encode> method can convert a
462Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>, 462Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>,
463resulting in an invalid JSON text: 463resulting in an invalid JSON text:
464 464
465 JSON::XS->new->allow_nonref->encode ("Hello, World!") 465 JSON::XS->new->allow_nonref->encode ("Hello, World!")
466 => "Hello, World!" 466 => "Hello, World!"
467
468=item $json = $json->allow_unknown ([$enable])
469
470=item $enabled = $json->get_allow_unknown
471
472If C<$enable> is true (or missing), then C<encode> will I<not> throw an
473exception when it encounters values it cannot represent in JSON (for
474example, filehandles) but instead will encode a JSON C<null> value. Note
475that blessed objects are not included here and are handled separately by
476c<allow_nonref>.
477
478If C<$enable> is false (the default), then C<encode> will throw an
479exception when it encounters anything it cannot encode as JSON.
480
481This option does not affect C<decode> in any way, and it is recommended to
482leave it off unless you know your communications partner.
467 483
468=item $json = $json->allow_blessed ([$enable]) 484=item $json = $json->allow_blessed ([$enable])
469 485
470=item $enabled = $json->get_allow_blessed 486=item $enabled = $json->get_allow_blessed
471 487
612=item $json = $json->max_depth ([$maximum_nesting_depth]) 628=item $json = $json->max_depth ([$maximum_nesting_depth])
613 629
614=item $max_depth = $json->get_max_depth 630=item $max_depth = $json->get_max_depth
615 631
616Sets the maximum nesting level (default C<512>) accepted while encoding 632Sets the maximum nesting level (default C<512>) accepted while encoding
617or decoding. If the JSON text or Perl data structure has an equal or 633or decoding. If a higher nesting level is detected in JSON text or a Perl
618higher nesting level then this limit, then the encoder and decoder will 634data structure, then the encoder and decoder will stop and croak at that
619stop and croak at that point. 635point.
620 636
621Nesting level is defined by number of hash- or arrayrefs that the encoder 637Nesting level is defined by number of hash- or arrayrefs that the encoder
622needs to traverse to reach a given point or the number of C<{> or C<[> 638needs to traverse to reach a given point or the number of C<{> or C<[>
623characters without their matching closing parenthesis crossed to reach a 639characters without their matching closing parenthesis crossed to reach a
624given character in a string. 640given character in a string.
625 641
626Setting the maximum depth to one disallows any nesting, so that ensures 642Setting the maximum depth to one disallows any nesting, so that ensures
627that the object is only a single hash/object or array. 643that the object is only a single hash/object or array.
628 644
629The argument to C<max_depth> will be rounded up to the next highest power
630of two. If no argument is given, the highest possible setting will be 645If no argument is given, the highest possible setting will be used, which
631used, which is rarely useful. 646is rarely useful.
647
648Note that nesting is implemented by recursion in C. The default value has
649been chosen to be as large as typical operating systems allow without
650crashing.
632 651
633See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 652See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
634 653
635=item $json = $json->max_size ([$maximum_string_size]) 654=item $json = $json->max_size ([$maximum_string_size])
636 655
637=item $max_size = $json->get_max_size 656=item $max_size = $json->get_max_size
638 657
639Set the maximum length a JSON text may have (in bytes) where decoding is 658Set the maximum length a JSON text may have (in bytes) where decoding is
640being attempted. The default is C<0>, meaning no limit. When C<decode> 659being attempted. The default is C<0>, meaning no limit. When C<decode>
641is called on a string longer then this number of characters it will not 660is called on a string that is longer then this many bytes, it will not
642attempt to decode the string but throw an exception. This setting has no 661attempt to decode the string but throw an exception. This setting has no
643effect on C<encode> (yet). 662effect on C<encode> (yet).
644 663
645The argument to C<max_size> will be rounded up to the next B<highest> 664If no argument is given, the limit check will be deactivated (same as when
646power of two (so may be more than requested). If no argument is given, the 665C<0> is specified).
647limit check will be deactivated (same as when C<0> is specified).
648 666
649See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 667See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
650 668
651=item $json_text = $json->encode ($perl_scalar) 669=item $json_text = $json->encode ($perl_scalar)
652 670
679 697
680 JSON::XS->new->decode_prefix ("[1] the tail") 698 JSON::XS->new->decode_prefix ("[1] the tail")
681 => ([], 3) 699 => ([], 3)
682 700
683=back 701=back
702
703
704=head1 INCREMENTAL PARSING
705
706In some cases, there is the need for incremental parsing of JSON
707texts. While this module always has to keep both JSON text and resulting
708Perl data structure in memory at one time, it does allow you to parse a
709JSON stream incrementally. It does so by accumulating text until it has
710a full JSON object, which it then can decode. This process is similar to
711using C<decode_prefix> to see if a full JSON object is available, but
712is much more efficient (and can be implemented with a minimum of method
713calls).
714
715JSON::XS will only attempt to parse the JSON text once it is sure it
716has enough text to get a decisive result, using a very simple but
717truly incremental parser. This means that it sometimes won't stop as
718early as the full parser, for example, it doesn't detect parenthese
719mismatches. The only thing it guarantees is that it starts decoding as
720soon as a syntactically valid JSON text has been seen. This means you need
721to set resource limits (e.g. C<max_size>) to ensure the parser will stop
722parsing in the presence if syntax errors.
723
724The following methods implement this incremental parser.
725
726=over 4
727
728=item [void, scalar or list context] = $json->incr_parse ([$string])
729
730This is the central parsing function. It can both append new text and
731extract objects from the stream accumulated so far (both of these
732functions are optional).
733
734If C<$string> is given, then this string is appended to the already
735existing JSON fragment stored in the C<$json> object.
736
737After that, if the function is called in void context, it will simply
738return without doing anything further. This can be used to add more text
739in as many chunks as you want.
740
741If the method is called in scalar context, then it will try to extract
742exactly I<one> JSON object. If that is successful, it will return this
743object, otherwise it will return C<undef>. If there is a parse error,
744this method will croak just as C<decode> would do (one can then use
745C<incr_skip> to skip the errornous part). This is the most common way of
746using the method.
747
748And finally, in list context, it will try to extract as many objects
749from the stream as it can find and return them, or the empty list
750otherwise. For this to work, there must be no separators between the JSON
751objects or arrays, instead they must be concatenated back-to-back. If
752an error occurs, an exception will be raised as in the scalar context
753case. Note that in this case, any previously-parsed JSON texts will be
754lost.
755
756=item $lvalue_string = $json->incr_text
757
758This method returns the currently stored JSON fragment as an lvalue, that
759is, you can manipulate it. This I<only> works when a preceding call to
760C<incr_parse> in I<scalar context> successfully returned an object. Under
761all other circumstances you must not call this function (I mean it.
762although in simple tests it might actually work, it I<will> fail under
763real world conditions). As a special exception, you can also call this
764method before having parsed anything.
765
766This function is useful in two cases: a) finding the trailing text after a
767JSON object or b) parsing multiple JSON objects separated by non-JSON text
768(such as commas).
769
770=item $json->incr_skip
771
772This will reset the state of the incremental parser and will remove
773the parsed text from the input buffer so far. This is useful after
774C<incr_parse> died, in which case the input buffer and incremental parser
775state is left unchanged, to skip the text parsed so far and to reset the
776parse state.
777
778The difference to C<incr_reset> is that only text until the parse error
779occured is removed.
780
781=item $json->incr_reset
782
783This completely resets the incremental parser, that is, after this call,
784it will be as if the parser had never parsed anything.
785
786This is useful if you want to repeatedly parse JSON objects and want to
787ignore any trailing data, which means you have to reset the parser after
788each successful decode.
789
790=back
791
792=head2 LIMITATIONS
793
794All options that affect decoding are supported, except
795C<allow_nonref>. The reason for this is that it cannot be made to
796work sensibly: JSON objects and arrays are self-delimited, i.e. you can concatenate
797them back to back and still decode them perfectly. This does not hold true
798for JSON numbers, however.
799
800For example, is the string C<1> a single JSON number, or is it simply the
801start of C<12>? Or is C<12> a single JSON number, or the concatenation
802of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS
803takes the conservative route and disallows this case.
804
805=head2 EXAMPLES
806
807Some examples will make all this clearer. First, a simple example that
808works similarly to C<decode_prefix>: We want to decode the JSON object at
809the start of a string and identify the portion after the JSON object:
810
811 my $text = "[1,2,3] hello";
812
813 my $json = new JSON::XS;
814
815 my $obj = $json->incr_parse ($text)
816 or die "expected JSON object or array at beginning of string";
817
818 my $tail = $json->incr_text;
819 # $tail now contains " hello"
820
821Easy, isn't it?
822
823Now for a more complicated example: Imagine a hypothetical protocol where
824you read some requests from a TCP stream, and each request is a JSON
825array, without any separation between them (in fact, it is often useful to
826use newlines as "separators", as these get interpreted as whitespace at
827the start of the JSON text, which makes it possible to test said protocol
828with C<telnet>...).
829
830Here is how you'd do it (it is trivial to write this in an event-based
831manner):
832
833 my $json = new JSON::XS;
834
835 # read some data from the socket
836 while (sysread $socket, my $buf, 4096) {
837
838 # split and decode as many requests as possible
839 for my $request ($json->incr_parse ($buf)) {
840 # act on the $request
841 }
842 }
843
844Another complicated example: Assume you have a string with JSON objects
845or arrays, all separated by (optional) comma characters (e.g. C<[1],[2],
846[3]>). To parse them, we have to skip the commas between the JSON texts,
847and here is where the lvalue-ness of C<incr_text> comes in useful:
848
849 my $text = "[1],[2], [3]";
850 my $json = new JSON::XS;
851
852 # void context, so no parsing done
853 $json->incr_parse ($text);
854
855 # now extract as many objects as possible. note the
856 # use of scalar context so incr_text can be called.
857 while (my $obj = $json->incr_parse) {
858 # do something with $obj
859
860 # now skip the optional comma
861 $json->incr_text =~ s/^ \s* , //x;
862 }
863
864Now lets go for a very complex example: Assume that you have a gigantic
865JSON array-of-objects, many gigabytes in size, and you want to parse it,
866but you cannot load it into memory fully (this has actually happened in
867the real world :).
868
869Well, you lost, you have to implement your own JSON parser. But JSON::XS
870can still help you: You implement a (very simple) array parser and let
871JSON decode the array elements, which are all full JSON objects on their
872own (this wouldn't work if the array elements could be JSON numbers, for
873example):
874
875 my $json = new JSON::XS;
876
877 # open the monster
878 open my $fh, "<bigfile.json"
879 or die "bigfile: $!";
880
881 # first parse the initial "["
882 for (;;) {
883 sysread $fh, my $buf, 65536
884 or die "read error: $!";
885 $json->incr_parse ($buf); # void context, so no parsing
886
887 # Exit the loop once we found and removed(!) the initial "[".
888 # In essence, we are (ab-)using the $json object as a simple scalar
889 # we append data to.
890 last if $json->incr_text =~ s/^ \s* \[ //x;
891 }
892
893 # now we have the skipped the initial "[", so continue
894 # parsing all the elements.
895 for (;;) {
896 # in this loop we read data until we got a single JSON object
897 for (;;) {
898 if (my $obj = $json->incr_parse) {
899 # do something with $obj
900 last;
901 }
902
903 # add more data
904 sysread $fh, my $buf, 65536
905 or die "read error: $!";
906 $json->incr_parse ($buf); # void context, so no parsing
907 }
908
909 # in this loop we read data until we either found and parsed the
910 # separating "," between elements, or the final "]"
911 for (;;) {
912 # first skip whitespace
913 $json->incr_text =~ s/^\s*//;
914
915 # if we find "]", we are done
916 if ($json->incr_text =~ s/^\]//) {
917 print "finished.\n";
918 exit;
919 }
920
921 # if we find ",", we can continue with the next element
922 if ($json->incr_text =~ s/^,//) {
923 last;
924 }
925
926 # if we find anything else, we have a parse error!
927 if (length $json->incr_text) {
928 die "parse error near ", $json->incr_text;
929 }
930
931 # else add more data
932 sysread $fh, my $buf, 65536
933 or die "read error: $!";
934 $json->incr_parse ($buf); # void context, so no parsing
935 }
936
937This is a complex example, but most of the complexity comes from the fact
938that we are trying to be correct (bear with me if I am wrong, I never ran
939the above example :).
940
684 941
685 942
686=head1 MAPPING 943=head1 MAPPING
687 944
688This section describes how JSON::XS maps Perl values to JSON values and 945This section describes how JSON::XS maps Perl values to JSON values and
777Other unblessed references are generally not allowed and will cause an 1034Other unblessed references are generally not allowed and will cause an
778exception to be thrown, except for references to the integers C<0> and 1035exception to be thrown, except for references to the integers C<0> and
779C<1>, which get turned into C<false> and C<true> atoms in JSON. You can 1036C<1>, which get turned into C<false> and C<true> atoms in JSON. You can
780also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability. 1037also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
781 1038
782 encode_json [\0,JSON::XS::true] # yields [false,true] 1039 encode_json [\0, JSON::XS::true] # yields [false,true]
783 1040
784=item JSON::XS::true, JSON::XS::false 1041=item JSON::XS::true, JSON::XS::false
785 1042
786These special values become JSON true and JSON false values, 1043These special values become JSON true and JSON false values,
787respectively. You can also use C<\1> and C<\0> directly if you want. 1044respectively. You can also use C<\1> and C<\0> directly if you want.
929proper subset of most 8-bit and multibyte encodings in use in the world. 1186proper subset of most 8-bit and multibyte encodings in use in the world.
930 1187
931=back 1188=back
932 1189
933 1190
934=head1 COMPARISON 1191=head2 JSON and ECMAscript
935 1192
936As already mentioned, this module was created because none of the existing 1193JSON syntax is based on how literals are represented in javascript (the
937JSON modules could be made to work correctly. First I will describe the 1194not-standardised predecessor of ECMAscript) which is presumably why it is
938problems (or pleasures) I encountered with various existing JSON modules, 1195called "JavaScript Object Notation".
939followed by some benchmark values. JSON::XS was designed not to suffer
940from any of these problems or limitations.
941 1196
942=over 4 1197However, JSON is not a subset (and also not a superset of course) of
1198ECMAscript (the standard) or javascript (whatever browsers actually
1199implement).
943 1200
944=item JSON 2.xx 1201If you want to use javascript's C<eval> function to "parse" JSON, you
1202might run into parse errors for valid JSON texts, or the resulting data
1203structure might not be queryable:
945 1204
946A marvellous piece of engineering, this module either uses JSON::XS 1205One of the problems is that U+2028 and U+2029 are valid characters inside
947directly when available (so will be 100% compatible with it, including 1206JSON strings, but are not allowed in ECMAscript string literals, so the
948speed), or it uses JSON::PP, which is basically JSON::XS translated to 1207following Perl fragment will not output something that can be guaranteed
949Pure Perl, which should be 100% compatible with JSON::XS, just a bit 1208to be parsable by javascript's C<eval>:
950slower.
951 1209
952You cannot really lose by using this module, especially as it tries very 1210 use JSON::XS;
953hard to work even with ancient Perl versions, while JSON::XS does not.
954 1211
955=item JSON 1.07 1212 print encode_json [chr 0x2028];
956 1213
957Slow (but very portable, as it is written in pure Perl). 1214The right fix for this is to use a proper JSON parser in your javascript
1215programs, and not rely on C<eval> (see for example Douglas Crockford's
1216F<json2.js> parser).
958 1217
959Undocumented/buggy Unicode handling (how JSON handles Unicode values is 1218If this is not an option, you can, as a stop-gap measure, simply encode to
960undocumented. One can get far by feeding it Unicode strings and doing 1219ASCII-only JSON:
961en-/decoding oneself, but Unicode escapes are not working properly).
962 1220
963No round-tripping (strings get clobbered if they look like numbers, e.g. 1221 use JSON::XS;
964the string C<2.0> will encode to C<2.0> instead of C<"2.0">, and that will
965decode into the number 2.
966 1222
967=item JSON::PC 0.01 1223 print JSON::XS->new->ascii->encode ([chr 0x2028]);
968 1224
969Very fast. 1225Note that this will enlarge the resulting JSON text quite a bit if you
1226have many non-ASCII characters. You might be tempted to run some regexes
1227to only escape U+2028 and U+2029, e.g.:
970 1228
971Undocumented/buggy Unicode handling. 1229 # DO NOT USE THIS!
1230 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1231 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1232 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1233 print $json;
972 1234
973No round-tripping. 1235Note that I<this is a bad idea>: the above only works for U+2028 and
1236U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1237javascript implementations, however, have issues with other characters as
1238well - using C<eval> naively simply I<will> cause problems.
974 1239
975Has problems handling many Perl values (e.g. regex results and other magic 1240Another problem is that some javascript implementations reserve
976values will make it croak). 1241some property names for their own purposes (which probably makes
1242them non-ECMAscript-compliant). For example, Iceweasel reserves the
1243C<__proto__> property name for it's own purposes.
977 1244
978Does not even generate valid JSON (C<{1,2}> gets converted to C<{1:2}> 1245If that is a problem, you could parse try to filter the resulting JSON
979which is not a valid JSON text. 1246output for these property strings, e.g.:
980 1247
981Unmaintained (maintainer unresponsive for many months, bugs are not 1248 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
982getting fixed).
983 1249
984=item JSON::Syck 0.21 1250This works because C<__proto__> is not valid outside of strings, so every
1251occurence of C<"__proto__"\s*:> must be a string used as property name.
985 1252
986Very buggy (often crashes). 1253If you know of other incompatibilities, please let me know.
987
988Very inflexible (no human-readable format supported, format pretty much
989undocumented. I need at least a format for easy reading by humans and a
990single-line compact format for use in a protocol, and preferably a way to
991generate ASCII-only JSON texts).
992
993Completely broken (and confusingly documented) Unicode handling (Unicode
994escapes are not working properly, you need to set ImplicitUnicode to
995I<different> values on en- and decoding to get symmetric behaviour).
996
997No round-tripping (simple cases work, but this depends on whether the scalar
998value was used in a numeric context or not).
999
1000Dumping hashes may skip hash values depending on iterator state.
1001
1002Unmaintained (maintainer unresponsive for many months, bugs are not
1003getting fixed).
1004
1005Does not check input for validity (i.e. will accept non-JSON input and
1006return "something" instead of raising an exception. This is a security
1007issue: imagine two banks transferring money between each other using
1008JSON. One bank might parse a given non-JSON request and deduct money,
1009while the other might reject the transaction with a syntax error. While a
1010good protocol will at least recover, that is extra unnecessary work and
1011the transaction will still not succeed).
1012
1013=item JSON::DWIW 0.04
1014
1015Very fast. Very natural. Very nice.
1016
1017Undocumented Unicode handling (but the best of the pack. Unicode escapes
1018still don't get parsed properly).
1019
1020Very inflexible.
1021
1022No round-tripping.
1023
1024Does not generate valid JSON texts (key strings are often unquoted, empty keys
1025result in nothing being output)
1026
1027Does not check input for validity.
1028
1029=back
1030 1254
1031 1255
1032=head2 JSON and YAML 1256=head2 JSON and YAML
1033 1257
1034You often hear that JSON is a subset of YAML. This is, however, a mass 1258You often hear that JSON is a subset of YAML. This is, however, a mass
1044 my $yaml = $to_yaml->encode ($ref) . "\n"; 1268 my $yaml = $to_yaml->encode ($ref) . "\n";
1045 1269
1046This will I<usually> generate JSON texts that also parse as valid 1270This will I<usually> generate JSON texts that also parse as valid
1047YAML. Please note that YAML has hardcoded limits on (simple) object key 1271YAML. Please note that YAML has hardcoded limits on (simple) object key
1048lengths that JSON doesn't have and also has different and incompatible 1272lengths that JSON doesn't have and also has different and incompatible
1049unicode handling, so you should make sure that your hash keys are 1273unicode character escape syntax, so you should make sure that your hash
1050noticeably shorter than the 1024 "stream characters" YAML allows and that 1274keys are noticeably shorter than the 1024 "stream characters" YAML allows
1051you do not have characters with codepoint values outside the Unicode BMP 1275and that you do not have characters with codepoint values outside the
1052(basic multilingual page). YAML also does not allow C<\/> sequences in 1276Unicode BMP (basic multilingual page). YAML also does not allow C<\/>
1053strings (which JSON::XS does not I<currently> generate, but other JSON 1277sequences in strings (which JSON::XS does not I<currently> generate, but
1054generators might). 1278other JSON generators might).
1055 1279
1056There might be other incompatibilities that I am not aware of (or the YAML 1280There might be other incompatibilities that I am not aware of (or the YAML
1057specification has been changed yet again - it does so quite often). In 1281specification has been changed yet again - it does so quite often). In
1058general you should not try to generate YAML with a JSON generator or vice 1282general you should not try to generate YAML with a JSON generator or vice
1059versa, or try to parse JSON with a YAML parser or vice versa: chances are 1283versa, or try to parse JSON with a YAML parser or vice versa: chances are
1078that difficult or long) and finally make YAML compatible to it, and 1302that difficult or long) and finally make YAML compatible to it, and
1079educating users about the changes, instead of spreading lies about the 1303educating users about the changes, instead of spreading lies about the
1080real compatibility for many I<years> and trying to silence people who 1304real compatibility for many I<years> and trying to silence people who
1081point out that it isn't true. 1305point out that it isn't true.
1082 1306
1307Addendum/2009: the YAML 1.2 spec is still incomaptible with JSON, even
1308though the incompatibilities have been documented (and are known to
1309Brian) for many years and the spec makes explicit claims that YAML is a
1310superset of JSON. It would be so easy to fix, but apparently, bullying and
1311corrupting userdata is so much easier.
1312
1083=back 1313=back
1084 1314
1085 1315
1086=head2 SPEED 1316=head2 SPEED
1087 1317
1092 1322
1093First comes a comparison between various modules using 1323First comes a comparison between various modules using
1094a very short single-line JSON string (also available at 1324a very short single-line JSON string (also available at
1095L<http://dist.schmorp.de/misc/json/short.json>). 1325L<http://dist.schmorp.de/misc/json/short.json>).
1096 1326
1097 {"method": "handleMessage", "params": ["user1", "we were just talking"], \ 1327 {"method": "handleMessage", "params": ["user1",
1098 "id": null, "array":[1,11,234,-5,1e5,1e7, true, false]} 1328 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1329 true, false]}
1099 1330
1100It shows the number of encodes/decodes per second (JSON::XS uses 1331It shows the number of encodes/decodes per second (JSON::XS uses
1101the functional interface, while JSON::XS/2 uses the OO interface 1332the functional interface, while JSON::XS/2 uses the OO interface
1102with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables 1333with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1103shrink). Higher is better: 1334shrink). Higher is better:
1181information you might want to make sure that exceptions thrown by JSON::XS 1412information you might want to make sure that exceptions thrown by JSON::XS
1182will not end up in front of untrusted eyes. 1413will not end up in front of untrusted eyes.
1183 1414
1184If you are using JSON::XS to return packets to consumption 1415If you are using JSON::XS to return packets to consumption
1185by JavaScript scripts in a browser you should have a look at 1416by JavaScript scripts in a browser you should have a look at
1186L<http://jpsykes.com/47/practical-csrf-and-json-security> to see whether 1417L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to
1187you are vulnerable to some common attack vectors (which really are browser 1418see whether you are vulnerable to some common attack vectors (which really
1188design bugs, but it is still you who will have to deal with it, as major 1419are browser design bugs, but it is still you who will have to deal with
1189browser developers care only for features, not about getting security 1420it, as major browser developers care only for features, not about getting
1190right). 1421security right).
1191 1422
1192 1423
1193=head1 THREADS 1424=head1 THREADS
1194 1425
1195This module is I<not> guaranteed to be thread safe and there are no 1426This module is I<not> guaranteed to be thread safe and there are no
1201 1432
1202 1433
1203=head1 BUGS 1434=head1 BUGS
1204 1435
1205While the goal of this module is to be correct, that unfortunately does 1436While the goal of this module is to be correct, that unfortunately does
1206not mean it's bug-free, only that I think its design is bug-free. It is 1437not mean it's bug-free, only that I think its design is bug-free. If you
1207still relatively early in its development. If you keep reporting bugs they 1438keep reporting bugs they will be fixed swiftly, though.
1208will be fixed swiftly, though.
1209 1439
1210Please refrain from using rt.cpan.org or any other bug reporting 1440Please refrain from using rt.cpan.org or any other bug reporting
1211service. I put the contact address into my modules for a reason. 1441service. I put the contact address into my modules for a reason.
1212 1442
1213=cut 1443=cut
1233 "--" => sub { $_[0] = ${$_[0]} - 1 }, 1463 "--" => sub { $_[0] = ${$_[0]} - 1 },
1234 fallback => 1; 1464 fallback => 1;
1235 1465
12361; 14661;
1237 1467
1468=head1 SEE ALSO
1469
1470The F<json_xs> command line utility for quick experiments.
1471
1238=head1 AUTHOR 1472=head1 AUTHOR
1239 1473
1240 Marc Lehmann <schmorp@schmorp.de> 1474 Marc Lehmann <schmorp@schmorp.de>
1241 http://home.schmorp.de/ 1475 http://home.schmorp.de/
1242 1476

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines