ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.91 by root, Thu Mar 20 02:11:21 2008 UTC vs.
Revision 1.130 by root, Thu Mar 11 17:36:09 2010 UTC

1=head1 NAME
2
3JSON::XS - JSON serialising/deserialising, done correctly and fast
4
1=encoding utf-8 5=encoding utf-8
2
3=head1 NAME
4
5JSON::XS - JSON serialising/deserialising, done correctly and fast
6 6
7JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ 7JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ
8 (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html) 8 (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html)
9 9
10=head1 SYNOPSIS 10=head1 SYNOPSIS
37primary goal is to be I<correct> and its secondary goal is to be 37primary goal is to be I<correct> and its secondary goal is to be
38I<fast>. To reach the latter goal it was written in C. 38I<fast>. To reach the latter goal it was written in C.
39 39
40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overriden) with no overhead due to emulation (by inheritign constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need and doesn't
46require a C compiler when that is a problem. 46require a C compiler when that is a problem.
47 47
49to write yet another JSON module? While it seems there are many JSON 49to write yet another JSON module? While it seems there are many JSON
50modules, none of them correctly handle all corner cases, and in most cases 50modules, none of them correctly handle all corner cases, and in most cases
51their maintainers are unresponsive, gone missing, or not listening to bug 51their maintainers are unresponsive, gone missing, or not listening to bug
52reports for other reasons. 52reports for other reasons.
53 53
54See COMPARISON, below, for a comparison to some other JSON modules.
55
56See MAPPING, below, on how JSON::XS maps perl values to JSON values and 54See MAPPING, below, on how JSON::XS maps perl values to JSON values and
57vice versa. 55vice versa.
58 56
59=head2 FEATURES 57=head2 FEATURES
60 58
65This module knows how to handle Unicode, documents how and when it does 63This module knows how to handle Unicode, documents how and when it does
66so, and even documents what "correct" means. 64so, and even documents what "correct" means.
67 65
68=item * round-trip integrity 66=item * round-trip integrity
69 67
70When you serialise a perl data structure using only datatypes supported 68When you serialise a perl data structure using only data types supported
71by JSON, the deserialised data structure is identical on the Perl level. 69by JSON, the deserialised data structure is identical on the Perl level.
72(e.g. the string "2.0" doesn't suddenly become "2" just because it looks 70(e.g. the string "2.0" doesn't suddenly become "2" just because it looks
73like a number). There minor I<are> exceptions to this, read the MAPPING 71like a number). There minor I<are> exceptions to this, read the MAPPING
74section below to learn about those. 72section below to learn about those.
75 73
84Compared to other JSON modules and other serialisers such as Storable, 82Compared to other JSON modules and other serialisers such as Storable,
85this module usually compares favourably in terms of speed, too. 83this module usually compares favourably in terms of speed, too.
86 84
87=item * simple to use 85=item * simple to use
88 86
89This module has both a simple functional interface as well as an objetc 87This module has both a simple functional interface as well as an object
90oriented interface interface. 88oriented interface interface.
91 89
92=item * reasonably versatile output formats 90=item * reasonably versatile output formats
93 91
94You can choose between the most compact guaranteed-single-line format 92You can choose between the most compact guaranteed-single-line format
95possible (nice for simple line-based protocols), a pure-ascii format 93possible (nice for simple line-based protocols), a pure-ASCII format
96(for when your transport is not 8-bit clean, still supports the whole 94(for when your transport is not 8-bit clean, still supports the whole
97Unicode range), or a pretty-printed format (for when you want to read that 95Unicode range), or a pretty-printed format (for when you want to read that
98stuff). Or you can combine those features in whatever way you like. 96stuff). Or you can combine those features in whatever way you like.
99 97
100=back 98=back
101 99
102=cut 100=cut
103 101
104package JSON::XS; 102package JSON::XS;
105 103
106use strict; 104use common::sense;
107 105
108our $VERSION = '2.1'; 106our $VERSION = '2.28';
109our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
110 108
111our @EXPORT = qw(encode_json decode_json to_json from_json); 109our @EXPORT = qw(encode_json decode_json to_json from_json);
112 110
113sub to_json($) { 111sub to_json($) {
137 135
138This function call is functionally identical to: 136This function call is functionally identical to:
139 137
140 $json_text = JSON::XS->new->utf8->encode ($perl_scalar) 138 $json_text = JSON::XS->new->utf8->encode ($perl_scalar)
141 139
142except being faster. 140Except being faster.
143 141
144=item $perl_scalar = decode_json $json_text 142=item $perl_scalar = decode_json $json_text
145 143
146The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 144The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries
147to parse that as an UTF-8 encoded JSON text, returning the resulting 145to parse that as an UTF-8 encoded JSON text, returning the resulting
149 147
150This function call is functionally identical to: 148This function call is functionally identical to:
151 149
152 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 150 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
153 151
154except being faster. 152Except being faster.
155 153
156=item $is_boolean = JSON::XS::is_bool $scalar 154=item $is_boolean = JSON::XS::is_bool $scalar
157 155
158Returns true if the passed scalar represents either JSON::XS::true or 156Returns true if the passed scalar represents either JSON::XS::true or
159JSON::XS::false, two constants that act like C<1> and C<0>, respectively 157JSON::XS::false, two constants that act like C<1> and C<0>, respectively
197 195
198If you didn't know about that flag, just the better, pretend it doesn't 196If you didn't know about that flag, just the better, pretend it doesn't
199exist. 197exist.
200 198
201=item 4. A "Unicode String" is simply a string where each character can be 199=item 4. A "Unicode String" is simply a string where each character can be
202validly interpreted as a Unicode codepoint. 200validly interpreted as a Unicode code point.
203 201
204If you have UTF-8 encoded data, it is no longer a Unicode string, but a 202If you have UTF-8 encoded data, it is no longer a Unicode string, but a
205Unicode string encoded in UTF-8, giving you a binary string. 203Unicode string encoded in UTF-8, giving you a binary string.
206 204
207=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string. 205=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string.
443the same hash might be encoded differently even if contains the same data, 441the same hash might be encoded differently even if contains the same data,
444as key-value pairs have no inherent ordering in Perl. 442as key-value pairs have no inherent ordering in Perl.
445 443
446This setting has no effect when decoding JSON texts. 444This setting has no effect when decoding JSON texts.
447 445
446This setting has currently no effect on tied hashes.
447
448=item $json = $json->allow_nonref ([$enable]) 448=item $json = $json->allow_nonref ([$enable])
449 449
450=item $enabled = $json->get_allow_nonref 450=item $enabled = $json->get_allow_nonref
451 451
452If C<$enable> is true (or missing), then the C<encode> method can convert a 452If C<$enable> is true (or missing), then the C<encode> method can convert a
462Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>, 462Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>,
463resulting in an invalid JSON text: 463resulting in an invalid JSON text:
464 464
465 JSON::XS->new->allow_nonref->encode ("Hello, World!") 465 JSON::XS->new->allow_nonref->encode ("Hello, World!")
466 => "Hello, World!" 466 => "Hello, World!"
467
468=item $json = $json->allow_unknown ([$enable])
469
470=item $enabled = $json->get_allow_unknown
471
472If C<$enable> is true (or missing), then C<encode> will I<not> throw an
473exception when it encounters values it cannot represent in JSON (for
474example, filehandles) but instead will encode a JSON C<null> value. Note
475that blessed objects are not included here and are handled separately by
476c<allow_nonref>.
477
478If C<$enable> is false (the default), then C<encode> will throw an
479exception when it encounters anything it cannot encode as JSON.
480
481This option does not affect C<decode> in any way, and it is recommended to
482leave it off unless you know your communications partner.
467 483
468=item $json = $json->allow_blessed ([$enable]) 484=item $json = $json->allow_blessed ([$enable])
469 485
470=item $enabled = $json->get_allow_blessed 486=item $enabled = $json->get_allow_blessed
471 487
612=item $json = $json->max_depth ([$maximum_nesting_depth]) 628=item $json = $json->max_depth ([$maximum_nesting_depth])
613 629
614=item $max_depth = $json->get_max_depth 630=item $max_depth = $json->get_max_depth
615 631
616Sets the maximum nesting level (default C<512>) accepted while encoding 632Sets the maximum nesting level (default C<512>) accepted while encoding
617or decoding. If the JSON text or Perl data structure has an equal or 633or decoding. If a higher nesting level is detected in JSON text or a Perl
618higher nesting level then this limit, then the encoder and decoder will 634data structure, then the encoder and decoder will stop and croak at that
619stop and croak at that point. 635point.
620 636
621Nesting level is defined by number of hash- or arrayrefs that the encoder 637Nesting level is defined by number of hash- or arrayrefs that the encoder
622needs to traverse to reach a given point or the number of C<{> or C<[> 638needs to traverse to reach a given point or the number of C<{> or C<[>
623characters without their matching closing parenthesis crossed to reach a 639characters without their matching closing parenthesis crossed to reach a
624given character in a string. 640given character in a string.
625 641
626Setting the maximum depth to one disallows any nesting, so that ensures 642Setting the maximum depth to one disallows any nesting, so that ensures
627that the object is only a single hash/object or array. 643that the object is only a single hash/object or array.
628 644
629The argument to C<max_depth> will be rounded up to the next highest power
630of two. If no argument is given, the highest possible setting will be 645If no argument is given, the highest possible setting will be used, which
631used, which is rarely useful. 646is rarely useful.
647
648Note that nesting is implemented by recursion in C. The default value has
649been chosen to be as large as typical operating systems allow without
650crashing.
632 651
633See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 652See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
634 653
635=item $json = $json->max_size ([$maximum_string_size]) 654=item $json = $json->max_size ([$maximum_string_size])
636 655
637=item $max_size = $json->get_max_size 656=item $max_size = $json->get_max_size
638 657
639Set the maximum length a JSON text may have (in bytes) where decoding is 658Set the maximum length a JSON text may have (in bytes) where decoding is
640being attempted. The default is C<0>, meaning no limit. When C<decode> 659being attempted. The default is C<0>, meaning no limit. When C<decode>
641is called on a string longer then this number of characters it will not 660is called on a string that is longer then this many bytes, it will not
642attempt to decode the string but throw an exception. This setting has no 661attempt to decode the string but throw an exception. This setting has no
643effect on C<encode> (yet). 662effect on C<encode> (yet).
644 663
645The argument to C<max_size> will be rounded up to the next B<highest> 664If no argument is given, the limit check will be deactivated (same as when
646power of two (so may be more than requested). If no argument is given, the 665C<0> is specified).
647limit check will be deactivated (same as when C<0> is specified).
648 666
649See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 667See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
650 668
651=item $json_text = $json->encode ($perl_scalar) 669=item $json_text = $json->encode ($perl_scalar)
652 670
679 697
680 JSON::XS->new->decode_prefix ("[1] the tail") 698 JSON::XS->new->decode_prefix ("[1] the tail")
681 => ([], 3) 699 => ([], 3)
682 700
683=back 701=back
702
703
704=head1 INCREMENTAL PARSING
705
706In some cases, there is the need for incremental parsing of JSON
707texts. While this module always has to keep both JSON text and resulting
708Perl data structure in memory at one time, it does allow you to parse a
709JSON stream incrementally. It does so by accumulating text until it has
710a full JSON object, which it then can decode. This process is similar to
711using C<decode_prefix> to see if a full JSON object is available, but
712is much more efficient (and can be implemented with a minimum of method
713calls).
714
715JSON::XS will only attempt to parse the JSON text once it is sure it
716has enough text to get a decisive result, using a very simple but
717truly incremental parser. This means that it sometimes won't stop as
718early as the full parser, for example, it doesn't detect parenthese
719mismatches. The only thing it guarantees is that it starts decoding as
720soon as a syntactically valid JSON text has been seen. This means you need
721to set resource limits (e.g. C<max_size>) to ensure the parser will stop
722parsing in the presence if syntax errors.
723
724The following methods implement this incremental parser.
725
726=over 4
727
728=item [void, scalar or list context] = $json->incr_parse ([$string])
729
730This is the central parsing function. It can both append new text and
731extract objects from the stream accumulated so far (both of these
732functions are optional).
733
734If C<$string> is given, then this string is appended to the already
735existing JSON fragment stored in the C<$json> object.
736
737After that, if the function is called in void context, it will simply
738return without doing anything further. This can be used to add more text
739in as many chunks as you want.
740
741If the method is called in scalar context, then it will try to extract
742exactly I<one> JSON object. If that is successful, it will return this
743object, otherwise it will return C<undef>. If there is a parse error,
744this method will croak just as C<decode> would do (one can then use
745C<incr_skip> to skip the errornous part). This is the most common way of
746using the method.
747
748And finally, in list context, it will try to extract as many objects
749from the stream as it can find and return them, or the empty list
750otherwise. For this to work, there must be no separators between the JSON
751objects or arrays, instead they must be concatenated back-to-back. If
752an error occurs, an exception will be raised as in the scalar context
753case. Note that in this case, any previously-parsed JSON texts will be
754lost.
755
756Example: Parse some JSON arrays/objects in a given string and return
757them.
758
759 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
760
761=item $lvalue_string = $json->incr_text
762
763This method returns the currently stored JSON fragment as an lvalue, that
764is, you can manipulate it. This I<only> works when a preceding call to
765C<incr_parse> in I<scalar context> successfully returned an object. Under
766all other circumstances you must not call this function (I mean it.
767although in simple tests it might actually work, it I<will> fail under
768real world conditions). As a special exception, you can also call this
769method before having parsed anything.
770
771This function is useful in two cases: a) finding the trailing text after a
772JSON object or b) parsing multiple JSON objects separated by non-JSON text
773(such as commas).
774
775=item $json->incr_skip
776
777This will reset the state of the incremental parser and will remove
778the parsed text from the input buffer so far. This is useful after
779C<incr_parse> died, in which case the input buffer and incremental parser
780state is left unchanged, to skip the text parsed so far and to reset the
781parse state.
782
783The difference to C<incr_reset> is that only text until the parse error
784occured is removed.
785
786=item $json->incr_reset
787
788This completely resets the incremental parser, that is, after this call,
789it will be as if the parser had never parsed anything.
790
791This is useful if you want to repeatedly parse JSON objects and want to
792ignore any trailing data, which means you have to reset the parser after
793each successful decode.
794
795=back
796
797=head2 LIMITATIONS
798
799All options that affect decoding are supported, except
800C<allow_nonref>. The reason for this is that it cannot be made to
801work sensibly: JSON objects and arrays are self-delimited, i.e. you can concatenate
802them back to back and still decode them perfectly. This does not hold true
803for JSON numbers, however.
804
805For example, is the string C<1> a single JSON number, or is it simply the
806start of C<12>? Or is C<12> a single JSON number, or the concatenation
807of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS
808takes the conservative route and disallows this case.
809
810=head2 EXAMPLES
811
812Some examples will make all this clearer. First, a simple example that
813works similarly to C<decode_prefix>: We want to decode the JSON object at
814the start of a string and identify the portion after the JSON object:
815
816 my $text = "[1,2,3] hello";
817
818 my $json = new JSON::XS;
819
820 my $obj = $json->incr_parse ($text)
821 or die "expected JSON object or array at beginning of string";
822
823 my $tail = $json->incr_text;
824 # $tail now contains " hello"
825
826Easy, isn't it?
827
828Now for a more complicated example: Imagine a hypothetical protocol where
829you read some requests from a TCP stream, and each request is a JSON
830array, without any separation between them (in fact, it is often useful to
831use newlines as "separators", as these get interpreted as whitespace at
832the start of the JSON text, which makes it possible to test said protocol
833with C<telnet>...).
834
835Here is how you'd do it (it is trivial to write this in an event-based
836manner):
837
838 my $json = new JSON::XS;
839
840 # read some data from the socket
841 while (sysread $socket, my $buf, 4096) {
842
843 # split and decode as many requests as possible
844 for my $request ($json->incr_parse ($buf)) {
845 # act on the $request
846 }
847 }
848
849Another complicated example: Assume you have a string with JSON objects
850or arrays, all separated by (optional) comma characters (e.g. C<[1],[2],
851[3]>). To parse them, we have to skip the commas between the JSON texts,
852and here is where the lvalue-ness of C<incr_text> comes in useful:
853
854 my $text = "[1],[2], [3]";
855 my $json = new JSON::XS;
856
857 # void context, so no parsing done
858 $json->incr_parse ($text);
859
860 # now extract as many objects as possible. note the
861 # use of scalar context so incr_text can be called.
862 while (my $obj = $json->incr_parse) {
863 # do something with $obj
864
865 # now skip the optional comma
866 $json->incr_text =~ s/^ \s* , //x;
867 }
868
869Now lets go for a very complex example: Assume that you have a gigantic
870JSON array-of-objects, many gigabytes in size, and you want to parse it,
871but you cannot load it into memory fully (this has actually happened in
872the real world :).
873
874Well, you lost, you have to implement your own JSON parser. But JSON::XS
875can still help you: You implement a (very simple) array parser and let
876JSON decode the array elements, which are all full JSON objects on their
877own (this wouldn't work if the array elements could be JSON numbers, for
878example):
879
880 my $json = new JSON::XS;
881
882 # open the monster
883 open my $fh, "<bigfile.json"
884 or die "bigfile: $!";
885
886 # first parse the initial "["
887 for (;;) {
888 sysread $fh, my $buf, 65536
889 or die "read error: $!";
890 $json->incr_parse ($buf); # void context, so no parsing
891
892 # Exit the loop once we found and removed(!) the initial "[".
893 # In essence, we are (ab-)using the $json object as a simple scalar
894 # we append data to.
895 last if $json->incr_text =~ s/^ \s* \[ //x;
896 }
897
898 # now we have the skipped the initial "[", so continue
899 # parsing all the elements.
900 for (;;) {
901 # in this loop we read data until we got a single JSON object
902 for (;;) {
903 if (my $obj = $json->incr_parse) {
904 # do something with $obj
905 last;
906 }
907
908 # add more data
909 sysread $fh, my $buf, 65536
910 or die "read error: $!";
911 $json->incr_parse ($buf); # void context, so no parsing
912 }
913
914 # in this loop we read data until we either found and parsed the
915 # separating "," between elements, or the final "]"
916 for (;;) {
917 # first skip whitespace
918 $json->incr_text =~ s/^\s*//;
919
920 # if we find "]", we are done
921 if ($json->incr_text =~ s/^\]//) {
922 print "finished.\n";
923 exit;
924 }
925
926 # if we find ",", we can continue with the next element
927 if ($json->incr_text =~ s/^,//) {
928 last;
929 }
930
931 # if we find anything else, we have a parse error!
932 if (length $json->incr_text) {
933 die "parse error near ", $json->incr_text;
934 }
935
936 # else add more data
937 sysread $fh, my $buf, 65536
938 or die "read error: $!";
939 $json->incr_parse ($buf); # void context, so no parsing
940 }
941
942This is a complex example, but most of the complexity comes from the fact
943that we are trying to be correct (bear with me if I am wrong, I never ran
944the above example :).
945
684 946
685 947
686=head1 MAPPING 948=head1 MAPPING
687 949
688This section describes how JSON::XS maps Perl values to JSON values and 950This section describes how JSON::XS maps Perl values to JSON values and
777Other unblessed references are generally not allowed and will cause an 1039Other unblessed references are generally not allowed and will cause an
778exception to be thrown, except for references to the integers C<0> and 1040exception to be thrown, except for references to the integers C<0> and
779C<1>, which get turned into C<false> and C<true> atoms in JSON. You can 1041C<1>, which get turned into C<false> and C<true> atoms in JSON. You can
780also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability. 1042also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
781 1043
782 encode_json [\0,JSON::XS::true] # yields [false,true] 1044 encode_json [\0, JSON::XS::true] # yields [false,true]
783 1045
784=item JSON::XS::true, JSON::XS::false 1046=item JSON::XS::true, JSON::XS::false
785 1047
786These special values become JSON true and JSON false values, 1048These special values become JSON true and JSON false values,
787respectively. You can also use C<\1> and C<\0> directly if you want. 1049respectively. You can also use C<\1> and C<\0> directly if you want.
929proper subset of most 8-bit and multibyte encodings in use in the world. 1191proper subset of most 8-bit and multibyte encodings in use in the world.
930 1192
931=back 1193=back
932 1194
933 1195
934=head1 COMPARISON 1196=head2 JSON and ECMAscript
935 1197
936As already mentioned, this module was created because none of the existing 1198JSON syntax is based on how literals are represented in javascript (the
937JSON modules could be made to work correctly. First I will describe the 1199not-standardised predecessor of ECMAscript) which is presumably why it is
938problems (or pleasures) I encountered with various existing JSON modules, 1200called "JavaScript Object Notation".
939followed by some benchmark values. JSON::XS was designed not to suffer
940from any of these problems or limitations.
941 1201
942=over 4 1202However, JSON is not a subset (and also not a superset of course) of
1203ECMAscript (the standard) or javascript (whatever browsers actually
1204implement).
943 1205
944=item JSON 2.xx 1206If you want to use javascript's C<eval> function to "parse" JSON, you
1207might run into parse errors for valid JSON texts, or the resulting data
1208structure might not be queryable:
945 1209
946A marvellous piece of engineering, this module either uses JSON::XS 1210One of the problems is that U+2028 and U+2029 are valid characters inside
947directly when available (so will be 100% compatible with it, including 1211JSON strings, but are not allowed in ECMAscript string literals, so the
948speed), or it uses JSON::PP, which is basically JSON::XS translated to 1212following Perl fragment will not output something that can be guaranteed
949Pure Perl, which should be 100% compatible with JSON::XS, just a bit 1213to be parsable by javascript's C<eval>:
950slower.
951 1214
952You cannot really lose by using this module, especially as it tries very 1215 use JSON::XS;
953hard to work even with ancient Perl versions, while JSON::XS does not.
954 1216
955=item JSON 1.07 1217 print encode_json [chr 0x2028];
956 1218
957Slow (but very portable, as it is written in pure Perl). 1219The right fix for this is to use a proper JSON parser in your javascript
1220programs, and not rely on C<eval> (see for example Douglas Crockford's
1221F<json2.js> parser).
958 1222
959Undocumented/buggy Unicode handling (how JSON handles Unicode values is 1223If this is not an option, you can, as a stop-gap measure, simply encode to
960undocumented. One can get far by feeding it Unicode strings and doing 1224ASCII-only JSON:
961en-/decoding oneself, but Unicode escapes are not working properly).
962 1225
963No round-tripping (strings get clobbered if they look like numbers, e.g. 1226 use JSON::XS;
964the string C<2.0> will encode to C<2.0> instead of C<"2.0">, and that will
965decode into the number 2.
966 1227
967=item JSON::PC 0.01 1228 print JSON::XS->new->ascii->encode ([chr 0x2028]);
968 1229
969Very fast. 1230Note that this will enlarge the resulting JSON text quite a bit if you
1231have many non-ASCII characters. You might be tempted to run some regexes
1232to only escape U+2028 and U+2029, e.g.:
970 1233
971Undocumented/buggy Unicode handling. 1234 # DO NOT USE THIS!
1235 my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1236 $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1237 $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1238 print $json;
972 1239
973No round-tripping. 1240Note that I<this is a bad idea>: the above only works for U+2028 and
1241U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1242javascript implementations, however, have issues with other characters as
1243well - using C<eval> naively simply I<will> cause problems.
974 1244
975Has problems handling many Perl values (e.g. regex results and other magic 1245Another problem is that some javascript implementations reserve
976values will make it croak). 1246some property names for their own purposes (which probably makes
1247them non-ECMAscript-compliant). For example, Iceweasel reserves the
1248C<__proto__> property name for it's own purposes.
977 1249
978Does not even generate valid JSON (C<{1,2}> gets converted to C<{1:2}> 1250If that is a problem, you could parse try to filter the resulting JSON
979which is not a valid JSON text. 1251output for these property strings, e.g.:
980 1252
981Unmaintained (maintainer unresponsive for many months, bugs are not 1253 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
982getting fixed).
983 1254
984=item JSON::Syck 0.21 1255This works because C<__proto__> is not valid outside of strings, so every
1256occurence of C<"__proto__"\s*:> must be a string used as property name.
985 1257
986Very buggy (often crashes). 1258If you know of other incompatibilities, please let me know.
987
988Very inflexible (no human-readable format supported, format pretty much
989undocumented. I need at least a format for easy reading by humans and a
990single-line compact format for use in a protocol, and preferably a way to
991generate ASCII-only JSON texts).
992
993Completely broken (and confusingly documented) Unicode handling (Unicode
994escapes are not working properly, you need to set ImplicitUnicode to
995I<different> values on en- and decoding to get symmetric behaviour).
996
997No round-tripping (simple cases work, but this depends on whether the scalar
998value was used in a numeric context or not).
999
1000Dumping hashes may skip hash values depending on iterator state.
1001
1002Unmaintained (maintainer unresponsive for many months, bugs are not
1003getting fixed).
1004
1005Does not check input for validity (i.e. will accept non-JSON input and
1006return "something" instead of raising an exception. This is a security
1007issue: imagine two banks transferring money between each other using
1008JSON. One bank might parse a given non-JSON request and deduct money,
1009while the other might reject the transaction with a syntax error. While a
1010good protocol will at least recover, that is extra unnecessary work and
1011the transaction will still not succeed).
1012
1013=item JSON::DWIW 0.04
1014
1015Very fast. Very natural. Very nice.
1016
1017Undocumented Unicode handling (but the best of the pack. Unicode escapes
1018still don't get parsed properly).
1019
1020Very inflexible.
1021
1022No round-tripping.
1023
1024Does not generate valid JSON texts (key strings are often unquoted, empty keys
1025result in nothing being output)
1026
1027Does not check input for validity.
1028
1029=back
1030 1259
1031 1260
1032=head2 JSON and YAML 1261=head2 JSON and YAML
1033 1262
1034You often hear that JSON is a subset of YAML. This is, however, a mass 1263You often hear that JSON is a subset of YAML. This is, however, a mass
1044 my $yaml = $to_yaml->encode ($ref) . "\n"; 1273 my $yaml = $to_yaml->encode ($ref) . "\n";
1045 1274
1046This will I<usually> generate JSON texts that also parse as valid 1275This will I<usually> generate JSON texts that also parse as valid
1047YAML. Please note that YAML has hardcoded limits on (simple) object key 1276YAML. Please note that YAML has hardcoded limits on (simple) object key
1048lengths that JSON doesn't have and also has different and incompatible 1277lengths that JSON doesn't have and also has different and incompatible
1049unicode handling, so you should make sure that your hash keys are 1278unicode character escape syntax, so you should make sure that your hash
1050noticeably shorter than the 1024 "stream characters" YAML allows and that 1279keys are noticeably shorter than the 1024 "stream characters" YAML allows
1051you do not have characters with codepoint values outside the Unicode BMP 1280and that you do not have characters with codepoint values outside the
1052(basic multilingual page). YAML also does not allow C<\/> sequences in 1281Unicode BMP (basic multilingual page). YAML also does not allow C<\/>
1053strings (which JSON::XS does not I<currently> generate, but other JSON 1282sequences in strings (which JSON::XS does not I<currently> generate, but
1054generators might). 1283other JSON generators might).
1055 1284
1056There might be other incompatibilities that I am not aware of (or the YAML 1285There might be other incompatibilities that I am not aware of (or the YAML
1057specification has been changed yet again - it does so quite often). In 1286specification has been changed yet again - it does so quite often). In
1058general you should not try to generate YAML with a JSON generator or vice 1287general you should not try to generate YAML with a JSON generator or vice
1059versa, or try to parse JSON with a YAML parser or vice versa: chances are 1288versa, or try to parse JSON with a YAML parser or vice versa: chances are
1078that difficult or long) and finally make YAML compatible to it, and 1307that difficult or long) and finally make YAML compatible to it, and
1079educating users about the changes, instead of spreading lies about the 1308educating users about the changes, instead of spreading lies about the
1080real compatibility for many I<years> and trying to silence people who 1309real compatibility for many I<years> and trying to silence people who
1081point out that it isn't true. 1310point out that it isn't true.
1082 1311
1312Addendum/2009: the YAML 1.2 spec is still incomaptible with JSON, even
1313though the incompatibilities have been documented (and are known to
1314Brian) for many years and the spec makes explicit claims that YAML is a
1315superset of JSON. It would be so easy to fix, but apparently, bullying and
1316corrupting userdata is so much easier.
1317
1083=back 1318=back
1084 1319
1085 1320
1086=head2 SPEED 1321=head2 SPEED
1087 1322
1092 1327
1093First comes a comparison between various modules using 1328First comes a comparison between various modules using
1094a very short single-line JSON string (also available at 1329a very short single-line JSON string (also available at
1095L<http://dist.schmorp.de/misc/json/short.json>). 1330L<http://dist.schmorp.de/misc/json/short.json>).
1096 1331
1097 {"method": "handleMessage", "params": ["user1", "we were just talking"], \ 1332 {"method": "handleMessage", "params": ["user1",
1098 "id": null, "array":[1,11,234,-5,1e5,1e7, true, false]} 1333 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1334 1, 0]}
1099 1335
1100It shows the number of encodes/decodes per second (JSON::XS uses 1336It shows the number of encodes/decodes per second (JSON::XS uses
1101the functional interface, while JSON::XS/2 uses the OO interface 1337the functional interface, while JSON::XS/2 uses the OO interface
1102with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables 1338with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1103shrink). Higher is better: 1339shrink. JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ
1340uses the from_json method). Higher is better:
1104 1341
1105 module | encode | decode | 1342 module | encode | decode |
1106 -----------|------------|------------| 1343 --------------|------------|------------|
1107 JSON 1.x | 4990.842 | 4088.813 | 1344 JSON::DWIW/DS | 86302.551 | 102300.098 |
1108 JSON::DWIW | 51653.990 | 71575.154 | 1345 JSON::DWIW/FJ | 86302.551 | 75983.768 |
1109 JSON::PC | 65948.176 | 74631.744 | 1346 JSON::PP | 15827.562 | 6638.658 |
1110 JSON::PP | 8931.652 | 3817.168 | 1347 JSON::Syck | 63358.066 | 47662.545 |
1111 JSON::Syck | 24877.248 | 27776.848 | 1348 JSON::XS | 511500.488 | 511500.488 |
1112 JSON::XS | 388361.481 | 227951.304 | 1349 JSON::XS/2 | 291271.111 | 388361.481 |
1113 JSON::XS/2 | 227951.304 | 218453.333 | 1350 JSON::XS/3 | 361577.931 | 361577.931 |
1114 JSON::XS/3 | 338250.323 | 218453.333 | 1351 Storable | 66788.280 | 265462.278 |
1115 Storable | 16500.016 | 135300.129 |
1116 -----------+------------+------------+ 1352 --------------+------------+------------+
1117 1353
1118That is, JSON::XS is about five times faster than JSON::DWIW on encoding, 1354That is, JSON::XS is almost six times faster than JSON::DWIW on encoding,
1119about three times faster on decoding, and over forty times faster 1355about five times faster on decoding, and over thirty to seventy times
1120than JSON, even with pretty-printing and key sorting. It also compares 1356faster than JSON's pure perl implementation. It also compares favourably
1121favourably to Storable for small amounts of data. 1357to Storable for small amounts of data.
1122 1358
1123Using a longer test string (roughly 18KB, generated from Yahoo! Locals 1359Using a longer test string (roughly 18KB, generated from Yahoo! Locals
1124search API (L<http://dist.schmorp.de/misc/json/long.json>). 1360search API (L<http://dist.schmorp.de/misc/json/long.json>).
1125 1361
1126 module | encode | decode | 1362 module | encode | decode |
1127 -----------|------------|------------| 1363 --------------|------------|------------|
1128 JSON 1.x | 55.260 | 34.971 | 1364 JSON::DWIW/DS | 1647.927 | 2673.916 |
1129 JSON::DWIW | 825.228 | 1082.513 | 1365 JSON::DWIW/FJ | 1630.249 | 2596.128 |
1130 JSON::PC | 3571.444 | 2394.829 |
1131 JSON::PP | 210.987 | 32.574 | 1366 JSON::PP | 400.640 | 62.311 |
1132 JSON::Syck | 552.551 | 787.544 | 1367 JSON::Syck | 1481.040 | 1524.869 |
1133 JSON::XS | 5780.463 | 4854.519 | 1368 JSON::XS | 20661.596 | 9541.183 |
1134 JSON::XS/2 | 3869.998 | 4798.975 | 1369 JSON::XS/2 | 10683.403 | 9416.938 |
1135 JSON::XS/3 | 5862.880 | 4798.975 | 1370 JSON::XS/3 | 20661.596 | 9400.054 |
1136 Storable | 4445.002 | 5235.027 | 1371 Storable | 19765.806 | 10000.725 |
1137 -----------+------------+------------+ 1372 --------------+------------+------------+
1138 1373
1139Again, JSON::XS leads by far (except for Storable which non-surprisingly 1374Again, JSON::XS leads by far (except for Storable which non-surprisingly
1140decodes faster). 1375decodes a bit faster).
1141 1376
1142On large strings containing lots of high Unicode characters, some modules 1377On large strings containing lots of high Unicode characters, some modules
1143(such as JSON::PC) seem to decode faster than JSON::XS, but the result 1378(such as JSON::PC) seem to decode faster than JSON::XS, but the result
1144will be broken due to missing (or wrong) Unicode handling. Others refuse 1379will be broken due to missing (or wrong) Unicode handling. Others refuse
1145to decode or encode properly, so it was impossible to prepare a fair 1380to decode or encode properly, so it was impossible to prepare a fair
1181information you might want to make sure that exceptions thrown by JSON::XS 1416information you might want to make sure that exceptions thrown by JSON::XS
1182will not end up in front of untrusted eyes. 1417will not end up in front of untrusted eyes.
1183 1418
1184If you are using JSON::XS to return packets to consumption 1419If you are using JSON::XS to return packets to consumption
1185by JavaScript scripts in a browser you should have a look at 1420by JavaScript scripts in a browser you should have a look at
1186L<http://jpsykes.com/47/practical-csrf-and-json-security> to see whether 1421L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to
1187you are vulnerable to some common attack vectors (which really are browser 1422see whether you are vulnerable to some common attack vectors (which really
1188design bugs, but it is still you who will have to deal with it, as major 1423are browser design bugs, but it is still you who will have to deal with
1189browser developers care only for features, not about getting security 1424it, as major browser developers care only for features, not about getting
1190right). 1425security right).
1191 1426
1192 1427
1193=head1 THREADS 1428=head1 THREADS
1194 1429
1195This module is I<not> guaranteed to be thread safe and there are no 1430This module is I<not> guaranteed to be thread safe and there are no
1201 1436
1202 1437
1203=head1 BUGS 1438=head1 BUGS
1204 1439
1205While the goal of this module is to be correct, that unfortunately does 1440While the goal of this module is to be correct, that unfortunately does
1206not mean it's bug-free, only that I think its design is bug-free. It is 1441not mean it's bug-free, only that I think its design is bug-free. If you
1207still relatively early in its development. If you keep reporting bugs they 1442keep reporting bugs they will be fixed swiftly, though.
1208will be fixed swiftly, though.
1209 1443
1210Please refrain from using rt.cpan.org or any other bug reporting 1444Please refrain from using rt.cpan.org or any other bug reporting
1211service. I put the contact address into my modules for a reason. 1445service. I put the contact address into my modules for a reason.
1212 1446
1213=cut 1447=cut
1233 "--" => sub { $_[0] = ${$_[0]} - 1 }, 1467 "--" => sub { $_[0] = ${$_[0]} - 1 },
1234 fallback => 1; 1468 fallback => 1;
1235 1469
12361; 14701;
1237 1471
1472=head1 SEE ALSO
1473
1474The F<json_xs> command line utility for quick experiments.
1475
1238=head1 AUTHOR 1476=head1 AUTHOR
1239 1477
1240 Marc Lehmann <schmorp@schmorp.de> 1478 Marc Lehmann <schmorp@schmorp.de>
1241 http://home.schmorp.de/ 1479 http://home.schmorp.de/
1242 1480

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines