ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.117 by root, Wed Feb 18 00:08:28 2009 UTC vs.
Revision 1.154 by root, Sun Mar 2 22:09:38 2014 UTC

64so, and even documents what "correct" means. 64so, and even documents what "correct" means.
65 65
66=item * round-trip integrity 66=item * round-trip integrity
67 67
68When you serialise a perl data structure using only data types supported 68When you serialise a perl data structure using only data types supported
69by JSON, the deserialised data structure is identical on the Perl level. 69by JSON and Perl, the deserialised data structure is identical on the Perl
70(e.g. the string "2.0" doesn't suddenly become "2" just because it looks 70level. (e.g. the string "2.0" doesn't suddenly become "2" just because
71like a number). There minor I<are> exceptions to this, read the MAPPING 71it looks like a number). There I<are> minor exceptions to this, read the
72section below to learn about those. 72MAPPING section below to learn about those.
73 73
74=item * strict checking of JSON correctness 74=item * strict checking of JSON correctness
75 75
76There is no guessing, no generating of illegal JSON texts by default, 76There is no guessing, no generating of illegal JSON texts by default,
77and only JSON is accepted as input by default (the latter is a security 77and only JSON is accepted as input by default (the latter is a security
83this module usually compares favourably in terms of speed, too. 83this module usually compares favourably in terms of speed, too.
84 84
85=item * simple to use 85=item * simple to use
86 86
87This module has both a simple functional interface as well as an object 87This module has both a simple functional interface as well as an object
88oriented interface interface. 88oriented interface.
89 89
90=item * reasonably versatile output formats 90=item * reasonably versatile output formats
91 91
92You can choose between the most compact guaranteed-single-line format 92You can choose between the most compact guaranteed-single-line format
93possible (nice for simple line-based protocols), a pure-ASCII format 93possible (nice for simple line-based protocols), a pure-ASCII format
99 99
100=cut 100=cut
101 101
102package JSON::XS; 102package JSON::XS;
103 103
104no warnings; 104use common::sense;
105use strict;
106 105
107our $VERSION = '2.232'; 106our $VERSION = 3.01;
108our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
109 108
110our @EXPORT = qw(encode_json decode_json to_json from_json); 109our @EXPORT = qw(encode_json decode_json);
111
112sub to_json($) {
113 require Carp;
114 Carp::croak ("JSON::XS::to_json has been renamed to encode_json, either downgrade to pre-2.0 versions of JSON::XS or rename the call");
115}
116
117sub from_json($) {
118 require Carp;
119 Carp::croak ("JSON::XS::from_json has been renamed to decode_json, either downgrade to pre-2.0 versions of JSON::XS or rename the call");
120}
121 110
122use Exporter; 111use Exporter;
123use XSLoader; 112use XSLoader;
113
114use Types::Serialiser ();
124 115
125=head1 FUNCTIONAL INTERFACE 116=head1 FUNCTIONAL INTERFACE
126 117
127The following convenience methods are provided by this module. They are 118The following convenience methods are provided by this module. They are
128exported by default: 119exported by default:
149This function call is functionally identical to: 140This function call is functionally identical to:
150 141
151 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 142 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
152 143
153Except being faster. 144Except being faster.
154
155=item $is_boolean = JSON::XS::is_bool $scalar
156
157Returns true if the passed scalar represents either JSON::XS::true or
158JSON::XS::false, two constants that act like C<1> and C<0>, respectively
159and are used to represent JSON C<true> and C<false> values in Perl.
160
161See MAPPING, below, for more information on how JSON values are mapped to
162Perl.
163 145
164=back 146=back
165 147
166 148
167=head1 A FEW NOTES ON UNICODE AND PERL 149=head1 A FEW NOTES ON UNICODE AND PERL
433If C<$enable> is true (or missing), then the C<encode> method will output JSON objects 415If C<$enable> is true (or missing), then the C<encode> method will output JSON objects
434by sorting their keys. This is adding a comparatively high overhead. 416by sorting their keys. This is adding a comparatively high overhead.
435 417
436If C<$enable> is false, then the C<encode> method will output key-value 418If C<$enable> is false, then the C<encode> method will output key-value
437pairs in the order Perl stores them (which will likely change between runs 419pairs in the order Perl stores them (which will likely change between runs
438of the same script). 420of the same script, and can change even within the same run from 5.18
421onwards).
439 422
440This option is useful if you want the same data structure to be encoded as 423This option is useful if you want the same data structure to be encoded as
441the same JSON text (given the same overall settings). If it is disabled, 424the same JSON text (given the same overall settings). If it is disabled,
442the same hash might be encoded differently even if contains the same data, 425the same hash might be encoded differently even if contains the same data,
443as key-value pairs have no inherent ordering in Perl. 426as key-value pairs have no inherent ordering in Perl.
444 427
445This setting has no effect when decoding JSON texts. 428This setting has no effect when decoding JSON texts.
429
430This setting has currently no effect on tied hashes.
446 431
447=item $json = $json->allow_nonref ([$enable]) 432=item $json = $json->allow_nonref ([$enable])
448 433
449=item $enabled = $json->get_allow_nonref 434=item $enabled = $json->get_allow_nonref
450 435
482 467
483=item $json = $json->allow_blessed ([$enable]) 468=item $json = $json->allow_blessed ([$enable])
484 469
485=item $enabled = $json->get_allow_blessed 470=item $enabled = $json->get_allow_blessed
486 471
472See L<OBJECT SERIALISATION> for details.
473
487If C<$enable> is true (or missing), then the C<encode> method will not 474If C<$enable> is true (or missing), then the C<encode> method will not
488barf when it encounters a blessed reference. Instead, the value of the 475barf when it encounters a blessed reference that it cannot convert
489B<convert_blessed> option will decide whether C<null> (C<convert_blessed> 476otherwise. Instead, a JSON C<null> value is encoded instead of the object.
490disabled or no C<TO_JSON> method found) or a representation of the
491object (C<convert_blessed> enabled and C<TO_JSON> method found) is being
492encoded. Has no effect on C<decode>.
493 477
494If C<$enable> is false (the default), then C<encode> will throw an 478If C<$enable> is false (the default), then C<encode> will throw an
495exception when it encounters a blessed object. 479exception when it encounters a blessed object that it cannot convert
480otherwise.
481
482This setting has no effect on C<decode>.
496 483
497=item $json = $json->convert_blessed ([$enable]) 484=item $json = $json->convert_blessed ([$enable])
498 485
499=item $enabled = $json->get_convert_blessed 486=item $enabled = $json->get_convert_blessed
487
488See L<OBJECT SERIALISATION> for details.
500 489
501If C<$enable> is true (or missing), then C<encode>, upon encountering a 490If C<$enable> is true (or missing), then C<encode>, upon encountering a
502blessed object, will check for the availability of the C<TO_JSON> method 491blessed object, will check for the availability of the C<TO_JSON> method
503on the object's class. If found, it will be called in scalar context 492on the object's class. If found, it will be called in scalar context and
504and the resulting scalar will be encoded instead of the object. If no 493the resulting scalar will be encoded instead of the object.
505C<TO_JSON> method is found, the value of C<allow_blessed> will decide what
506to do.
507 494
508The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON> 495The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON>
509returns other blessed objects, those will be handled in the same 496returns other blessed objects, those will be handled in the same
510way. C<TO_JSON> must take care of not causing an endless recursion cycle 497way. C<TO_JSON> must take care of not causing an endless recursion cycle
511(== crash) in this case. The name of C<TO_JSON> was chosen because other 498(== crash) in this case. The name of C<TO_JSON> was chosen because other
512methods called by the Perl core (== not by the user of the object) are 499methods called by the Perl core (== not by the user of the object) are
513usually in upper case letters and to avoid collisions with any C<to_json> 500usually in upper case letters and to avoid collisions with any C<to_json>
514function or method. 501function or method.
515 502
516This setting does not yet influence C<decode> in any way, but in the 503If C<$enable> is false (the default), then C<encode> will not consider
517future, global hooks might get installed that influence C<decode> and are 504this type of conversion.
518enabled by this setting.
519 505
520If C<$enable> is false, then the C<allow_blessed> setting will decide what 506This setting has no effect on C<decode>.
521to do when a blessed object is found. 507
508=item $json = $json->allow_tags ([$enable])
509
510=item $enabled = $json->allow_tags
511
512See L<OBJECT SERIALISATION> for details.
513
514If C<$enable> is true (or missing), then C<encode>, upon encountering a
515blessed object, will check for the availability of the C<FREEZE> method on
516the object's class. If found, it will be used to serialise the object into
517a nonstandard tagged JSON value (that JSON decoders cannot decode).
518
519It also causes C<decode> to parse such tagged JSON values and deserialise
520them via a call to the C<THAW> method.
521
522If C<$enable> is false (the default), then C<encode> will not consider
523this type of conversion, and tagged JSON values will cause a parse error
524in C<decode>, as if tags were not part of the grammar.
522 525
523=item $json = $json->filter_json_object ([$coderef->($hashref)]) 526=item $json = $json->filter_json_object ([$coderef->($hashref)])
524 527
525When C<$coderef> is specified, it will be called from C<decode> each 528When C<$coderef> is specified, it will be called from C<decode> each
526time it decodes a JSON object. The only argument is a reference to the 529time it decodes a JSON object. The only argument is a reference to the
665 668
666See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 669See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
667 670
668=item $json_text = $json->encode ($perl_scalar) 671=item $json_text = $json->encode ($perl_scalar)
669 672
670Converts the given Perl data structure (a simple scalar or a reference 673Converts the given Perl value or data structure to its JSON
671to a hash or array) to its JSON representation. Simple scalars will be 674representation. Croaks on error.
672converted into JSON string or number sequences, while references to arrays
673become JSON arrays and references to hashes become JSON objects. Undefined
674Perl values (e.g. C<undef>) become JSON C<null> values. Neither C<true>
675nor C<false> values will be generated.
676 675
677=item $perl_scalar = $json->decode ($json_text) 676=item $perl_scalar = $json->decode ($json_text)
678 677
679The opposite of C<encode>: expects a JSON text and tries to parse it, 678The opposite of C<encode>: expects a JSON text and tries to parse it,
680returning the resulting simple scalar or reference. Croaks on error. 679returning the resulting simple scalar or reference. Croaks on error.
681
682JSON numbers and strings become simple Perl scalars. JSON arrays become
683Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
684C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.
685 680
686=item ($perl_scalar, $characters) = $json->decode_prefix ($json_text) 681=item ($perl_scalar, $characters) = $json->decode_prefix ($json_text)
687 682
688This works like the C<decode> method, but instead of raising an exception 683This works like the C<decode> method, but instead of raising an exception
689when there is trailing garbage after the first JSON object, it will 684when there is trailing garbage after the first JSON object, it will
690silently stop parsing there and return the number of characters consumed 685silently stop parsing there and return the number of characters consumed
691so far. 686so far.
692 687
693This is useful if your JSON texts are not delimited by an outer protocol 688This is useful if your JSON texts are not delimited by an outer protocol
694(which is not the brightest thing to do in the first place) and you need
695to know where the JSON text ends. 689and you need to know where the JSON text ends.
696 690
697 JSON::XS->new->decode_prefix ("[1] the tail") 691 JSON::XS->new->decode_prefix ("[1] the tail")
698 => ([], 3) 692 => ([], 3)
699 693
700=back 694=back
712calls). 706calls).
713 707
714JSON::XS will only attempt to parse the JSON text once it is sure it 708JSON::XS will only attempt to parse the JSON text once it is sure it
715has enough text to get a decisive result, using a very simple but 709has enough text to get a decisive result, using a very simple but
716truly incremental parser. This means that it sometimes won't stop as 710truly incremental parser. This means that it sometimes won't stop as
717early as the full parser, for example, it doesn't detect parenthese 711early as the full parser, for example, it doesn't detect mismatched
718mismatches. The only thing it guarantees is that it starts decoding as 712parentheses. The only thing it guarantees is that it starts decoding as
719soon as a syntactically valid JSON text has been seen. This means you need 713soon as a syntactically valid JSON text has been seen. This means you need
720to set resource limits (e.g. C<max_size>) to ensure the parser will stop 714to set resource limits (e.g. C<max_size>) to ensure the parser will stop
721parsing in the presence if syntax errors. 715parsing in the presence if syntax errors.
722 716
723The following methods implement this incremental parser. 717The following methods implement this incremental parser.
739 733
740If the method is called in scalar context, then it will try to extract 734If the method is called in scalar context, then it will try to extract
741exactly I<one> JSON object. If that is successful, it will return this 735exactly I<one> JSON object. If that is successful, it will return this
742object, otherwise it will return C<undef>. If there is a parse error, 736object, otherwise it will return C<undef>. If there is a parse error,
743this method will croak just as C<decode> would do (one can then use 737this method will croak just as C<decode> would do (one can then use
744C<incr_skip> to skip the errornous part). This is the most common way of 738C<incr_skip> to skip the erroneous part). This is the most common way of
745using the method. 739using the method.
746 740
747And finally, in list context, it will try to extract as many objects 741And finally, in list context, it will try to extract as many objects
748from the stream as it can find and return them, or the empty list 742from the stream as it can find and return them, or the empty list
749otherwise. For this to work, there must be no separators between the JSON 743otherwise. For this to work, there must be no separators between the JSON
750objects or arrays, instead they must be concatenated back-to-back. If 744objects or arrays, instead they must be concatenated back-to-back. If
751an error occurs, an exception will be raised as in the scalar context 745an error occurs, an exception will be raised as in the scalar context
752case. Note that in this case, any previously-parsed JSON texts will be 746case. Note that in this case, any previously-parsed JSON texts will be
753lost. 747lost.
754 748
749Example: Parse some JSON arrays/objects in a given string and return
750them.
751
752 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
753
755=item $lvalue_string = $json->incr_text 754=item $lvalue_string = $json->incr_text
756 755
757This method returns the currently stored JSON fragment as an lvalue, that 756This method returns the currently stored JSON fragment as an lvalue, that
758is, you can manipulate it. This I<only> works when a preceding call to 757is, you can manipulate it. This I<only> works when a preceding call to
759C<incr_parse> in I<scalar context> successfully returned an object. Under 758C<incr_parse> in I<scalar context> successfully returned an object. Under
773C<incr_parse> died, in which case the input buffer and incremental parser 772C<incr_parse> died, in which case the input buffer and incremental parser
774state is left unchanged, to skip the text parsed so far and to reset the 773state is left unchanged, to skip the text parsed so far and to reset the
775parse state. 774parse state.
776 775
777The difference to C<incr_reset> is that only text until the parse error 776The difference to C<incr_reset> is that only text until the parse error
778occured is removed. 777occurred is removed.
779 778
780=item $json->incr_reset 779=item $json->incr_reset
781 780
782This completely resets the incremental parser, that is, after this call, 781This completely resets the incremental parser, that is, after this call,
783it will be as if the parser had never parsed anything. 782it will be as if the parser had never parsed anything.
789=back 788=back
790 789
791=head2 LIMITATIONS 790=head2 LIMITATIONS
792 791
793All options that affect decoding are supported, except 792All options that affect decoding are supported, except
794C<allow_nonref>. The reason for this is that it cannot be made to 793C<allow_nonref>. The reason for this is that it cannot be made to work
795work sensibly: JSON objects and arrays are self-delimited, i.e. you can concatenate 794sensibly: JSON objects and arrays are self-delimited, i.e. you can
796them back to back and still decode them perfectly. This does not hold true 795concatenate them back to back and still decode them perfectly. This does
797for JSON numbers, however. 796not hold true for JSON numbers, however.
798 797
799For example, is the string C<1> a single JSON number, or is it simply the 798For example, is the string C<1> a single JSON number, or is it simply the
800start of C<12>? Or is C<12> a single JSON number, or the concatenation 799start of C<12>? Or is C<12> a single JSON number, or the concatenation
801of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS 800of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS
802takes the conservative route and disallows this case. 801takes the conservative route and disallows this case.
981If the number consists of digits only, JSON::XS will try to represent 980If the number consists of digits only, JSON::XS will try to represent
982it as an integer value. If that fails, it will try to represent it as 981it as an integer value. If that fails, it will try to represent it as
983a numeric (floating point) value if that is possible without loss of 982a numeric (floating point) value if that is possible without loss of
984precision. Otherwise it will preserve the number as a string value (in 983precision. Otherwise it will preserve the number as a string value (in
985which case you lose roundtripping ability, as the JSON number will be 984which case you lose roundtripping ability, as the JSON number will be
986re-encoded toa JSON string). 985re-encoded to a JSON string).
987 986
988Numbers containing a fractional or exponential part will always be 987Numbers containing a fractional or exponential part will always be
989represented as numeric (floating point) values, possibly at a loss of 988represented as numeric (floating point) values, possibly at a loss of
990precision (in which case you might lose perfect roundtripping ability, but 989precision (in which case you might lose perfect roundtripping ability, but
991the JSON number will still be re-encoded as a JSON number). 990the JSON number will still be re-encoded as a JSON number).
992 991
992Note that precision is not accuracy - binary floating point values cannot
993represent most decimal fractions exactly, and when converting from and to
994floating point, JSON::XS only guarantees precision up to but not including
995the least significant bit.
996
993=item true, false 997=item true, false
994 998
995These JSON atoms become C<JSON::XS::true> and C<JSON::XS::false>, 999These JSON atoms become C<Types::Serialiser::true> and
996respectively. They are overloaded to act almost exactly like the numbers 1000C<Types::Serialiser::false>, respectively. They are overloaded to act
997C<1> and C<0>. You can check whether a scalar is a JSON boolean by using 1001almost exactly like the numbers C<1> and C<0>. You can check whether
998the C<JSON::XS::is_bool> function. 1002a scalar is a JSON boolean by using the C<Types::Serialiser::is_bool>
1003function (after C<use Types::Serialier>, of course).
999 1004
1000=item null 1005=item null
1001 1006
1002A JSON null atom becomes C<undef> in Perl. 1007A JSON null atom becomes C<undef> in Perl.
1008
1009=item shell-style comments (C<< # I<text> >>)
1010
1011As a nonstandard extension to the JSON syntax that is enabled by the
1012C<relaxed> setting, shell-style comments are allowed. They can start
1013anywhere outside strings and go till the end of the line.
1014
1015=item tagged values (C<< (I<tag>)I<value> >>).
1016
1017Another nonstandard extension to the JSON syntax, enabled with the
1018C<allow_tags> setting, are tagged values. In this implementation, the
1019I<tag> must be a perl package/class name encoded as a JSON string, and the
1020I<value> must be a JSON array encoding optional constructor arguments.
1021
1022See L<OBJECT SERIALISATION>, below, for details.
1003 1023
1004=back 1024=back
1005 1025
1006 1026
1007=head2 PERL -> JSON 1027=head2 PERL -> JSON
1012 1032
1013=over 4 1033=over 4
1014 1034
1015=item hash references 1035=item hash references
1016 1036
1017Perl hash references become JSON objects. As there is no inherent ordering 1037Perl hash references become JSON objects. As there is no inherent
1018in hash keys (or JSON objects), they will usually be encoded in a 1038ordering in hash keys (or JSON objects), they will usually be encoded
1019pseudo-random order that can change between runs of the same program but 1039in a pseudo-random order. JSON::XS can optionally sort the hash keys
1020stays generally the same within a single run of a program. JSON::XS can 1040(determined by the I<canonical> flag), so the same datastructure will
1021optionally sort the hash keys (determined by the I<canonical> flag), so 1041serialise to the same JSON text (given same settings and version of
1022the same datastructure will serialise to the same JSON text (given same 1042JSON::XS), but this incurs a runtime overhead and is only rarely useful,
1023settings and version of JSON::XS), but this incurs a runtime overhead 1043e.g. when you want to compare some JSON text against another for equality.
1024and is only rarely useful, e.g. when you want to compare some JSON text
1025against another for equality.
1026 1044
1027=item array references 1045=item array references
1028 1046
1029Perl array references become JSON arrays. 1047Perl array references become JSON arrays.
1030 1048
1031=item other references 1049=item other references
1032 1050
1033Other unblessed references are generally not allowed and will cause an 1051Other unblessed references are generally not allowed and will cause an
1034exception to be thrown, except for references to the integers C<0> and 1052exception to be thrown, except for references to the integers C<0> and
1035C<1>, which get turned into C<false> and C<true> atoms in JSON. You can 1053C<1>, which get turned into C<false> and C<true> atoms in JSON.
1036also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
1037 1054
1055Since C<JSON::XS> uses the boolean model from L<Types::Serialiser>, you
1056can also C<use Types::Serialiser> and then use C<Types::Serialiser::false>
1057and C<Types::Serialiser::true> to improve readability.
1058
1059 use Types::Serialiser;
1038 encode_json [\0, JSON::XS::true] # yields [false,true] 1060 encode_json [\0, Types::Serialiser::true] # yields [false,true]
1039 1061
1040=item JSON::XS::true, JSON::XS::false 1062=item Types::Serialiser::true, Types::Serialiser::false
1041 1063
1042These special values become JSON true and JSON false values, 1064These special values from the L<Types::Serialiser> module become JSON true
1043respectively. You can also use C<\1> and C<\0> directly if you want. 1065and JSON false values, respectively. You can also use C<\1> and C<\0>
1066directly if you want.
1044 1067
1045=item blessed objects 1068=item blessed objects
1046 1069
1047Blessed objects are not directly representable in JSON. See the 1070Blessed objects are not directly representable in JSON, but C<JSON::XS>
1048C<allow_blessed> and C<convert_blessed> methods on various options on 1071allows various ways of handling objects. See L<OBJECT SERIALISATION>,
1049how to deal with this: basically, you can choose between throwing an 1072below, for details.
1050exception, encoding the reference as if it weren't blessed, or provide
1051your own serialiser method.
1052 1073
1053=item simple scalars 1074=item simple scalars
1054 1075
1055Simple Perl scalars (any scalar that is not a reference) are the most 1076Simple Perl scalars (any scalar that is not a reference) are the most
1056difficult objects to encode: JSON::XS will encode undefined scalars as 1077difficult objects to encode: JSON::XS will encode undefined scalars as
1084 1105
1085You can not currently force the type in other, less obscure, ways. Tell me 1106You can not currently force the type in other, less obscure, ways. Tell me
1086if you need this capability (but don't forget to explain why it's needed 1107if you need this capability (but don't forget to explain why it's needed
1087:). 1108:).
1088 1109
1110Note that numerical precision has the same meaning as under Perl (so
1111binary to decimal conversion follows the same rules as in Perl, which
1112can differ to other languages). Also, your perl interpreter might expose
1113extensions to the floating point numbers of your platform, such as
1114infinities or NaN's - these cannot be represented in JSON, and it is an
1115error to pass those in.
1116
1089=back 1117=back
1118
1119=head2 OBJECT SERIALISATION
1120
1121As JSON cannot directly represent Perl objects, you have to choose between
1122a pure JSON representation (without the ability to deserialise the object
1123automatically again), and a nonstandard extension to the JSON syntax,
1124tagged values.
1125
1126=head3 SERIALISATION
1127
1128What happens when C<JSON::XS> encounters a Perl object depends on the
1129C<allow_blessed>, C<convert_blessed> and C<allow_tags> settings, which are
1130used in this order:
1131
1132=over 4
1133
1134=item 1. C<allow_tags> is enabled and the object has a C<FREEZE> method.
1135
1136In this case, C<JSON::XS> uses the L<Types::Serialiser> object
1137serialisation protocol to create a tagged JSON value, using a nonstandard
1138extension to the JSON syntax.
1139
1140This works by invoking the C<FREEZE> method on the object, with the first
1141argument being the object to serialise, and the second argument being the
1142constant string C<JSON> to distinguish it from other serialisers.
1143
1144The C<FREEZE> method can return any number of values (i.e. zero or
1145more). These values and the paclkage/classname of the object will then be
1146encoded as a tagged JSON value in the following format:
1147
1148 ("classname")[FREEZE return values...]
1149
1150e.g.:
1151
1152 ("URI")["http://www.google.com/"]
1153 ("MyDate")[2013,10,29]
1154 ("ImageData::JPEG")["Z3...VlCg=="]
1155
1156For example, the hypothetical C<My::Object> C<FREEZE> method might use the
1157objects C<type> and C<id> members to encode the object:
1158
1159 sub My::Object::FREEZE {
1160 my ($self, $serialiser) = @_;
1161
1162 ($self->{type}, $self->{id})
1163 }
1164
1165=item 2. C<convert_blessed> is enabled and the object has a C<TO_JSON> method.
1166
1167In this case, the C<TO_JSON> method of the object is invoked in scalar
1168context. It must return a single scalar that can be directly encoded into
1169JSON. This scalar replaces the object in the JSON text.
1170
1171For example, the following C<TO_JSON> method will convert all L<URI>
1172objects to JSON strings when serialised. The fatc that these values
1173originally were L<URI> objects is lost.
1174
1175 sub URI::TO_JSON {
1176 my ($uri) = @_;
1177 $uri->as_string
1178 }
1179
1180=item 3. C<allow_blessed> is enabled.
1181
1182The object will be serialised as a JSON null value.
1183
1184=item 4. none of the above
1185
1186If none of the settings are enabled or the respective methods are missing,
1187C<JSON::XS> throws an exception.
1188
1189=back
1190
1191=head3 DESERIALISATION
1192
1193For deserialisation there are only two cases to consider: either
1194nonstandard tagging was used, in which case C<allow_tags> decides,
1195or objects cannot be automatically be deserialised, in which
1196case you can use postprocessing or the C<filter_json_object> or
1197C<filter_json_single_key_object> callbacks to get some real objects our of
1198your JSON.
1199
1200This section only considers the tagged value case: I a tagged JSON object
1201is encountered during decoding and C<allow_tags> is disabled, a parse
1202error will result (as if tagged values were not part of the grammar).
1203
1204If C<allow_tags> is enabled, C<JSON::XS> will look up the C<THAW> method
1205of the package/classname used during serialisation (it will not attempt
1206to load the package as a Perl module). If there is no such method, the
1207decoding will fail with an error.
1208
1209Otherwise, the C<THAW> method is invoked with the classname as first
1210argument, the constant string C<JSON> as second argument, and all the
1211values from the JSON array (the values originally returned by the
1212C<FREEZE> method) as remaining arguments.
1213
1214The method must then return the object. While technically you can return
1215any Perl scalar, you might have to enable the C<enable_nonref> setting to
1216make that work in all cases, so better return an actual blessed reference.
1217
1218As an example, let's implement a C<THAW> function that regenerates the
1219C<My::Object> from the C<FREEZE> example earlier:
1220
1221 sub My::Object::THAW {
1222 my ($class, $serialiser, $type, $id) = @_;
1223
1224 $class->new (type => $type, id => $id)
1225 }
1090 1226
1091 1227
1092=head1 ENCODING/CODESET FLAG NOTES 1228=head1 ENCODING/CODESET FLAG NOTES
1093 1229
1094The interested reader might have seen a number of flags that signify 1230The interested reader might have seen a number of flags that signify
1119=item C<utf8> flag disabled 1255=item C<utf8> flag disabled
1120 1256
1121When C<utf8> is disabled (the default), then C<encode>/C<decode> generate 1257When C<utf8> is disabled (the default), then C<encode>/C<decode> generate
1122and expect Unicode strings, that is, characters with high ordinal Unicode 1258and expect Unicode strings, that is, characters with high ordinal Unicode
1123values (> 255) will be encoded as such characters, and likewise such 1259values (> 255) will be encoded as such characters, and likewise such
1124characters are decoded as-is, no canges to them will be done, except 1260characters are decoded as-is, no changes to them will be done, except
1125"(re-)interpreting" them as Unicode codepoints or Unicode characters, 1261"(re-)interpreting" them as Unicode codepoints or Unicode characters,
1126respectively (to Perl, these are the same thing in strings unless you do 1262respectively (to Perl, these are the same thing in strings unless you do
1127funny/weird/dumb stuff). 1263funny/weird/dumb stuff).
1128 1264
1129This is useful when you want to do the encoding yourself (e.g. when you 1265This is useful when you want to do the encoding yourself (e.g. when you
1237well - using C<eval> naively simply I<will> cause problems. 1373well - using C<eval> naively simply I<will> cause problems.
1238 1374
1239Another problem is that some javascript implementations reserve 1375Another problem is that some javascript implementations reserve
1240some property names for their own purposes (which probably makes 1376some property names for their own purposes (which probably makes
1241them non-ECMAscript-compliant). For example, Iceweasel reserves the 1377them non-ECMAscript-compliant). For example, Iceweasel reserves the
1242C<__proto__> property name for it's own purposes. 1378C<__proto__> property name for its own purposes.
1243 1379
1244If that is a problem, you could parse try to filter the resulting JSON 1380If that is a problem, you could parse try to filter the resulting JSON
1245output for these property strings, e.g.: 1381output for these property strings, e.g.:
1246 1382
1247 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g; 1383 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1248 1384
1249This works because C<__proto__> is not valid outside of strings, so every 1385This works because C<__proto__> is not valid outside of strings, so every
1250occurence of C<"__proto__"\s*:> must be a string used as property name. 1386occurrence of C<"__proto__"\s*:> must be a string used as property name.
1251 1387
1252If you know of other incompatibilities, please let me know. 1388If you know of other incompatibilities, please let me know.
1253 1389
1254 1390
1255=head2 JSON and YAML 1391=head2 JSON and YAML
1267 my $yaml = $to_yaml->encode ($ref) . "\n"; 1403 my $yaml = $to_yaml->encode ($ref) . "\n";
1268 1404
1269This will I<usually> generate JSON texts that also parse as valid 1405This will I<usually> generate JSON texts that also parse as valid
1270YAML. Please note that YAML has hardcoded limits on (simple) object key 1406YAML. Please note that YAML has hardcoded limits on (simple) object key
1271lengths that JSON doesn't have and also has different and incompatible 1407lengths that JSON doesn't have and also has different and incompatible
1272unicode handling, so you should make sure that your hash keys are 1408unicode character escape syntax, so you should make sure that your hash
1273noticeably shorter than the 1024 "stream characters" YAML allows and that 1409keys are noticeably shorter than the 1024 "stream characters" YAML allows
1274you do not have characters with codepoint values outside the Unicode BMP 1410and that you do not have characters with codepoint values outside the
1275(basic multilingual page). YAML also does not allow C<\/> sequences in 1411Unicode BMP (basic multilingual page). YAML also does not allow C<\/>
1276strings (which JSON::XS does not I<currently> generate, but other JSON 1412sequences in strings (which JSON::XS does not I<currently> generate, but
1277generators might). 1413other JSON generators might).
1278 1414
1279There might be other incompatibilities that I am not aware of (or the YAML 1415There might be other incompatibilities that I am not aware of (or the YAML
1280specification has been changed yet again - it does so quite often). In 1416specification has been changed yet again - it does so quite often). In
1281general you should not try to generate YAML with a JSON generator or vice 1417general you should not try to generate YAML with a JSON generator or vice
1282versa, or try to parse JSON with a YAML parser or vice versa: chances are 1418versa, or try to parse JSON with a YAML parser or vice versa: chances are
1301that difficult or long) and finally make YAML compatible to it, and 1437that difficult or long) and finally make YAML compatible to it, and
1302educating users about the changes, instead of spreading lies about the 1438educating users about the changes, instead of spreading lies about the
1303real compatibility for many I<years> and trying to silence people who 1439real compatibility for many I<years> and trying to silence people who
1304point out that it isn't true. 1440point out that it isn't true.
1305 1441
1442Addendum/2009: the YAML 1.2 spec is still incompatible with JSON, even
1443though the incompatibilities have been documented (and are known to Brian)
1444for many years and the spec makes explicit claims that YAML is a superset
1445of JSON. It would be so easy to fix, but apparently, bullying people and
1446corrupting userdata is so much easier.
1447
1306=back 1448=back
1307 1449
1308 1450
1309=head2 SPEED 1451=head2 SPEED
1310 1452
1317a very short single-line JSON string (also available at 1459a very short single-line JSON string (also available at
1318L<http://dist.schmorp.de/misc/json/short.json>). 1460L<http://dist.schmorp.de/misc/json/short.json>).
1319 1461
1320 {"method": "handleMessage", "params": ["user1", 1462 {"method": "handleMessage", "params": ["user1",
1321 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7, 1463 "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1322 true, false]} 1464 1, 0]}
1323 1465
1324It shows the number of encodes/decodes per second (JSON::XS uses 1466It shows the number of encodes/decodes per second (JSON::XS uses
1325the functional interface, while JSON::XS/2 uses the OO interface 1467the functional interface, while JSON::XS/2 uses the OO interface
1326with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables 1468with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1327shrink). Higher is better: 1469shrink. JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ
1470uses the from_json method). Higher is better:
1328 1471
1329 module | encode | decode | 1472 module | encode | decode |
1330 -----------|------------|------------| 1473 --------------|------------|------------|
1331 JSON 1.x | 4990.842 | 4088.813 | 1474 JSON::DWIW/DS | 86302.551 | 102300.098 |
1332 JSON::DWIW | 51653.990 | 71575.154 | 1475 JSON::DWIW/FJ | 86302.551 | 75983.768 |
1333 JSON::PC | 65948.176 | 74631.744 | 1476 JSON::PP | 15827.562 | 6638.658 |
1334 JSON::PP | 8931.652 | 3817.168 | 1477 JSON::Syck | 63358.066 | 47662.545 |
1335 JSON::Syck | 24877.248 | 27776.848 | 1478 JSON::XS | 511500.488 | 511500.488 |
1336 JSON::XS | 388361.481 | 227951.304 | 1479 JSON::XS/2 | 291271.111 | 388361.481 |
1337 JSON::XS/2 | 227951.304 | 218453.333 | 1480 JSON::XS/3 | 361577.931 | 361577.931 |
1338 JSON::XS/3 | 338250.323 | 218453.333 | 1481 Storable | 66788.280 | 265462.278 |
1339 Storable | 16500.016 | 135300.129 |
1340 -----------+------------+------------+ 1482 --------------+------------+------------+
1341 1483
1342That is, JSON::XS is about five times faster than JSON::DWIW on encoding, 1484That is, JSON::XS is almost six times faster than JSON::DWIW on encoding,
1343about three times faster on decoding, and over forty times faster 1485about five times faster on decoding, and over thirty to seventy times
1344than JSON, even with pretty-printing and key sorting. It also compares 1486faster than JSON's pure perl implementation. It also compares favourably
1345favourably to Storable for small amounts of data. 1487to Storable for small amounts of data.
1346 1488
1347Using a longer test string (roughly 18KB, generated from Yahoo! Locals 1489Using a longer test string (roughly 18KB, generated from Yahoo! Locals
1348search API (L<http://dist.schmorp.de/misc/json/long.json>). 1490search API (L<http://dist.schmorp.de/misc/json/long.json>).
1349 1491
1350 module | encode | decode | 1492 module | encode | decode |
1351 -----------|------------|------------| 1493 --------------|------------|------------|
1352 JSON 1.x | 55.260 | 34.971 | 1494 JSON::DWIW/DS | 1647.927 | 2673.916 |
1353 JSON::DWIW | 825.228 | 1082.513 | 1495 JSON::DWIW/FJ | 1630.249 | 2596.128 |
1354 JSON::PC | 3571.444 | 2394.829 |
1355 JSON::PP | 210.987 | 32.574 | 1496 JSON::PP | 400.640 | 62.311 |
1356 JSON::Syck | 552.551 | 787.544 | 1497 JSON::Syck | 1481.040 | 1524.869 |
1357 JSON::XS | 5780.463 | 4854.519 | 1498 JSON::XS | 20661.596 | 9541.183 |
1358 JSON::XS/2 | 3869.998 | 4798.975 | 1499 JSON::XS/2 | 10683.403 | 9416.938 |
1359 JSON::XS/3 | 5862.880 | 4798.975 | 1500 JSON::XS/3 | 20661.596 | 9400.054 |
1360 Storable | 4445.002 | 5235.027 | 1501 Storable | 19765.806 | 10000.725 |
1361 -----------+------------+------------+ 1502 --------------+------------+------------+
1362 1503
1363Again, JSON::XS leads by far (except for Storable which non-surprisingly 1504Again, JSON::XS leads by far (except for Storable which non-surprisingly
1364decodes faster). 1505decodes a bit faster).
1365 1506
1366On large strings containing lots of high Unicode characters, some modules 1507On large strings containing lots of high Unicode characters, some modules
1367(such as JSON::PC) seem to decode faster than JSON::XS, but the result 1508(such as JSON::PC) seem to decode faster than JSON::XS, but the result
1368will be broken due to missing (or wrong) Unicode handling. Others refuse 1509will be broken due to missing (or wrong) Unicode handling. Others refuse
1369to decode or encode properly, so it was impossible to prepare a fair 1510to decode or encode properly, so it was impossible to prepare a fair
1405information you might want to make sure that exceptions thrown by JSON::XS 1546information you might want to make sure that exceptions thrown by JSON::XS
1406will not end up in front of untrusted eyes. 1547will not end up in front of untrusted eyes.
1407 1548
1408If you are using JSON::XS to return packets to consumption 1549If you are using JSON::XS to return packets to consumption
1409by JavaScript scripts in a browser you should have a look at 1550by JavaScript scripts in a browser you should have a look at
1410L<http://jpsykes.com/47/practical-csrf-and-json-security> to see whether 1551L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to
1411you are vulnerable to some common attack vectors (which really are browser 1552see whether you are vulnerable to some common attack vectors (which really
1412design bugs, but it is still you who will have to deal with it, as major 1553are browser design bugs, but it is still you who will have to deal with
1413browser developers care only for features, not about getting security 1554it, as major browser developers care only for features, not about getting
1414right). 1555security right).
1556
1557
1558=head1 INTEROPERABILITY WITH OTHER MODULES
1559
1560C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean
1561constants. That means that the JSON true and false values will be
1562comaptible to true and false values of iother modules that do the same,
1563such as L<JSON::PP> and L<CBOR::XS>.
1564
1565
1566=head1 INTEROPERABILITY WITH OTHER JSON DECODERS
1567
1568As long as you only serialise data that can be directly expressed in JSON,
1569C<JSON::XS> is incapable of generating invalid JSON output (modulo bugs,
1570but C<JSON::XS> has found more bugs in the official JSON testsuite (1)
1571than the official JSON testsuite has found in C<JSON::XS> (0)).
1572
1573When you have trouble decoding JSON generated by this module using other
1574decoders, then it is very likely that you have an encoding mismatch or the
1575other decoder is broken.
1576
1577When decoding, C<JSON::XS> is strict by default and will likely catch all
1578errors. There are currently two settings that change this: C<relaxed>
1579makes C<JSON::XS> accept (but not generate) some non-standard extensions,
1580and C<allow_tags> will allow you to encode and decode Perl objects, at the
1581cost of not outputting valid JSON anymore.
1582
1583=head2 TAGGED VALUE SYNTAX AND STANDARD JSON EN/DECODERS
1584
1585When you use C<allow_tags> to use the extended (and also nonstandard and
1586invalid) JSON syntax for serialised objects, and you still want to decode
1587the generated When you want to serialise objects, you can run a regex
1588to replace the tagged syntax by standard JSON arrays (it only works for
1589"normal" packagesnames without comma, newlines or single colons). First,
1590the readable Perl version:
1591
1592 # if your FREEZE methods return no values, you need this replace first:
1593 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[\s*\]/[$1]/gx;
1594
1595 # this works for non-empty constructor arg lists:
1596 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[/[$1,/gx;
1597
1598And here is a less readable version that is easy to adapt to other
1599languages:
1600
1601 $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/[$1,/g;
1602
1603Here is an ECMAScript version (same regex):
1604
1605 json = json.replace (/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/g, "[$1,");
1606
1607Since this syntax converts to standard JSON arrays, it might be hard to
1608distinguish serialised objects from normal arrays. You can prepend a
1609"magic number" as first array element to reduce chances of a collision:
1610
1611 $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/["XU1peReLzT4ggEllLanBYq4G9VzliwKF",$1,/g;
1612
1613And after decoding the JSON text, you could walk the data
1614structure looking for arrays with a first element of
1615C<XU1peReLzT4ggEllLanBYq4G9VzliwKF>.
1616
1617The same approach can be used to create the tagged format with another
1618encoder. First, you create an array with the magic string as first member,
1619the classname as second, and constructor arguments last, encode it as part
1620of your JSON structure, and then:
1621
1622 $json =~ s/\[\s*"XU1peReLzT4ggEllLanBYq4G9VzliwKF"\s*,\s*("([^\\":,]+|\\.|::)*")\s*,/($1)[/g;
1623
1624Again, this has some limitations - the magic string must not be encoded
1625with character escapes, and the constructor arguments must be non-empty.
1626
1627
1628=head1 RFC7158
1629
1630Since this module was written, Google has written a new JSON RFC, RFC
16317158. Unfortunately, this RFC breaks compatibility with both the original
1632JSON specification on www.json.org and RFC4627.
1633
1634As far as I can see, you can get partial compatibility when parsing by
1635using C<< ->allow_nonref >>. However, consider thew security implications
1636of doing so.
1637
1638I haven't decided yet whether to break compatibility with RFC4627 by
1639default (and potentially leave applications insecure), or change the
1640default to follow RFC7158.
1415 1641
1416 1642
1417=head1 THREADS 1643=head1 THREADS
1418 1644
1419This module is I<not> guaranteed to be thread safe and there are no 1645This module is I<not> guaranteed to be thread safe and there are no
1422process simulations - use fork, it's I<much> faster, cheaper, better). 1648process simulations - use fork, it's I<much> faster, cheaper, better).
1423 1649
1424(It might actually work, but you have been warned). 1650(It might actually work, but you have been warned).
1425 1651
1426 1652
1653=head1 THE PERILS OF SETLOCALE
1654
1655Sometimes people avoid the Perl locale support and directly call the
1656system's setlocale function with C<LC_ALL>.
1657
1658This breaks both perl and modules such as JSON::XS, as stringification of
1659numbers no longer works correctly (e.g. C<$x = 0.1; print "$x"+1> might
1660print C<1>, and JSON::XS might output illegal JSON as JSON::XS relies on
1661perl to stringify numbers).
1662
1663The solution is simple: don't call C<setlocale>, or use it for only those
1664categories you need, such as C<LC_MESSAGES> or C<LC_CTYPE>.
1665
1666If you need C<LC_NUMERIC>, you should enable it only around the code that
1667actually needs it (avoiding stringification of numbers), and restore it
1668afterwards.
1669
1670
1427=head1 BUGS 1671=head1 BUGS
1428 1672
1429While the goal of this module is to be correct, that unfortunately does 1673While the goal of this module is to be correct, that unfortunately does
1430not mean it's bug-free, only that I think its design is bug-free. If you 1674not mean it's bug-free, only that I think its design is bug-free. If you
1431keep reporting bugs they will be fixed swiftly, though. 1675keep reporting bugs they will be fixed swiftly, though.
1433Please refrain from using rt.cpan.org or any other bug reporting 1677Please refrain from using rt.cpan.org or any other bug reporting
1434service. I put the contact address into my modules for a reason. 1678service. I put the contact address into my modules for a reason.
1435 1679
1436=cut 1680=cut
1437 1681
1438our $true = do { bless \(my $dummy = 1), "JSON::XS::Boolean" }; 1682BEGIN {
1439our $false = do { bless \(my $dummy = 0), "JSON::XS::Boolean" }; 1683 *true = \$Types::Serialiser::true;
1684 *true = \&Types::Serialiser::true;
1685 *false = \$Types::Serialiser::false;
1686 *false = \&Types::Serialiser::false;
1687 *is_bool = \&Types::Serialiser::is_bool;
1440 1688
1441sub true() { $true } 1689 *JSON::XS::Boolean:: = *Types::Serialiser::Boolean::;
1442sub false() { $false }
1443
1444sub is_bool($) {
1445 UNIVERSAL::isa $_[0], "JSON::XS::Boolean"
1446# or UNIVERSAL::isa $_[0], "JSON::Literal"
1447} 1690}
1448 1691
1449XSLoader::load "JSON::XS", $VERSION; 1692XSLoader::load "JSON::XS", $VERSION;
1450
1451package JSON::XS::Boolean;
1452
1453use overload
1454 "0+" => sub { ${$_[0]} },
1455 "++" => sub { $_[0] = ${$_[0]} + 1 },
1456 "--" => sub { $_[0] = ${$_[0]} - 1 },
1457 fallback => 1;
1458
14591;
1460 1693
1461=head1 SEE ALSO 1694=head1 SEE ALSO
1462 1695
1463The F<json_xs> command line utility for quick experiments. 1696The F<json_xs> command line utility for quick experiments.
1464 1697
1467 Marc Lehmann <schmorp@schmorp.de> 1700 Marc Lehmann <schmorp@schmorp.de>
1468 http://home.schmorp.de/ 1701 http://home.schmorp.de/
1469 1702
1470=cut 1703=cut
1471 1704
17051
1706

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines