ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.142 by root, Fri Oct 25 19:57:42 2013 UTC vs.
Revision 1.153 by root, Sun Mar 2 20:41:14 2014 UTC

101 101
102package JSON::XS; 102package JSON::XS;
103 103
104use common::sense; 104use common::sense;
105 105
106our $VERSION = 2.34; 106our $VERSION = 3.01;
107our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
108 108
109our @EXPORT = qw(encode_json decode_json); 109our @EXPORT = qw(encode_json decode_json);
110 110
111use Exporter; 111use Exporter;
112use XSLoader; 112use XSLoader;
113
114use Types::Serialiser ();
113 115
114=head1 FUNCTIONAL INTERFACE 116=head1 FUNCTIONAL INTERFACE
115 117
116The following convenience methods are provided by this module. They are 118The following convenience methods are provided by this module. They are
117exported by default: 119exported by default:
138This function call is functionally identical to: 140This function call is functionally identical to:
139 141
140 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 142 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
141 143
142Except being faster. 144Except being faster.
143
144=item $is_boolean = JSON::XS::is_bool $scalar
145
146Returns true if the passed scalar represents either JSON::XS::true or
147JSON::XS::false, two constants that act like C<1> and C<0>, respectively
148and are used to represent JSON C<true> and C<false> values in Perl.
149
150See MAPPING, below, for more information on how JSON values are mapped to
151Perl.
152 145
153=back 146=back
154 147
155 148
156=head1 A FEW NOTES ON UNICODE AND PERL 149=head1 A FEW NOTES ON UNICODE AND PERL
474 467
475=item $json = $json->allow_blessed ([$enable]) 468=item $json = $json->allow_blessed ([$enable])
476 469
477=item $enabled = $json->get_allow_blessed 470=item $enabled = $json->get_allow_blessed
478 471
472See L<OBJECT SERIALISATION> for details.
473
479If C<$enable> is true (or missing), then the C<encode> method will not 474If C<$enable> is true (or missing), then the C<encode> method will not
480barf when it encounters a blessed reference. Instead, the value of the 475barf when it encounters a blessed reference that it cannot convert
481B<convert_blessed> option will decide whether C<null> (C<convert_blessed> 476otherwise. Instead, a JSON C<null> value is encoded instead of the object.
482disabled or no C<TO_JSON> method found) or a representation of the
483object (C<convert_blessed> enabled and C<TO_JSON> method found) is being
484encoded. Has no effect on C<decode>.
485 477
486If C<$enable> is false (the default), then C<encode> will throw an 478If C<$enable> is false (the default), then C<encode> will throw an
487exception when it encounters a blessed object. 479exception when it encounters a blessed object that it cannot convert
480otherwise.
481
482This setting has no effect on C<decode>.
488 483
489=item $json = $json->convert_blessed ([$enable]) 484=item $json = $json->convert_blessed ([$enable])
490 485
491=item $enabled = $json->get_convert_blessed 486=item $enabled = $json->get_convert_blessed
487
488See L<OBJECT SERIALISATION> for details.
492 489
493If C<$enable> is true (or missing), then C<encode>, upon encountering a 490If C<$enable> is true (or missing), then C<encode>, upon encountering a
494blessed object, will check for the availability of the C<TO_JSON> method 491blessed object, will check for the availability of the C<TO_JSON> method
495on the object's class. If found, it will be called in scalar context 492on the object's class. If found, it will be called in scalar context and
496and the resulting scalar will be encoded instead of the object. If no 493the resulting scalar will be encoded instead of the object.
497C<TO_JSON> method is found, the value of C<allow_blessed> will decide what
498to do.
499 494
500The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON> 495The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON>
501returns other blessed objects, those will be handled in the same 496returns other blessed objects, those will be handled in the same
502way. C<TO_JSON> must take care of not causing an endless recursion cycle 497way. C<TO_JSON> must take care of not causing an endless recursion cycle
503(== crash) in this case. The name of C<TO_JSON> was chosen because other 498(== crash) in this case. The name of C<TO_JSON> was chosen because other
504methods called by the Perl core (== not by the user of the object) are 499methods called by the Perl core (== not by the user of the object) are
505usually in upper case letters and to avoid collisions with any C<to_json> 500usually in upper case letters and to avoid collisions with any C<to_json>
506function or method. 501function or method.
507 502
508This setting does not yet influence C<decode> in any way, but in the 503If C<$enable> is false (the default), then C<encode> will not consider
509future, global hooks might get installed that influence C<decode> and are 504this type of conversion.
510enabled by this setting.
511 505
512If C<$enable> is false, then the C<allow_blessed> setting will decide what 506This setting has no effect on C<decode>.
513to do when a blessed object is found. 507
508=item $json = $json->allow_tags ([$enable])
509
510=item $enabled = $json->allow_tags
511
512See L<OBJECT SERIALISATION> for details.
513
514If C<$enable> is true (or missing), then C<encode>, upon encountering a
515blessed object, will check for the availability of the C<FREEZE> method on
516the object's class. If found, it will be used to serialise the object into
517a nonstandard tagged JSON value (that JSON decoders cannot decode).
518
519It also causes C<decode> to parse such tagged JSON values and deserialise
520them via a call to the C<THAW> method.
521
522If C<$enable> is false (the default), then C<encode> will not consider
523this type of conversion, and tagged JSON values will cause a parse error
524in C<decode>, as if tags were not part of the grammar.
514 525
515=item $json = $json->filter_json_object ([$coderef->($hashref)]) 526=item $json = $json->filter_json_object ([$coderef->($hashref)])
516 527
517When C<$coderef> is specified, it will be called from C<decode> each 528When C<$coderef> is specified, it will be called from C<decode> each
518time it decodes a JSON object. The only argument is a reference to the 529time it decodes a JSON object. The only argument is a reference to the
673when there is trailing garbage after the first JSON object, it will 684when there is trailing garbage after the first JSON object, it will
674silently stop parsing there and return the number of characters consumed 685silently stop parsing there and return the number of characters consumed
675so far. 686so far.
676 687
677This is useful if your JSON texts are not delimited by an outer protocol 688This is useful if your JSON texts are not delimited by an outer protocol
678(which is not the brightest thing to do in the first place) and you need
679to know where the JSON text ends. 689and you need to know where the JSON text ends.
680 690
681 JSON::XS->new->decode_prefix ("[1] the tail") 691 JSON::XS->new->decode_prefix ("[1] the tail")
682 => ([], 3) 692 => ([], 3)
683 693
684=back 694=back
778=back 788=back
779 789
780=head2 LIMITATIONS 790=head2 LIMITATIONS
781 791
782All options that affect decoding are supported, except 792All options that affect decoding are supported, except
783C<allow_nonref>. The reason for this is that it cannot be made to 793C<allow_nonref>. The reason for this is that it cannot be made to work
784work sensibly: JSON objects and arrays are self-delimited, i.e. you can concatenate 794sensibly: JSON objects and arrays are self-delimited, i.e. you can
785them back to back and still decode them perfectly. This does not hold true 795concatenate them back to back and still decode them perfectly. This does
786for JSON numbers, however. 796not hold true for JSON numbers, however.
787 797
788For example, is the string C<1> a single JSON number, or is it simply the 798For example, is the string C<1> a single JSON number, or is it simply the
789start of C<12>? Or is C<12> a single JSON number, or the concatenation 799start of C<12>? Or is C<12> a single JSON number, or the concatenation
790of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS 800of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS
791takes the conservative route and disallows this case. 801takes the conservative route and disallows this case.
984floating point, JSON::XS only guarantees precision up to but not including 994floating point, JSON::XS only guarantees precision up to but not including
985the least significant bit. 995the least significant bit.
986 996
987=item true, false 997=item true, false
988 998
989These JSON atoms become C<JSON::XS::true> and C<JSON::XS::false>, 999These JSON atoms become C<Types::Serialiser::true> and
990respectively. They are overloaded to act almost exactly like the numbers 1000C<Types::Serialiser::false>, respectively. They are overloaded to act
991C<1> and C<0>. You can check whether a scalar is a JSON boolean by using 1001almost exactly like the numbers C<1> and C<0>. You can check whether
992the C<JSON::XS::is_bool> function. 1002a scalar is a JSON boolean by using the C<Types::Serialiser::is_bool>
1003function (after C<use Types::Serialier>, of course).
993 1004
994=item null 1005=item null
995 1006
996A JSON null atom becomes C<undef> in Perl. 1007A JSON null atom becomes C<undef> in Perl.
1008
1009=item shell-style comments (C<< # I<text> >>)
1010
1011As a nonstandard extension to the JSON syntax that is enabled by the
1012C<relaxed> setting, shell-style comments are allowed. They can start
1013anywhere outside strings and go till the end of the line.
1014
1015=item tagged values (C<< (I<tag>)I<value> >>).
1016
1017Another nonstandard extension to the JSON syntax, enabled with the
1018C<allow_tags> setting, are tagged values. In this implementation, the
1019I<tag> must be a perl package/class name encoded as a JSON string, and the
1020I<value> must be a JSON array encoding optional constructor arguments.
1021
1022See L<OBJECT SERIALISATION>, below, for details.
997 1023
998=back 1024=back
999 1025
1000 1026
1001=head2 PERL -> JSON 1027=head2 PERL -> JSON
1022 1048
1023=item other references 1049=item other references
1024 1050
1025Other unblessed references are generally not allowed and will cause an 1051Other unblessed references are generally not allowed and will cause an
1026exception to be thrown, except for references to the integers C<0> and 1052exception to be thrown, except for references to the integers C<0> and
1027C<1>, which get turned into C<false> and C<true> atoms in JSON. You can 1053C<1>, which get turned into C<false> and C<true> atoms in JSON.
1028also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
1029 1054
1055Since C<JSON::XS> uses the boolean model from L<Types::Serialiser>, you
1056can also C<use Types::Serialiser> and then use C<Types::Serialiser::false>
1057and C<Types::Serialiser::true> to improve readability.
1058
1059 use Types::Serialiser;
1030 encode_json [\0, JSON::XS::true] # yields [false,true] 1060 encode_json [\0, Types::Serialiser::true] # yields [false,true]
1031 1061
1032=item JSON::XS::true, JSON::XS::false 1062=item Types::Serialiser::true, Types::Serialiser::false
1033 1063
1034These special values become JSON true and JSON false values, 1064These special values from the L<Types::Serialiser> module become JSON true
1035respectively. You can also use C<\1> and C<\0> directly if you want. 1065and JSON false values, respectively. You can also use C<\1> and C<\0>
1066directly if you want.
1036 1067
1037=item blessed objects 1068=item blessed objects
1038 1069
1039Blessed objects are not directly representable in JSON. See the 1070Blessed objects are not directly representable in JSON, but C<JSON::XS>
1040C<allow_blessed> and C<convert_blessed> methods on various options on 1071allows various ways of handling objects. See L<OBJECT SERIALISATION>,
1041how to deal with this: basically, you can choose between throwing an 1072below, for details.
1042exception, encoding the reference as if it weren't blessed, or provide
1043your own serialiser method.
1044 1073
1045=item simple scalars 1074=item simple scalars
1046 1075
1047Simple Perl scalars (any scalar that is not a reference) are the most 1076Simple Perl scalars (any scalar that is not a reference) are the most
1048difficult objects to encode: JSON::XS will encode undefined scalars as 1077difficult objects to encode: JSON::XS will encode undefined scalars as
1084extensions to the floating point numbers of your platform, such as 1113extensions to the floating point numbers of your platform, such as
1085infinities or NaN's - these cannot be represented in JSON, and it is an 1114infinities or NaN's - these cannot be represented in JSON, and it is an
1086error to pass those in. 1115error to pass those in.
1087 1116
1088=back 1117=back
1118
1119=head2 OBJECT SERIALISATION
1120
1121As JSON cannot directly represent Perl objects, you have to choose between
1122a pure JSON representation (without the ability to deserialise the object
1123automatically again), and a nonstandard extension to the JSON syntax,
1124tagged values.
1125
1126=head3 SERIALISATION
1127
1128What happens when C<JSON::XS> encounters a Perl object depends on the
1129C<allow_blessed>, C<convert_blessed> and C<allow_tags> settings, which are
1130used in this order:
1131
1132=over 4
1133
1134=item 1. C<allow_tags> is enabled and the object has a C<FREEZE> method.
1135
1136In this case, C<JSON::XS> uses the L<Types::Serialiser> object
1137serialisation protocol to create a tagged JSON value, using a nonstandard
1138extension to the JSON syntax.
1139
1140This works by invoking the C<FREEZE> method on the object, with the first
1141argument being the object to serialise, and the second argument being the
1142constant string C<JSON> to distinguish it from other serialisers.
1143
1144The C<FREEZE> method can return any number of values (i.e. zero or
1145more). These values and the paclkage/classname of the object will then be
1146encoded as a tagged JSON value in the following format:
1147
1148 ("classname")[FREEZE return values...]
1149
1150e.g.:
1151
1152 ("URI")["http://www.google.com/"]
1153 ("MyDate")[2013,10,29]
1154 ("ImageData::JPEG")["Z3...VlCg=="]
1155
1156For example, the hypothetical C<My::Object> C<FREEZE> method might use the
1157objects C<type> and C<id> members to encode the object:
1158
1159 sub My::Object::FREEZE {
1160 my ($self, $serialiser) = @_;
1161
1162 ($self->{type}, $self->{id})
1163 }
1164
1165=item 2. C<convert_blessed> is enabled and the object has a C<TO_JSON> method.
1166
1167In this case, the C<TO_JSON> method of the object is invoked in scalar
1168context. It must return a single scalar that can be directly encoded into
1169JSON. This scalar replaces the object in the JSON text.
1170
1171For example, the following C<TO_JSON> method will convert all L<URI>
1172objects to JSON strings when serialised. The fatc that these values
1173originally were L<URI> objects is lost.
1174
1175 sub URI::TO_JSON {
1176 my ($uri) = @_;
1177 $uri->as_string
1178 }
1179
1180=item 3. C<allow_blessed> is enabled.
1181
1182The object will be serialised as a JSON null value.
1183
1184=item 4. none of the above
1185
1186If none of the settings are enabled or the respective methods are missing,
1187C<JSON::XS> throws an exception.
1188
1189=back
1190
1191=head3 DESERIALISATION
1192
1193For deserialisation there are only two cases to consider: either
1194nonstandard tagging was used, in which case C<allow_tags> decides,
1195or objects cannot be automatically be deserialised, in which
1196case you can use postprocessing or the C<filter_json_object> or
1197C<filter_json_single_key_object> callbacks to get some real objects our of
1198your JSON.
1199
1200This section only considers the tagged value case: I a tagged JSON object
1201is encountered during decoding and C<allow_tags> is disabled, a parse
1202error will result (as if tagged values were not part of the grammar).
1203
1204If C<allow_tags> is enabled, C<JSON::XS> will look up the C<THAW> method
1205of the package/classname used during serialisation (it will not attempt
1206to load the package as a Perl module). If there is no such method, the
1207decoding will fail with an error.
1208
1209Otherwise, the C<THAW> method is invoked with the classname as first
1210argument, the constant string C<JSON> as second argument, and all the
1211values from the JSON array (the values originally returned by the
1212C<FREEZE> method) as remaining arguments.
1213
1214The method must then return the object. While technically you can return
1215any Perl scalar, you might have to enable the C<enable_nonref> setting to
1216make that work in all cases, so better return an actual blessed reference.
1217
1218As an example, let's implement a C<THAW> function that regenerates the
1219C<My::Object> from the C<FREEZE> example earlier:
1220
1221 sub My::Object::THAW {
1222 my ($class, $serialiser, $type, $id) = @_;
1223
1224 $class->new (type => $type, id => $id)
1225 }
1089 1226
1090 1227
1091=head1 ENCODING/CODESET FLAG NOTES 1228=head1 ENCODING/CODESET FLAG NOTES
1092 1229
1093The interested reader might have seen a number of flags that signify 1230The interested reader might have seen a number of flags that signify
1416are browser design bugs, but it is still you who will have to deal with 1553are browser design bugs, but it is still you who will have to deal with
1417it, as major browser developers care only for features, not about getting 1554it, as major browser developers care only for features, not about getting
1418security right). 1555security right).
1419 1556
1420 1557
1558=head1 INTEROPERABILITY WITH OTHER MODULES
1559
1560C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean
1561constants. That means that the JSON true and false values will be
1562comaptible to true and false values of iother modules that do the same,
1563such as L<JSON::PP> and L<CBOR::XS>.
1564
1565
1566=head1 INTEROPERABILITY WITH OTHER JSON DECODERS
1567
1568As long as you only serialise data that can be directly expressed in JSON,
1569C<JSON::XS> is incapable of generating invalid JSON output (modulo bugs,
1570but C<JSON::XS> has found more bugs in the official JSON testsuite (1)
1571than the official JSON testsuite has found in C<JSON::XS> (0)).
1572
1573When you have trouble decoding JSON generated by this module using other
1574decoders, then it is very likely that you have an encoding mismatch or the
1575other decoder is broken.
1576
1577When decoding, C<JSON::XS> is strict by default and will likely catch all
1578errors. There are currently two settings that change this: C<relaxed>
1579makes C<JSON::XS> accept (but not generate) some non-standard extensions,
1580and C<allow_tags> will allow you to encode and decode Perl objects, at the
1581cost of not outputting valid JSON anymore.
1582
1583=head2 TAGGED VALUE SYNTAX AND STANDARD JSON EN/DECODERS
1584
1585When you use C<allow_tags> to use the extended (and also nonstandard and
1586invalid) JSON syntax for serialised objects, and you still want to decode
1587the generated When you want to serialise objects, you can run a regex
1588to replace the tagged syntax by standard JSON arrays (it only works for
1589"normal" packagesnames without comma, newlines or single colons). First,
1590the readable Perl version:
1591
1592 # if your FREEZE methods return no values, you need this replace first:
1593 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[\s*\]/[$1]/gx;
1594
1595 # this works for non-empty constructor arg lists:
1596 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[/[$1,/gx;
1597
1598And here is a less readable version that is easy to adapt to other
1599languages:
1600
1601 $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/[$1,/g;
1602
1603Here is an ECMAScript version (same regex):
1604
1605 json = json.replace (/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/g, "[$1,");
1606
1607Since this syntax converts to standard JSON arrays, it might be hard to
1608distinguish serialised objects from normal arrays. You can prepend a
1609"magic number" as first array element to reduce chances of a collision:
1610
1611 $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/["XU1peReLzT4ggEllLanBYq4G9VzliwKF",$1,/g;
1612
1613And after decoding the JSON text, you could walk the data
1614structure looking for arrays with a first element of
1615C<XU1peReLzT4ggEllLanBYq4G9VzliwKF>.
1616
1617The same approach cna be used to create the tagged format with another
1618encoder. First, you create an array with the magic string as first member,
1619the classname as second, and constructor arguments last, encode it as part
1620of your JSON structure, and then:
1621
1622 $json =~ s/\[\s*"XU1peReLzT4ggEllLanBYq4G9VzliwKF"\s*,\s*("([^\\":,]+|\\.|::)*")\s*,/($1)[/g;
1623
1624Again, this has some limitations - the magic string must not be encoded
1625with character escapes, and the constructor arguments must be non-empty.
1626
1627
1628=head1 RFC7158
1629
1630Since this module was written, Google has written a new JSON RFC, RFC
16317158. Unfortunately, this RFC breaks compatibility with both the original
1632JSON specification on www.json.org and RFC4627.
1633
1634As far as I can see, you can get partial compatibility when parsing by
1635using C<< ->allow_nonref >>. However, consider thew security implications
1636of doing so.
1637
1638I haven't decided yet whether to break compatibility with RFC4627 by
1639default (and potentially leave applications insecure), or change the
1640default to follow RFC7158.
1641
1642
1421=head1 THREADS 1643=head1 THREADS
1422 1644
1423This module is I<not> guaranteed to be thread safe and there are no 1645This module is I<not> guaranteed to be thread safe and there are no
1424plans to change this until Perl gets thread support (as opposed to the 1646plans to change this until Perl gets thread support (as opposed to the
1425horribly slow so-called "threads" which are simply slow and bloated 1647horribly slow so-called "threads" which are simply slow and bloated
1455Please refrain from using rt.cpan.org or any other bug reporting 1677Please refrain from using rt.cpan.org or any other bug reporting
1456service. I put the contact address into my modules for a reason. 1678service. I put the contact address into my modules for a reason.
1457 1679
1458=cut 1680=cut
1459 1681
1460our $true = do { bless \(my $dummy = 1), "JSON::XS::Boolean" }; 1682BEGIN {
1461our $false = do { bless \(my $dummy = 0), "JSON::XS::Boolean" }; 1683 *true = \$Types::Serialiser::true;
1684 *true = \&Types::Serialiser::true;
1685 *false = \$Types::Serialiser::false;
1686 *false = \&Types::Serialiser::false;
1687 *is_bool = \&Types::Serialiser::is_bool;
1462 1688
1463sub true() { $true } 1689 *JSON::XS::Boolean:: = *Types::Serialiser::Boolean::;
1464sub false() { $false }
1465
1466sub is_bool($) {
1467 UNIVERSAL::isa $_[0], "JSON::XS::Boolean"
1468# or UNIVERSAL::isa $_[0], "JSON::Literal"
1469} 1690}
1470 1691
1471XSLoader::load "JSON::XS", $VERSION; 1692XSLoader::load "JSON::XS", $VERSION;
1472
1473package JSON::XS::Boolean;
1474
1475use overload
1476 "0+" => sub { ${$_[0]} },
1477 "++" => sub { $_[0] = ${$_[0]} + 1 },
1478 "--" => sub { $_[0] = ${$_[0]} - 1 },
1479 fallback => 1;
1480
14811;
1482 1693
1483=head1 SEE ALSO 1694=head1 SEE ALSO
1484 1695
1485The F<json_xs> command line utility for quick experiments. 1696The F<json_xs> command line utility for quick experiments.
1486 1697
1489 Marc Lehmann <schmorp@schmorp.de> 1700 Marc Lehmann <schmorp@schmorp.de>
1490 http://home.schmorp.de/ 1701 http://home.schmorp.de/
1491 1702
1492=cut 1703=cut
1493 1704
17051
1706

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines