ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.154 by root, Sun Mar 2 22:09:38 2014 UTC vs.
Revision 1.165 by root, Tue Sep 5 13:07:09 2017 UTC

… …
40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overridden) with no overhead due to emulation (by inheriting constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need it and
46require a C compiler when that is a problem. 46doesn't require a C compiler when that is a problem.
47 47
48As this is the n-th-something JSON module on CPAN, what was the reason 48As this is the n-th-something JSON module on CPAN, what was the reason
49to write yet another JSON module? While it seems there are many JSON 49to write yet another JSON module? While it seems there are many JSON
50modules, none of them correctly handle all corner cases, and in most cases 50modules, none of them correctly handle all corner cases, and in most cases
51their maintainers are unresponsive, gone missing, or not listening to bug 51their maintainers are unresponsive, gone missing, or not listening to bug
… …
101 101
102package JSON::XS; 102package JSON::XS;
103 103
104use common::sense; 104use common::sense;
105 105
106our $VERSION = 3.01; 106our $VERSION = 3.04;
107our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
108 108
109our @EXPORT = qw(encode_json decode_json); 109our @EXPORT = qw(encode_json decode_json);
110 110
111use Exporter; 111use Exporter;
… …
131 131
132Except being faster. 132Except being faster.
133 133
134=item $perl_scalar = decode_json $json_text 134=item $perl_scalar = decode_json $json_text
135 135
136The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 136The opposite of C<encode_json>: expects a UTF-8 (binary) string and tries
137to parse that as an UTF-8 encoded JSON text, returning the resulting 137to parse that as a UTF-8 encoded JSON text, returning the resulting
138reference. Croaks on error. 138reference. Croaks on error.
139 139
140This function call is functionally identical to: 140This function call is functionally identical to:
141 141
142 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 142 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
… …
270 270
271=item $enabled = $json->get_utf8 271=item $enabled = $json->get_utf8
272 272
273If C<$enable> is true (or missing), then the C<encode> method will encode 273If C<$enable> is true (or missing), then the C<encode> method will encode
274the JSON result into UTF-8, as required by many protocols, while the 274the JSON result into UTF-8, as required by many protocols, while the
275C<decode> method expects to be handled an UTF-8-encoded string. Please 275C<decode> method expects to be handed a UTF-8-encoded string. Please
276note that UTF-8-encoded strings do not contain any characters outside the 276note that UTF-8-encoded strings do not contain any characters outside the
277range C<0..255>, they are thus useful for bytewise/binary I/O. In future 277range C<0..255>, they are thus useful for bytewise/binary I/O. In future
278versions, enabling this option might enable autodetection of the UTF-16 278versions, enabling this option might enable autodetection of the UTF-16
279and UTF-32 encoding families, as described in RFC4627. 279and UTF-32 encoding families, as described in RFC4627.
280 280
… …
402character, after which more white-space and comments are allowed. 402character, after which more white-space and comments are allowed.
403 403
404 [ 404 [
405 1, # this comment not allowed in JSON 405 1, # this comment not allowed in JSON
406 # neither this one... 406 # neither this one...
407 ]
408
409=item * literal ASCII TAB characters in strings
410
411Literal ASCII TAB characters are now allowed in strings (and treated as
412C<\t>).
413
414 [
415 "Hello\tWorld",
416 "Hello<TAB>World", # literal <TAB> would not normally be allowed
407 ] 417 ]
408 418
409=back 419=back
410 420
411=item $json = $json->canonical ([$enable]) 421=item $json = $json->canonical ([$enable])
… …
687 697
688This is useful if your JSON texts are not delimited by an outer protocol 698This is useful if your JSON texts are not delimited by an outer protocol
689and you need to know where the JSON text ends. 699and you need to know where the JSON text ends.
690 700
691 JSON::XS->new->decode_prefix ("[1] the tail") 701 JSON::XS->new->decode_prefix ("[1] the tail")
692 => ([], 3) 702 => ([1], 3)
693 703
694=back 704=back
695 705
696 706
697=head1 INCREMENTAL PARSING 707=head1 INCREMENTAL PARSING
… …
738C<incr_skip> to skip the erroneous part). This is the most common way of 748C<incr_skip> to skip the erroneous part). This is the most common way of
739using the method. 749using the method.
740 750
741And finally, in list context, it will try to extract as many objects 751And finally, in list context, it will try to extract as many objects
742from the stream as it can find and return them, or the empty list 752from the stream as it can find and return them, or the empty list
743otherwise. For this to work, there must be no separators between the JSON 753otherwise. For this to work, there must be no separators (other than
744objects or arrays, instead they must be concatenated back-to-back. If 754whitespace) between the JSON objects or arrays, instead they must be
745an error occurs, an exception will be raised as in the scalar context 755concatenated back-to-back. If an error occurs, an exception will be
746case. Note that in this case, any previously-parsed JSON texts will be 756raised as in the scalar context case. Note that in this case, any
747lost. 757previously-parsed JSON texts will be lost.
748 758
749Example: Parse some JSON arrays/objects in a given string and return 759Example: Parse some JSON arrays/objects in a given string and return
750them. 760them.
751 761
752 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]"); 762 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
… …
758C<incr_parse> in I<scalar context> successfully returned an object. Under 768C<incr_parse> in I<scalar context> successfully returned an object. Under
759all other circumstances you must not call this function (I mean it. 769all other circumstances you must not call this function (I mean it.
760although in simple tests it might actually work, it I<will> fail under 770although in simple tests it might actually work, it I<will> fail under
761real world conditions). As a special exception, you can also call this 771real world conditions). As a special exception, you can also call this
762method before having parsed anything. 772method before having parsed anything.
773
774That means you can only use this function to look at or manipulate text
775before or after complete JSON objects, not while the parser is in the
776middle of parsing a JSON object.
763 777
764This function is useful in two cases: a) finding the trailing text after a 778This function is useful in two cases: a) finding the trailing text after a
765JSON object or b) parsing multiple JSON objects separated by non-JSON text 779JSON object or b) parsing multiple JSON objects separated by non-JSON text
766(such as commas). 780(such as commas).
767 781
… …
1275expect your input strings to be encoded as UTF-8, that is, no "character" 1289expect your input strings to be encoded as UTF-8, that is, no "character"
1276of the input string must have any value > 255, as UTF-8 does not allow 1290of the input string must have any value > 255, as UTF-8 does not allow
1277that. 1291that.
1278 1292
1279The C<utf8> flag therefore switches between two modes: disabled means you 1293The C<utf8> flag therefore switches between two modes: disabled means you
1280will get a Unicode string in Perl, enabled means you get an UTF-8 encoded 1294will get a Unicode string in Perl, enabled means you get a UTF-8 encoded
1281octet/binary string in Perl. 1295octet/binary string in Perl.
1282 1296
1283=item C<latin1> or C<ascii> flags enabled 1297=item C<latin1> or C<ascii> flags enabled
1284 1298
1285With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters 1299With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters
… …
1553are browser design bugs, but it is still you who will have to deal with 1567are browser design bugs, but it is still you who will have to deal with
1554it, as major browser developers care only for features, not about getting 1568it, as major browser developers care only for features, not about getting
1555security right). 1569security right).
1556 1570
1557 1571
1572=head1 "OLD" VS. "NEW" JSON (RFC 4627 VS. RFC 7159)
1573
1574TL;DR: Due to security concerns, JSON::XS will not allow scalar data in
1575JSON texts by default - you need to create your own JSON::XS object and
1576enable C<allow_nonref>:
1577
1578
1579 my $json = JSON::XS->new->allow_nonref;
1580
1581 $text = $json->encode ($data);
1582 $data = $json->decode ($text);
1583
1584The long version: JSON being an important and supposedly stable format,
1585the IETF standardised it as RFC 4627 in 2006. Unfortunately, the inventor
1586of JSON, Dougles Crockford, unilaterally changed the definition of JSON in
1587javascript. Rather than create a fork, the IETF decided to standardise the
1588new syntax (apparently, so Iw as told, without finding it very amusing).
1589
1590The biggest difference between thed original JSON and the new JSON is that
1591the new JSON supports scalars (anything other than arrays and objects) at
1592the toplevel of a JSON text. While this is strictly backwards compatible
1593to older versions, it breaks a number of protocols that relied on sending
1594JSON back-to-back, and is a minor security concern.
1595
1596For example, imagine you have two banks communicating, and on one side,
1597trhe JSON coder gets upgraded. Two messages, such as C<10> and C<1000>
1598might then be confused to mean C<101000>, something that couldn't happen
1599in the original JSON, because niether of these messages would be valid
1600JSON.
1601
1602If one side accepts these messages, then an upgrade in the coder on either
1603side could result in this becoming exploitable.
1604
1605This module has always allowed these messages as an optional extension, by
1606default disabled. The security concerns are the reason why the default is
1607still disabled, but future versions might/will likely upgrade to the newer
1608RFC as default format, so you are advised to check your implementation
1609and/or override the default with C<< ->allow_nonref (0) >> to ensure that
1610future versions are safe.
1611
1612
1558=head1 INTEROPERABILITY WITH OTHER MODULES 1613=head1 INTEROPERABILITY WITH OTHER MODULES
1559 1614
1560C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean 1615C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean
1561constants. That means that the JSON true and false values will be 1616constants. That means that the JSON true and false values will be
1562comaptible to true and false values of iother modules that do the same, 1617comaptible to true and false values of other modules that do the same,
1563such as L<JSON::PP> and L<CBOR::XS>. 1618such as L<JSON::PP> and L<CBOR::XS>.
1564 1619
1565 1620
1566=head1 INTEROPERABILITY WITH OTHER JSON DECODERS 1621=head1 INTEROPERABILITY WITH OTHER JSON DECODERS
1567 1622
… …
1584 1639
1585When you use C<allow_tags> to use the extended (and also nonstandard and 1640When you use C<allow_tags> to use the extended (and also nonstandard and
1586invalid) JSON syntax for serialised objects, and you still want to decode 1641invalid) JSON syntax for serialised objects, and you still want to decode
1587the generated When you want to serialise objects, you can run a regex 1642the generated When you want to serialise objects, you can run a regex
1588to replace the tagged syntax by standard JSON arrays (it only works for 1643to replace the tagged syntax by standard JSON arrays (it only works for
1589"normal" packagesnames without comma, newlines or single colons). First, 1644"normal" package names without comma, newlines or single colons). First,
1590the readable Perl version: 1645the readable Perl version:
1591 1646
1592 # if your FREEZE methods return no values, you need this replace first: 1647 # if your FREEZE methods return no values, you need this replace first:
1593 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[\s*\]/[$1]/gx; 1648 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[\s*\]/[$1]/gx;
1594 1649
… …
1623 1678
1624Again, this has some limitations - the magic string must not be encoded 1679Again, this has some limitations - the magic string must not be encoded
1625with character escapes, and the constructor arguments must be non-empty. 1680with character escapes, and the constructor arguments must be non-empty.
1626 1681
1627 1682
1628=head1 RFC7158 1683=head1 RFC7159
1629 1684
1630Since this module was written, Google has written a new JSON RFC, RFC 1685Since this module was written, Google has written a new JSON RFC, RFC 7159
16317158. Unfortunately, this RFC breaks compatibility with both the original 1686(and RFC7158). Unfortunately, this RFC breaks compatibility with both the
1632JSON specification on www.json.org and RFC4627. 1687original JSON specification on www.json.org and RFC4627.
1633 1688
1634As far as I can see, you can get partial compatibility when parsing by 1689As far as I can see, you can get partial compatibility when parsing by
1635using C<< ->allow_nonref >>. However, consider thew security implications 1690using C<< ->allow_nonref >>. However, consider the security implications
1636of doing so. 1691of doing so.
1637 1692
1638I haven't decided yet whether to break compatibility with RFC4627 by 1693I haven't decided yet when to break compatibility with RFC4627 by default
1639default (and potentially leave applications insecure), or change the 1694(and potentially leave applications insecure) and change the default to
1640default to follow RFC7158. 1695follow RFC7159, but application authors are well advised to call C<<
1696->allow_nonref(0) >> even if this is the current default, if they cannot
1697handle non-reference values, in preparation for the day when the default
1698will change.
1641 1699
1642 1700
1643=head1 THREADS 1701=head1 (I-)THREADS
1644 1702
1645This module is I<not> guaranteed to be thread safe and there are no 1703This module is I<not> guaranteed to be ithread (or MULTIPLICITY-) safe
1646plans to change this until Perl gets thread support (as opposed to the 1704and there are no plans to change this. Note that perl's builtin so-called
1647horribly slow so-called "threads" which are simply slow and bloated 1705theeads/ithreads are officially deprecated and should not be used.
1648process simulations - use fork, it's I<much> faster, cheaper, better).
1649
1650(It might actually work, but you have been warned).
1651 1706
1652 1707
1653=head1 THE PERILS OF SETLOCALE 1708=head1 THE PERILS OF SETLOCALE
1654 1709
1655Sometimes people avoid the Perl locale support and directly call the 1710Sometimes people avoid the Perl locale support and directly call the

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines