ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.156 by root, Sun Feb 21 15:37:53 2016 UTC vs.
Revision 1.169 by root, Thu Nov 15 20:49:12 2018 UTC

… …
40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overridden) with no overhead due to emulation (by inheriting constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need it and
46require a C compiler when that is a problem. 46doesn't require a C compiler when that is a problem.
47 47
48As this is the n-th-something JSON module on CPAN, what was the reason 48As this is the n-th-something JSON module on CPAN, what was the reason
49to write yet another JSON module? While it seems there are many JSON 49to write yet another JSON module? While it seems there are many JSON
50modules, none of them correctly handle all corner cases, and in most cases 50modules, none of them correctly handle all corner cases, and in most cases
51their maintainers are unresponsive, gone missing, or not listening to bug 51their maintainers are unresponsive, gone missing, or not listening to bug
… …
101 101
102package JSON::XS; 102package JSON::XS;
103 103
104use common::sense; 104use common::sense;
105 105
106our $VERSION = 3.01; 106our $VERSION = 3.04;
107our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
108 108
109our @EXPORT = qw(encode_json decode_json); 109our @EXPORT = qw(encode_json decode_json);
110 110
111use Exporter; 111use Exporter;
… …
131 131
132Except being faster. 132Except being faster.
133 133
134=item $perl_scalar = decode_json $json_text 134=item $perl_scalar = decode_json $json_text
135 135
136The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 136The opposite of C<encode_json>: expects a UTF-8 (binary) string and tries
137to parse that as an UTF-8 encoded JSON text, returning the resulting 137to parse that as a UTF-8 encoded JSON text, returning the resulting
138reference. Croaks on error. 138reference. Croaks on error.
139 139
140This function call is functionally identical to: 140This function call is functionally identical to:
141 141
142 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 142 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
… …
202=over 4 202=over 4
203 203
204=item $json = new JSON::XS 204=item $json = new JSON::XS
205 205
206Creates a new JSON::XS object that can be used to de/encode JSON 206Creates a new JSON::XS object that can be used to de/encode JSON
207strings. All boolean flags described below are by default I<disabled>. 207strings. All boolean flags described below are by default I<disabled>
208(with the exception of C<allow_nonref>, which defaults to I<enabled> since
209version C<4.0>).
208 210
209The mutators for flags all return the JSON object again and thus calls can 211The mutators for flags all return the JSON object again and thus calls can
210be chained: 212be chained:
211 213
212 my $json = JSON::XS->new->utf8->space_after->encode ({a => [1,2]}) 214 my $json = JSON::XS->new->utf8->space_after->encode ({a => [1,2]})
… …
270 272
271=item $enabled = $json->get_utf8 273=item $enabled = $json->get_utf8
272 274
273If C<$enable> is true (or missing), then the C<encode> method will encode 275If C<$enable> is true (or missing), then the C<encode> method will encode
274the JSON result into UTF-8, as required by many protocols, while the 276the JSON result into UTF-8, as required by many protocols, while the
275C<decode> method expects to be handled an UTF-8-encoded string. Please 277C<decode> method expects to be handed a UTF-8-encoded string. Please
276note that UTF-8-encoded strings do not contain any characters outside the 278note that UTF-8-encoded strings do not contain any characters outside the
277range C<0..255>, they are thus useful for bytewise/binary I/O. In future 279range C<0..255>, they are thus useful for bytewise/binary I/O. In future
278versions, enabling this option might enable autodetection of the UTF-16 280versions, enabling this option might enable autodetection of the UTF-16
279and UTF-32 encoding families, as described in RFC4627. 281and UTF-32 encoding families, as described in RFC4627.
280 282
… …
365 367
366=item $enabled = $json->get_relaxed 368=item $enabled = $json->get_relaxed
367 369
368If C<$enable> is true (or missing), then C<decode> will accept some 370If C<$enable> is true (or missing), then C<decode> will accept some
369extensions to normal JSON syntax (see below). C<encode> will not be 371extensions to normal JSON syntax (see below). C<encode> will not be
370affected in anyway. I<Be aware that this option makes you accept invalid 372affected in any way. I<Be aware that this option makes you accept invalid
371JSON texts as if they were valid!>. I suggest only to use this option to 373JSON texts as if they were valid!>. I suggest only to use this option to
372parse application-specific files written by humans (configuration files, 374parse application-specific files written by humans (configuration files,
373resource files etc.) 375resource files etc.)
374 376
375If C<$enable> is false (the default), then C<decode> will only accept 377If C<$enable> is false (the default), then C<decode> will only accept
… …
441 443
442=item $json = $json->allow_nonref ([$enable]) 444=item $json = $json->allow_nonref ([$enable])
443 445
444=item $enabled = $json->get_allow_nonref 446=item $enabled = $json->get_allow_nonref
445 447
448Unlike other boolean options, this opotion is enabled by default beginning
449with version C<4.0>. See L<SECURITY CONSIDERATIONS> for the gory details.
450
446If C<$enable> is true (or missing), then the C<encode> method can convert a 451If C<$enable> is true (or missing), then the C<encode> method can convert a
447non-reference into its corresponding string, number or null JSON value, 452non-reference into its corresponding string, number or null JSON value,
448which is an extension to RFC4627. Likewise, C<decode> will accept those JSON 453which is an extension to RFC4627. Likewise, C<decode> will accept those JSON
449values instead of croaking. 454values instead of croaking.
450 455
451If C<$enable> is false, then the C<encode> method will croak if it isn't 456If C<$enable> is false, then the C<encode> method will croak if it isn't
452passed an arrayref or hashref, as JSON texts must either be an object 457passed an arrayref or hashref, as JSON texts must either be an object
453or array. Likewise, C<decode> will croak if given something that is not a 458or array. Likewise, C<decode> will croak if given something that is not a
454JSON object or array. 459JSON object or array.
455 460
456Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>, 461Example, encode a Perl scalar as JSON value without enabled C<allow_nonref>,
457resulting in an invalid JSON text: 462resulting in an error:
458 463
459 JSON::XS->new->allow_nonref->encode ("Hello, World!") 464 JSON::XS->new->allow_nonref (0)->encode ("Hello, World!")
460 => "Hello, World!" 465 => hash- or arrayref expected...
461 466
462=item $json = $json->allow_unknown ([$enable]) 467=item $json = $json->allow_unknown ([$enable])
463 468
464=item $enabled = $json->get_allow_unknown 469=item $enabled = $json->get_allow_unknown
465 470
… …
515 520
516This setting has no effect on C<decode>. 521This setting has no effect on C<decode>.
517 522
518=item $json = $json->allow_tags ([$enable]) 523=item $json = $json->allow_tags ([$enable])
519 524
520=item $enabled = $json->allow_tags 525=item $enabled = $json->get_allow_tags
521 526
522See L<OBJECT SERIALISATION> for details. 527See L<OBJECT SERIALISATION> for details.
523 528
524If C<$enable> is true (or missing), then C<encode>, upon encountering a 529If C<$enable> is true (or missing), then C<encode>, upon encountering a
525blessed object, will check for the availability of the C<FREEZE> method on 530blessed object, will check for the availability of the C<FREEZE> method on
… …
534in C<decode>, as if tags were not part of the grammar. 539in C<decode>, as if tags were not part of the grammar.
535 540
536=item $json = $json->filter_json_object ([$coderef->($hashref)]) 541=item $json = $json->filter_json_object ([$coderef->($hashref)])
537 542
538When C<$coderef> is specified, it will be called from C<decode> each 543When C<$coderef> is specified, it will be called from C<decode> each
539time it decodes a JSON object. The only argument is a reference to the 544time it decodes a JSON object. The only argument is a reference to
540newly-created hash. If the code references returns a single scalar (which 545the newly-created hash. If the code reference returns a single scalar
541need not be a reference), this value (i.e. a copy of that scalar to avoid 546(which need not be a reference), this value (or rather a copy of it) is
542aliasing) is inserted into the deserialised data structure. If it returns 547inserted into the deserialised data structure. If it returns an empty
543an empty list (NOTE: I<not> C<undef>, which is a valid scalar), the 548list (NOTE: I<not> C<undef>, which is a valid scalar), the original
544original deserialised hash will be inserted. This setting can slow down 549deserialised hash will be inserted. This setting can slow down decoding
545decoding considerably. 550considerably.
546 551
547When C<$coderef> is omitted or undefined, any existing callback will 552When C<$coderef> is omitted or undefined, any existing callback will
548be removed and C<decode> will not change the deserialised hash in any 553be removed and C<decode> will not change the deserialised hash in any
549way. 554way.
550 555
… …
748C<incr_skip> to skip the erroneous part). This is the most common way of 753C<incr_skip> to skip the erroneous part). This is the most common way of
749using the method. 754using the method.
750 755
751And finally, in list context, it will try to extract as many objects 756And finally, in list context, it will try to extract as many objects
752from the stream as it can find and return them, or the empty list 757from the stream as it can find and return them, or the empty list
753otherwise. For this to work, there must be no separators between the JSON 758otherwise. For this to work, there must be no separators (other than
754objects or arrays, instead they must be concatenated back-to-back. If 759whitespace) between the JSON objects or arrays, instead they must be
755an error occurs, an exception will be raised as in the scalar context 760concatenated back-to-back. If an error occurs, an exception will be
756case. Note that in this case, any previously-parsed JSON texts will be 761raised as in the scalar context case. Note that in this case, any
757lost. 762previously-parsed JSON texts will be lost.
758 763
759Example: Parse some JSON arrays/objects in a given string and return 764Example: Parse some JSON arrays/objects in a given string and return
760them. 765them.
761 766
762 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]"); 767 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
… …
768C<incr_parse> in I<scalar context> successfully returned an object. Under 773C<incr_parse> in I<scalar context> successfully returned an object. Under
769all other circumstances you must not call this function (I mean it. 774all other circumstances you must not call this function (I mean it.
770although in simple tests it might actually work, it I<will> fail under 775although in simple tests it might actually work, it I<will> fail under
771real world conditions). As a special exception, you can also call this 776real world conditions). As a special exception, you can also call this
772method before having parsed anything. 777method before having parsed anything.
778
779That means you can only use this function to look at or manipulate text
780before or after complete JSON objects, not while the parser is in the
781middle of parsing a JSON object.
773 782
774This function is useful in two cases: a) finding the trailing text after a 783This function is useful in two cases: a) finding the trailing text after a
775JSON object or b) parsing multiple JSON objects separated by non-JSON text 784JSON object or b) parsing multiple JSON objects separated by non-JSON text
776(such as commas). 785(such as commas).
777 786
… …
1285expect your input strings to be encoded as UTF-8, that is, no "character" 1294expect your input strings to be encoded as UTF-8, that is, no "character"
1286of the input string must have any value > 255, as UTF-8 does not allow 1295of the input string must have any value > 255, as UTF-8 does not allow
1287that. 1296that.
1288 1297
1289The C<utf8> flag therefore switches between two modes: disabled means you 1298The C<utf8> flag therefore switches between two modes: disabled means you
1290will get a Unicode string in Perl, enabled means you get an UTF-8 encoded 1299will get a Unicode string in Perl, enabled means you get a UTF-8 encoded
1291octet/binary string in Perl. 1300octet/binary string in Perl.
1292 1301
1293=item C<latin1> or C<ascii> flags enabled 1302=item C<latin1> or C<ascii> flags enabled
1294 1303
1295With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters 1304With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters
… …
1563are browser design bugs, but it is still you who will have to deal with 1572are browser design bugs, but it is still you who will have to deal with
1564it, as major browser developers care only for features, not about getting 1573it, as major browser developers care only for features, not about getting
1565security right). 1574security right).
1566 1575
1567 1576
1577=head2 "OLD" VS. "NEW" JSON (RFC 4627 VS. RFC 7159)
1578
1579JSON originally required JSON texts to represent an array or object -
1580scalar values were explicitly not allowed. This has changed, and versions
1581of JSON::XS beginning with C<4.0> reflect this by allowing scalar values
1582by default.
1583
1584One reason why one might not want this is that this removes a fundamental
1585property of JSON texts, namely that they are self-delimited and
1586self-contained, or in other words, you could take any number of "old"
1587JSON texts and paste them together, and the result would be unambiguously
1588parseable:
1589
1590 [1,3]{"k":5}[][null] # four JSON texts, without doubt
1591
1592By allowing scalars, this property is lost: in the following example, is
1593this one JSON text (the number 12) or two JSON texts (the numbers 1 and
15942):
1595
1596 12 # could be 12, or 1 and 2
1597
1598Another lost property of "old" JSON is that no lookahead is required to
1599know the end of a JSON text, i.e. the JSON text definitely ended at the
1600last C<]> or C<}> character, there was no need to read extra characters.
1601
1602For example, a viable network protocol with "old" JSON was to simply
1603exchange JSON texts without delimiter. For "new" JSON, you have to use a
1604suitable delimiter (such as a newline) after every JSON text or ensure you
1605never encode/decode scalar values.
1606
1607Most protocols do work by only transferring arrays or objects, and the
1608easiest way to avoid problems with the "new" JSON definition is to
1609explicitly disallow scalar values in your encoder and decoder:
1610
1611 $json_coder = JSON::XS->new->allow_nonref (0)
1612
1613This is a somewhat unhappy situation, and the blame can fully be put on
1614JSON's inmventor, Douglas Crockford, who unilaterally changed the format
1615in 2006 without consulting the IETF, forcing the IETF to either fork the
1616format or go with it (as I was told, the IETF wasn't amused).
1617
1618
1568=head1 INTEROPERABILITY WITH OTHER MODULES 1619=head1 INTEROPERABILITY WITH OTHER MODULES
1569 1620
1570C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean 1621C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean
1571constants. That means that the JSON true and false values will be 1622constants. That means that the JSON true and false values will be
1572comaptible to true and false values of iother modules that do the same, 1623comaptible to true and false values of other modules that do the same,
1573such as L<JSON::PP> and L<CBOR::XS>. 1624such as L<JSON::PP> and L<CBOR::XS>.
1574 1625
1575 1626
1576=head1 INTEROPERABILITY WITH OTHER JSON DECODERS 1627=head1 INTEROPERABILITY WITH OTHER JSON DECODERS
1577 1628
… …
1594 1645
1595When you use C<allow_tags> to use the extended (and also nonstandard and 1646When you use C<allow_tags> to use the extended (and also nonstandard and
1596invalid) JSON syntax for serialised objects, and you still want to decode 1647invalid) JSON syntax for serialised objects, and you still want to decode
1597the generated When you want to serialise objects, you can run a regex 1648the generated When you want to serialise objects, you can run a regex
1598to replace the tagged syntax by standard JSON arrays (it only works for 1649to replace the tagged syntax by standard JSON arrays (it only works for
1599"normal" packagesnames without comma, newlines or single colons). First, 1650"normal" package names without comma, newlines or single colons). First,
1600the readable Perl version: 1651the readable Perl version:
1601 1652
1602 # if your FREEZE methods return no values, you need this replace first: 1653 # if your FREEZE methods return no values, you need this replace first:
1603 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[\s*\]/[$1]/gx; 1654 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[\s*\]/[$1]/gx;
1604 1655
… …
1640Since this module was written, Google has written a new JSON RFC, RFC 7159 1691Since this module was written, Google has written a new JSON RFC, RFC 7159
1641(and RFC7158). Unfortunately, this RFC breaks compatibility with both the 1692(and RFC7158). Unfortunately, this RFC breaks compatibility with both the
1642original JSON specification on www.json.org and RFC4627. 1693original JSON specification on www.json.org and RFC4627.
1643 1694
1644As far as I can see, you can get partial compatibility when parsing by 1695As far as I can see, you can get partial compatibility when parsing by
1645using C<< ->allow_nonref >>. However, consider thew security implications 1696using C<< ->allow_nonref >>. However, consider the security implications
1646of doing so. 1697of doing so.
1647 1698
1648I haven't decided yet when to break compatibility with RFC4627 by default 1699I haven't decided yet when to break compatibility with RFC4627 by default
1649(and potentially leave applications insecure) and change the default to 1700(and potentially leave applications insecure) and change the default to
1650follow RFC7159, but application authors are well advised to call C<< 1701follow RFC7159, but application authors are well advised to call C<<
1651->allow_nonref(0) >> even if this is the current default, if they cannot 1702->allow_nonref(0) >> even if this is the current default, if they cannot
1652handle non-reference values, in preparation for the day when the4 default 1703handle non-reference values, in preparation for the day when the default
1653will change. 1704will change.
1654 1705
1655 1706
1656=head1 THREADS 1707=head1 (I-)THREADS
1657 1708
1658This module is I<not> guaranteed to be thread safe and there are no 1709This module is I<not> guaranteed to be ithread (or MULTIPLICITY-) safe
1659plans to change this until Perl gets thread support (as opposed to the 1710and there are no plans to change this. Note that perl's builtin so-called
1660horribly slow so-called "threads" which are simply slow and bloated 1711threads/ithreads are officially deprecated and should not be used.
1661process simulations - use fork, it's I<much> faster, cheaper, better).
1662
1663(It might actually work, but you have been warned).
1664 1712
1665 1713
1666=head1 THE PERILS OF SETLOCALE 1714=head1 THE PERILS OF SETLOCALE
1667 1715
1668Sometimes people avoid the Perl locale support and directly call the 1716Sometimes people avoid the Perl locale support and directly call the

Diff Legend

– Removed lines
+ Added lines
< Changed lines
> Changed lines