ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/JSON-XS/XS.pm
(Generate patch)

Comparing JSON-XS/XS.pm (file contents):
Revision 1.136 by root, Wed Jul 27 15:53:40 2011 UTC vs.
Revision 1.169 by root, Thu Nov 15 20:49:12 2018 UTC

40Beginning with version 2.0 of the JSON module, when both JSON and 40Beginning with version 2.0 of the JSON module, when both JSON and
41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be 41JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42overridden) with no overhead due to emulation (by inheriting constructor 42overridden) with no overhead due to emulation (by inheriting constructor
43and methods). If JSON::XS is not available, it will fall back to the 43and methods). If JSON::XS is not available, it will fall back to the
44compatible JSON::PP module as backend, so using JSON instead of JSON::XS 44compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45gives you a portable JSON API that can be fast when you need and doesn't 45gives you a portable JSON API that can be fast when you need it and
46require a C compiler when that is a problem. 46doesn't require a C compiler when that is a problem.
47 47
48As this is the n-th-something JSON module on CPAN, what was the reason 48As this is the n-th-something JSON module on CPAN, what was the reason
49to write yet another JSON module? While it seems there are many JSON 49to write yet another JSON module? While it seems there are many JSON
50modules, none of them correctly handle all corner cases, and in most cases 50modules, none of them correctly handle all corner cases, and in most cases
51their maintainers are unresponsive, gone missing, or not listening to bug 51their maintainers are unresponsive, gone missing, or not listening to bug
83this module usually compares favourably in terms of speed, too. 83this module usually compares favourably in terms of speed, too.
84 84
85=item * simple to use 85=item * simple to use
86 86
87This module has both a simple functional interface as well as an object 87This module has both a simple functional interface as well as an object
88oriented interface interface. 88oriented interface.
89 89
90=item * reasonably versatile output formats 90=item * reasonably versatile output formats
91 91
92You can choose between the most compact guaranteed-single-line format 92You can choose between the most compact guaranteed-single-line format
93possible (nice for simple line-based protocols), a pure-ASCII format 93possible (nice for simple line-based protocols), a pure-ASCII format
101 101
102package JSON::XS; 102package JSON::XS;
103 103
104use common::sense; 104use common::sense;
105 105
106our $VERSION = '2.31'; 106our $VERSION = 3.04;
107our @ISA = qw(Exporter); 107our @ISA = qw(Exporter);
108 108
109our @EXPORT = qw(encode_json decode_json to_json from_json); 109our @EXPORT = qw(encode_json decode_json);
110
111sub to_json($) {
112 require Carp;
113 Carp::croak ("JSON::XS::to_json has been renamed to encode_json, either downgrade to pre-2.0 versions of JSON::XS or rename the call");
114}
115
116sub from_json($) {
117 require Carp;
118 Carp::croak ("JSON::XS::from_json has been renamed to decode_json, either downgrade to pre-2.0 versions of JSON::XS or rename the call");
119}
120 110
121use Exporter; 111use Exporter;
122use XSLoader; 112use XSLoader;
123 113
114use Types::Serialiser ();
115
124=head1 FUNCTIONAL INTERFACE 116=head1 FUNCTIONAL INTERFACE
125 117
126The following convenience methods are provided by this module. They are 118The following convenience methods are provided by this module. They are
127exported by default: 119exported by default:
128 120
139 131
140Except being faster. 132Except being faster.
141 133
142=item $perl_scalar = decode_json $json_text 134=item $perl_scalar = decode_json $json_text
143 135
144The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries 136The opposite of C<encode_json>: expects a UTF-8 (binary) string and tries
145to parse that as an UTF-8 encoded JSON text, returning the resulting 137to parse that as a UTF-8 encoded JSON text, returning the resulting
146reference. Croaks on error. 138reference. Croaks on error.
147 139
148This function call is functionally identical to: 140This function call is functionally identical to:
149 141
150 $perl_scalar = JSON::XS->new->utf8->decode ($json_text) 142 $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
151 143
152Except being faster. 144Except being faster.
153
154=item $is_boolean = JSON::XS::is_bool $scalar
155
156Returns true if the passed scalar represents either JSON::XS::true or
157JSON::XS::false, two constants that act like C<1> and C<0>, respectively
158and are used to represent JSON C<true> and C<false> values in Perl.
159
160See MAPPING, below, for more information on how JSON values are mapped to
161Perl.
162 145
163=back 146=back
164 147
165 148
166=head1 A FEW NOTES ON UNICODE AND PERL 149=head1 A FEW NOTES ON UNICODE AND PERL
219=over 4 202=over 4
220 203
221=item $json = new JSON::XS 204=item $json = new JSON::XS
222 205
223Creates a new JSON::XS object that can be used to de/encode JSON 206Creates a new JSON::XS object that can be used to de/encode JSON
224strings. All boolean flags described below are by default I<disabled>. 207strings. All boolean flags described below are by default I<disabled>
208(with the exception of C<allow_nonref>, which defaults to I<enabled> since
209version C<4.0>).
225 210
226The mutators for flags all return the JSON object again and thus calls can 211The mutators for flags all return the JSON object again and thus calls can
227be chained: 212be chained:
228 213
229 my $json = JSON::XS->new->utf8->space_after->encode ({a => [1,2]}) 214 my $json = JSON::XS->new->utf8->space_after->encode ({a => [1,2]})
287 272
288=item $enabled = $json->get_utf8 273=item $enabled = $json->get_utf8
289 274
290If C<$enable> is true (or missing), then the C<encode> method will encode 275If C<$enable> is true (or missing), then the C<encode> method will encode
291the JSON result into UTF-8, as required by many protocols, while the 276the JSON result into UTF-8, as required by many protocols, while the
292C<decode> method expects to be handled an UTF-8-encoded string. Please 277C<decode> method expects to be handed a UTF-8-encoded string. Please
293note that UTF-8-encoded strings do not contain any characters outside the 278note that UTF-8-encoded strings do not contain any characters outside the
294range C<0..255>, they are thus useful for bytewise/binary I/O. In future 279range C<0..255>, they are thus useful for bytewise/binary I/O. In future
295versions, enabling this option might enable autodetection of the UTF-16 280versions, enabling this option might enable autodetection of the UTF-16
296and UTF-32 encoding families, as described in RFC4627. 281and UTF-32 encoding families, as described in RFC4627.
297 282
382 367
383=item $enabled = $json->get_relaxed 368=item $enabled = $json->get_relaxed
384 369
385If C<$enable> is true (or missing), then C<decode> will accept some 370If C<$enable> is true (or missing), then C<decode> will accept some
386extensions to normal JSON syntax (see below). C<encode> will not be 371extensions to normal JSON syntax (see below). C<encode> will not be
387affected in anyway. I<Be aware that this option makes you accept invalid 372affected in any way. I<Be aware that this option makes you accept invalid
388JSON texts as if they were valid!>. I suggest only to use this option to 373JSON texts as if they were valid!>. I suggest only to use this option to
389parse application-specific files written by humans (configuration files, 374parse application-specific files written by humans (configuration files,
390resource files etc.) 375resource files etc.)
391 376
392If C<$enable> is false (the default), then C<decode> will only accept 377If C<$enable> is false (the default), then C<decode> will only accept
421 [ 406 [
422 1, # this comment not allowed in JSON 407 1, # this comment not allowed in JSON
423 # neither this one... 408 # neither this one...
424 ] 409 ]
425 410
411=item * literal ASCII TAB characters in strings
412
413Literal ASCII TAB characters are now allowed in strings (and treated as
414C<\t>).
415
416 [
417 "Hello\tWorld",
418 "Hello<TAB>World", # literal <TAB> would not normally be allowed
419 ]
420
426=back 421=back
427 422
428=item $json = $json->canonical ([$enable]) 423=item $json = $json->canonical ([$enable])
429 424
430=item $enabled = $json->get_canonical 425=item $enabled = $json->get_canonical
432If C<$enable> is true (or missing), then the C<encode> method will output JSON objects 427If C<$enable> is true (or missing), then the C<encode> method will output JSON objects
433by sorting their keys. This is adding a comparatively high overhead. 428by sorting their keys. This is adding a comparatively high overhead.
434 429
435If C<$enable> is false, then the C<encode> method will output key-value 430If C<$enable> is false, then the C<encode> method will output key-value
436pairs in the order Perl stores them (which will likely change between runs 431pairs in the order Perl stores them (which will likely change between runs
437of the same script). 432of the same script, and can change even within the same run from 5.18
433onwards).
438 434
439This option is useful if you want the same data structure to be encoded as 435This option is useful if you want the same data structure to be encoded as
440the same JSON text (given the same overall settings). If it is disabled, 436the same JSON text (given the same overall settings). If it is disabled,
441the same hash might be encoded differently even if contains the same data, 437the same hash might be encoded differently even if contains the same data,
442as key-value pairs have no inherent ordering in Perl. 438as key-value pairs have no inherent ordering in Perl.
447 443
448=item $json = $json->allow_nonref ([$enable]) 444=item $json = $json->allow_nonref ([$enable])
449 445
450=item $enabled = $json->get_allow_nonref 446=item $enabled = $json->get_allow_nonref
451 447
448Unlike other boolean options, this opotion is enabled by default beginning
449with version C<4.0>. See L<SECURITY CONSIDERATIONS> for the gory details.
450
452If C<$enable> is true (or missing), then the C<encode> method can convert a 451If C<$enable> is true (or missing), then the C<encode> method can convert a
453non-reference into its corresponding string, number or null JSON value, 452non-reference into its corresponding string, number or null JSON value,
454which is an extension to RFC4627. Likewise, C<decode> will accept those JSON 453which is an extension to RFC4627. Likewise, C<decode> will accept those JSON
455values instead of croaking. 454values instead of croaking.
456 455
457If C<$enable> is false, then the C<encode> method will croak if it isn't 456If C<$enable> is false, then the C<encode> method will croak if it isn't
458passed an arrayref or hashref, as JSON texts must either be an object 457passed an arrayref or hashref, as JSON texts must either be an object
459or array. Likewise, C<decode> will croak if given something that is not a 458or array. Likewise, C<decode> will croak if given something that is not a
460JSON object or array. 459JSON object or array.
461 460
462Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>, 461Example, encode a Perl scalar as JSON value without enabled C<allow_nonref>,
463resulting in an invalid JSON text: 462resulting in an error:
464 463
465 JSON::XS->new->allow_nonref->encode ("Hello, World!") 464 JSON::XS->new->allow_nonref (0)->encode ("Hello, World!")
466 => "Hello, World!" 465 => hash- or arrayref expected...
467 466
468=item $json = $json->allow_unknown ([$enable]) 467=item $json = $json->allow_unknown ([$enable])
469 468
470=item $enabled = $json->get_allow_unknown 469=item $enabled = $json->get_allow_unknown
471 470
483 482
484=item $json = $json->allow_blessed ([$enable]) 483=item $json = $json->allow_blessed ([$enable])
485 484
486=item $enabled = $json->get_allow_blessed 485=item $enabled = $json->get_allow_blessed
487 486
487See L<OBJECT SERIALISATION> for details.
488
488If C<$enable> is true (or missing), then the C<encode> method will not 489If C<$enable> is true (or missing), then the C<encode> method will not
489barf when it encounters a blessed reference. Instead, the value of the 490barf when it encounters a blessed reference that it cannot convert
490B<convert_blessed> option will decide whether C<null> (C<convert_blessed> 491otherwise. Instead, a JSON C<null> value is encoded instead of the object.
491disabled or no C<TO_JSON> method found) or a representation of the
492object (C<convert_blessed> enabled and C<TO_JSON> method found) is being
493encoded. Has no effect on C<decode>.
494 492
495If C<$enable> is false (the default), then C<encode> will throw an 493If C<$enable> is false (the default), then C<encode> will throw an
496exception when it encounters a blessed object. 494exception when it encounters a blessed object that it cannot convert
495otherwise.
496
497This setting has no effect on C<decode>.
497 498
498=item $json = $json->convert_blessed ([$enable]) 499=item $json = $json->convert_blessed ([$enable])
499 500
500=item $enabled = $json->get_convert_blessed 501=item $enabled = $json->get_convert_blessed
502
503See L<OBJECT SERIALISATION> for details.
501 504
502If C<$enable> is true (or missing), then C<encode>, upon encountering a 505If C<$enable> is true (or missing), then C<encode>, upon encountering a
503blessed object, will check for the availability of the C<TO_JSON> method 506blessed object, will check for the availability of the C<TO_JSON> method
504on the object's class. If found, it will be called in scalar context 507on the object's class. If found, it will be called in scalar context and
505and the resulting scalar will be encoded instead of the object. If no 508the resulting scalar will be encoded instead of the object.
506C<TO_JSON> method is found, the value of C<allow_blessed> will decide what
507to do.
508 509
509The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON> 510The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON>
510returns other blessed objects, those will be handled in the same 511returns other blessed objects, those will be handled in the same
511way. C<TO_JSON> must take care of not causing an endless recursion cycle 512way. C<TO_JSON> must take care of not causing an endless recursion cycle
512(== crash) in this case. The name of C<TO_JSON> was chosen because other 513(== crash) in this case. The name of C<TO_JSON> was chosen because other
513methods called by the Perl core (== not by the user of the object) are 514methods called by the Perl core (== not by the user of the object) are
514usually in upper case letters and to avoid collisions with any C<to_json> 515usually in upper case letters and to avoid collisions with any C<to_json>
515function or method. 516function or method.
516 517
517This setting does not yet influence C<decode> in any way, but in the 518If C<$enable> is false (the default), then C<encode> will not consider
518future, global hooks might get installed that influence C<decode> and are 519this type of conversion.
519enabled by this setting.
520 520
521If C<$enable> is false, then the C<allow_blessed> setting will decide what 521This setting has no effect on C<decode>.
522to do when a blessed object is found. 522
523=item $json = $json->allow_tags ([$enable])
524
525=item $enabled = $json->get_allow_tags
526
527See L<OBJECT SERIALISATION> for details.
528
529If C<$enable> is true (or missing), then C<encode>, upon encountering a
530blessed object, will check for the availability of the C<FREEZE> method on
531the object's class. If found, it will be used to serialise the object into
532a nonstandard tagged JSON value (that JSON decoders cannot decode).
533
534It also causes C<decode> to parse such tagged JSON values and deserialise
535them via a call to the C<THAW> method.
536
537If C<$enable> is false (the default), then C<encode> will not consider
538this type of conversion, and tagged JSON values will cause a parse error
539in C<decode>, as if tags were not part of the grammar.
523 540
524=item $json = $json->filter_json_object ([$coderef->($hashref)]) 541=item $json = $json->filter_json_object ([$coderef->($hashref)])
525 542
526When C<$coderef> is specified, it will be called from C<decode> each 543When C<$coderef> is specified, it will be called from C<decode> each
527time it decodes a JSON object. The only argument is a reference to the 544time it decodes a JSON object. The only argument is a reference to
528newly-created hash. If the code references returns a single scalar (which 545the newly-created hash. If the code reference returns a single scalar
529need not be a reference), this value (i.e. a copy of that scalar to avoid 546(which need not be a reference), this value (or rather a copy of it) is
530aliasing) is inserted into the deserialised data structure. If it returns 547inserted into the deserialised data structure. If it returns an empty
531an empty list (NOTE: I<not> C<undef>, which is a valid scalar), the 548list (NOTE: I<not> C<undef>, which is a valid scalar), the original
532original deserialised hash will be inserted. This setting can slow down 549deserialised hash will be inserted. This setting can slow down decoding
533decoding considerably. 550considerably.
534 551
535When C<$coderef> is omitted or undefined, any existing callback will 552When C<$coderef> is omitted or undefined, any existing callback will
536be removed and C<decode> will not change the deserialised hash in any 553be removed and C<decode> will not change the deserialised hash in any
537way. 554way.
538 555
666 683
667See SECURITY CONSIDERATIONS, below, for more info on why this is useful. 684See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
668 685
669=item $json_text = $json->encode ($perl_scalar) 686=item $json_text = $json->encode ($perl_scalar)
670 687
671Converts the given Perl data structure (a simple scalar or a reference 688Converts the given Perl value or data structure to its JSON
672to a hash or array) to its JSON representation. Simple scalars will be 689representation. Croaks on error.
673converted into JSON string or number sequences, while references to arrays
674become JSON arrays and references to hashes become JSON objects. Undefined
675Perl values (e.g. C<undef>) become JSON C<null> values. Neither C<true>
676nor C<false> values will be generated.
677 690
678=item $perl_scalar = $json->decode ($json_text) 691=item $perl_scalar = $json->decode ($json_text)
679 692
680The opposite of C<encode>: expects a JSON text and tries to parse it, 693The opposite of C<encode>: expects a JSON text and tries to parse it,
681returning the resulting simple scalar or reference. Croaks on error. 694returning the resulting simple scalar or reference. Croaks on error.
682
683JSON numbers and strings become simple Perl scalars. JSON arrays become
684Perl arrayrefs and JSON objects become Perl hashrefs. C<true> becomes
685C<1>, C<false> becomes C<0> and C<null> becomes C<undef>.
686 695
687=item ($perl_scalar, $characters) = $json->decode_prefix ($json_text) 696=item ($perl_scalar, $characters) = $json->decode_prefix ($json_text)
688 697
689This works like the C<decode> method, but instead of raising an exception 698This works like the C<decode> method, but instead of raising an exception
690when there is trailing garbage after the first JSON object, it will 699when there is trailing garbage after the first JSON object, it will
691silently stop parsing there and return the number of characters consumed 700silently stop parsing there and return the number of characters consumed
692so far. 701so far.
693 702
694This is useful if your JSON texts are not delimited by an outer protocol 703This is useful if your JSON texts are not delimited by an outer protocol
695(which is not the brightest thing to do in the first place) and you need
696to know where the JSON text ends. 704and you need to know where the JSON text ends.
697 705
698 JSON::XS->new->decode_prefix ("[1] the tail") 706 JSON::XS->new->decode_prefix ("[1] the tail")
699 => ([], 3) 707 => ([1], 3)
700 708
701=back 709=back
702 710
703 711
704=head1 INCREMENTAL PARSING 712=head1 INCREMENTAL PARSING
740 748
741If the method is called in scalar context, then it will try to extract 749If the method is called in scalar context, then it will try to extract
742exactly I<one> JSON object. If that is successful, it will return this 750exactly I<one> JSON object. If that is successful, it will return this
743object, otherwise it will return C<undef>. If there is a parse error, 751object, otherwise it will return C<undef>. If there is a parse error,
744this method will croak just as C<decode> would do (one can then use 752this method will croak just as C<decode> would do (one can then use
745C<incr_skip> to skip the errornous part). This is the most common way of 753C<incr_skip> to skip the erroneous part). This is the most common way of
746using the method. 754using the method.
747 755
748And finally, in list context, it will try to extract as many objects 756And finally, in list context, it will try to extract as many objects
749from the stream as it can find and return them, or the empty list 757from the stream as it can find and return them, or the empty list
750otherwise. For this to work, there must be no separators between the JSON 758otherwise. For this to work, there must be no separators (other than
751objects or arrays, instead they must be concatenated back-to-back. If 759whitespace) between the JSON objects or arrays, instead they must be
752an error occurs, an exception will be raised as in the scalar context 760concatenated back-to-back. If an error occurs, an exception will be
753case. Note that in this case, any previously-parsed JSON texts will be 761raised as in the scalar context case. Note that in this case, any
754lost. 762previously-parsed JSON texts will be lost.
755 763
756Example: Parse some JSON arrays/objects in a given string and return 764Example: Parse some JSON arrays/objects in a given string and return
757them. 765them.
758 766
759 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]"); 767 my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
766all other circumstances you must not call this function (I mean it. 774all other circumstances you must not call this function (I mean it.
767although in simple tests it might actually work, it I<will> fail under 775although in simple tests it might actually work, it I<will> fail under
768real world conditions). As a special exception, you can also call this 776real world conditions). As a special exception, you can also call this
769method before having parsed anything. 777method before having parsed anything.
770 778
779That means you can only use this function to look at or manipulate text
780before or after complete JSON objects, not while the parser is in the
781middle of parsing a JSON object.
782
771This function is useful in two cases: a) finding the trailing text after a 783This function is useful in two cases: a) finding the trailing text after a
772JSON object or b) parsing multiple JSON objects separated by non-JSON text 784JSON object or b) parsing multiple JSON objects separated by non-JSON text
773(such as commas). 785(such as commas).
774 786
775=item $json->incr_skip 787=item $json->incr_skip
779C<incr_parse> died, in which case the input buffer and incremental parser 791C<incr_parse> died, in which case the input buffer and incremental parser
780state is left unchanged, to skip the text parsed so far and to reset the 792state is left unchanged, to skip the text parsed so far and to reset the
781parse state. 793parse state.
782 794
783The difference to C<incr_reset> is that only text until the parse error 795The difference to C<incr_reset> is that only text until the parse error
784occured is removed. 796occurred is removed.
785 797
786=item $json->incr_reset 798=item $json->incr_reset
787 799
788This completely resets the incremental parser, that is, after this call, 800This completely resets the incremental parser, that is, after this call,
789it will be as if the parser had never parsed anything. 801it will be as if the parser had never parsed anything.
795=back 807=back
796 808
797=head2 LIMITATIONS 809=head2 LIMITATIONS
798 810
799All options that affect decoding are supported, except 811All options that affect decoding are supported, except
800C<allow_nonref>. The reason for this is that it cannot be made to 812C<allow_nonref>. The reason for this is that it cannot be made to work
801work sensibly: JSON objects and arrays are self-delimited, i.e. you can concatenate 813sensibly: JSON objects and arrays are self-delimited, i.e. you can
802them back to back and still decode them perfectly. This does not hold true 814concatenate them back to back and still decode them perfectly. This does
803for JSON numbers, however. 815not hold true for JSON numbers, however.
804 816
805For example, is the string C<1> a single JSON number, or is it simply the 817For example, is the string C<1> a single JSON number, or is it simply the
806start of C<12>? Or is C<12> a single JSON number, or the concatenation 818start of C<12>? Or is C<12> a single JSON number, or the concatenation
807of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS 819of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS
808takes the conservative route and disallows this case. 820takes the conservative route and disallows this case.
987If the number consists of digits only, JSON::XS will try to represent 999If the number consists of digits only, JSON::XS will try to represent
988it as an integer value. If that fails, it will try to represent it as 1000it as an integer value. If that fails, it will try to represent it as
989a numeric (floating point) value if that is possible without loss of 1001a numeric (floating point) value if that is possible without loss of
990precision. Otherwise it will preserve the number as a string value (in 1002precision. Otherwise it will preserve the number as a string value (in
991which case you lose roundtripping ability, as the JSON number will be 1003which case you lose roundtripping ability, as the JSON number will be
992re-encoded toa JSON string). 1004re-encoded to a JSON string).
993 1005
994Numbers containing a fractional or exponential part will always be 1006Numbers containing a fractional or exponential part will always be
995represented as numeric (floating point) values, possibly at a loss of 1007represented as numeric (floating point) values, possibly at a loss of
996precision (in which case you might lose perfect roundtripping ability, but 1008precision (in which case you might lose perfect roundtripping ability, but
997the JSON number will still be re-encoded as a JSON number). 1009the JSON number will still be re-encoded as a JSON number).
998 1010
999Note that precision is not accuracy - binary floating point values cannot 1011Note that precision is not accuracy - binary floating point values cannot
1000represent most decimal fractions exactly, and when converting from and to 1012represent most decimal fractions exactly, and when converting from and to
1001floating point, JSON::XS only guarantees precision up to but not including 1013floating point, JSON::XS only guarantees precision up to but not including
1002the leats significant bit. 1014the least significant bit.
1003 1015
1004=item true, false 1016=item true, false
1005 1017
1006These JSON atoms become C<JSON::XS::true> and C<JSON::XS::false>, 1018These JSON atoms become C<Types::Serialiser::true> and
1007respectively. They are overloaded to act almost exactly like the numbers 1019C<Types::Serialiser::false>, respectively. They are overloaded to act
1008C<1> and C<0>. You can check whether a scalar is a JSON boolean by using 1020almost exactly like the numbers C<1> and C<0>. You can check whether
1009the C<JSON::XS::is_bool> function. 1021a scalar is a JSON boolean by using the C<Types::Serialiser::is_bool>
1022function (after C<use Types::Serialier>, of course).
1010 1023
1011=item null 1024=item null
1012 1025
1013A JSON null atom becomes C<undef> in Perl. 1026A JSON null atom becomes C<undef> in Perl.
1027
1028=item shell-style comments (C<< # I<text> >>)
1029
1030As a nonstandard extension to the JSON syntax that is enabled by the
1031C<relaxed> setting, shell-style comments are allowed. They can start
1032anywhere outside strings and go till the end of the line.
1033
1034=item tagged values (C<< (I<tag>)I<value> >>).
1035
1036Another nonstandard extension to the JSON syntax, enabled with the
1037C<allow_tags> setting, are tagged values. In this implementation, the
1038I<tag> must be a perl package/class name encoded as a JSON string, and the
1039I<value> must be a JSON array encoding optional constructor arguments.
1040
1041See L<OBJECT SERIALISATION>, below, for details.
1014 1042
1015=back 1043=back
1016 1044
1017 1045
1018=head2 PERL -> JSON 1046=head2 PERL -> JSON
1023 1051
1024=over 4 1052=over 4
1025 1053
1026=item hash references 1054=item hash references
1027 1055
1028Perl hash references become JSON objects. As there is no inherent ordering 1056Perl hash references become JSON objects. As there is no inherent
1029in hash keys (or JSON objects), they will usually be encoded in a 1057ordering in hash keys (or JSON objects), they will usually be encoded
1030pseudo-random order that can change between runs of the same program but 1058in a pseudo-random order. JSON::XS can optionally sort the hash keys
1031stays generally the same within a single run of a program. JSON::XS can 1059(determined by the I<canonical> flag), so the same datastructure will
1032optionally sort the hash keys (determined by the I<canonical> flag), so 1060serialise to the same JSON text (given same settings and version of
1033the same datastructure will serialise to the same JSON text (given same 1061JSON::XS), but this incurs a runtime overhead and is only rarely useful,
1034settings and version of JSON::XS), but this incurs a runtime overhead 1062e.g. when you want to compare some JSON text against another for equality.
1035and is only rarely useful, e.g. when you want to compare some JSON text
1036against another for equality.
1037 1063
1038=item array references 1064=item array references
1039 1065
1040Perl array references become JSON arrays. 1066Perl array references become JSON arrays.
1041 1067
1042=item other references 1068=item other references
1043 1069
1044Other unblessed references are generally not allowed and will cause an 1070Other unblessed references are generally not allowed and will cause an
1045exception to be thrown, except for references to the integers C<0> and 1071exception to be thrown, except for references to the integers C<0> and
1046C<1>, which get turned into C<false> and C<true> atoms in JSON. You can 1072C<1>, which get turned into C<false> and C<true> atoms in JSON.
1047also use C<JSON::XS::false> and C<JSON::XS::true> to improve readability.
1048 1073
1074Since C<JSON::XS> uses the boolean model from L<Types::Serialiser>, you
1075can also C<use Types::Serialiser> and then use C<Types::Serialiser::false>
1076and C<Types::Serialiser::true> to improve readability.
1077
1078 use Types::Serialiser;
1049 encode_json [\0, JSON::XS::true] # yields [false,true] 1079 encode_json [\0, Types::Serialiser::true] # yields [false,true]
1050 1080
1051=item JSON::XS::true, JSON::XS::false 1081=item Types::Serialiser::true, Types::Serialiser::false
1052 1082
1053These special values become JSON true and JSON false values, 1083These special values from the L<Types::Serialiser> module become JSON true
1054respectively. You can also use C<\1> and C<\0> directly if you want. 1084and JSON false values, respectively. You can also use C<\1> and C<\0>
1085directly if you want.
1055 1086
1056=item blessed objects 1087=item blessed objects
1057 1088
1058Blessed objects are not directly representable in JSON. See the 1089Blessed objects are not directly representable in JSON, but C<JSON::XS>
1059C<allow_blessed> and C<convert_blessed> methods on various options on 1090allows various ways of handling objects. See L<OBJECT SERIALISATION>,
1060how to deal with this: basically, you can choose between throwing an 1091below, for details.
1061exception, encoding the reference as if it weren't blessed, or provide
1062your own serialiser method.
1063 1092
1064=item simple scalars 1093=item simple scalars
1065 1094
1066Simple Perl scalars (any scalar that is not a reference) are the most 1095Simple Perl scalars (any scalar that is not a reference) are the most
1067difficult objects to encode: JSON::XS will encode undefined scalars as 1096difficult objects to encode: JSON::XS will encode undefined scalars as
1104infinities or NaN's - these cannot be represented in JSON, and it is an 1133infinities or NaN's - these cannot be represented in JSON, and it is an
1105error to pass those in. 1134error to pass those in.
1106 1135
1107=back 1136=back
1108 1137
1138=head2 OBJECT SERIALISATION
1139
1140As JSON cannot directly represent Perl objects, you have to choose between
1141a pure JSON representation (without the ability to deserialise the object
1142automatically again), and a nonstandard extension to the JSON syntax,
1143tagged values.
1144
1145=head3 SERIALISATION
1146
1147What happens when C<JSON::XS> encounters a Perl object depends on the
1148C<allow_blessed>, C<convert_blessed> and C<allow_tags> settings, which are
1149used in this order:
1150
1151=over 4
1152
1153=item 1. C<allow_tags> is enabled and the object has a C<FREEZE> method.
1154
1155In this case, C<JSON::XS> uses the L<Types::Serialiser> object
1156serialisation protocol to create a tagged JSON value, using a nonstandard
1157extension to the JSON syntax.
1158
1159This works by invoking the C<FREEZE> method on the object, with the first
1160argument being the object to serialise, and the second argument being the
1161constant string C<JSON> to distinguish it from other serialisers.
1162
1163The C<FREEZE> method can return any number of values (i.e. zero or
1164more). These values and the paclkage/classname of the object will then be
1165encoded as a tagged JSON value in the following format:
1166
1167 ("classname")[FREEZE return values...]
1168
1169e.g.:
1170
1171 ("URI")["http://www.google.com/"]
1172 ("MyDate")[2013,10,29]
1173 ("ImageData::JPEG")["Z3...VlCg=="]
1174
1175For example, the hypothetical C<My::Object> C<FREEZE> method might use the
1176objects C<type> and C<id> members to encode the object:
1177
1178 sub My::Object::FREEZE {
1179 my ($self, $serialiser) = @_;
1180
1181 ($self->{type}, $self->{id})
1182 }
1183
1184=item 2. C<convert_blessed> is enabled and the object has a C<TO_JSON> method.
1185
1186In this case, the C<TO_JSON> method of the object is invoked in scalar
1187context. It must return a single scalar that can be directly encoded into
1188JSON. This scalar replaces the object in the JSON text.
1189
1190For example, the following C<TO_JSON> method will convert all L<URI>
1191objects to JSON strings when serialised. The fatc that these values
1192originally were L<URI> objects is lost.
1193
1194 sub URI::TO_JSON {
1195 my ($uri) = @_;
1196 $uri->as_string
1197 }
1198
1199=item 3. C<allow_blessed> is enabled.
1200
1201The object will be serialised as a JSON null value.
1202
1203=item 4. none of the above
1204
1205If none of the settings are enabled or the respective methods are missing,
1206C<JSON::XS> throws an exception.
1207
1208=back
1209
1210=head3 DESERIALISATION
1211
1212For deserialisation there are only two cases to consider: either
1213nonstandard tagging was used, in which case C<allow_tags> decides,
1214or objects cannot be automatically be deserialised, in which
1215case you can use postprocessing or the C<filter_json_object> or
1216C<filter_json_single_key_object> callbacks to get some real objects our of
1217your JSON.
1218
1219This section only considers the tagged value case: I a tagged JSON object
1220is encountered during decoding and C<allow_tags> is disabled, a parse
1221error will result (as if tagged values were not part of the grammar).
1222
1223If C<allow_tags> is enabled, C<JSON::XS> will look up the C<THAW> method
1224of the package/classname used during serialisation (it will not attempt
1225to load the package as a Perl module). If there is no such method, the
1226decoding will fail with an error.
1227
1228Otherwise, the C<THAW> method is invoked with the classname as first
1229argument, the constant string C<JSON> as second argument, and all the
1230values from the JSON array (the values originally returned by the
1231C<FREEZE> method) as remaining arguments.
1232
1233The method must then return the object. While technically you can return
1234any Perl scalar, you might have to enable the C<enable_nonref> setting to
1235make that work in all cases, so better return an actual blessed reference.
1236
1237As an example, let's implement a C<THAW> function that regenerates the
1238C<My::Object> from the C<FREEZE> example earlier:
1239
1240 sub My::Object::THAW {
1241 my ($class, $serialiser, $type, $id) = @_;
1242
1243 $class->new (type => $type, id => $id)
1244 }
1245
1109 1246
1110=head1 ENCODING/CODESET FLAG NOTES 1247=head1 ENCODING/CODESET FLAG NOTES
1111 1248
1112The interested reader might have seen a number of flags that signify 1249The interested reader might have seen a number of flags that signify
1113encodings or codesets - C<utf8>, C<latin1> and C<ascii>. There seems to be 1250encodings or codesets - C<utf8>, C<latin1> and C<ascii>. There seems to be
1137=item C<utf8> flag disabled 1274=item C<utf8> flag disabled
1138 1275
1139When C<utf8> is disabled (the default), then C<encode>/C<decode> generate 1276When C<utf8> is disabled (the default), then C<encode>/C<decode> generate
1140and expect Unicode strings, that is, characters with high ordinal Unicode 1277and expect Unicode strings, that is, characters with high ordinal Unicode
1141values (> 255) will be encoded as such characters, and likewise such 1278values (> 255) will be encoded as such characters, and likewise such
1142characters are decoded as-is, no canges to them will be done, except 1279characters are decoded as-is, no changes to them will be done, except
1143"(re-)interpreting" them as Unicode codepoints or Unicode characters, 1280"(re-)interpreting" them as Unicode codepoints or Unicode characters,
1144respectively (to Perl, these are the same thing in strings unless you do 1281respectively (to Perl, these are the same thing in strings unless you do
1145funny/weird/dumb stuff). 1282funny/weird/dumb stuff).
1146 1283
1147This is useful when you want to do the encoding yourself (e.g. when you 1284This is useful when you want to do the encoding yourself (e.g. when you
1157expect your input strings to be encoded as UTF-8, that is, no "character" 1294expect your input strings to be encoded as UTF-8, that is, no "character"
1158of the input string must have any value > 255, as UTF-8 does not allow 1295of the input string must have any value > 255, as UTF-8 does not allow
1159that. 1296that.
1160 1297
1161The C<utf8> flag therefore switches between two modes: disabled means you 1298The C<utf8> flag therefore switches between two modes: disabled means you
1162will get a Unicode string in Perl, enabled means you get an UTF-8 encoded 1299will get a Unicode string in Perl, enabled means you get a UTF-8 encoded
1163octet/binary string in Perl. 1300octet/binary string in Perl.
1164 1301
1165=item C<latin1> or C<ascii> flags enabled 1302=item C<latin1> or C<ascii> flags enabled
1166 1303
1167With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters 1304With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters
1263output for these property strings, e.g.: 1400output for these property strings, e.g.:
1264 1401
1265 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g; 1402 $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1266 1403
1267This works because C<__proto__> is not valid outside of strings, so every 1404This works because C<__proto__> is not valid outside of strings, so every
1268occurence of C<"__proto__"\s*:> must be a string used as property name. 1405occurrence of C<"__proto__"\s*:> must be a string used as property name.
1269 1406
1270If you know of other incompatibilities, please let me know. 1407If you know of other incompatibilities, please let me know.
1271 1408
1272 1409
1273=head2 JSON and YAML 1410=head2 JSON and YAML
1435are browser design bugs, but it is still you who will have to deal with 1572are browser design bugs, but it is still you who will have to deal with
1436it, as major browser developers care only for features, not about getting 1573it, as major browser developers care only for features, not about getting
1437security right). 1574security right).
1438 1575
1439 1576
1577=head2 "OLD" VS. "NEW" JSON (RFC 4627 VS. RFC 7159)
1578
1579JSON originally required JSON texts to represent an array or object -
1580scalar values were explicitly not allowed. This has changed, and versions
1581of JSON::XS beginning with C<4.0> reflect this by allowing scalar values
1582by default.
1583
1584One reason why one might not want this is that this removes a fundamental
1585property of JSON texts, namely that they are self-delimited and
1586self-contained, or in other words, you could take any number of "old"
1587JSON texts and paste them together, and the result would be unambiguously
1588parseable:
1589
1590 [1,3]{"k":5}[][null] # four JSON texts, without doubt
1591
1592By allowing scalars, this property is lost: in the following example, is
1593this one JSON text (the number 12) or two JSON texts (the numbers 1 and
15942):
1595
1596 12 # could be 12, or 1 and 2
1597
1598Another lost property of "old" JSON is that no lookahead is required to
1599know the end of a JSON text, i.e. the JSON text definitely ended at the
1600last C<]> or C<}> character, there was no need to read extra characters.
1601
1602For example, a viable network protocol with "old" JSON was to simply
1603exchange JSON texts without delimiter. For "new" JSON, you have to use a
1604suitable delimiter (such as a newline) after every JSON text or ensure you
1605never encode/decode scalar values.
1606
1607Most protocols do work by only transferring arrays or objects, and the
1608easiest way to avoid problems with the "new" JSON definition is to
1609explicitly disallow scalar values in your encoder and decoder:
1610
1611 $json_coder = JSON::XS->new->allow_nonref (0)
1612
1613This is a somewhat unhappy situation, and the blame can fully be put on
1614JSON's inmventor, Douglas Crockford, who unilaterally changed the format
1615in 2006 without consulting the IETF, forcing the IETF to either fork the
1616format or go with it (as I was told, the IETF wasn't amused).
1617
1618
1619=head1 INTEROPERABILITY WITH OTHER MODULES
1620
1621C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean
1622constants. That means that the JSON true and false values will be
1623comaptible to true and false values of other modules that do the same,
1624such as L<JSON::PP> and L<CBOR::XS>.
1625
1626
1627=head1 INTEROPERABILITY WITH OTHER JSON DECODERS
1628
1629As long as you only serialise data that can be directly expressed in JSON,
1630C<JSON::XS> is incapable of generating invalid JSON output (modulo bugs,
1631but C<JSON::XS> has found more bugs in the official JSON testsuite (1)
1632than the official JSON testsuite has found in C<JSON::XS> (0)).
1633
1634When you have trouble decoding JSON generated by this module using other
1635decoders, then it is very likely that you have an encoding mismatch or the
1636other decoder is broken.
1637
1638When decoding, C<JSON::XS> is strict by default and will likely catch all
1639errors. There are currently two settings that change this: C<relaxed>
1640makes C<JSON::XS> accept (but not generate) some non-standard extensions,
1641and C<allow_tags> will allow you to encode and decode Perl objects, at the
1642cost of not outputting valid JSON anymore.
1643
1644=head2 TAGGED VALUE SYNTAX AND STANDARD JSON EN/DECODERS
1645
1646When you use C<allow_tags> to use the extended (and also nonstandard and
1647invalid) JSON syntax for serialised objects, and you still want to decode
1648the generated When you want to serialise objects, you can run a regex
1649to replace the tagged syntax by standard JSON arrays (it only works for
1650"normal" package names without comma, newlines or single colons). First,
1651the readable Perl version:
1652
1653 # if your FREEZE methods return no values, you need this replace first:
1654 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[\s*\]/[$1]/gx;
1655
1656 # this works for non-empty constructor arg lists:
1657 $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[/[$1,/gx;
1658
1659And here is a less readable version that is easy to adapt to other
1660languages:
1661
1662 $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/[$1,/g;
1663
1664Here is an ECMAScript version (same regex):
1665
1666 json = json.replace (/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/g, "[$1,");
1667
1668Since this syntax converts to standard JSON arrays, it might be hard to
1669distinguish serialised objects from normal arrays. You can prepend a
1670"magic number" as first array element to reduce chances of a collision:
1671
1672 $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/["XU1peReLzT4ggEllLanBYq4G9VzliwKF",$1,/g;
1673
1674And after decoding the JSON text, you could walk the data
1675structure looking for arrays with a first element of
1676C<XU1peReLzT4ggEllLanBYq4G9VzliwKF>.
1677
1678The same approach can be used to create the tagged format with another
1679encoder. First, you create an array with the magic string as first member,
1680the classname as second, and constructor arguments last, encode it as part
1681of your JSON structure, and then:
1682
1683 $json =~ s/\[\s*"XU1peReLzT4ggEllLanBYq4G9VzliwKF"\s*,\s*("([^\\":,]+|\\.|::)*")\s*,/($1)[/g;
1684
1685Again, this has some limitations - the magic string must not be encoded
1686with character escapes, and the constructor arguments must be non-empty.
1687
1688
1689=head1 RFC7159
1690
1691Since this module was written, Google has written a new JSON RFC, RFC 7159
1692(and RFC7158). Unfortunately, this RFC breaks compatibility with both the
1693original JSON specification on www.json.org and RFC4627.
1694
1695As far as I can see, you can get partial compatibility when parsing by
1696using C<< ->allow_nonref >>. However, consider the security implications
1697of doing so.
1698
1699I haven't decided yet when to break compatibility with RFC4627 by default
1700(and potentially leave applications insecure) and change the default to
1701follow RFC7159, but application authors are well advised to call C<<
1702->allow_nonref(0) >> even if this is the current default, if they cannot
1703handle non-reference values, in preparation for the day when the default
1704will change.
1705
1706
1440=head1 THREADS 1707=head1 (I-)THREADS
1441 1708
1442This module is I<not> guaranteed to be thread safe and there are no 1709This module is I<not> guaranteed to be ithread (or MULTIPLICITY-) safe
1443plans to change this until Perl gets thread support (as opposed to the 1710and there are no plans to change this. Note that perl's builtin so-called
1444horribly slow so-called "threads" which are simply slow and bloated 1711threads/ithreads are officially deprecated and should not be used.
1445process simulations - use fork, it's I<much> faster, cheaper, better).
1446 1712
1447(It might actually work, but you have been warned). 1713
1714=head1 THE PERILS OF SETLOCALE
1715
1716Sometimes people avoid the Perl locale support and directly call the
1717system's setlocale function with C<LC_ALL>.
1718
1719This breaks both perl and modules such as JSON::XS, as stringification of
1720numbers no longer works correctly (e.g. C<$x = 0.1; print "$x"+1> might
1721print C<1>, and JSON::XS might output illegal JSON as JSON::XS relies on
1722perl to stringify numbers).
1723
1724The solution is simple: don't call C<setlocale>, or use it for only those
1725categories you need, such as C<LC_MESSAGES> or C<LC_CTYPE>.
1726
1727If you need C<LC_NUMERIC>, you should enable it only around the code that
1728actually needs it (avoiding stringification of numbers), and restore it
1729afterwards.
1448 1730
1449 1731
1450=head1 BUGS 1732=head1 BUGS
1451 1733
1452While the goal of this module is to be correct, that unfortunately does 1734While the goal of this module is to be correct, that unfortunately does
1456Please refrain from using rt.cpan.org or any other bug reporting 1738Please refrain from using rt.cpan.org or any other bug reporting
1457service. I put the contact address into my modules for a reason. 1739service. I put the contact address into my modules for a reason.
1458 1740
1459=cut 1741=cut
1460 1742
1461our $true = do { bless \(my $dummy = 1), "JSON::XS::Boolean" }; 1743BEGIN {
1462our $false = do { bless \(my $dummy = 0), "JSON::XS::Boolean" }; 1744 *true = \$Types::Serialiser::true;
1745 *true = \&Types::Serialiser::true;
1746 *false = \$Types::Serialiser::false;
1747 *false = \&Types::Serialiser::false;
1748 *is_bool = \&Types::Serialiser::is_bool;
1463 1749
1464sub true() { $true } 1750 *JSON::XS::Boolean:: = *Types::Serialiser::Boolean::;
1465sub false() { $false }
1466
1467sub is_bool($) {
1468 UNIVERSAL::isa $_[0], "JSON::XS::Boolean"
1469# or UNIVERSAL::isa $_[0], "JSON::Literal"
1470} 1751}
1471 1752
1472XSLoader::load "JSON::XS", $VERSION; 1753XSLoader::load "JSON::XS", $VERSION;
1473
1474package JSON::XS::Boolean;
1475
1476use overload
1477 "0+" => sub { ${$_[0]} },
1478 "++" => sub { $_[0] = ${$_[0]} + 1 },
1479 "--" => sub { $_[0] = ${$_[0]} - 1 },
1480 fallback => 1;
1481
14821;
1483 1754
1484=head1 SEE ALSO 1755=head1 SEE ALSO
1485 1756
1486The F<json_xs> command line utility for quick experiments. 1757The F<json_xs> command line utility for quick experiments.
1487 1758
1490 Marc Lehmann <schmorp@schmorp.de> 1761 Marc Lehmann <schmorp@schmorp.de>
1491 http://home.schmorp.de/ 1762 http://home.schmorp.de/
1492 1763
1493=cut 1764=cut
1494 1765
17661
1767

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines