[ViewVC] Diff of: cvs/JSON-XS/XS.pm

Comparing JSON-XS/XS.pm (file contents):
Revision 1.145 by root, Tue Oct 29 00:06:40 2013 UTC vs.
Revision 1.159 by root, Wed Sep 7 17:14:56 2016 UTC

…		…
101		101
102	package JSON::XS;	102	package JSON::XS;
103		103
104	use common::sense;	104	use common::sense;
105		105
106	our $VERSION = '3.0';	106	our $VERSION = 3.02;
107	our @ISA = qw(Exporter);	107	our @ISA = qw(Exporter);
108		108
109	our @EXPORT = qw(encode_json decode_json);	109	our @EXPORT = qw(encode_json decode_json);
110		110
111	use Exporter;	111	use Exporter;
…		…
404	[	404	[
405	1, # this comment not allowed in JSON	405	1, # this comment not allowed in JSON
406	# neither this one...	406	# neither this one...
407	]	407	]
408		408
		409	=item * literal ASCII TAB characters in strings
		410
		411	Literal ASCII TAB characters are now allowed in strings (and treated as
		412	C<\t>).
		413
		414	[
		415	"Hello\tWorld",
		416	"Hello<TAB>World", # literal <TAB> would not normally be allowed
		417	]
		418
409	=back	419	=back
410		420
411	=item $json = $json->canonical ([$enable])	421	=item $json = $json->canonical ([$enable])
412		422
413	=item $enabled = $json->get_canonical	423	=item $enabled = $json->get_canonical
…		…
467		477
468	=item $json = $json->allow_blessed ([$enable])	478	=item $json = $json->allow_blessed ([$enable])
469		479
470	=item $enabled = $json->get_allow_blessed	480	=item $enabled = $json->get_allow_blessed
471		481
		482	See L<OBJECT SERIALISATION> for details.
		483
472	If C<$enable> is true (or missing), then the C<encode> method will not	484	If C<$enable> is true (or missing), then the C<encode> method will not
473	barf when it encounters a blessed reference. Instead, the value of the	485	barf when it encounters a blessed reference that it cannot convert
474	B<convert_blessed> option will decide whether C<null> (C<convert_blessed>	486	otherwise. Instead, a JSON C<null> value is encoded instead of the object.
475	disabled or no C<TO_JSON> method found) or a representation of the
476	object (C<convert_blessed> enabled and C<TO_JSON> method found) is being
477	encoded. Has no effect on C<decode>.
478		487
479	If C<$enable> is false (the default), then C<encode> will throw an	488	If C<$enable> is false (the default), then C<encode> will throw an
480	exception when it encounters a blessed object.	489	exception when it encounters a blessed object that it cannot convert
		490	otherwise.
		491
		492	This setting has no effect on C<decode>.
481		493
482	=item $json = $json->convert_blessed ([$enable])	494	=item $json = $json->convert_blessed ([$enable])
483		495
484	=item $enabled = $json->get_convert_blessed	496	=item $enabled = $json->get_convert_blessed
		497
		498	See L<OBJECT SERIALISATION> for details.
485		499
486	If C<$enable> is true (or missing), then C<encode>, upon encountering a	500	If C<$enable> is true (or missing), then C<encode>, upon encountering a
487	blessed object, will check for the availability of the C<TO_JSON> method	501	blessed object, will check for the availability of the C<TO_JSON> method
488	on the object's class. If found, it will be called in scalar context	502	on the object's class. If found, it will be called in scalar context and
489	and the resulting scalar will be encoded instead of the object. If no	503	the resulting scalar will be encoded instead of the object.
490	C<TO_JSON> method is found, the value of C<allow_blessed> will decide what
491	to do.
492		504
493	The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON>	505	The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON>
494	returns other blessed objects, those will be handled in the same	506	returns other blessed objects, those will be handled in the same
495	way. C<TO_JSON> must take care of not causing an endless recursion cycle	507	way. C<TO_JSON> must take care of not causing an endless recursion cycle
496	(== crash) in this case. The name of C<TO_JSON> was chosen because other	508	(== crash) in this case. The name of C<TO_JSON> was chosen because other
497	methods called by the Perl core (== not by the user of the object) are	509	methods called by the Perl core (== not by the user of the object) are
498	usually in upper case letters and to avoid collisions with any C<to_json>	510	usually in upper case letters and to avoid collisions with any C<to_json>
499	function or method.	511	function or method.
500		512
501	This setting does not yet influence C<decode> in any way, but in the	513	If C<$enable> is false (the default), then C<encode> will not consider
502	future, global hooks might get installed that influence C<decode> and are	514	this type of conversion.
503	enabled by this setting.
504		515
505	If C<$enable> is false, then the C<allow_blessed> setting will decide what	516	This setting has no effect on C<decode>.
506	to do when a blessed object is found.	517
		518	=item $json = $json->allow_tags ([$enable])
		519
		520	=item $enabled = $json->allow_tags
		521
		522	See L<OBJECT SERIALISATION> for details.
		523
		524	If C<$enable> is true (or missing), then C<encode>, upon encountering a
		525	blessed object, will check for the availability of the C<FREEZE> method on
		526	the object's class. If found, it will be used to serialise the object into
		527	a nonstandard tagged JSON value (that JSON decoders cannot decode).
		528
		529	It also causes C<decode> to parse such tagged JSON values and deserialise
		530	them via a call to the C<THAW> method.
		531
		532	If C<$enable> is false (the default), then C<encode> will not consider
		533	this type of conversion, and tagged JSON values will cause a parse error
		534	in C<decode>, as if tags were not part of the grammar.
507		535
508	=item $json = $json->filter_json_object ([$coderef->($hashref)])	536	=item $json = $json->filter_json_object ([$coderef->($hashref)])
509		537
510	When C<$coderef> is specified, it will be called from C<decode> each	538	When C<$coderef> is specified, it will be called from C<decode> each
511	time it decodes a JSON object. The only argument is a reference to the	539	time it decodes a JSON object. The only argument is a reference to the
…		…
669		697
670	This is useful if your JSON texts are not delimited by an outer protocol	698	This is useful if your JSON texts are not delimited by an outer protocol
671	and you need to know where the JSON text ends.	699	and you need to know where the JSON text ends.
672		700
673	JSON::XS->new->decode_prefix ("[1] the tail")	701	JSON::XS->new->decode_prefix ("[1] the tail")
674	=> ([], 3)	702	=> ([1], 3)
675		703
676	=back	704	=back
677		705
678		706
679	=head1 INCREMENTAL PARSING	707	=head1 INCREMENTAL PARSING
…		…
720	C<incr_skip> to skip the erroneous part). This is the most common way of	748	C<incr_skip> to skip the erroneous part). This is the most common way of
721	using the method.	749	using the method.
722		750
723	And finally, in list context, it will try to extract as many objects	751	And finally, in list context, it will try to extract as many objects
724	from the stream as it can find and return them, or the empty list	752	from the stream as it can find and return them, or the empty list
725	otherwise. For this to work, there must be no separators between the JSON	753	otherwise. For this to work, there must be no separators (other than
726	objects or arrays, instead they must be concatenated back-to-back. If	754	whitespace) between the JSON objects or arrays, instead they must be
727	an error occurs, an exception will be raised as in the scalar context	755	concatenated back-to-back. If an error occurs, an exception will be
728	case. Note that in this case, any previously-parsed JSON texts will be	756	raised as in the scalar context case. Note that in this case, any
729	lost.	757	previously-parsed JSON texts will be lost.
730		758
731	Example: Parse some JSON arrays/objects in a given string and return	759	Example: Parse some JSON arrays/objects in a given string and return
732	them.	760	them.
733		761
734	my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");	762	my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
…		…
999	Another nonstandard extension to the JSON syntax, enabled with the	1027	Another nonstandard extension to the JSON syntax, enabled with the
1000	C<allow_tags> setting, are tagged values. In this implementation, the	1028	C<allow_tags> setting, are tagged values. In this implementation, the
1001	I<tag> must be a perl package/class name encoded as a JSON string, and the	1029	I<tag> must be a perl package/class name encoded as a JSON string, and the
1002	I<value> must be a JSON array encoding optional constructor arguments.	1030	I<value> must be a JSON array encoding optional constructor arguments.
1003		1031
1004	See "OBJECT SERIALISATION", below, for details.	1032	See L<OBJECT SERIALISATION>, below, for details.
1005		1033
1006	=back	1034	=back
1007		1035
1008		1036
1009	=head2 PERL -> JSON	1037	=head2 PERL -> JSON
…		…
1048	directly if you want.	1076	directly if you want.
1049		1077
1050	=item blessed objects	1078	=item blessed objects
1051		1079
1052	Blessed objects are not directly representable in JSON, but C<JSON::XS>	1080	Blessed objects are not directly representable in JSON, but C<JSON::XS>
1053	allows various ways of handling objects. See "OBJECT SERIALISATION",	1081	allows various ways of handling objects. See L<OBJECT SERIALISATION>,
1054	below, for details.	1082	below, for details.
1055		1083
1056	=item simple scalars	1084	=item simple scalars
1057		1085
1058	Simple Perl scalars (any scalar that is not a reference) are the most	1086	Simple Perl scalars (any scalar that is not a reference) are the most
…		…
1111	C<allow_blessed>, C<convert_blessed> and C<allow_tags> settings, which are	1139	C<allow_blessed>, C<convert_blessed> and C<allow_tags> settings, which are
1112	used in this order:	1140	used in this order:
1113		1141
1114	=over 4	1142	=over 4
1115		1143
1116	=item 1. C<allow_tags> is enabled and object has a C<FREEZE> method.	1144	=item 1. C<allow_tags> is enabled and the object has a C<FREEZE> method.
1117		1145
1118	In this case, C<JSON::XS> uses the L<Types::Serialiser> object	1146	In this case, C<JSON::XS> uses the L<Types::Serialiser> object
1119	serialisation protocol to create a tagged JSON value, using a nonstandard	1147	serialisation protocol to create a tagged JSON value, using a nonstandard
1120	extension to the JSON syntax.	1148	extension to the JSON syntax.
1121		1149
…		…
1127	more). These values and the paclkage/classname of the object will then be	1155	more). These values and the paclkage/classname of the object will then be
1128	encoded as a tagged JSON value in the following format:	1156	encoded as a tagged JSON value in the following format:
1129		1157
1130	("classname")[FREEZE return values...]	1158	("classname")[FREEZE return values...]
1131		1159
		1160	e.g.:
		1161
		1162	("URI")["http://www.google.com/"]
		1163	("MyDate")[2013,10,29]
		1164	("ImageData::JPEG")["Z3...VlCg=="]
		1165
1132	For example, the hypothetical C<My::Object> C<FREEZE> method might use the	1166	For example, the hypothetical C<My::Object> C<FREEZE> method might use the
1133	objects C<type> and C<id> members to encode the object:	1167	objects C<type> and C<id> members to encode the object:
1134		1168
1135	sub My::Object::FREEZE {	1169	sub My::Object::FREEZE {
1136	my ($self, $serialiser) = @_;	1170	my ($self, $serialiser) = @_;
1137		1171
1138	($self->{type}, $self->{id})	1172	($self->{type}, $self->{id})
1139	}	1173	}
1140		1174
1141	=item 2. C<convert_blessed> is enabled and object has a C<TO_JSON> method.	1175	=item 2. C<convert_blessed> is enabled and the object has a C<TO_JSON> method.
1142		1176
1143	In this case, the C<TO_JSON> method of the object is invoked in scalar	1177	In this case, the C<TO_JSON> method of the object is invoked in scalar
1144	context. It must return a single scalar that can be directly encoded into	1178	context. It must return a single scalar that can be directly encoded into
1145	JSON. This scalar replaces the object in the JSON text.	1179	JSON. This scalar replaces the object in the JSON text.
1146		1180
…		…
1176	This section only considers the tagged value case: I a tagged JSON object	1210	This section only considers the tagged value case: I a tagged JSON object
1177	is encountered during decoding and C<allow_tags> is disabled, a parse	1211	is encountered during decoding and C<allow_tags> is disabled, a parse
1178	error will result (as if tagged values were not part of the grammar).	1212	error will result (as if tagged values were not part of the grammar).
1179		1213
1180	If C<allow_tags> is enabled, C<JSON::XS> will look up the C<THAW> method	1214	If C<allow_tags> is enabled, C<JSON::XS> will look up the C<THAW> method
1181	of the package/classname used during serialisation. If there is no such	1215	of the package/classname used during serialisation (it will not attempt
		1216	to load the package as a Perl module). If there is no such method, the
1182	method, the decoding will fail with an error.	1217	decoding will fail with an error.
1183		1218
1184	Otherwise, the C<THAW> method is invoked with the classname as first	1219	Otherwise, the C<THAW> method is invoked with the classname as first
1185	argument, the constant string C<JSON> as second argument, and all the	1220	argument, the constant string C<JSON> as second argument, and all the
1186	values from the JSON array (the values originally returned by the	1221	values from the JSON array (the values originally returned by the
1187	C<FREEZE> method) as remaining arguments.	1222	C<FREEZE> method) as remaining arguments.
…		…
1528	are browser design bugs, but it is still you who will have to deal with	1563	are browser design bugs, but it is still you who will have to deal with
1529	it, as major browser developers care only for features, not about getting	1564	it, as major browser developers care only for features, not about getting
1530	security right).	1565	security right).
1531		1566
1532		1567
		1568	=head1 "OLD" VS. "NEW" JSON (RFC 4627 VS. RFC 7159)
		1569
		1570	TL;DR: Due to security concerns, JSON::XS will not allow scalar data in
		1571	JSON texts by default - you need to create your own JSON::XS object and
		1572	enable C<allow_nonref>:
		1573
		1574
		1575	my $json = JSON::XS->new->allow_nonref;
		1576
		1577	$text = $json->encode ($data);
		1578	$data = $json->decode ($text);
		1579
		1580	The long version: JSON being an important and supposedly stable format,
		1581	the IETF standardised it as RFC 4627 in 2006. Unfortunately, the inventor
		1582	of JSON, Dougles Crockford, unilaterally changed the definition of JSON in
		1583	javascript. Rather than create a fork, the IETF decided to standardise the
		1584	new syntax (apparently, so Iw as told, without finding it very amusing).
		1585
		1586	The biggest difference between thed original JSON and the new JSON is that
		1587	the new JSON supports scalars (anything other than arrays and objects) at
		1588	the toplevel of a JSON text. While this is strictly backwards compatible
		1589	to older versions, it breaks a number of protocols that relied on sending
		1590	JSON back-to-back, and is a minor security concern.
		1591
		1592	For example, imagine you have two banks communicating, and on one side,
		1593	trhe JSON coder gets upgraded. Two messages, such as C<10> and C<1000>
		1594	might then be confused to mean C<101000>, something that couldn't happen
		1595	in the original JSON, because niether of these messages would be valid
		1596	JSON.
		1597
		1598	If one side accepts these messages, then an upgrade in the coder on either
		1599	side could result in this becoming exploitable.
		1600
		1601	This module has always allowed these messages as an optional extension, by
		1602	default disabled. The security concerns are the reason why the default is
		1603	still disabled, but future versions might/will likely upgrade to the newer
		1604	RFC as default format, so you are advised to check your implementation
		1605	and/or override the default with C<< ->allow_nonref (0) >> to ensure that
		1606	future versions are safe.
		1607
		1608
1533	=head1 INTEROPERABILITY WITH OTHER MODULES	1609	=head1 INTEROPERABILITY WITH OTHER MODULES
1534		1610
1535	C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean	1611	C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean
1536	constants. That means that the JSON true and false values will be	1612	constants. That means that the JSON true and false values will be
1537	comaptible to true and false values of iother modules that do the same,	1613	comaptible to true and false values of other modules that do the same,
1538	such as L<JSON::PP> and L<CBOR::XS>.	1614	such as L<JSON::PP> and L<CBOR::XS>.
		1615
		1616
		1617	=head1 INTEROPERABILITY WITH OTHER JSON DECODERS
		1618
		1619	As long as you only serialise data that can be directly expressed in JSON,
		1620	C<JSON::XS> is incapable of generating invalid JSON output (modulo bugs,
		1621	but C<JSON::XS> has found more bugs in the official JSON testsuite (1)
		1622	than the official JSON testsuite has found in C<JSON::XS> (0)).
		1623
		1624	When you have trouble decoding JSON generated by this module using other
		1625	decoders, then it is very likely that you have an encoding mismatch or the
		1626	other decoder is broken.
		1627
		1628	When decoding, C<JSON::XS> is strict by default and will likely catch all
		1629	errors. There are currently two settings that change this: C<relaxed>
		1630	makes C<JSON::XS> accept (but not generate) some non-standard extensions,
		1631	and C<allow_tags> will allow you to encode and decode Perl objects, at the
		1632	cost of not outputting valid JSON anymore.
		1633
		1634	=head2 TAGGED VALUE SYNTAX AND STANDARD JSON EN/DECODERS
		1635
		1636	When you use C<allow_tags> to use the extended (and also nonstandard and
		1637	invalid) JSON syntax for serialised objects, and you still want to decode
		1638	the generated When you want to serialise objects, you can run a regex
		1639	to replace the tagged syntax by standard JSON arrays (it only works for
		1640	"normal" package names without comma, newlines or single colons). First,
		1641	the readable Perl version:
		1642
		1643	# if your FREEZE methods return no values, you need this replace first:
		1644	$json =~ s/$ \s* (" (?: [^\\":,]+\|\\.\|::)* ") \s* $ \s* \[\s*\]/[$1]/gx;
		1645
		1646	# this works for non-empty constructor arg lists:
		1647	$json =~ s/$ \s* (" (?: [^\\":,]+\|\\.\|::)* ") \s* $ \s* \[/[$1,/gx;
		1648
		1649	And here is a less readable version that is easy to adapt to other
		1650	languages:
		1651
		1652	$json =~ s/$\s("([^\\":,]+\|\\.\|::)")\s$\s\[/[$1,/g;
		1653
		1654	Here is an ECMAScript version (same regex):
		1655
		1656	json = json.replace (/$\s("([^\\":,]+\|\\.\|::)")\s$\s\[/g, "[$1,");
		1657
		1658	Since this syntax converts to standard JSON arrays, it might be hard to
		1659	distinguish serialised objects from normal arrays. You can prepend a
		1660	"magic number" as first array element to reduce chances of a collision:
		1661
		1662	$json =~ s/$\s("([^\\":,]+\|\\.\|::)")\s$\s\[/["XU1peReLzT4ggEllLanBYq4G9VzliwKF",$1,/g;
		1663
		1664	And after decoding the JSON text, you could walk the data
		1665	structure looking for arrays with a first element of
		1666	C<XU1peReLzT4ggEllLanBYq4G9VzliwKF>.
		1667
		1668	The same approach can be used to create the tagged format with another
		1669	encoder. First, you create an array with the magic string as first member,
		1670	the classname as second, and constructor arguments last, encode it as part
		1671	of your JSON structure, and then:
		1672
		1673	$json =~ s/\[\s"XU1peReLzT4ggEllLanBYq4G9VzliwKF"\s,\s("([^\\":,]+\|\\.\|::)")\s*,/($1)[/g;
		1674
		1675	Again, this has some limitations - the magic string must not be encoded
		1676	with character escapes, and the constructor arguments must be non-empty.
		1677
		1678
		1679	=head1 RFC7159
		1680
		1681	Since this module was written, Google has written a new JSON RFC, RFC 7159
		1682	(and RFC7158). Unfortunately, this RFC breaks compatibility with both the
		1683	original JSON specification on www.json.org and RFC4627.
		1684
		1685	As far as I can see, you can get partial compatibility when parsing by
		1686	using C<< ->allow_nonref >>. However, consider the security implications
		1687	of doing so.
		1688
		1689	I haven't decided yet when to break compatibility with RFC4627 by default
		1690	(and potentially leave applications insecure) and change the default to
		1691	follow RFC7159, but application authors are well advised to call C<<
		1692	->allow_nonref(0) >> even if this is the current default, if they cannot
		1693	handle non-reference values, in preparation for the day when the default
		1694	will change.
1539		1695
1540		1696
1541	=head1 THREADS	1697	=head1 THREADS
1542		1698
1543	This module is I<not> guaranteed to be thread safe and there are no	1699	This module is I<not> guaranteed to be thread safe and there are no

Diff Legend

-–
+Removed lines
-+
+Added lines
-<
+Changed lines
->
+Changed lines

Comparing JSON-XS/XS.pm (file contents): Revision 1.145 by root, Tue Oct 29 00:06:40 2013 UTC vs. Revision 1.159 by root, Wed Sep 7 17:14:56 2016 UTC

Diff Legend

Comparing JSON-XS/XS.pm (file contents):
Revision 1.145 by root, Tue Oct 29 00:06:40 2013 UTC vs.
Revision 1.159 by root, Wed Sep 7 17:14:56 2016 UTC