ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/CBOR-XS/XS.pm
(Generate patch)

Comparing CBOR-XS/XS.pm (file contents):
Revision 1.56 by root, Mon Apr 25 21:44:13 2016 UTC vs.
Revision 1.64 by root, Fri Nov 25 23:37:27 2016 UTC

64 64
65package CBOR::XS; 65package CBOR::XS;
66 66
67use common::sense; 67use common::sense;
68 68
69our $VERSION = 1.5; 69our $VERSION = 1.51;
70our @ISA = qw(Exporter); 70our @ISA = qw(Exporter);
71 71
72our @EXPORT = qw(encode_cbor decode_cbor); 72our @EXPORT = qw(encode_cbor decode_cbor);
73 73
74use Exporter; 74use Exporter;
180reference to the earlier value. 180reference to the earlier value.
181 181
182This means that such values will only be encoded once, and will not result 182This means that such values will only be encoded once, and will not result
183in a deep cloning of the value on decode, in decoders supporting the value 183in a deep cloning of the value on decode, in decoders supporting the value
184sharing extension. This also makes it possible to encode cyclic data 184sharing extension. This also makes it possible to encode cyclic data
185structures (which need C<allow_cycles> to ne enabled to be decoded by this 185structures (which need C<allow_cycles> to be enabled to be decoded by this
186module). 186module).
187 187
188It is recommended to leave it off unless you know your 188It is recommended to leave it off unless you know your
189communication partner supports the value sharing extensions to CBOR 189communication partner supports the value sharing extensions to CBOR
190(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the 190(L<http://cbor.schmorp.de/value-sharing>), as without decoder support, the
440 440
441Resets the incremental decoder. This throws away any saved state, so that 441Resets the incremental decoder. This throws away any saved state, so that
442subsequent calls to C<incr_parse> or C<incr_parse_multiple> start to parse 442subsequent calls to C<incr_parse> or C<incr_parse_multiple> start to parse
443a new CBOR value from the beginning of the C<$buffer> again. 443a new CBOR value from the beginning of the C<$buffer> again.
444 444
445This method can be caled at any time, but it I<must> be called if you want 445This method can be called at any time, but it I<must> be called if you want
446to change your C<$buffer> or there was a decoding error and you want to 446to change your C<$buffer> or there was a decoding error and you want to
447reuse the C<$cbor> object for future incremental parsings. 447reuse the C<$cbor> object for future incremental parsings.
448 448
449=back 449=back
450 450
935objects. The corresponding C<Math::BigFloat::TO_CBOR> method I<always> 935objects. The corresponding C<Math::BigFloat::TO_CBOR> method I<always>
936encodes into a decimal fraction (either tag 4 or 264). 936encodes into a decimal fraction (either tag 4 or 264).
937 937
938NaN and infinities are not encoded properly, as they cannot be represented 938NaN and infinities are not encoded properly, as they cannot be represented
939in CBOR. 939in CBOR.
940
941See L<BIGNUM SECURITY CONSIDERATIONS> for more info.
942
943=item 30 (rational numbers)
944
945These tags are decoded into L<Math::BigRat> objects. The corresponding
946C<Math::BigRat::TO_CBOR> method encodes rational numbers with denominator
947C<1> via their numerator only, i.e., they become normal integers or
948C<bignums>.
940 949
941See L<BIGNUM SECURITY CONSIDERATIONS> for more info. 950See L<BIGNUM SECURITY CONSIDERATIONS> for more info.
942 951
943=item 21, 22, 23 (expected later JSON conversion) 952=item 21, 22, 23 (expected later JSON conversion)
944 953
976 985
977First of all, your CBOR decoder should be secure, that is, should not have 986First of all, your CBOR decoder should be secure, that is, should not have
978any buffer overflows. Obviously, this module should ensure that and I am 987any buffer overflows. Obviously, this module should ensure that and I am
979trying hard on making that true, but you never know. 988trying hard on making that true, but you never know.
980 989
990Second, CBOR::XS supports object serialisation - decoding CBOR can cause
991calls to I<any> C<THAW> method in I<any> package that exists in your
992process (that is, CBOR::XS will not try to load modules, but any existing
993C<THAW> method or function can be called, so they all have to be secure).
994
981Second, you need to avoid resource-starving attacks. That means you should 995Third, you need to avoid resource-starving attacks. That means you should
982limit the size of CBOR data you accept, or make sure then when your 996limit the size of CBOR data you accept, or make sure then when your
983resources run out, that's just fine (e.g. by using a separate process that 997resources run out, that's just fine (e.g. by using a separate process that
984can crash safely). The size of a CBOR string in octets is usually a good 998can crash safely). The size of a CBOR string in octets is usually a good
985indication of the size of the resources required to decode it into a Perl 999indication of the size of the resources required to decode it into a Perl
986structure. While CBOR::XS can check the size of the CBOR text, it might be 1000structure. While CBOR::XS can check the size of the CBOR text, it might be
987too late when you already have it in memory, so you might want to check 1001too late when you already have it in memory, so you might want to check
988the size before you accept the string. 1002the size before you accept the string.
989 1003
990Third, CBOR::XS recurses using the C stack when decoding objects and 1004Fourth, CBOR::XS recurses using the C stack when decoding objects and
991arrays. The C stack is a limited resource: for instance, on my amd64 1005arrays. The C stack is a limited resource: for instance, on my amd64
992machine with 8MB of stack size I can decode around 180k nested arrays but 1006machine with 8MB of stack size I can decode around 180k nested arrays but
993only 14k nested CBOR objects (due to perl itself recursing deeply on croak 1007only 14k nested CBOR objects (due to perl itself recursing deeply on croak
994to free the temporary). If that is exceeded, the program crashes. To be 1008to free the temporary). If that is exceeded, the program crashes. To be
995conservative, the default nesting limit is set to 512. If your process 1009conservative, the default nesting limit is set to 512. If your process
1008=head1 BIGNUM SECURITY CONSIDERATIONS 1022=head1 BIGNUM SECURITY CONSIDERATIONS
1009 1023
1010CBOR::XS provides a C<TO_CBOR> method for both L<Math::BigInt> and 1024CBOR::XS provides a C<TO_CBOR> method for both L<Math::BigInt> and
1011L<Math::BigFloat> that tries to encode the number in the simplest possible 1025L<Math::BigFloat> that tries to encode the number in the simplest possible
1012way, that is, either a CBOR integer, a CBOR bigint/decimal fraction (tag 1026way, that is, either a CBOR integer, a CBOR bigint/decimal fraction (tag
10134) or an arbitrary-exponent decimal fraction (tag 264). 10274) or an arbitrary-exponent decimal fraction (tag 264). Rational numbers
1028(L<Math::BigRat>, tag 30) can also contain bignums as members.
1014 1029
1015It will also understand base-2 bigfloat or arbitrary-exponent bigfloats 1030CBOR::XS will also understand base-2 bigfloat or arbitrary-exponent
1016(tags 5 and 265), but it will never generate these on its own. 1031bigfloats (tags 5 and 265), but it will never generate these on its own.
1017 1032
1018Using the built-in L<Math::BigInt::Calc> support, encoding and decoding 1033Using the built-in L<Math::BigInt::Calc> support, encoding and decoding
1019decimal fractions is generally fast. Decoding bigints can be slow for very 1034decimal fractions is generally fast. Decoding bigints can be slow for very
1020big numbers, and decoding bigfloats or arbitrary-exponent bigfloats can be 1035big numbers (tens of thousands of digits, something that could potentially
1021extremely slow (minutes, decades) for large exponents. 1036be caught by limiting the size of CBOR texts), and decoding bigfloats or
1037arbitrary-exponent bigfloats can be I<extremely> slow (minutes, decades)
1038for large exponents (roughly 40 bit and longer).
1022 1039
1023Additionally, L<Math::BigInt> can take advantage of other bignum 1040Additionally, L<Math::BigInt> can take advantage of other bignum
1024libraries, such as L<Math::GMP>, which cannot handle big 1041libraries, such as L<Math::GMP>, which cannot handle big floats with large
1025floats with large exponents, and might simply abort or crash your program, 1042exponents, and might simply abort or crash your program, due to their code
1026due to their code quality. 1043quality.
1027 1044
1028This can be a concern if you want to parse untrusted CBOR. If it is, you 1045This can be a concern if you want to parse untrusted CBOR. If it is, you
1029need to disable decoding of tag 2 (bigint) and 3 (negative bigint) types, 1046might want to disable decoding of tag 2 (bigint) and 3 (negative bigint)
1030which will also disable bigfloat support (to be sure, you can also disable 1047types. You should also disable types 5 and 265, as these can be slow even
1031types 4, 5, 264 and 265). 1048without bigints.
1049
1050Disabling bigints will also partially or fully disable types that rely on
1051them, e.g. rational numbers that use bignums.
1032 1052
1033 1053
1034=head1 CBOR IMPLEMENTATION NOTES 1054=head1 CBOR IMPLEMENTATION NOTES
1035 1055
1036This section contains some random implementation notes. They do not 1056This section contains some random implementation notes. They do not
1078 1098
1079Please refrain from using rt.cpan.org or any other bug reporting 1099Please refrain from using rt.cpan.org or any other bug reporting
1080service. I put the contact address into my modules for a reason. 1100service. I put the contact address into my modules for a reason.
1081 1101
1082=cut 1102=cut
1103
1104# clumsy hv_store-in-perl
1105sub _hv_store {
1106 $_[0]{$_[1]} = $_[2];
1107}
1083 1108
1084our %FILTER = ( 1109our %FILTER = (
1085 0 => sub { # rfc4287 datetime, utf-8 1110 0 => sub { # rfc4287 datetime, utf-8
1086 require Time::Piece; 1111 require Time::Piece;
1087 # Time::Piece::Strptime uses the "incredibly flexible date parsing routine" 1112 # Time::Piece::Strptime uses the "incredibly flexible date parsing routine"
1110 scalar Time::Piece::gmtime (pop) 1135 scalar Time::Piece::gmtime (pop)
1111 }, 1136 },
1112 1137
1113 2 => sub { # pos bigint 1138 2 => sub { # pos bigint
1114 require Math::BigInt; 1139 require Math::BigInt;
1115 Math::BigInt->from_hex ("0x" . unpack "H*", pop) 1140 Math::BigInt->new ("0x" . unpack "H*", pop)
1116 }, 1141 },
1117 1142
1118 3 => sub { # neg bigint 1143 3 => sub { # neg bigint
1119 require Math::BigInt; 1144 require Math::BigInt;
1120 -Math::BigInt->from_hex ("0x" . unpack "H*", pop) 1145 -Math::BigInt->new ("0x" . unpack "H*", pop)
1121 }, 1146 },
1122 1147
1123 4 => sub { # decimal fraction, array 1148 4 => sub { # decimal fraction, array
1124 require Math::BigFloat; 1149 require Math::BigFloat;
1125 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0]) 1150 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
1126 }, 1151 },
1127 1152
1153 264 => sub { # decimal fraction with arbitrary exponent
1154 require Math::BigFloat;
1155 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
1156 },
1157
1128 5 => sub { # bigfloat, array 1158 5 => sub { # bigfloat, array
1129 require Math::BigFloat; 1159 require Math::BigFloat;
1130 scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0]) 1160 scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
1131 }, 1161 },
1132 1162
1163 265 => sub { # bigfloat with arbitrary exponent
1164 require Math::BigFloat;
1165 scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
1166 },
1167
1168 30 => sub { # rational number
1169 require Math::BigRat;
1170 Math::BigRat->new ("$_[1][0]/$_[1][1]") # separate parameters only work in recent versons
1171 },
1172
1133 21 => sub { pop }, # expected conversion to base64url encoding 1173 21 => sub { pop }, # expected conversion to base64url encoding
1134 22 => sub { pop }, # expected conversion to base64 encoding 1174 22 => sub { pop }, # expected conversion to base64 encoding
1135 23 => sub { pop }, # expected conversion to base16 encoding 1175 23 => sub { pop }, # expected conversion to base16 encoding
1136 1176
1137 # 24 # embedded cbor, byte string 1177 # 24 # embedded cbor, byte string
1143 1183
1144 # 33 # base64url rfc4648, utf-8 1184 # 33 # base64url rfc4648, utf-8
1145 # 34 # base64 rfc46484, utf-8 1185 # 34 # base64 rfc46484, utf-8
1146 # 35 # regex pcre/ecma262, utf-8 1186 # 35 # regex pcre/ecma262, utf-8
1147 # 36 # mime message rfc2045, utf-8 1187 # 36 # mime message rfc2045, utf-8
1148
1149 264 => sub { # decimal fraction with arbitrary exponent
1150 require Math::BigFloat;
1151 Math::BigFloat->new ($_[1][1] . "E" . $_[1][0])
1152 },
1153
1154 265 => sub { # bigfloat with arbitrary exponent
1155 require Math::BigFloat;
1156 scalar Math::BigFloat->new ($_[1][1]) * Math::BigFloat->new (2)->bpow ($_[1][0])
1157 },
1158); 1188);
1159 1189
1160sub CBOR::XS::default_filter { 1190sub CBOR::XS::default_filter {
1161 &{ $FILTER{$_[0]} or return } 1191 &{ $FILTER{$_[0]} or return }
1162} 1192}
1183 -9223372036854775808 <= $e && $e <= 18446744073709551615 1213 -9223372036854775808 <= $e && $e <= 18446744073709551615
1184 ? tag 4, [$e->numify, $m] 1214 ? tag 4, [$e->numify, $m]
1185 : tag 264, [$e, $m] 1215 : tag 264, [$e, $m]
1186} 1216}
1187 1217
1218sub Math::BigRat::TO_CBOR {
1219 my ($n, $d) = $_[0]->parts;
1220
1221 # older versions of BigRat need *1, as they not always return numbers
1222
1223 $d*1 == 1
1224 ? $n*1
1225 : tag 30, [$n*1, $d*1]
1226}
1227
1188sub Time::Piece::TO_CBOR { 1228sub Time::Piece::TO_CBOR {
1189 tag 1, 0 + $_[0]->epoch 1229 tag 1, 0 + $_[0]->epoch
1190} 1230}
1191 1231
1192XSLoader::load "CBOR::XS", $VERSION; 1232XSLoader::load "CBOR::XS", $VERSION;

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines