ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Convert-BER-XS/XS.pm
Revision: 1.13
Committed: Sat Apr 20 01:03:59 2019 UTC (5 years, 1 month ago) by root
Branch: MAIN
Changes since 1.12: +192 -15 lines
Log Message:
*** empty log message ***

File Contents

# Content
1 =head1 NAME
2
3 Convert::BER::XS - I<very> low level BER en-/decoding
4
5 =head1 SYNOPSIS
6
7 use Convert::BER::XS ':all';
8
9 my $ber = ber_decode $buf
10 or die "unable to decode SNMP message";
11
12 # The above results in a data structure consisting of
13 # (class, tag, # constructed, data)
14 # tuples. Below is such a message, SNMPv1 trap
15 # with a Cisco mac change notification.
16 # Did you know that Cisco is in the news almost
17 # every week because # of some backdoor password
18 # or other extremely stupid security bug?
19
20 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
21 [
22 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
23 [ ASN_UNIVERSAL, 4, 0, "public" ], # community
24 [ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
25 [
26 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
27 [ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1
28 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
29 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
30 [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
31 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
32 [
33 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
34 [
35 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
36 [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
37 ]
38 ]
39 ],
40 ...
41
42 # let's decode it a bit with some helper functions
43
44 my $msg = ber_is_seq $ber
45 or die "SNMP message does not start with a sequence";
46
47 ber_is $msg->[0], ASN_UNIVERSAL, ASN_INTEGER32, 0
48 or die "SNMP message does not start with snmp version\n";
49
50 # message is SNMP v1 or v2c?
51 if ($msg->[0][BER_DATA] == 0 || $msg->[0][BER_DATA] == 1) {
52
53 # message is v1 trap?
54 if (ber_is $msg->[2], ASN_CONTEXT, 4, 1) {
55 my $trap = $msg->[2][BER_DATA];
56
57 # check whether trap is a cisco mac notification mac changed message
58 if (
59 (ber_is_oid $trap->[0], "1.3.6.1.4.1.9.9.215.2") # cmnInterfaceObjects
60 and (ber_is_i32 $trap->[2], 6)
61 and (ber_is_i32 $trap->[3], 1) # mac changed msg
62 ) {
63 ... and so on
64
65 # finally, let's encode it again and hope it results in the same bit pattern
66
67 my $buf = ber_encode $ber;
68
69 =head1 DESCRIPTION
70
71 WARNING: Before release 1.0, the API is not considered stable in any way.
72
73 This module implements a I<very> low level BER/DER en-/decoder.
74
75 If is tuned for low memory and high speed, while still maintaining some
76 level of user-friendlyness.
77
78 Currently, not much is documented, as this is an initial release to
79 reserve CPAN namespace, stay tuned for a few days.
80
81 =head2 ASN.1/BER/DER/... BASICS
82
83 ASN.1 is a strange language that can be sed to describe protocols and
84 data structures. It supports various mappings to JSON, XML, but most
85 importantly, to a various binary encodings such as BER, that is the topic
86 of this module, and is used in SNMP or LDAP for example.
87
88 While ASN.1 defines a schema that is useful to interpret encoded data,
89 the BER encoding is actually somewhat self-describing: you might not know
90 whether something is a string or a number or a sequence or something else,
91 but you can nevertheless decode the overall structure, even if you end up
92 with just a binary blob for the actual value.
93
94 This works because BER values are tagged with a type and a namespace,
95 and also have a flag that says whther a value consists of subvalues (is
96 "constructed") or not (is "primitive").
97
98 Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
99 those - for example, you have 32 bit signed integers and 16(!) different
100 string types, but there is no unsigned32 type for example. Different
101 applications work around this in different ways, for example, SNMP defines
102 application-specific Gauge32, Counter32 and Unsigned32, which are mapped
103 to two different tags: you can distinguish between Counter32 and the
104 others, but not between Gause32 and Unsigned32, without the ASN.1 schema.
105
106 Ugh.
107
108 =head2 DECODED BER REPRESENTATION
109
110 This module represents every BER value as a 4-element tuple (actually an
111 array-reference):
112
113 [CLASS, TAG, CONSTRUCTED, DATA]
114
115 To avoid non-descriptive hardcoded array index numbers, this module
116 defines symbolic constants to access these members: C<BER_CLASS>,
117 C<BER_TAG>, C<BER_CONSTRUCTED> and C<BER_DATA>.
118
119 Also, the first three members are integers with a little caveat: for
120 performance reasons, these are readonly and shared, so you must not modify
121 them (increment, assign to them etc.) in any way. You may modify the
122 I<DATA> member, and you may re-assign the array itself, e.g.:
123
124 $ber = ber_decode $binbuf;
125
126 # the following is NOT legal:
127 $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)
128
129 # but all of the following are fine:
130 $ber->[BER_DATA] = "string";
131 $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
132 @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);
133
134 I<CLASS> is something like a namespace for I<TAG>s - there is the
135 C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
136 implementations, the C<ASN_APPLICATION> namespace which defines tags for
137 specific applications (for example, the SNMP C<Unsigned32> type is in this
138 namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g.
139 for C<CHOICE>) and a private namespace (C<ASN_PRIVATE>).
140
141 The meaning of the I<TAG> depends on the namespace, and defines a
142 (partial) interpretation of the data value. For example, right now, SNMP
143 application namespace knowledge ix hardcoded into this module, so it
144 knows that SNMP C<Unsigned32> values need to be decoded into actual perl
145 integers.
146
147 The most common tags in the C<ASN_UNIVERSAL> namespace are
148 C<ASN_INTEGER32>, C<ASN_BIT_STRING>, C<ASN_NULL>, C<ASN_OCTET_STRING>,
149 C<ASN_OBJECT_IDENTIFIER>, C<ASN_SEQUENCE>, C<ASN_SET> and
150 C<ASN_IA5_STRING>.
151
152 The most common tags in SNMP's C<ASN_APPLICATION> namespace
153 are C<SNMP_IPADDRESS>, C<SNMP_COUNTER32>, C<SNMP_UNSIGNED32>,
154 C<SNMP_TIMETICKS>, C<SNMP_OPAQUE> and C<SNMP_COUNTER64>.
155
156 The I<CONSTRUCTED> flag is really just a boolean - if it is false, the
157 the value is "primitive" and contains no subvalues, kind of like a
158 non-reference perl scalar. IF it is true, then the value is "constructed"
159 which just means it contains a list of subvalues which this module will
160 en-/decode as BER tuples themselves.
161
162 The I<DATA> value is either a reference to an array of further tuples (if
163 the value is I<CONSTRUCTED>), some decoded representation of the value,
164 if this module knows how to decode it (e.g. for the integer types above)
165 or a binary string with the raw octets if this module doesn't know how to
166 interpret the namespace/tag.
167
168 Thus, you can always decode a BER data structure and at worst you get a
169 string in place of some nice decoded value.
170
171 See the SYNOPSIS for an example of such an encoded tuple representation.
172
173 =head2 DECODING AND ENCODING
174
175 =over
176
177 =item $tuple = ber_decoded $bindata
178
179 Decodes binary BER data in C<$bindata> and returns the resulting BER
180 tuple. Croaks on any decoding error, so the returned C<$tuple> is always
181 valid.
182
183 =item $bindata = ber_encode $tuple
184
185 Encodes the BER tuple into a BER/DER data structure.
186
187 =back
188
189 =head2 HELPER FUNCTIONS
190
191 Working with a 4-tuple for every value can be annoying. Or, rather, I<is>
192 annoying. To reduce this a bit, this module defines a number of helper
193 functions, both to match BER tuples and to conmstruct BER tuples:
194
195 =head3 MATCH HELPERS
196
197 Thse functions accept a BER tuple as first argument and either paertially
198 or fully match it. They often come in two forms, one which exactly matches
199 a value, and one which only matches the type and returns the value.
200
201 They do check whether valid tuples are passed in and croak otherwise. As
202 a ease-of-use exception, they usually also accept C<undef> instead of a
203 tuple reference. in which case they silently fail to match.
204
205 =over
206
207 =item $bool = ber_is $tuple, $class, $tag, $constructed, $data
208
209 This takes a BER C<$tuple> and matches its elements agains the privded
210 values, all of which are optional - values that are either missing or
211 C<undef> will be ignored, the others will be matched exactly (e.g. as if
212 you used C<==> or C<eq> (for C<$data>)).
213
214 Some examples:
215
216 ber_is $tuple, ASN_UNIVERSAL, ASN_SEQUENCE, 1
217 orf die "tuple is not an ASN SEQUENCE";
218
219 ber_is $tuple, ASN_UNIVERSAL, ASN_NULL
220 or die "tuple is not an ASN NULL value";
221
222 ber_is $tuple, ASN_UNIVERSAL, ASN_INTEGER32, 0, 50
223 or die "BER integer must be 50";
224
225 =item $seq = ber_is_seq $tuple
226
227 Returns the sequence members (the array of subvalues) if the C<$tuple> is
228 an ASN SEQUENCE, i.e. the C<BER_DATA> member. If the C<$tuple> is not a
229 sequence it returns C<undef>. For example, SNMP version 1/2c/3 packets all
230 consist of an outer SEQUENCE value:
231
232 my $ber = ber_decode $snmp_data;
233
234 my $snmp = ber_is_seq $ber
235 or die "SNMP packet invalid: does not start with SEQUENCE";
236
237 # now we know $snmp is a sequence, so decode the SNMP version
238
239 my $version = ber_is_i32 $snmp->[0]
240 or die "SNMP packet invalid: does not start with version number";
241
242 =item $bool = ber_is_i32 $tuple, $i32
243
244 Returns a true value if the C<$tuple> represents an ASN INTEGER32 with
245 the value C<$i32>.
246
247 =item $i32 = ber_is_i32 $tuple
248
249 Returns true (and extracts the integer value) if the C<$tuple> is an ASN
250 INTEGER32. For C<0>, this function returns a special value that is 0 but
251 true.
252
253 =item $bool = ber_is_oid $tuple, $oid_string
254
255 Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
256 that exactly matches C<$oid_string>. Example:
257
258 ber_is_oid $tuple, "1.3.6.1.4"
259 or die "oid must be 1.3.6.1.4";
260
261 =item $oid = ber_is_oid $tuple
262
263 Returns true (and extracts the OID string) if the C<$tuple> is an ASN
264 OBJECT IDENTIFIER. Otherwise, it returns C<undef>.
265
266 =back
267
268 =head3 CONSTRUCTION HELPERS
269
270 =over
271
272 =item $tuple = ber_i32 $value
273
274 Constructs a new C<ASN_INTEGER32> tuple.
275
276 =back
277
278 =head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1>
279
280 This module is I<not> the XS version of L<Convert::BER>, but a different
281 take at doing the same thing. I imagine this module would be a good base
282 for speeding up either of these, or write a similar module, or write your
283 own LDAP or SNMP module for example.
284
285 =cut
286
287 package Convert::BER::XS;
288
289 use common::sense;
290
291 use XSLoader ();
292 use Exporter qw(import);
293
294 our $VERSION;
295
296 BEGIN {
297 $VERSION = 0.7;
298 XSLoader::load __PACKAGE__, $VERSION;
299 }
300
301 our %EXPORT_TAGS = (
302 const => [qw(
303 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
304
305 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
306 ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
307 ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
308 ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
309 ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
310 ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
311
312 ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
313
314 BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
315 BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
316 BER_TYPE_IPADDRESS BER_TYPE_CROAK
317 )],
318 const_snmp => [qw(
319 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
320 )],
321 encode => [qw(
322 ber_decode
323 ber_is ber_is_seq ber_is_i32 ber_is_oid
324 )],
325 decode => [qw(
326 ber_encode
327 ber_i32
328 )],
329 );
330
331 our @EXPORT_OK = map @$_, values %EXPORT_TAGS;
332
333 $EXPORT_TAGS{all} = \@EXPORT_OK;
334
335 =head1 PROFILES
336
337 While any BER data can be correctly encoded and decoded out of the box, it
338 can be inconvenient to have to manually decode some values into a "better"
339 format: for instance, SNMP TimeTicks values are decoded into the raw octet
340 strings of their BER representation, which is quite hard to decode. With
341 profiles, you can change which class/tag combinations map to which decoder
342 function inside C<ber_decode> (and of course also which encoder functions
343 are used in C<ber_encode>).
344
345 This works by mapping specific class/tag combinations to an internal "ber
346 type".
347
348 The default profile supports the standard ASN.1 types, but no
349 application-specific ones. This means that class/tag combinations not in
350 the base set of ASN.1 are decoded into their raw octet strings.
351
352 C<Convert::BER::XS> defines two profile variables you cna use out of the box:
353
354 =over
355
356 =item C<$Convert::BER::XS::DEFAULT_PROFILE>
357
358 This is the default profile, i.e. the profile that is used when no
359 profile is specified for de-/encoding.
360
361 You cna modify it, but remember that this modifies the defaults for all
362 callers that rely on the defauit profile.
363
364 =item C<$Convert::BER::XS::SNMP_PROFILE>
365
366 A profile with mappings for SNMP-specific application tags added. This is
367 useful when de-/encoding SNMP data.
368
369 Example:
370 $ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;
371
372 =back
373
374 =head2 The Convert::BER::XS::Profile class
375
376 =over
377
378 =item $profile = new Convert::BER::XS::Profile
379
380 Create a new profile. The profile will be identical to the default
381 profile.
382
383 =item $profile->set ($class, $tag, $type)
384
385 Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
386 which must be one of the C<BER_TYPE_*> constants.
387
388 Note that currently, the mapping is stored in a flat array, so large
389 values of C<$tag> will consume large amounts of memory.
390
391 Example:
392 $profile = new Convert::BER::XS::Profile;
393 $profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
394 $ber = ber_decode $data, $profile;
395
396 =item $type = $profile->get ($class, $tag)
397
398 Returns the BER type mapped to the given C<$class>/C<$tag> combination.
399
400 =back
401
402 =head2 BER TYPES
403
404 This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
405 combination to any C<BER_TYPE_*>.
406
407 =over
408
409 =item C<BER_TYPE_BYTES>
410
411 The raw octets of the value. This is the default type for unknown tags and
412 de-/encodes the value as if it were an octet string, i.e. by copying the
413 raw bytes.
414
415 =item C<BER_TYPE_UTF8>
416
417 Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
418 (without validation!) and encodes a perl unicode string into a UTF-8 BER
419 string.
420
421 =item C<BER_TYPE_UCS2>
422
423 Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
424 string. NOT IMPLEMENTED.
425
426 =item C<BER_TYPE_UCS4>
427
428 Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
429 string. NOT IMPLEMENTED.
430
431 =item C<BER_TYPE_INT>
432
433 Encodes and decodes a BER integer value to a perl integer scalar. This
434 should correctly handle 64 bit signed and unsigned values.
435
436 =item C<BER_TYPE_OID>
437
438 Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
439 dot, e.g. C<1.3.6.1.213>.
440
441 =item C<BER_TYPE_RELOID>
442
443 Same as C<BER_TYPE_OID> but uses relative OID encoding: ASN.1 has this
444 hack of encoding the first two OID components into a single integer in a
445 weird attempt to save an insignificant amount of space in an otherwise
446 wasteful encoding, and relative OIDs are basically OIDs without this
447 hack. The practical difference is that the second component of an OID
448 can only have the values 1..40, while relative OIDs do not have this
449 restriction.
450
451 =item C<BER_TYPE_NULL>
452
453 Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
454 C<ASN_NULL> type, regardless of the perl value.
455
456 =item C<BER_TYPE_BOOL>
457
458 Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
459 boolean value into an C<ASN_BOOLEAN>.
460
461 =item C<BER_TYPE_REAL>
462
463 Decodes/encodes a BER real value. NOT IMPLEMENTED.
464
465 =item C<BER_TYPE_IPADDRESS>
466
467 Decodes/encodes a four byte string into an IOv4 dotted-quad address string
468 in perl. Given ther obsolete nature of this type, this is a low-effort
469 implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
470 so it won't handle all string forms supported by C<inet_aton>.
471
472 =item C<BER_TYPE_CROAK>
473
474 Always croaks when encountered during encoding or decoding - the
475 default behaviour when encountering an unknown type is to treat it as
476 C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
477 error for some types, then CyBER_TYPE_CROAK> is for you.
478
479 =back
480
481 =cut
482
483 our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
484 our $SNMP_PROFILE = new Convert::BER::XS::Profile;
485
486 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
487 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
488 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
489 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
490 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE , BER_TYPE_IPADDRESS);
491 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);
492
493 $DEFAULT_PROFILE->_set_default;
494
495 1;
496
497 =head2 LIMITATIONS
498
499 This module can only en-/decode 64 bit signed and unsigned integers, and
500 only when your perl supports those.
501
502 OBJECT IDENTIFIEERS cannot have unlimited length, although the limit is
503 much larger than e.g. the one imposed by SNMP or other protocols.
504
505 =head1 AUTHOR
506
507 Marc Lehmann <schmorp@schmorp.de>
508 http://software.schmorp.de/pkg/Convert-BER-XS
509
510 =cut
511