ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Convert-BER-XS/XS.pm
Revision: 1.17
Committed: Sat Apr 20 13:46:14 2019 UTC (5 years, 1 month ago) by root
Branch: MAIN
Changes since 1.16: +5 -0 lines
Log Message:
*** empty log message ***

File Contents

# Content
1 =head1 NAME
2
3 Convert::BER::XS - I<very> low level BER en-/decoding
4
5 =head1 SYNOPSIS
6
7 use Convert::BER::XS ':all';
8
9 my $ber = ber_decode $buf
10 or die "unable to decode SNMP message";
11
12 # The above results in a data structure consisting of
13 # (class, tag, # constructed, data)
14 # tuples. Below is such a message, SNMPv1 trap
15 # with a Cisco mac change notification.
16 # Did you know that Cisco is in the news almost
17 # every week because # of some backdoor password
18 # or other extremely stupid security bug?
19
20 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
21 [
22 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
23 [ ASN_UNIVERSAL, 4, 0, "public" ], # community
24 [ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
25 [
26 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
27 [ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1
28 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
29 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
30 [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
31 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
32 [
33 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
34 [
35 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
36 [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
37 ]
38 ]
39 ],
40 ...
41
42 # let's decode it a bit with some helper functions
43
44 my $msg = ber_is_seq $ber
45 or die "SNMP message does not start with a sequence";
46
47 ber_is $msg->[0], ASN_UNIVERSAL, ASN_INTEGER32, 0
48 or die "SNMP message does not start with snmp version\n";
49
50 # message is SNMP v1 or v2c?
51 if ($msg->[0][BER_DATA] == 0 || $msg->[0][BER_DATA] == 1) {
52
53 # message is v1 trap?
54 if (ber_is $msg->[2], ASN_CONTEXT, 4, 1) {
55 my $trap = $msg->[2][BER_DATA];
56
57 # check whether trap is a cisco mac notification mac changed message
58 if (
59 (ber_is_oid $trap->[0], "1.3.6.1.4.1.9.9.215.2") # cmnInterfaceObjects
60 and (ber_is_i32 $trap->[2], 6)
61 and (ber_is_i32 $trap->[3], 1) # mac changed msg
62 ) {
63 ... and so on
64
65 # finally, let's encode it again and hope it results in the same bit pattern
66
67 my $buf = ber_encode $ber;
68
69 =head1 DESCRIPTION
70
71 WARNING: Before release 1.0, the API is not considered stable in any way.
72
73 This module implements a I<very> low level BER/DER en-/decoder.
74
75 If is tuned for low memory and high speed, while still maintaining some
76 level of user-friendlyness.
77
78 =head2 ASN.1/BER/DER/... BASICS
79
80 ASN.1 is a strange language that can be used to describe protocols and
81 data structures. It supports various mappings to JSON, XML, but most
82 importantly, to a various binary encodings such as BER, that is the topic
83 of this module, and is used in SNMP or LDAP for example.
84
85 While ASN.1 defines a schema that is useful to interpret encoded data,
86 the BER encoding is actually somewhat self-describing: you might not know
87 whether something is a string or a number or a sequence or something else,
88 but you can nevertheless decode the overall structure, even if you end up
89 with just a binary blob for the actual value.
90
91 This works because BER values are tagged with a type and a namespace,
92 and also have a flag that says whether a value consists of subvalues (is
93 "constructed") or not (is "primitive").
94
95 Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
96 those - for example, you have 32 bit signed integers and 16(!) different
97 string types, but there is no unsigned32 type for example. Different
98 applications work around this in different ways, for example, SNMP defines
99 application-specific Gauge32, Counter32 and Unsigned32, which are mapped
100 to two different tags: you can distinguish between Counter32 and the
101 others, but not between Gause32 and Unsigned32, without the ASN.1 schema.
102
103 Ugh.
104
105 =head2 DECODED BER REPRESENTATION
106
107 This module represents every BER value as a 4-element tuple (actually an
108 array-reference):
109
110 [CLASS, TAG, CONSTRUCTED, DATA]
111
112 To avoid non-descriptive hardcoded array index numbers, this module
113 defines symbolic constants to access these members: C<BER_CLASS>,
114 C<BER_TAG>, C<BER_CONSTRUCTED> and C<BER_DATA>.
115
116 Also, the first three members are integers with a little caveat: for
117 performance reasons, these are readonly and shared, so you must not modify
118 them (increment, assign to them etc.) in any way. You may modify the
119 I<DATA> member, and you may re-assign the array itself, e.g.:
120
121 $ber = ber_decode $binbuf;
122
123 # the following is NOT legal:
124 $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)
125
126 # but all of the following are fine:
127 $ber->[BER_DATA] = "string";
128 $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
129 @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);
130
131 I<CLASS> is something like a namespace for I<TAG>s - there is the
132 C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
133 implementations, the C<ASN_APPLICATION> namespace which defines tags for
134 specific applications (for example, the SNMP C<Unsigned32> type is in this
135 namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g.
136 for C<CHOICE>) and a private namespace (C<ASN_PRIVATE>).
137
138 The meaning of the I<TAG> depends on the namespace, and defines a
139 (partial) interpretation of the data value. For example, right now, SNMP
140 application namespace knowledge ix hardcoded into this module, so it
141 knows that SNMP C<Unsigned32> values need to be decoded into actual perl
142 integers.
143
144 The most common tags in the C<ASN_UNIVERSAL> namespace are
145 C<ASN_INTEGER32>, C<ASN_BIT_STRING>, C<ASN_NULL>, C<ASN_OCTET_STRING>,
146 C<ASN_OBJECT_IDENTIFIER>, C<ASN_SEQUENCE>, C<ASN_SET> and
147 C<ASN_IA5_STRING>.
148
149 The most common tags in SNMP's C<ASN_APPLICATION> namespace
150 are C<SNMP_IPADDRESS>, C<SNMP_COUNTER32>, C<SNMP_UNSIGNED32>,
151 C<SNMP_TIMETICKS>, C<SNMP_OPAQUE> and C<SNMP_COUNTER64>.
152
153 The I<CONSTRUCTED> flag is really just a boolean - if it is false, the
154 the value is "primitive" and contains no subvalues, kind of like a
155 non-reference perl scalar. IF it is true, then the value is "constructed"
156 which just means it contains a list of subvalues which this module will
157 en-/decode as BER tuples themselves.
158
159 The I<DATA> value is either a reference to an array of further tuples (if
160 the value is I<CONSTRUCTED>), some decoded representation of the value,
161 if this module knows how to decode it (e.g. for the integer types above)
162 or a binary string with the raw octets if this module doesn't know how to
163 interpret the namespace/tag.
164
165 Thus, you can always decode a BER data structure and at worst you get a
166 string in place of some nice decoded value.
167
168 See the SYNOPSIS for an example of such an encoded tuple representation.
169
170 =head2 DECODING AND ENCODING
171
172 =over
173
174 =item $tuple = ber_decoded $bindata
175
176 Decodes binary BER data in C<$bindata> and returns the resulting BER
177 tuple. Croaks on any decoding error, so the returned C<$tuple> is always
178 valid.
179
180 =item $bindata = ber_encode $tuple
181
182 Encodes the BER tuple into a BER/DER data structure.
183
184 =back
185
186 =head2 HELPER FUNCTIONS
187
188 Working with a 4-tuple for every value can be annoying. Or, rather, I<is>
189 annoying. To reduce this a bit, this module defines a number of helper
190 functions, both to match BER tuples and to conmstruct BER tuples:
191
192 =head3 MATCH HELPERS
193
194 Thse functions accept a BER tuple as first argument and either paertially
195 or fully match it. They often come in two forms, one which exactly matches
196 a value, and one which only matches the type and returns the value.
197
198 They do check whether valid tuples are passed in and croak otherwise. As
199 a ease-of-use exception, they usually also accept C<undef> instead of a
200 tuple reference. in which case they silently fail to match.
201
202 =over
203
204 =item $bool = ber_is $tuple, $class, $tag, $constructed, $data
205
206 This takes a BER C<$tuple> and matches its elements agains the privded
207 values, all of which are optional - values that are either missing or
208 C<undef> will be ignored, the others will be matched exactly (e.g. as if
209 you used C<==> or C<eq> (for C<$data>)).
210
211 Some examples:
212
213 ber_is $tuple, ASN_UNIVERSAL, ASN_SEQUENCE, 1
214 orf die "tuple is not an ASN SEQUENCE";
215
216 ber_is $tuple, ASN_UNIVERSAL, ASN_NULL
217 or die "tuple is not an ASN NULL value";
218
219 ber_is $tuple, ASN_UNIVERSAL, ASN_INTEGER32, 0, 50
220 or die "BER integer must be 50";
221
222 =item $seq = ber_is_seq $tuple
223
224 Returns the sequence members (the array of subvalues) if the C<$tuple> is
225 an ASN SEQUENCE, i.e. the C<BER_DATA> member. If the C<$tuple> is not a
226 sequence it returns C<undef>. For example, SNMP version 1/2c/3 packets all
227 consist of an outer SEQUENCE value:
228
229 my $ber = ber_decode $snmp_data;
230
231 my $snmp = ber_is_seq $ber
232 or die "SNMP packet invalid: does not start with SEQUENCE";
233
234 # now we know $snmp is a sequence, so decode the SNMP version
235
236 my $version = ber_is_i32 $snmp->[0]
237 or die "SNMP packet invalid: does not start with version number";
238
239 =item $bool = ber_is_i32 $tuple, $i32
240
241 Returns a true value if the C<$tuple> represents an ASN INTEGER32 with
242 the value C<$i32>.
243
244 =item $i32 = ber_is_i32 $tuple
245
246 Returns true (and extracts the integer value) if the C<$tuple> is an ASN
247 INTEGER32. For C<0>, this function returns a special value that is 0 but
248 true.
249
250 =item $bool = ber_is_oid $tuple, $oid_string
251
252 Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
253 that exactly matches C<$oid_string>. Example:
254
255 ber_is_oid $tuple, "1.3.6.1.4"
256 or die "oid must be 1.3.6.1.4";
257
258 =item $oid = ber_is_oid $tuple
259
260 Returns true (and extracts the OID string) if the C<$tuple> is an ASN
261 OBJECT IDENTIFIER. Otherwise, it returns C<undef>.
262
263 =back
264
265 =head3 CONSTRUCTION HELPERS
266
267 =over
268
269 =item $tuple = ber_i32 $value
270
271 Constructs a new C<ASN_INTEGER32> tuple.
272
273 =back
274
275 =head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1>
276
277 This module is I<not> the XS version of L<Convert::BER>, but a different
278 take at doing the same thing. I imagine this module would be a good base
279 for speeding up either of these, or write a similar module, or write your
280 own LDAP or SNMP module for example.
281
282 =cut
283
284 package Convert::BER::XS;
285
286 use common::sense;
287
288 use XSLoader ();
289 use Exporter qw(import);
290
291 our $VERSION;
292
293 BEGIN {
294 $VERSION = 0.7;
295 XSLoader::load __PACKAGE__, $VERSION;
296 }
297
298 our %EXPORT_TAGS = (
299 const => [qw(
300 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
301
302 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
303 ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
304 ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
305 ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
306 ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
307 ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
308
309 ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
310
311 BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
312 BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
313 BER_TYPE_IPADDRESS BER_TYPE_CROAK
314 )],
315 const_snmp => [qw(
316 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
317 )],
318 encode => [qw(
319 ber_decode
320 ber_is ber_is_seq ber_is_i32 ber_is_oid
321 )],
322 decode => [qw(
323 ber_encode
324 ber_i32
325 )],
326 );
327
328 our @EXPORT_OK = map @$_, values %EXPORT_TAGS;
329
330 $EXPORT_TAGS{all} = \@EXPORT_OK;
331
332 =head1 PROFILES
333
334 While any BER data can be correctly encoded and decoded out of the box, it
335 can be inconvenient to have to manually decode some values into a "better"
336 format: for instance, SNMP TimeTicks values are decoded into the raw octet
337 strings of their BER representation, which is quite hard to decode. With
338 profiles, you can change which class/tag combinations map to which decoder
339 function inside C<ber_decode> (and of course also which encoder functions
340 are used in C<ber_encode>).
341
342 This works by mapping specific class/tag combinations to an internal "ber
343 type".
344
345 The default profile supports the standard ASN.1 types, but no
346 application-specific ones. This means that class/tag combinations not in
347 the base set of ASN.1 are decoded into their raw octet strings.
348
349 C<Convert::BER::XS> defines two profile variables you can use out of the box:
350
351 =over
352
353 =item C<$Convert::BER::XS::DEFAULT_PROFILE>
354
355 This is the default profile, i.e. the profile that is used when no
356 profile is specified for de-/encoding.
357
358 You can modify it, but remember that this modifies the defaults for all
359 callers that rely on the default profile.
360
361 =item C<$Convert::BER::XS::SNMP_PROFILE>
362
363 A profile with mappings for SNMP-specific application tags added. This is
364 useful when de-/encoding SNMP data.
365
366 Example:
367
368 $ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;
369
370 =back
371
372 =head2 The Convert::BER::XS::Profile class
373
374 =over
375
376 =item $profile = new Convert::BER::XS::Profile
377
378 Create a new profile. The profile will be identical to the default
379 profile.
380
381 =item $profile->set ($class, $tag, $type)
382
383 Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
384 which must be one of the C<BER_TYPE_*> constants.
385
386 Note that currently, the mapping is stored in a flat array, so large
387 values of C<$tag> will consume large amounts of memory.
388
389 Example:
390
391 $profile = new Convert::BER::XS::Profile;
392 $profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
393 $ber = ber_decode $data, $profile;
394
395 =item $type = $profile->get ($class, $tag)
396
397 Returns the BER type mapped to the given C<$class>/C<$tag> combination.
398
399 =back
400
401 =head2 BER TYPES
402
403 This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
404 combination to any C<BER_TYPE_*>.
405
406 =over
407
408 =item C<BER_TYPE_BYTES>
409
410 The raw octets of the value. This is the default type for unknown tags and
411 de-/encodes the value as if it were an octet string, i.e. by copying the
412 raw bytes.
413
414 =item C<BER_TYPE_UTF8>
415
416 Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
417 (without validation!) and encodes a perl unicode string into a UTF-8 BER
418 string.
419
420 =item C<BER_TYPE_UCS2>
421
422 Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
423 string.
424
425 =item C<BER_TYPE_UCS4>
426
427 Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
428 string.
429
430 =item C<BER_TYPE_INT>
431
432 Encodes and decodes a BER integer value to a perl integer scalar. This
433 should correctly handle 64 bit signed and unsigned values.
434
435 =item C<BER_TYPE_OID>
436
437 Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
438 dot, e.g. C<1.3.6.1.213>.
439
440 =item C<BER_TYPE_RELOID>
441
442 Same as C<BER_TYPE_OID> but uses relative object identifier
443 encoding: ASN.1 has this hack of encoding the first two OID components
444 into a single integer in a weird attempt to save an insignificant amount
445 of space in an otherwise wasteful encoding, and relative OIDs are
446 basically OIDs without this hack. The practical difference is that the
447 second component of an OID can only have the values 1..40, while relative
448 OIDs do not have this restriction.
449
450 =item C<BER_TYPE_NULL>
451
452 Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
453 C<ASN_NULL> type, regardless of the perl value.
454
455 =item C<BER_TYPE_BOOL>
456
457 Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
458 boolean value into an C<ASN_BOOLEAN>.
459
460 =item C<BER_TYPE_REAL>
461
462 Decodes/encodes a BER real value. NOT IMPLEMENTED.
463
464 =item C<BER_TYPE_IPADDRESS>
465
466 Decodes/encodes a four byte string into an IPv4 dotted-quad address string
467 in Perl. Given the obsolete nature of this type, this is a low-effort
468 implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
469 so it won't handle all string forms supported by C<inet_aton> for example.
470
471 =item C<BER_TYPE_CROAK>
472
473 Always croaks when encountered during encoding or decoding - the
474 default behaviour when encountering an unknown type is to treat it as
475 C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
476 error for some types, then C<BER_TYPE_CROAK> is for you.
477
478 =back
479
480 =cut
481
482 our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
483 our $SNMP_PROFILE = new Convert::BER::XS::Profile;
484
485 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
486 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
487 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
488 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
489 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE , BER_TYPE_IPADDRESS);
490 $SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);
491
492 $DEFAULT_PROFILE->_set_default;
493
494 1;
495
496 =head2 LIMITATIONS
497
498 This module can only en-/decode 64 bit signed and unsigned integers, and
499 only when your perl supports those.
500
501 OBJECT IDENTIFIEERs cannot have unlimited length, although the limit is
502 much larger than e.g. the one imposed by SNMP or other protocols.
503
504 REAL values are not supported and will croak.
505
506 This module has undergone little to no testing so far.
507
508 =head2 ITHREADS SUPPORT
509
510 This module is unlikely to work when the (officially discouraged) ithreads
511 are in use.
512
513 =head1 AUTHOR
514
515 Marc Lehmann <schmorp@schmorp.de>
516 http://software.schmorp.de/pkg/Convert-BER-XS
517
518 =cut
519