ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Convert-BER-XS/XS.pm
(Generate patch)

Comparing Convert-BER-XS/XS.pm (file contents):
Revision 1.1 by root, Fri Apr 19 16:19:36 2019 UTC vs.
Revision 1.13 by root, Sat Apr 20 01:03:59 2019 UTC

1=head1 NAME 1=head1 NAME
2 2
3Convert::BER::XS - I<very> low level BER decoding 3Convert::BER::XS - I<very> low level BER en-/decoding
4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use Convert::BER::XS ':all'; 7 use Convert::BER::XS ':all';
8 8
9 my $ber = ber_decode $buf 9 my $ber = ber_decode $buf
10 or die "unable to decode SNMP v1/v2c Message"; 10 or die "unable to decode SNMP message";
11
12 # The above results in a data structure consisting of
13 # (class, tag, # constructed, data)
14 # tuples. Below is such a message, SNMPv1 trap
15 # with a Cisco mac change notification.
16 # Did you know that Cisco is in the news almost
17 # every week because # of some backdoor password
18 # or other extremely stupid security bug?
19
20 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
21 [
22 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
23 [ ASN_UNIVERSAL, 4, 0, "public" ], # community
24 [ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
25 [
26 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
27 [ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1
28 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
29 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
30 [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
31 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
32 [
33 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
34 [
35 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
36 [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
37 ]
38 ]
39 ],
40 ...
41
42 # let's decode it a bit with some helper functions
11 43
12 my $msg = ber_is_seq $ber 44 my $msg = ber_is_seq $ber
13 or die "SNMP message does not start with a sequence"; 45 or die "SNMP message does not start with a sequence";
14 46
15 ber_is $msg->[0], ASN_UNIVERSAL, ASN_INTEGER32, 0 47 ber_is $msg->[0], ASN_UNIVERSAL, ASN_INTEGER32, 0
16 or die "SNMP message does not start with snmp version\n"; 48 or die "SNMP message does not start with snmp version\n";
17 49
50 # message is SNMP v1 or v2c?
18 if ($msg->[0][BER_DATA] == 0 || $msg->[0][BER_DATA] == 1) { 51 if ($msg->[0][BER_DATA] == 0 || $msg->[0][BER_DATA] == 1) {
52
19 # message is SNMP v1 or v2c 53 # message is v1 trap?
20
21 if (ber_is $msg->[2], ASN_CONTEXT, 4, 1) { 54 if (ber_is $msg->[2], ASN_CONTEXT, 4, 1) {
22 # message is v1 trap
23 my $trap = $msg->[2][BER_DATA]; 55 my $trap = $msg->[2][BER_DATA];
24 56
25 # check whether trap is a cisco mac notification mac changed message 57 # check whether trap is a cisco mac notification mac changed message
26 if ( 58 if (
27 (ber_is_oid $trap->[0], "1.3.6.1.4.1.9.9.215.2") # cmnInterfaceObjects 59 (ber_is_oid $trap->[0], "1.3.6.1.4.1.9.9.215.2") # cmnInterfaceObjects
28 and (ber_is_i32 $trap->[2], 6) 60 and (ber_is_i32 $trap->[2], 6)
29 and (ber_is_i32 $trap->[3], 1) # mac changed msg 61 and (ber_is_i32 $trap->[3], 1) # mac changed msg
30 ) { 62 ) {
31 ... and so on 63 ... and so on
32 64
65 # finally, let's encode it again and hope it results in the same bit pattern
66
67 my $buf = ber_encode $ber;
68
33=head1 DESCRIPTION 69=head1 DESCRIPTION
34 70
71WARNING: Before release 1.0, the API is not considered stable in any way.
72
35This module implements a I<very> low level BER/DER decoder, and in the 73This module implements a I<very> low level BER/DER en-/decoder.
36future, probably also an encoder (tell me if you want an encoder, this
37might speed up the process of getting one).
38 74
39If is tuned for low memory and high speed, while still maintaining some 75If is tuned for low memory and high speed, while still maintaining some
40level of user-friendlyness. 76level of user-friendlyness.
41 77
42Currently, not much is documented, as this is an initial release to 78Currently, not much is documented, as this is an initial release to
43reserve CPAN namespace, stay tuned for a few days. 79reserve CPAN namespace, stay tuned for a few days.
44 80
81=head2 ASN.1/BER/DER/... BASICS
82
83ASN.1 is a strange language that can be sed to describe protocols and
84data structures. It supports various mappings to JSON, XML, but most
85importantly, to a various binary encodings such as BER, that is the topic
86of this module, and is used in SNMP or LDAP for example.
87
88While ASN.1 defines a schema that is useful to interpret encoded data,
89the BER encoding is actually somewhat self-describing: you might not know
90whether something is a string or a number or a sequence or something else,
91but you can nevertheless decode the overall structure, even if you end up
92with just a binary blob for the actual value.
93
94This works because BER values are tagged with a type and a namespace,
95and also have a flag that says whther a value consists of subvalues (is
96"constructed") or not (is "primitive").
97
98Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
99those - for example, you have 32 bit signed integers and 16(!) different
100string types, but there is no unsigned32 type for example. Different
101applications work around this in different ways, for example, SNMP defines
102application-specific Gauge32, Counter32 and Unsigned32, which are mapped
103to two different tags: you can distinguish between Counter32 and the
104others, but not between Gause32 and Unsigned32, without the ASN.1 schema.
105
106Ugh.
107
108=head2 DECODED BER REPRESENTATION
109
110This module represents every BER value as a 4-element tuple (actually an
111array-reference):
112
113 [CLASS, TAG, CONSTRUCTED, DATA]
114
115To avoid non-descriptive hardcoded array index numbers, this module
116defines symbolic constants to access these members: C<BER_CLASS>,
117C<BER_TAG>, C<BER_CONSTRUCTED> and C<BER_DATA>.
118
119Also, the first three members are integers with a little caveat: for
120performance reasons, these are readonly and shared, so you must not modify
121them (increment, assign to them etc.) in any way. You may modify the
122I<DATA> member, and you may re-assign the array itself, e.g.:
123
124 $ber = ber_decode $binbuf;
125
126 # the following is NOT legal:
127 $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)
128
129 # but all of the following are fine:
130 $ber->[BER_DATA] = "string";
131 $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
132 @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);
133
134I<CLASS> is something like a namespace for I<TAG>s - there is the
135C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
136implementations, the C<ASN_APPLICATION> namespace which defines tags for
137specific applications (for example, the SNMP C<Unsigned32> type is in this
138namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g.
139for C<CHOICE>) and a private namespace (C<ASN_PRIVATE>).
140
141The meaning of the I<TAG> depends on the namespace, and defines a
142(partial) interpretation of the data value. For example, right now, SNMP
143application namespace knowledge ix hardcoded into this module, so it
144knows that SNMP C<Unsigned32> values need to be decoded into actual perl
145integers.
146
147The most common tags in the C<ASN_UNIVERSAL> namespace are
148C<ASN_INTEGER32>, C<ASN_BIT_STRING>, C<ASN_NULL>, C<ASN_OCTET_STRING>,
149C<ASN_OBJECT_IDENTIFIER>, C<ASN_SEQUENCE>, C<ASN_SET> and
150C<ASN_IA5_STRING>.
151
152The most common tags in SNMP's C<ASN_APPLICATION> namespace
153are C<SNMP_IPADDRESS>, C<SNMP_COUNTER32>, C<SNMP_UNSIGNED32>,
154C<SNMP_TIMETICKS>, C<SNMP_OPAQUE> and C<SNMP_COUNTER64>.
155
156The I<CONSTRUCTED> flag is really just a boolean - if it is false, the
157the value is "primitive" and contains no subvalues, kind of like a
158non-reference perl scalar. IF it is true, then the value is "constructed"
159which just means it contains a list of subvalues which this module will
160en-/decode as BER tuples themselves.
161
162The I<DATA> value is either a reference to an array of further tuples (if
163the value is I<CONSTRUCTED>), some decoded representation of the value,
164if this module knows how to decode it (e.g. for the integer types above)
165or a binary string with the raw octets if this module doesn't know how to
166interpret the namespace/tag.
167
168Thus, you can always decode a BER data structure and at worst you get a
169string in place of some nice decoded value.
170
171See the SYNOPSIS for an example of such an encoded tuple representation.
172
173=head2 DECODING AND ENCODING
174
175=over
176
177=item $tuple = ber_decoded $bindata
178
179Decodes binary BER data in C<$bindata> and returns the resulting BER
180tuple. Croaks on any decoding error, so the returned C<$tuple> is always
181valid.
182
183=item $bindata = ber_encode $tuple
184
185Encodes the BER tuple into a BER/DER data structure.
186
187=back
188
189=head2 HELPER FUNCTIONS
190
191Working with a 4-tuple for every value can be annoying. Or, rather, I<is>
192annoying. To reduce this a bit, this module defines a number of helper
193functions, both to match BER tuples and to conmstruct BER tuples:
194
195=head3 MATCH HELPERS
196
197Thse functions accept a BER tuple as first argument and either paertially
198or fully match it. They often come in two forms, one which exactly matches
199a value, and one which only matches the type and returns the value.
200
201They do check whether valid tuples are passed in and croak otherwise. As
202a ease-of-use exception, they usually also accept C<undef> instead of a
203tuple reference. in which case they silently fail to match.
204
205=over
206
207=item $bool = ber_is $tuple, $class, $tag, $constructed, $data
208
209This takes a BER C<$tuple> and matches its elements agains the privded
210values, all of which are optional - values that are either missing or
211C<undef> will be ignored, the others will be matched exactly (e.g. as if
212you used C<==> or C<eq> (for C<$data>)).
213
214Some examples:
215
216 ber_is $tuple, ASN_UNIVERSAL, ASN_SEQUENCE, 1
217 orf die "tuple is not an ASN SEQUENCE";
218
219 ber_is $tuple, ASN_UNIVERSAL, ASN_NULL
220 or die "tuple is not an ASN NULL value";
221
222 ber_is $tuple, ASN_UNIVERSAL, ASN_INTEGER32, 0, 50
223 or die "BER integer must be 50";
224
225=item $seq = ber_is_seq $tuple
226
227Returns the sequence members (the array of subvalues) if the C<$tuple> is
228an ASN SEQUENCE, i.e. the C<BER_DATA> member. If the C<$tuple> is not a
229sequence it returns C<undef>. For example, SNMP version 1/2c/3 packets all
230consist of an outer SEQUENCE value:
231
232 my $ber = ber_decode $snmp_data;
233
234 my $snmp = ber_is_seq $ber
235 or die "SNMP packet invalid: does not start with SEQUENCE";
236
237 # now we know $snmp is a sequence, so decode the SNMP version
238
239 my $version = ber_is_i32 $snmp->[0]
240 or die "SNMP packet invalid: does not start with version number";
241
242=item $bool = ber_is_i32 $tuple, $i32
243
244Returns a true value if the C<$tuple> represents an ASN INTEGER32 with
245the value C<$i32>.
246
247=item $i32 = ber_is_i32 $tuple
248
249Returns true (and extracts the integer value) if the C<$tuple> is an ASN
250INTEGER32. For C<0>, this function returns a special value that is 0 but
251true.
252
253=item $bool = ber_is_oid $tuple, $oid_string
254
255Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
256that exactly matches C<$oid_string>. Example:
257
258 ber_is_oid $tuple, "1.3.6.1.4"
259 or die "oid must be 1.3.6.1.4";
260
261=item $oid = ber_is_oid $tuple
262
263Returns true (and extracts the OID string) if the C<$tuple> is an ASN
264OBJECT IDENTIFIER. Otherwise, it returns C<undef>.
265
266=back
267
268=head3 CONSTRUCTION HELPERS
269
270=over
271
272=item $tuple = ber_i32 $value
273
274Constructs a new C<ASN_INTEGER32> tuple.
275
276=back
277
278=head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1>
279
280This module is I<not> the XS version of L<Convert::BER>, but a different
281take at doing the same thing. I imagine this module would be a good base
282for speeding up either of these, or write a similar module, or write your
283own LDAP or SNMP module for example.
284
45=cut 285=cut
46 286
47package Convert::BER::XS; 287package Convert::BER::XS;
48 288
49use common::sense; 289use common::sense;
50 290
51use XSLoader (); 291use XSLoader ();
52use Exporter qw(import); 292use Exporter qw(import);
53 293
294our $VERSION;
295
296BEGIN {
54our $VERSION = '0.0'; 297 $VERSION = 0.7;
55
56XSLoader::load __PACKAGE__, $VERSION; 298 XSLoader::load __PACKAGE__, $VERSION;
299}
57 300
58our %EXPORT_TAGS = ( 301our %EXPORT_TAGS = (
59 all => [qw( 302 const => [qw(
303 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
304
305 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
306 ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
307 ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
308 ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
309 ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
310 ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
311
312 ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
313
314 BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
315 BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
316 BER_TYPE_IPADDRESS BER_TYPE_CROAK
317 )],
318 const_snmp => [qw(
319 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
320 )],
321 encode => [qw(
60 ber_decode 322 ber_decode
61 ber_is ber_is_seq ber_is_i32 ber_is_oid 323 ber_is ber_is_seq ber_is_i32 ber_is_oid
62 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA 324 )],
63 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER ASN_TAG_BER ASN_TAG_MASK 325 decode => [qw(
64 ASN_CONSTRUCTED ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE ASN_CLASS_MASK ASN_CLASS_SHIFT 326 ber_encode
65 ASN_SEQUENCE ASN_IPADDRESS ASN_COUNTER32 ASN_UNSIGNED32 ASN_TIMETICKS ASN_OPAQUE ASN_COUNTER64 327 ber_i32
66 )], 328 )],
67); 329);
68 330
69our @EXPORT_OK = map @$_, values %EXPORT_TAGS; 331our @EXPORT_OK = map @$_, values %EXPORT_TAGS;
70 332
333$EXPORT_TAGS{all} = \@EXPORT_OK;
334
335=head1 PROFILES
336
337While any BER data can be correctly encoded and decoded out of the box, it
338can be inconvenient to have to manually decode some values into a "better"
339format: for instance, SNMP TimeTicks values are decoded into the raw octet
340strings of their BER representation, which is quite hard to decode. With
341profiles, you can change which class/tag combinations map to which decoder
342function inside C<ber_decode> (and of course also which encoder functions
343are used in C<ber_encode>).
344
345This works by mapping specific class/tag combinations to an internal "ber
346type".
347
348The default profile supports the standard ASN.1 types, but no
349application-specific ones. This means that class/tag combinations not in
350the base set of ASN.1 are decoded into their raw octet strings.
351
352C<Convert::BER::XS> defines two profile variables you cna use out of the box:
353
354=over
355
356=item C<$Convert::BER::XS::DEFAULT_PROFILE>
357
358This is the default profile, i.e. the profile that is used when no
359profile is specified for de-/encoding.
360
361You cna modify it, but remember that this modifies the defaults for all
362callers that rely on the defauit profile.
363
364=item C<$Convert::BER::XS::SNMP_PROFILE>
365
366A profile with mappings for SNMP-specific application tags added. This is
367useful when de-/encoding SNMP data.
368
369Example:
370 $ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;
371
372=back
373
374=head2 The Convert::BER::XS::Profile class
375
376=over
377
378=item $profile = new Convert::BER::XS::Profile
379
380Create a new profile. The profile will be identical to the default
381profile.
382
383=item $profile->set ($class, $tag, $type)
384
385Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
386which must be one of the C<BER_TYPE_*> constants.
387
388Note that currently, the mapping is stored in a flat array, so large
389values of C<$tag> will consume large amounts of memory.
390
391Example:
392 $profile = new Convert::BER::XS::Profile;
393 $profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
394 $ber = ber_decode $data, $profile;
395
396=item $type = $profile->get ($class, $tag)
397
398Returns the BER type mapped to the given C<$class>/C<$tag> combination.
399
400=back
401
402=head2 BER TYPES
403
404This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
405combination to any C<BER_TYPE_*>.
406
407=over
408
409=item C<BER_TYPE_BYTES>
410
411The raw octets of the value. This is the default type for unknown tags and
412de-/encodes the value as if it were an octet string, i.e. by copying the
413raw bytes.
414
415=item C<BER_TYPE_UTF8>
416
417Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
418(without validation!) and encodes a perl unicode string into a UTF-8 BER
419string.
420
421=item C<BER_TYPE_UCS2>
422
423Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
424string. NOT IMPLEMENTED.
425
426=item C<BER_TYPE_UCS4>
427
428Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
429string. NOT IMPLEMENTED.
430
431=item C<BER_TYPE_INT>
432
433Encodes and decodes a BER integer value to a perl integer scalar. This
434should correctly handle 64 bit signed and unsigned values.
435
436=item C<BER_TYPE_OID>
437
438Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
439dot, e.g. C<1.3.6.1.213>.
440
441=item C<BER_TYPE_RELOID>
442
443Same as C<BER_TYPE_OID> but uses relative OID encoding: ASN.1 has this
444hack of encoding the first two OID components into a single integer in a
445weird attempt to save an insignificant amount of space in an otherwise
446wasteful encoding, and relative OIDs are basically OIDs without this
447hack. The practical difference is that the second component of an OID
448can only have the values 1..40, while relative OIDs do not have this
449restriction.
450
451=item C<BER_TYPE_NULL>
452
453Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
454C<ASN_NULL> type, regardless of the perl value.
455
456=item C<BER_TYPE_BOOL>
457
458Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
459boolean value into an C<ASN_BOOLEAN>.
460
461=item C<BER_TYPE_REAL>
462
463Decodes/encodes a BER real value. NOT IMPLEMENTED.
464
465=item C<BER_TYPE_IPADDRESS>
466
467Decodes/encodes a four byte string into an IOv4 dotted-quad address string
468in perl. Given ther obsolete nature of this type, this is a low-effort
469implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
470so it won't handle all string forms supported by C<inet_aton>.
471
472=item C<BER_TYPE_CROAK>
473
474Always croaks when encountered during encoding or decoding - the
475default behaviour when encountering an unknown type is to treat it as
476C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
477error for some types, then CyBER_TYPE_CROAK> is for you.
478
479=back
480
481=cut
482
483our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
484our $SNMP_PROFILE = new Convert::BER::XS::Profile;
485
486$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
487$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
488$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
489$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
490$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE , BER_TYPE_IPADDRESS);
491$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);
492
493$DEFAULT_PROFILE->_set_default;
494
711; 4951;
496
497=head2 LIMITATIONS
498
499This module can only en-/decode 64 bit signed and unsigned integers, and
500only when your perl supports those.
501
502OBJECT IDENTIFIEERS cannot have unlimited length, although the limit is
503much larger than e.g. the one imposed by SNMP or other protocols.
72 504
73=head1 AUTHOR 505=head1 AUTHOR
74 506
75 Marc Lehmann <schmorp@schmorp.de> 507 Marc Lehmann <schmorp@schmorp.de>
76 http://software.schmorp.de/pkg/Convert-BER-XS 508 http://software.schmorp.de/pkg/Convert-BER-XS

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines