ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Convert-BER-XS/XS.pm
(Generate patch)

Comparing Convert-BER-XS/XS.pm (file contents):
Revision 1.8 by root, Fri Apr 19 20:42:31 2019 UTC vs.
Revision 1.20 by root, Sat Apr 20 14:47:23 2019 UTC

4 4
5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use Convert::BER::XS ':all'; 7 use Convert::BER::XS ':all';
8 8
9 my $ber = ber_decode $buf 9 my $ber = ber_decode $buf, $Convert::BER::XS::SNMP_PROFILE
10 or die "unable to decode SNMP message"; 10 or die "unable to decode SNMP message";
11 11
12 # The above results in a data structure consisting of (class, tag, 12 # The above results in a data structure consisting of
13 # (class, tag, # constructed, data)
13 # constructed, data) tuples. Below is such a message, SNMPv1 trap 14 # tuples. Below is such a message, SNMPv1 trap
14 # with a Cisco mac change notification. 15 # with a Cisco mac change notification.
15 # Did you know that Cisco is in the news almost every week because 16 # Did you know that Cisco is in the news almost
17 # every week because of some backdoor password
16 # of some backdoor password or other extremely stupid security bug? 18 # or other extremely stupid security bug?
17 19
18 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, 20 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
19 [ 21 [
20 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1 22 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
21 [ ASN_UNIVERSAL, 4, 0, "public" ], # community 23 [ ASN_UNIVERSAL, 4, 0, "public" ], # community
22 [ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU 24 [ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
23 [ 25 [
24 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid 26 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
25 [ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1 27 [ ASN_APPLICATION, SNMP_IPADDRESS, 0, "10.0.0.1" ], # SNMP IpAddress
26 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap 28 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
27 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap 29 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
28 [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks 30 [ ASN_APPLICATION, SNMP_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
29 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist 31 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
30 [ 32 [
31 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair 33 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
32 [ 34 [
33 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ], 35 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
60 ) { 62 ) {
61 ... and so on 63 ... and so on
62 64
63 # finally, let's encode it again and hope it results in the same bit pattern 65 # finally, let's encode it again and hope it results in the same bit pattern
64 66
65 my $buf = ber_encode $ber; 67 my $buf = ber_encode $ber, $Convert::BER::XS::SNMP_PROFILE;
66 68
67=head1 DESCRIPTION 69=head1 DESCRIPTION
68 70
69WARNING: Before release 1.0, the API is not considered stable in any way. 71WARNING: Before release 1.0, the API is not considered stable in any way.
70 72
71This module implements a I<very> low level BER/DER en-/decoder. 73This module implements a I<very> low level BER/DER en-/decoder.
72 74
73If is tuned for low memory and high speed, while still maintaining some 75It is tuned for low memory and high speed, while still maintaining some
74level of user-friendlyness. 76level of user-friendlyness.
75 77
76Currently, not much is documented, as this is an initial release to 78=head2 EXPORT TAGS AND CONSTANTS
77reserve CPAN namespace, stay tuned for a few days. 79
80By default this module doesn't export any symbols, but if you don't want
81to break your keyboard, editor or eyesigh with extreemly long names, I
82recommend importing the C<:all> tag. Still, you can selectively import
83things:
84
85=over
86
87=item :all
88
89All of the below. Really. Rcommended for at least first steps, or if you
90don't care about a few kilobytes of wasted memory (and namespace).
91
92=item :const
93
94All of the stricly ASN.1-related constants defined by this module, the
95same as C<:const_asn :const_index>. Notably, this does not contain
96C<:const_ber_type> and C<:const_snmp>.
97
98A good set to get everything you need to decode and match BER data would be
99C<:decode :const>.
100
101=item C<:const_index>>
102
103The BER tuple array index constants:
104
105 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
106
107=item C<:const_asn>
108
109ASN class values (these are C<0>, C<1>, C<2> and C<3>, reespectively -
110exactly thw two topmost bits from the identifdier octet shifted 6 bits to
111the right):
112
113 ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
114
115ASN tag values (some of which are aliases, such as C<ASN_OID>). Their
116numerical value corresponds exactly to the numbers used in BER/X.690.
117
118 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
119 ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
120 ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
121 ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
122 ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
123 ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
124
125=item C<:const_ber_type>
126
127The BER type constants, explained in the PROFILES section.
128
129 BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
130 BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
131 BER_TYPE_IPADDRESS BER_TYPE_CROAK
132
133=item C<:const_snmp>
134
135Constants only relevant to SNMP. These are the tag values used by SNMP in
136the C<ASN_APPLICATION> namespace and have the exact numerical value as in
137BER/RFC 2578.
138
139 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
140
141=item C<:decode>
142
143C<ber_decode> and the match helper functions:
144
145 ber_decode ber_is ber_is_seq ber_is_i32 ber_is_oid
146
147=item C<:encode>
148
149C<ber_encode> and the construction helper functions:
150
151 ber_encode ber_i32
152
153=back
78 154
79=head2 ASN.1/BER/DER/... BASICS 155=head2 ASN.1/BER/DER/... BASICS
80 156
81ASN.1 is a strange language that can be sed to describe protocols and 157ASN.1 is a strange language that can be used to describe protocols and
82data structures. It supports various mappings to JSON, XML, but most 158data structures. It supports various mappings to JSON, XML, but most
83importantly, to a various binary encodings such as BER, that is the topic 159importantly, to a various binary encodings such as BER, that is the topic
84of this module, and is used in SNMP or LDAP for example. 160of this module, and is used in SNMP or LDAP for example.
85 161
86While ASN.1 defines a schema that is useful to interpret encoded data, 162While ASN.1 defines a schema that is useful to interpret encoded data,
87the BER encoding is actually somehat self-describing: you might not know 163the BER encoding is actually somewhat self-describing: you might not know
88whether something is a string or a number or a sequence or something else, 164whether something is a string or a number or a sequence or something else,
89but you can nevertheless decode the overall structure, even if you end up 165but you can nevertheless decode the overall structure, even if you end up
90with just a binary blob for the actual value. 166with just a binary blob for the actual value.
91 167
92This works because BER values are tagged with a type and a namespace, 168This works because BER values are tagged with a type and a namespace,
93and also have a flag that says whther a value consists of subvalues (is 169and also have a flag that says whether a value consists of subvalues (is
94"constructed") or not (is "primitive"). 170"constructed") or not (is "primitive").
95 171
96Tags are simple integers, and ASN.1 defines a somewhat weird assortment of 172Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
97those - for example, you have 32 bit signed integers and 16(!) different 173those - for example, you have 32 bit signed integers and 16(!) different
98string types, but there is no unsigned32 type for example. Different 174string types, but there is no unsigned32 type for example. Different
120I<DATA> member, and you may re-assign the array itself, e.g.: 196I<DATA> member, and you may re-assign the array itself, e.g.:
121 197
122 $ber = ber_decode $binbuf; 198 $ber = ber_decode $binbuf;
123 199
124 # the following is NOT legal: 200 # the following is NOT legal:
125 $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, readonly(!) 201 $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)
126 202
127 # but all of the following are fine: 203 # but all of the following are fine:
128 $ber->[BER_DATA] = "string"; 204 $ber->[BER_DATA] = "string";
129 $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123]; 205 $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
130 @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 1000); 206 @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);
131 207
132I<CLASS> is something like a namespace for I<TAG>s - there is the 208I<CLASS> is something like a namespace for I<TAG>s - there is the
133C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1 209C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
134implementations, the C<ASN_APPLICATION> namespace which defines tags for 210implementations, the C<ASN_APPLICATION> namespace which defines tags for
135specific applications (for example, the SNMP C<Unsigned32> type is in this 211specific applications (for example, the SNMP C<Unsigned32> type is in this
249true. 325true.
250 326
251=item $bool = ber_is_oid $tuple, $oid_string 327=item $bool = ber_is_oid $tuple, $oid_string
252 328
253Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER 329Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
254that exactly matches C$oid_string>. Exmaple: 330that exactly matches C<$oid_string>. Example:
255 331
256 ber_is_oid $tuple, "1.3.6.1.4" 332 ber_is_oid $tuple, "1.3.6.1.4"
257 or die "oid must be 1.3.6.1.4"; 333 or die "oid must be 1.3.6.1.4";
258 334
259=item $oid = ber_is_oid $tuple 335=item $oid = ber_is_oid $tuple
287use common::sense; 363use common::sense;
288 364
289use XSLoader (); 365use XSLoader ();
290use Exporter qw(import); 366use Exporter qw(import);
291 367
292our $VERSION = 0.2; 368our $VERSION;
293 369
370BEGIN {
371 $VERSION = 0.8;
294XSLoader::load __PACKAGE__, $VERSION; 372 XSLoader::load __PACKAGE__, $VERSION;
373}
295 374
296our %EXPORT_TAGS = ( 375our %EXPORT_TAGS = (
297 const => [qw( 376 const_index => [qw(
298 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA 377 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
299 378 )],
379 const_asn => [qw(
300 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER ASN_TAG_BER ASN_TAG_MASK 380 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
301 ASN_CONSTRUCTED ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE ASN_CLASS_MASK ASN_CLASS_SHIFT 381 ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
302 ASN_SEQUENCE 382 ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
303 383 ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
384 ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
385 ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
386
387 ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
388 )],
389 const_ber_type => [qw(
390 BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
391 BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
392 BER_TYPE_IPADDRESS BER_TYPE_CROAK
393 )],
394 const_snmp => [qw(
304 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64 395 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
305 )], 396 )],
306 encode => [qw( 397 decode => [qw(
307 ber_decode 398 ber_decode
308 ber_is ber_is_seq ber_is_i32 ber_is_oid 399 ber_is ber_is_seq ber_is_i32 ber_is_oid
309 )], 400 )],
310 decode => [qw( 401 encode => [qw(
311 ber_encode 402 ber_encode
403 ber_i32
312 )], 404 )],
313); 405);
314 406
315our @EXPORT_OK = map @$_, values %EXPORT_TAGS; 407our @EXPORT_OK = map @$_, values %EXPORT_TAGS;
316 408
317$EXPORT_TAGS{all} = \@EXPORT_OK; 409$EXPORT_TAGS{all} = \@EXPORT_OK;
410$EXPORT_TAGS{const} = [map @{ $EXPORT_TAGS{$_} }, qw(const_index const_asn)];
411use Data::Dump; ddx \%EXPORT_TAGS;
412
413=head1 PROFILES
414
415While any BER data can be correctly encoded and decoded out of the box, it
416can be inconvenient to have to manually decode some values into a "better"
417format: for instance, SNMP TimeTicks values are decoded into the raw octet
418strings of their BER representation, which is quite hard to decode. With
419profiles, you can change which class/tag combinations map to which decoder
420function inside C<ber_decode> (and of course also which encoder functions
421are used in C<ber_encode>).
422
423This works by mapping specific class/tag combinations to an internal "ber
424type".
425
426The default profile supports the standard ASN.1 types, but no
427application-specific ones. This means that class/tag combinations not in
428the base set of ASN.1 are decoded into their raw octet strings.
429
430C<Convert::BER::XS> defines two profile variables you can use out of the box:
431
432=over
433
434=item C<$Convert::BER::XS::DEFAULT_PROFILE>
435
436This is the default profile, i.e. the profile that is used when no
437profile is specified for de-/encoding.
438
439You can modify it, but remember that this modifies the defaults for all
440callers that rely on the default profile.
441
442=item C<$Convert::BER::XS::SNMP_PROFILE>
443
444A profile with mappings for SNMP-specific application tags added. This is
445useful when de-/encoding SNMP data.
446
447Example:
448
449 $ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;
450
451=back
452
453=head2 The Convert::BER::XS::Profile class
454
455=over
456
457=item $profile = new Convert::BER::XS::Profile
458
459Create a new profile. The profile will be identical to the default
460profile.
461
462=item $profile->set ($class, $tag, $type)
463
464Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
465which must be one of the C<BER_TYPE_*> constants.
466
467Note that currently, the mapping is stored in a flat array, so large
468values of C<$tag> will consume large amounts of memory.
469
470Example:
471
472 $profile = new Convert::BER::XS::Profile;
473 $profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
474 $ber = ber_decode $data, $profile;
475
476=item $type = $profile->get ($class, $tag)
477
478Returns the BER type mapped to the given C<$class>/C<$tag> combination.
479
480=back
481
482=head2 BER TYPES
483
484This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
485combination to any C<BER_TYPE_*>.
486
487=over
488
489=item C<BER_TYPE_BYTES>
490
491The raw octets of the value. This is the default type for unknown tags and
492de-/encodes the value as if it were an octet string, i.e. by copying the
493raw bytes.
494
495=item C<BER_TYPE_UTF8>
496
497Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
498(without validation!) and encodes a perl unicode string into a UTF-8 BER
499string.
500
501=item C<BER_TYPE_UCS2>
502
503Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
504string.
505
506=item C<BER_TYPE_UCS4>
507
508Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
509string.
510
511=item C<BER_TYPE_INT>
512
513Encodes and decodes a BER integer value to a perl integer scalar. This
514should correctly handle 64 bit signed and unsigned values.
515
516=item C<BER_TYPE_OID>
517
518Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
519dot, e.g. C<1.3.6.1.213>.
520
521=item C<BER_TYPE_RELOID>
522
523Same as C<BER_TYPE_OID> but uses relative object identifier
524encoding: ASN.1 has this hack of encoding the first two OID components
525into a single integer in a weird attempt to save an insignificant amount
526of space in an otherwise wasteful encoding, and relative OIDs are
527basically OIDs without this hack. The practical difference is that the
528second component of an OID can only have the values 1..40, while relative
529OIDs do not have this restriction.
530
531=item C<BER_TYPE_NULL>
532
533Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
534C<ASN_NULL> type, regardless of the perl value.
535
536=item C<BER_TYPE_BOOL>
537
538Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
539boolean value into an C<ASN_BOOLEAN>.
540
541=item C<BER_TYPE_REAL>
542
543Decodes/encodes a BER real value. NOT IMPLEMENTED.
544
545=item C<BER_TYPE_IPADDRESS>
546
547Decodes/encodes a four byte string into an IPv4 dotted-quad address string
548in Perl. Given the obsolete nature of this type, this is a low-effort
549implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
550so it won't handle all string forms supported by C<inet_aton> for example.
551
552=item C<BER_TYPE_CROAK>
553
554Always croaks when encountered during encoding or decoding - the
555default behaviour when encountering an unknown type is to treat it as
556C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
557error for some types, then C<BER_TYPE_CROAK> is for you.
558
559=back
560
561=cut
562
563our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
564our $SNMP_PROFILE = new Convert::BER::XS::Profile;
565
566# additional SNMP application types
567$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
568$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
569$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
570$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
571$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE , BER_TYPE_IPADDRESS);
572$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);
573
574$DEFAULT_PROFILE->_set_default;
318 575
3191; 5761;
320 577
321=head2 BUGS / SHORTCOMINGs 578=head2 LIMITATIONS/NOTES
322 579
323This module does have a number of SNMPisms hardcoded, such as the SNMP 580This module can only en-/decode 64 bit signed and unsigned integers, and
324tags for Unsigned32 and so on. More configurability is needed, and, if 581only when your perl supports those.
325ever implemented, will come in a form similar to how L<JSON::XS> and 582
326L<CBOR::XS> respresent things, namely with an object-oriented interface. 583This module does not generally care about ranges, i.e. it will happily
584de-/encode 64 bit integers into an C<ASN_INTEGER32> value, or a negative
585number into an C<SNMP_COUNTER64>.
586
587OBJECT IDENTIFIEERs cannot have unlimited length, although the limit is
588much larger than e.g. the one imposed by SNMP or other protocols,a nd is
589about 4kB.
590
591REAL values are not supported and will currently croak.
592
593This module has undergone little to no testing so far.
594
595=head2 ITHREADS SUPPORT
596
597This module is unlikely to work when the (officially discouraged) ithreads
598are in use.
327 599
328=head1 AUTHOR 600=head1 AUTHOR
329 601
330 Marc Lehmann <schmorp@schmorp.de> 602 Marc Lehmann <schmorp@schmorp.de>
331 http://software.schmorp.de/pkg/Convert-BER-XS 603 http://software.schmorp.de/pkg/Convert-BER-XS

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines