ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Convert-BER-XS/XS.pm
(Generate patch)

Comparing Convert-BER-XS/XS.pm (file contents):
Revision 1.6 by root, Fri Apr 19 20:38:38 2019 UTC vs.
Revision 1.19 by root, Sat Apr 20 14:45:03 2019 UTC

7 use Convert::BER::XS ':all'; 7 use Convert::BER::XS ':all';
8 8
9 my $ber = ber_decode $buf 9 my $ber = ber_decode $buf
10 or die "unable to decode SNMP message"; 10 or die "unable to decode SNMP message";
11 11
12 # The above results in a data structure consisting of (class, tag, 12 # The above results in a data structure consisting of
13 # (class, tag, # constructed, data)
13 # constructed, data) tuples. Below is such a message, SNMPv1 trap 14 # tuples. Below is such a message, SNMPv1 trap
14 # with a Cisco mac change notification. 15 # with a Cisco mac change notification.
15 # Did you know that Cisco is in the news almost every week because 16 # Did you know that Cisco is in the news almost
17 # every week because # of some backdoor password
16 # of some backdoor password or other extremely stupid security bug? 18 # or other extremely stupid security bug?
17 19
18 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, 20 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
19 [ 21 [
20 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1 22 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
21 [ ASN_UNIVERSAL, 4, 0, "public" ], # community 23 [ ASN_UNIVERSAL, 4, 0, "public" ], # community
28 [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks 30 [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
29 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist 31 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
30 [ 32 [
31 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair 33 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
32 [ 34 [
33 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ], # the oid 35 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
34 [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value 36 [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
35 ] 37 ]
36 ] 38 ]
37 ], 39 ],
38 ... 40 ...
64 66
65 my $buf = ber_encode $ber; 67 my $buf = ber_encode $ber;
66 68
67=head1 DESCRIPTION 69=head1 DESCRIPTION
68 70
71WARNING: Before release 1.0, the API is not considered stable in any way.
72
69This module implements a I<very> low level BER/DER en-/decoder. 73This module implements a I<very> low level BER/DER en-/decoder.
70 74
71If is tuned for low memory and high speed, while still maintaining some 75If is tuned for low memory and high speed, while still maintaining some
72level of user-friendlyness. 76level of user-friendlyness.
73 77
74Currently, not much is documented, as this is an initial release to 78=head2 EXPORT TAGS AND CONSTANTS
75reserve CPAN namespace, stay tuned for a few days. 79
80By default this module doesn't export any symbols, but if you don't want
81to break your keyboard, editor or eyesigh with extreemly long names, I
82recommend importing the C<:all> tag. Still, you can selectively import
83things:
84
85=over
86
87=item :all
88
89All of the below. Really. Rcommended for at least first steps, or if you
90don't care about a few kilobytes of wasted memory (and namespace).
91
92=item :const
93
94All of the stricly ASN.1-related constants defined by this module, the
95same as C<:const_asn :const_index>. Notably, this does not contain
96C<:const_ber_type> and C<:const_snmp>.
97
98A good set to get everything you need to decode and match BER data would be
99C<:decode :const>.
100
101=item C<:const_index>>
102
103The BER tuple array index constants:
104
105 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
106
107=item C<:const_asn>
108
109ASN class values (these are C<0>, C<1>, C<2> and C<3>, reespectively -
110exactly thw two topmost bits from the identifdier octet shifted 6 bits to
111the right):
112
113 ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
114
115ASN tag values (some of which are aliases, such as C<ASN_OID>). Their
116numerical value corresponds exactly to the numbers used in BER/X.690.
117
118 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
119 ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
120 ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
121 ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
122 ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
123 ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
124
125=item C<:const_ber_type>
126
127The BER type constants, explained in the PROFILES section.
128
129 BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
130 BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
131 BER_TYPE_IPADDRESS BER_TYPE_CROAK
132
133=item C<:const_snmp>
134
135Constants only relevant to SNMP. These are the tag values used by SNMP in
136the C<ASN_APPLICATION> namespace and have the exact numerical value as in
137BER/RFC 2578.
138
139 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
140
141=item C<:decode>
142
143C<ber_decode> and the match helper functions:
144
145 ber_decode ber_is ber_is_seq ber_is_i32 ber_is_oid
146
147=item C<:encode>
148
149C<ber_encode> and the construction helper functions:
150
151 ber_encode ber_i32
152
153=back
76 154
77=head2 ASN.1/BER/DER/... BASICS 155=head2 ASN.1/BER/DER/... BASICS
78 156
79ASN.1 is a strange language that can be sed to describe protocols and 157ASN.1 is a strange language that can be used to describe protocols and
80data structures. It supports various mappings to JSON, XML, but most 158data structures. It supports various mappings to JSON, XML, but most
81importantly, to a various binary encodings such as BER, that is the topic 159importantly, to a various binary encodings such as BER, that is the topic
82of this module, and is used in SNMP or LDAP for example. 160of this module, and is used in SNMP or LDAP for example.
83 161
84While ASN.1 defines a schema that is useful to interpret encoded data, 162While ASN.1 defines a schema that is useful to interpret encoded data,
85the BER encoding is actually somehat self-describing: you might not know 163the BER encoding is actually somewhat self-describing: you might not know
86whether something is a string or a number or a sequence or something else, 164whether something is a string or a number or a sequence or something else,
87but you can nevertheless decode the overall structure, even if you end up 165but you can nevertheless decode the overall structure, even if you end up
88with just a binary blob for the actual value. 166with just a binary blob for the actual value.
89 167
90This works because BER values are tagged with a type and a namespace, 168This works because BER values are tagged with a type and a namespace,
91and also have a flag that says whther a value consists of subvalues (is 169and also have a flag that says whether a value consists of subvalues (is
92"constructed") or not (is "primitive"). 170"constructed") or not (is "primitive").
93 171
94Tags are simple integers, and ASN.1 defines a somewhat weird assortment of 172Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
95those - for example, you have 32 bit signed integers and 16(!) different 173those - for example, you have 32 bit signed integers and 16(!) different
96string types, but there is no unsigned32 type for example. Different 174string types, but there is no unsigned32 type for example. Different
118I<DATA> member, and you may re-assign the array itself, e.g.: 196I<DATA> member, and you may re-assign the array itself, e.g.:
119 197
120 $ber = ber_decode $binbuf; 198 $ber = ber_decode $binbuf;
121 199
122 # the following is NOT legal: 200 # the following is NOT legal:
123 $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, readonly(!) 201 $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)
124 202
125 # but all of the following are fine: 203 # but all of the following are fine:
126 $ber->[BER_DATA] = "string"; 204 $ber->[BER_DATA] = "string";
127 $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123]; 205 $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
128 @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 1000); 206 @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);
129 207
130I<CLASS> is something like a namespace for I<TAG>s - there is the 208I<CLASS> is something like a namespace for I<TAG>s - there is the
131C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1 209C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
132implementations, the C<ASN_APPLICATION> namespace which defines tags for 210implementations, the C<ASN_APPLICATION> namespace which defines tags for
133specific applications (for example, the SNMP C<Unsigned32> type is in this 211specific applications (for example, the SNMP C<Unsigned32> type is in this
164Thus, you can always decode a BER data structure and at worst you get a 242Thus, you can always decode a BER data structure and at worst you get a
165string in place of some nice decoded value. 243string in place of some nice decoded value.
166 244
167See the SYNOPSIS for an example of such an encoded tuple representation. 245See the SYNOPSIS for an example of such an encoded tuple representation.
168 246
247=head2 DECODING AND ENCODING
248
249=over
250
251=item $tuple = ber_decoded $bindata
252
253Decodes binary BER data in C<$bindata> and returns the resulting BER
254tuple. Croaks on any decoding error, so the returned C<$tuple> is always
255valid.
256
257=item $bindata = ber_encode $tuple
258
259Encodes the BER tuple into a BER/DER data structure.
260
261=back
262
169=head2 HELPER FUNCTIONS 263=head2 HELPER FUNCTIONS
170 264
171Working with a 4-tuple for every value can be annoying. Or, rather, I<is> 265Working with a 4-tuple for every value can be annoying. Or, rather, I<is>
172annoying. To reduce this a bit, this module defines a number of helper 266annoying. To reduce this a bit, this module defines a number of helper
173functions, both to match BER tuples and to conmstruct BER tuples: 267functions, both to match BER tuples and to conmstruct BER tuples:
231true. 325true.
232 326
233=item $bool = ber_is_oid $tuple, $oid_string 327=item $bool = ber_is_oid $tuple, $oid_string
234 328
235Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER 329Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
236that exactly matches C$oid_string>. Exmaple: 330that exactly matches C<$oid_string>. Example:
237 331
238 ber_is_oid $tuple, "1.3.6.1.4" 332 ber_is_oid $tuple, "1.3.6.1.4"
239 or die "oid must be 1.3.6.1.4"; 333 or die "oid must be 1.3.6.1.4";
240 334
241=item $oid = ber_is_oid $tuple 335=item $oid = ber_is_oid $tuple
269use common::sense; 363use common::sense;
270 364
271use XSLoader (); 365use XSLoader ();
272use Exporter qw(import); 366use Exporter qw(import);
273 367
274our $VERSION = 0.1; 368our $VERSION;
275 369
370BEGIN {
371 $VERSION = 0.8;
276XSLoader::load __PACKAGE__, $VERSION; 372 XSLoader::load __PACKAGE__, $VERSION;
373}
277 374
278our %EXPORT_TAGS = ( 375our %EXPORT_TAGS = (
279 const => [qw( 376 const_index => [qw(
280 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA 377 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
281 378 )],
379 const_asn => [qw(
282 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER ASN_TAG_BER ASN_TAG_MASK 380 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
283 ASN_CONSTRUCTED ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE ASN_CLASS_MASK ASN_CLASS_SHIFT 381 ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
284 ASN_SEQUENCE 382 ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
285 383 ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
384 ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
385 ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
386
387 ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
388 )],
389 const_ber_type => [qw(
390 BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
391 BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
392 BER_TYPE_IPADDRESS BER_TYPE_CROAK
393 )],
394 const_snmp => [qw(
286 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64 395 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
287 )], 396 )],
288 encode => [qw( 397 decode => [qw(
289 ber_decode 398 ber_decode
290 ber_is ber_is_seq ber_is_i32 ber_is_oid 399 ber_is ber_is_seq ber_is_i32 ber_is_oid
291 )], 400 )],
292 decode => [qw( 401 encode => [qw(
293 ber_encode 402 ber_encode
403 ber_i32
294 )], 404 )],
295); 405);
296 406
297our @EXPORT_OK = map @$_, values %EXPORT_TAGS; 407our @EXPORT_OK = map @$_, values %EXPORT_TAGS;
298 408
299$EXPORT_TAGS{all} = \@EXPORT_OK; 409$EXPORT_TAGS{all} = \@EXPORT_OK;
410$EXPORT_TAGS{const} = [map @{ $EXPORT_TAGS{$_} }, qw(const_index const_asn)];
411use Data::Dump; ddx \%EXPORT_TAGS;
412
413=head1 PROFILES
414
415While any BER data can be correctly encoded and decoded out of the box, it
416can be inconvenient to have to manually decode some values into a "better"
417format: for instance, SNMP TimeTicks values are decoded into the raw octet
418strings of their BER representation, which is quite hard to decode. With
419profiles, you can change which class/tag combinations map to which decoder
420function inside C<ber_decode> (and of course also which encoder functions
421are used in C<ber_encode>).
422
423This works by mapping specific class/tag combinations to an internal "ber
424type".
425
426The default profile supports the standard ASN.1 types, but no
427application-specific ones. This means that class/tag combinations not in
428the base set of ASN.1 are decoded into their raw octet strings.
429
430C<Convert::BER::XS> defines two profile variables you can use out of the box:
431
432=over
433
434=item C<$Convert::BER::XS::DEFAULT_PROFILE>
435
436This is the default profile, i.e. the profile that is used when no
437profile is specified for de-/encoding.
438
439You can modify it, but remember that this modifies the defaults for all
440callers that rely on the default profile.
441
442=item C<$Convert::BER::XS::SNMP_PROFILE>
443
444A profile with mappings for SNMP-specific application tags added. This is
445useful when de-/encoding SNMP data.
446
447Example:
448
449 $ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;
450
451=back
452
453=head2 The Convert::BER::XS::Profile class
454
455=over
456
457=item $profile = new Convert::BER::XS::Profile
458
459Create a new profile. The profile will be identical to the default
460profile.
461
462=item $profile->set ($class, $tag, $type)
463
464Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
465which must be one of the C<BER_TYPE_*> constants.
466
467Note that currently, the mapping is stored in a flat array, so large
468values of C<$tag> will consume large amounts of memory.
469
470Example:
471
472 $profile = new Convert::BER::XS::Profile;
473 $profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
474 $ber = ber_decode $data, $profile;
475
476=item $type = $profile->get ($class, $tag)
477
478Returns the BER type mapped to the given C<$class>/C<$tag> combination.
479
480=back
481
482=head2 BER TYPES
483
484This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
485combination to any C<BER_TYPE_*>.
486
487=over
488
489=item C<BER_TYPE_BYTES>
490
491The raw octets of the value. This is the default type for unknown tags and
492de-/encodes the value as if it were an octet string, i.e. by copying the
493raw bytes.
494
495=item C<BER_TYPE_UTF8>
496
497Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
498(without validation!) and encodes a perl unicode string into a UTF-8 BER
499string.
500
501=item C<BER_TYPE_UCS2>
502
503Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
504string.
505
506=item C<BER_TYPE_UCS4>
507
508Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
509string.
510
511=item C<BER_TYPE_INT>
512
513Encodes and decodes a BER integer value to a perl integer scalar. This
514should correctly handle 64 bit signed and unsigned values.
515
516=item C<BER_TYPE_OID>
517
518Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
519dot, e.g. C<1.3.6.1.213>.
520
521=item C<BER_TYPE_RELOID>
522
523Same as C<BER_TYPE_OID> but uses relative object identifier
524encoding: ASN.1 has this hack of encoding the first two OID components
525into a single integer in a weird attempt to save an insignificant amount
526of space in an otherwise wasteful encoding, and relative OIDs are
527basically OIDs without this hack. The practical difference is that the
528second component of an OID can only have the values 1..40, while relative
529OIDs do not have this restriction.
530
531=item C<BER_TYPE_NULL>
532
533Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
534C<ASN_NULL> type, regardless of the perl value.
535
536=item C<BER_TYPE_BOOL>
537
538Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
539boolean value into an C<ASN_BOOLEAN>.
540
541=item C<BER_TYPE_REAL>
542
543Decodes/encodes a BER real value. NOT IMPLEMENTED.
544
545=item C<BER_TYPE_IPADDRESS>
546
547Decodes/encodes a four byte string into an IPv4 dotted-quad address string
548in Perl. Given the obsolete nature of this type, this is a low-effort
549implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
550so it won't handle all string forms supported by C<inet_aton> for example.
551
552=item C<BER_TYPE_CROAK>
553
554Always croaks when encountered during encoding or decoding - the
555default behaviour when encountering an unknown type is to treat it as
556C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
557error for some types, then C<BER_TYPE_CROAK> is for you.
558
559=back
560
561=cut
562
563our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
564our $SNMP_PROFILE = new Convert::BER::XS::Profile;
565
566# additional SNMP application types
567$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
568$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
569$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
570$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
571$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE , BER_TYPE_IPADDRESS);
572$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);
573
574$DEFAULT_PROFILE->_set_default;
300 575
3011; 5761;
302 577
303=head2 BUGS / SHORTCOMINGs 578=head2 LIMITATIONS/NOTES
304 579
305This module does have a number of SNMPisms hardcoded, such as the SNMP 580This module can only en-/decode 64 bit signed and unsigned integers, and
306tags for Unsigned32 and so on. More configurability is needed, and, if 581only when your perl supports those.
307ever implemented, will come in a form similar to how L<JSON::XS> and 582
308L<CBOR::XS> respresent things, namely with an object-oriented interface. 583This module does not generally care about ranges, i.e. it will happily
584de-/encode 64 bit integers into an C<ASN_INTEGER32> value, or a negative
585number into an C<SNMP_COUNTER64>.
586
587OBJECT IDENTIFIEERs cannot have unlimited length, although the limit is
588much larger than e.g. the one imposed by SNMP or other protocols,a nd is
589about 4kB.
590
591REAL values are not supported and will currently croak.
592
593This module has undergone little to no testing so far.
594
595=head2 ITHREADS SUPPORT
596
597This module is unlikely to work when the (officially discouraged) ithreads
598are in use.
309 599
310=head1 AUTHOR 600=head1 AUTHOR
311 601
312 Marc Lehmann <schmorp@schmorp.de> 602 Marc Lehmann <schmorp@schmorp.de>
313 http://software.schmorp.de/pkg/Convert-BER-XS 603 http://software.schmorp.de/pkg/Convert-BER-XS

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines