Convert-BER-XS/XS.pm

=head1 NAME

Convert::BER::XS - I<very> low level BER en-/decoding

=head1 SYNOPSIS

 use Convert::BER::XS ':all';

 my $ber = ber_decode $buf
    or die "unable to decode SNMP message";

 # The above results in a data structure consisting of
 # (class, tag, # constructed, data)
 # tuples. Below is such a message, SNMPv1 trap
 # with a Cisco mac change notification.
 # Did you know that Cisco is in the news almost
 # every week because # of some backdoor password
 # or other extremely stupid security bug?

 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
   [
      [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
      [ ASN_UNIVERSAL, 4, 0, "public" ], # community
      [ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
         [
            [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
            [ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1
            [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
            [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
            [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
            [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
               [
                  [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
                     [
                        [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
                        [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
                        ]
                     ]
                  ],
                  ...

 # let's decode it a bit with some helper functions

 my $msg = ber_is_seq $ber
    or die "SNMP message does not start with a sequence";

 ber_is $msg->[0], ASN_UNIVERSAL, ASN_INTEGER32, 0
    or die "SNMP message does not start with snmp version\n";

 # message is SNMP v1 or v2c?
 if ($msg->[0][BER_DATA] == 0 || $msg->[0][BER_DATA] == 1) {

    # message is v1 trap?
    if (ber_is $msg->[2], ASN_CONTEXT, 4, 1) {
       my $trap = $msg->[2][BER_DATA];

       # check whether trap is a cisco mac notification mac changed message
       if (
          (ber_is_oid $trap->[0], "1.3.6.1.4.1.9.9.215.2") # cmnInterfaceObjects
          and (ber_is_i32 $trap->[2], 6)
          and (ber_is_i32 $trap->[3], 1) # mac changed msg
       ) {
          ... and so on

 # finally, let's encode it again and hope it results in the same bit pattern

 my $buf = ber_encode $ber;

=head1 DESCRIPTION

WARNING: Before release 1.0, the API is not considered stable in any way.

This module implements a I<very> low level BER/DER en-/decoder.

If is tuned for low memory and high speed, while still maintaining some
level of user-friendlyness.

Currently, not much is documented, as this is an initial release to
reserve CPAN namespace, stay tuned for a few days.

=head2 ASN.1/BER/DER/... BASICS

ASN.1 is a strange language that can be sed to describe protocols and
data structures. It supports various mappings to JSON, XML, but most
importantly, to a various binary encodings such as BER, that is the topic
of this module, and is used in SNMP or LDAP for example.

While ASN.1 defines a schema that is useful to interpret encoded data,
the BER encoding is actually somewhat self-describing: you might not know
whether something is a string or a number or a sequence or something else,
but you can nevertheless decode the overall structure, even if you end up
with just a binary blob for the actual value.

This works because BER values are tagged with a type and a namespace,
and also have a flag that says whther a value consists of subvalues (is
"constructed") or not (is "primitive").

Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
those - for example, you have 32 bit signed integers and 16(!) different
string types, but there is no unsigned32 type for example. Different
applications work around this in different ways, for example, SNMP defines
application-specific Gauge32, Counter32 and Unsigned32, which are mapped
to two different tags: you can distinguish between Counter32 and the
others, but not between Gause32 and Unsigned32, without the ASN.1 schema.

Ugh.

=head2 DECODED BER REPRESENTATION

This module represents every BER value as a 4-element tuple (actually an
array-reference):

   [CLASS, TAG, CONSTRUCTED, DATA]

To avoid non-descriptive hardcoded array index numbers, this module
defines symbolic constants to access these members: C<BER_CLASS>,
C<BER_TAG>, C<BER_CONSTRUCTED> and C<BER_DATA>.

Also, the first three members are integers with a little caveat: for
performance reasons, these are readonly and shared, so you must not modify
them (increment, assign to them etc.) in any way. You may modify the
I<DATA> member, and you may re-assign the array itself, e.g.:

   $ber = ber_decode $binbuf;

   # the following is NOT legal:
   $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)

   # but all of the following are fine:
   $ber->[BER_DATA] = "string";
   $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
   @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);

I<CLASS> is something like a namespace for I<TAG>s - there is the
C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
implementations, the C<ASN_APPLICATION> namespace which defines tags for
specific applications (for example, the SNMP C<Unsigned32> type is in this
namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g.
for C<CHOICE>) and a private namespace (C<ASN_PRIVATE>).

The meaning of the I<TAG> depends on the namespace, and defines a
(partial) interpretation of the data value. For example, right now, SNMP
application namespace knowledge ix hardcoded into this module, so it
knows that SNMP C<Unsigned32> values need to be decoded into actual perl
integers.

The most common tags in the C<ASN_UNIVERSAL> namespace are
C<ASN_INTEGER32>, C<ASN_BIT_STRING>, C<ASN_NULL>, C<ASN_OCTET_STRING>,
C<ASN_OBJECT_IDENTIFIER>, C<ASN_SEQUENCE>, C<ASN_SET> and
C<ASN_IA5_STRING>.

The most common tags in SNMP's C<ASN_APPLICATION> namespace
are C<SNMP_IPADDRESS>, C<SNMP_COUNTER32>, C<SNMP_UNSIGNED32>,
C<SNMP_TIMETICKS>, C<SNMP_OPAQUE> and C<SNMP_COUNTER64>.

The I<CONSTRUCTED> flag is really just a boolean - if it is false, the
the value is "primitive" and contains no subvalues, kind of like a
non-reference perl scalar. IF it is true, then the value is "constructed"
which just means it contains a list of subvalues which this module will
en-/decode as BER tuples themselves.

The I<DATA> value is either a reference to an array of further tuples (if
the value is I<CONSTRUCTED>), some decoded representation of the value,
if this module knows how to decode it (e.g. for the integer types above)
or a binary string with the raw octets if this module doesn't know how to
interpret the namespace/tag.

Thus, you can always decode a BER data structure and at worst you get a
string in place of some nice decoded value.

See the SYNOPSIS for an example of such an encoded tuple representation.

=head2 DECODING AND ENCODING

=over

=item $tuple = ber_decoded $bindata

Decodes binary BER data in C<$bindata> and returns the resulting BER
tuple. Croaks on any decoding error, so the returned C<$tuple> is always
valid.

=item $bindata = ber_encode $tuple

Encodes the BER tuple into a BER/DER data structure.

=back

=head2 HELPER FUNCTIONS

Working with a 4-tuple for every value can be annoying. Or, rather, I<is>
annoying. To reduce this a bit, this module defines a number of helper
functions, both to match BER tuples and to conmstruct BER tuples:

=head3 MATCH HELPERS

Thse functions accept a BER tuple as first argument and either paertially
or fully match it. They often come in two forms, one which exactly matches
a value, and one which only matches the type and returns the value.

They do check whether valid tuples are passed in and croak otherwise. As
a ease-of-use exception, they usually also accept C<undef> instead of a
tuple reference. in which case they silently fail to match.

=over

=item $bool = ber_is $tuple, $class, $tag, $constructed, $data

This takes a BER C<$tuple> and matches its elements agains the privded
values, all of which are optional - values that are either missing or
C<undef> will be ignored, the others will be matched exactly (e.g. as if
you used C<==> or C<eq> (for C<$data>)).

Some examples:

   ber_is $tuple, ASN_UNIVERSAL, ASN_SEQUENCE, 1
      orf die "tuple is not an ASN SEQUENCE";

   ber_is $tuple, ASN_UNIVERSAL, ASN_NULL
      or die "tuple is not an ASN NULL value";

   ber_is $tuple, ASN_UNIVERSAL, ASN_INTEGER32, 0, 50
      or die "BER integer must be 50";

=item $seq = ber_is_seq $tuple

Returns the sequence members (the array of subvalues) if the C<$tuple> is
an ASN SEQUENCE, i.e. the C<BER_DATA> member. If the C<$tuple> is not a
sequence it returns C<undef>. For example, SNMP version 1/2c/3 packets all
consist of an outer SEQUENCE value:

   my $ber = ber_decode $snmp_data;

   my $snmp = ber_is_seq $ber
      or die "SNMP packet invalid: does not start with SEQUENCE";

   # now we know $snmp is a sequence, so decode the SNMP version

   my $version = ber_is_i32 $snmp->[0]
      or die "SNMP packet invalid: does not start with version number";

=item $bool = ber_is_i32 $tuple, $i32

Returns a true value if the C<$tuple> represents an ASN INTEGER32 with
the value C<$i32>.

=item $i32 = ber_is_i32 $tuple

Returns true (and extracts the integer value) if the C<$tuple> is an ASN
INTEGER32. For C<0>, this function returns a special value that is 0 but
true.

=item $bool = ber_is_oid $tuple, $oid_string

Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
that exactly matches C<$oid_string>. Example:

   ber_is_oid $tuple, "1.3.6.1.4"
      or die "oid must be 1.3.6.1.4";

=item $oid = ber_is_oid $tuple

Returns true (and extracts the OID string) if the C<$tuple> is an ASN
OBJECT IDENTIFIER. Otherwise, it returns C<undef>.

=back

=head3 CONSTRUCTION HELPERS

=over

=item $tuple = ber_i32 $value

Constructs a new C<ASN_INTEGER32> tuple.

=back

=head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1>

This module is I<not> the XS version of L<Convert::BER>, but a different
take at doing the same thing. I imagine this module would be a good base
for speeding up either of these, or write a similar module, or write your
own LDAP or SNMP module for example.

=cut

package Convert::BER::XS;

use common::sense;

use XSLoader ();
use Exporter qw(import);

our $VERSION;

BEGIN {
   $VERSION = 0.7;
   XSLoader::load __PACKAGE__, $VERSION;
}

our %EXPORT_TAGS = (
   const => [qw(
      BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA

      ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
      ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
      ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
      ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
      ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
      ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING

      ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE

      BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
      BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
      BER_TYPE_IPADDRESS BER_TYPE_CROAK
   )],
   const_snmp => [qw(
      SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
   )],
   encode => [qw(
      ber_decode
      ber_is ber_is_seq ber_is_i32 ber_is_oid
   )],
   decode => [qw(
      ber_encode
      ber_i32
   )],
);

our @EXPORT_OK = map @$_, values %EXPORT_TAGS;

$EXPORT_TAGS{all} = \@EXPORT_OK;

=head1 PROFILES

While any BER data can be correctly encoded and decoded out of the box, it
can be inconvenient to have to manually decode some values into a "better"
format: for instance, SNMP TimeTicks values are decoded into the raw octet
strings of their BER representation, which is quite hard to decode. With
profiles, you can change which class/tag combinations map to which decoder
function inside C<ber_decode> (and of course also which encoder functions
are used in C<ber_encode>).

This works by mapping specific class/tag combinations to an internal "ber
type".

The default profile supports the standard ASN.1 types, but no
application-specific ones. This means that class/tag combinations not in
the base set of ASN.1 are decoded into their raw octet strings.

C<Convert::BER::XS> defines two profile variables you cna use out of the box:

=over

=item C<$Convert::BER::XS::DEFAULT_PROFILE>

This is the default profile, i.e. the profile that is used when no
profile is specified for de-/encoding.

You cna modify it, but remember that this modifies the defaults for all
callers that rely on the defauit profile.

=item C<$Convert::BER::XS::SNMP_PROFILE>

A profile with mappings for SNMP-specific application tags added. This is
useful when de-/encoding SNMP data.

Example:
   $ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;

=back

=head2 The Convert::BER::XS::Profile class

=over

=item $profile = new Convert::BER::XS::Profile

Create a new profile. The profile will be identical to the default
profile.

=item $profile->set ($class, $tag, $type)

Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
which must be one of the C<BER_TYPE_*> constants.

Note that currently, the mapping is stored in a flat array, so large
values of C<$tag> will consume large amounts of memory.

Example:
   $profile = new Convert::BER::XS::Profile;
   $profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
   $ber = ber_decode $data, $profile;

=item $type = $profile->get ($class, $tag)

Returns the BER type mapped to the given C<$class>/C<$tag> combination.

=back

=head2 BER TYPES

This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
combination to any C<BER_TYPE_*>.

=over

=item C<BER_TYPE_BYTES>

The raw octets of the value. This is the default type for unknown tags and
de-/encodes the value as if it were an octet string, i.e. by copying the
raw bytes.

=item C<BER_TYPE_UTF8>

Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
(without validation!) and encodes a perl unicode string into a UTF-8 BER
string.

=item C<BER_TYPE_UCS2>

Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
string.

=item C<BER_TYPE_UCS4>

Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
string.

=item C<BER_TYPE_INT>

Encodes and decodes a BER integer value to a perl integer scalar. This
should correctly handle 64 bit signed and unsigned values.

=item C<BER_TYPE_OID>

Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
dot, e.g. C<1.3.6.1.213>.

=item C<BER_TYPE_RELOID>

Same as C<BER_TYPE_OID> but uses relative OID encoding: ASN.1 has this
hack of encoding the first two OID components into a single integer in a
weird attempt to save an insignificant amount of space in an otherwise
wasteful encoding, and relative OIDs are basically OIDs without this
hack. The practical difference is that the second component of an OID
can only have the values 1..40, while relative OIDs do not have this
restriction.

=item C<BER_TYPE_NULL>

Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
C<ASN_NULL> type, regardless of the perl value.

=item C<BER_TYPE_BOOL>

Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
boolean value into an C<ASN_BOOLEAN>.

=item C<BER_TYPE_REAL>

Decodes/encodes a BER real value. NOT IMPLEMENTED.

=item C<BER_TYPE_IPADDRESS>

Decodes/encodes a four byte string into an IOv4 dotted-quad address string
in perl. Given ther obsolete nature of this type, this is a low-effort
implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
so it won't handle all string forms supported by C<inet_aton>.

=item C<BER_TYPE_CROAK>

Always croaks when encountered during encoding or decoding - the
default behaviour when encountering an unknown type is to treat it as
C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
error for some types, then CyBER_TYPE_CROAK> is for you.

=back

=cut

our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
our $SNMP_PROFILE    = new Convert::BER::XS::Profile;

$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE    , BER_TYPE_IPADDRESS);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);

$DEFAULT_PROFILE->_set_default;

1;

=head2 LIMITATIONS

This module can only en-/decode 64 bit signed and unsigned integers, and
only when your perl supports those.

OBJECT IDENTIFIEERS cannot have unlimited length, although the limit is
much larger than e.g. the one imposed by SNMP or other protocols.

REAL values are not supported and will croak.

This module has undergone little to no testing so far.

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://software.schmorp.de/pkg/Convert-BER-XS

=cut

Revision:	1.14
Committed:	Sat Apr 20 01:31:07 2019 UTC (5 years, 1 month ago) by root
Branch:	MAIN
CVS Tags:	rel-0_7
Changes since 1.13:	+6 -2 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3	root	1.4	Convert::BER::XS - I<very> low level BER en-/decoding
4	root	1.1
5			=head1 SYNOPSIS
6
7			use Convert::BER::XS ':all';
8
9			my $ber = ber_decode $buf
10	root	1.6	or die "unable to decode SNMP message";
11	root	1.1
12	root	1.13	# The above results in a data structure consisting of
13			# (class, tag, # constructed, data)
14			# tuples. Below is such a message, SNMPv1 trap
15	root	1.6	# with a Cisco mac change notification.
16	root	1.13	# Did you know that Cisco is in the news almost
17			# every week because # of some backdoor password
18			# or other extremely stupid security bug?
19	root	1.3
20			[ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
21			[
22			[ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
23			[ ASN_UNIVERSAL, 4, 0, "public" ], # community
24	root	1.6	[ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
25	root	1.3	[
26			[ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
27			[ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1
28			[ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
29			[ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
30			[ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
31			[ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
32			[
33			[ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
34			[
35	root	1.8	[ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
36	root	1.3	[ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
37			]
38			]
39			],
40			...
41
42			# let's decode it a bit with some helper functions
43
44	root	1.1	my $msg = ber_is_seq $ber
45			or die "SNMP message does not start with a sequence";
46
47			ber_is $msg->[0], ASN_UNIVERSAL, ASN_INTEGER32, 0
48			or die "SNMP message does not start with snmp version\n";
49
50	root	1.3	# message is SNMP v1 or v2c?
51	root	1.1	if ($msg->[0][BER_DATA] == 0 \|\| $msg->[0][BER_DATA] == 1) {
52
53	root	1.3	# message is v1 trap?
54	root	1.1	if (ber_is $msg->[2], ASN_CONTEXT, 4, 1) {
55			my $trap = $msg->[2][BER_DATA];
56
57			# check whether trap is a cisco mac notification mac changed message
58			if (
59			(ber_is_oid $trap->[0], "1.3.6.1.4.1.9.9.215.2") # cmnInterfaceObjects
60			and (ber_is_i32 $trap->[2], 6)
61			and (ber_is_i32 $trap->[3], 1) # mac changed msg
62			) {
63			... and so on
64
65	root	1.4	# finally, let's encode it again and hope it results in the same bit pattern
66
67			my $buf = ber_encode $ber;
68
69	root	1.1	=head1 DESCRIPTION
70
71	root	1.7	WARNING: Before release 1.0, the API is not considered stable in any way.
72
73	root	1.4	This module implements a I<very> low level BER/DER en-/decoder.
74	root	1.1
75			If is tuned for low memory and high speed, while still maintaining some
76			level of user-friendlyness.
77
78			Currently, not much is documented, as this is an initial release to
79			reserve CPAN namespace, stay tuned for a few days.
80
81	root	1.4	=head2 ASN.1/BER/DER/... BASICS
82
83			ASN.1 is a strange language that can be sed to describe protocols and
84			data structures. It supports various mappings to JSON, XML, but most
85			importantly, to a various binary encodings such as BER, that is the topic
86			of this module, and is used in SNMP or LDAP for example.
87
88			While ASN.1 defines a schema that is useful to interpret encoded data,
89	root	1.12	the BER encoding is actually somewhat self-describing: you might not know
90	root	1.4	whether something is a string or a number or a sequence or something else,
91			but you can nevertheless decode the overall structure, even if you end up
92			with just a binary blob for the actual value.
93
94			This works because BER values are tagged with a type and a namespace,
95			and also have a flag that says whther a value consists of subvalues (is
96			"constructed") or not (is "primitive").
97
98			Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
99			those - for example, you have 32 bit signed integers and 16(!) different
100			string types, but there is no unsigned32 type for example. Different
101			applications work around this in different ways, for example, SNMP defines
102			application-specific Gauge32, Counter32 and Unsigned32, which are mapped
103			to two different tags: you can distinguish between Counter32 and the
104			others, but not between Gause32 and Unsigned32, without the ASN.1 schema.
105
106			Ugh.
107
108			=head2 DECODED BER REPRESENTATION
109
110			This module represents every BER value as a 4-element tuple (actually an
111			array-reference):
112
113			[CLASS, TAG, CONSTRUCTED, DATA]
114
115	root	1.6	To avoid non-descriptive hardcoded array index numbers, this module
116			defines symbolic constants to access these members: C<BER_CLASS>,
117			C<BER_TAG>, C<BER_CONSTRUCTED> and C<BER_DATA>.
118
119			Also, the first three members are integers with a little caveat: for
120			performance reasons, these are readonly and shared, so you must not modify
121			them (increment, assign to them etc.) in any way. You may modify the
122			I<DATA> member, and you may re-assign the array itself, e.g.:
123
124			$ber = ber_decode $binbuf;
125
126			# the following is NOT legal:
127	root	1.10	$ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)
128	root	1.6
129			# but all of the following are fine:
130			$ber->[BER_DATA] = "string";
131			$ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
132	root	1.11	@$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);
133	root	1.6
134	root	1.4	I<CLASS> is something like a namespace for I<TAG>s - there is the
135			C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
136			implementations, the C<ASN_APPLICATION> namespace which defines tags for
137			specific applications (for example, the SNMP C<Unsigned32> type is in this
138			namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g.
139			for C<CHOICE>) and a private namespace (C<ASN_PRIVATE>).
140
141			The meaning of the I<TAG> depends on the namespace, and defines a
142			(partial) interpretation of the data value. For example, right now, SNMP
143			application namespace knowledge ix hardcoded into this module, so it
144			knows that SNMP C<Unsigned32> values need to be decoded into actual perl
145			integers.
146
147			The most common tags in the C<ASN_UNIVERSAL> namespace are
148			C<ASN_INTEGER32>, C<ASN_BIT_STRING>, C<ASN_NULL>, C<ASN_OCTET_STRING>,
149			C<ASN_OBJECT_IDENTIFIER>, C<ASN_SEQUENCE>, C<ASN_SET> and
150			C<ASN_IA5_STRING>.
151
152			The most common tags in SNMP's C<ASN_APPLICATION> namespace
153			are C<SNMP_IPADDRESS>, C<SNMP_COUNTER32>, C<SNMP_UNSIGNED32>,
154			C<SNMP_TIMETICKS>, C<SNMP_OPAQUE> and C<SNMP_COUNTER64>.
155
156			The I<CONSTRUCTED> flag is really just a boolean - if it is false, the
157			the value is "primitive" and contains no subvalues, kind of like a
158			non-reference perl scalar. IF it is true, then the value is "constructed"
159			which just means it contains a list of subvalues which this module will
160			en-/decode as BER tuples themselves.
161
162			The I<DATA> value is either a reference to an array of further tuples (if
163			the value is I<CONSTRUCTED>), some decoded representation of the value,
164			if this module knows how to decode it (e.g. for the integer types above)
165			or a binary string with the raw octets if this module doesn't know how to
166			interpret the namespace/tag.
167
168			Thus, you can always decode a BER data structure and at worst you get a
169			string in place of some nice decoded value.
170
171			See the SYNOPSIS for an example of such an encoded tuple representation.
172
173	root	1.7	=head2 DECODING AND ENCODING
174
175			=over
176
177			=item $tuple = ber_decoded $bindata
178
179			Decodes binary BER data in C<$bindata> and returns the resulting BER
180			tuple. Croaks on any decoding error, so the returned C<$tuple> is always
181			valid.
182
183			=item $bindata = ber_encode $tuple
184
185			Encodes the BER tuple into a BER/DER data structure.
186
187			=back
188
189	root	1.6	=head2 HELPER FUNCTIONS
190
191			Working with a 4-tuple for every value can be annoying. Or, rather, I<is>
192			annoying. To reduce this a bit, this module defines a number of helper
193			functions, both to match BER tuples and to conmstruct BER tuples:
194
195			=head3 MATCH HELPERS
196
197			Thse functions accept a BER tuple as first argument and either paertially
198			or fully match it. They often come in two forms, one which exactly matches
199			a value, and one which only matches the type and returns the value.
200
201			They do check whether valid tuples are passed in and croak otherwise. As
202			a ease-of-use exception, they usually also accept C<undef> instead of a
203			tuple reference. in which case they silently fail to match.
204
205			=over
206
207			=item $bool = ber_is $tuple, $class, $tag, $constructed, $data
208
209			This takes a BER C<$tuple> and matches its elements agains the privded
210			values, all of which are optional - values that are either missing or
211			C<undef> will be ignored, the others will be matched exactly (e.g. as if
212			you used C<==> or C<eq> (for C<$data>)).
213
214			Some examples:
215
216			ber_is $tuple, ASN_UNIVERSAL, ASN_SEQUENCE, 1
217			orf die "tuple is not an ASN SEQUENCE";
218
219			ber_is $tuple, ASN_UNIVERSAL, ASN_NULL
220			or die "tuple is not an ASN NULL value";
221
222			ber_is $tuple, ASN_UNIVERSAL, ASN_INTEGER32, 0, 50
223			or die "BER integer must be 50";
224
225			=item $seq = ber_is_seq $tuple
226
227			Returns the sequence members (the array of subvalues) if the C<$tuple> is
228			an ASN SEQUENCE, i.e. the C<BER_DATA> member. If the C<$tuple> is not a
229			sequence it returns C<undef>. For example, SNMP version 1/2c/3 packets all
230			consist of an outer SEQUENCE value:
231
232			my $ber = ber_decode $snmp_data;
233
234			my $snmp = ber_is_seq $ber
235			or die "SNMP packet invalid: does not start with SEQUENCE";
236
237			# now we know $snmp is a sequence, so decode the SNMP version
238
239			my $version = ber_is_i32 $snmp->[0]
240			or die "SNMP packet invalid: does not start with version number";
241
242			=item $bool = ber_is_i32 $tuple, $i32
243
244			Returns a true value if the C<$tuple> represents an ASN INTEGER32 with
245			the value C<$i32>.
246
247			=item $i32 = ber_is_i32 $tuple
248
249			Returns true (and extracts the integer value) if the C<$tuple> is an ASN
250			INTEGER32. For C<0>, this function returns a special value that is 0 but
251			true.
252
253			=item $bool = ber_is_oid $tuple, $oid_string
254
255			Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
256	root	1.12	that exactly matches C<$oid_string>. Example:
257	root	1.6
258			ber_is_oid $tuple, "1.3.6.1.4"
259			or die "oid must be 1.3.6.1.4";
260
261			=item $oid = ber_is_oid $tuple
262
263			Returns true (and extracts the OID string) if the C<$tuple> is an ASN
264			OBJECT IDENTIFIER. Otherwise, it returns C<undef>.
265
266			=back
267
268			=head3 CONSTRUCTION HELPERS
269
270			=over
271
272			=item $tuple = ber_i32 $value
273
274			Constructs a new C<ASN_INTEGER32> tuple.
275
276			=back
277
278	root	1.2	=head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1>
279
280			This module is I<not> the XS version of L<Convert::BER>, but a different
281			take at doing the same thing. I imagine this module would be a good base
282	root	1.4	for speeding up either of these, or write a similar module, or write your
283	root	1.2	own LDAP or SNMP module for example.
284
285	root	1.1	=cut
286
287			package Convert::BER::XS;
288
289			use common::sense;
290
291			use XSLoader ();
292			use Exporter qw(import);
293
294	root	1.13	our $VERSION;
295	root	1.1
296	root	1.13	BEGIN {
297			$VERSION = 0.7;
298			XSLoader::load __PACKAGE__, $VERSION;
299			}
300	root	1.1
301			our %EXPORT_TAGS = (
302	root	1.4	const => [qw(
303	root	1.1	BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
304	root	1.4
305	root	1.13	ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
306			ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
307			ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
308			ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
309			ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
310			ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
311
312			ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
313
314			BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
315			BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
316			BER_TYPE_IPADDRESS BER_TYPE_CROAK
317			)],
318			const_snmp => [qw(
319	root	1.4	SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
320			)],
321			encode => [qw(
322			ber_decode
323			ber_is ber_is_seq ber_is_i32 ber_is_oid
324			)],
325			decode => [qw(
326			ber_encode
327	root	1.13	ber_i32
328	root	1.1	)],
329			);
330
331			our @EXPORT_OK = map @$_, values %EXPORT_TAGS;
332
333	root	1.4	$EXPORT_TAGS{all} = \@EXPORT_OK;
334
335	root	1.13	=head1 PROFILES
336
337			While any BER data can be correctly encoded and decoded out of the box, it
338			can be inconvenient to have to manually decode some values into a "better"
339			format: for instance, SNMP TimeTicks values are decoded into the raw octet
340			strings of their BER representation, which is quite hard to decode. With
341			profiles, you can change which class/tag combinations map to which decoder
342			function inside C<ber_decode> (and of course also which encoder functions
343			are used in C<ber_encode>).
344
345			This works by mapping specific class/tag combinations to an internal "ber
346			type".
347
348			The default profile supports the standard ASN.1 types, but no
349			application-specific ones. This means that class/tag combinations not in
350			the base set of ASN.1 are decoded into their raw octet strings.
351
352			C<Convert::BER::XS> defines two profile variables you cna use out of the box:
353
354			=over
355
356			=item C<$Convert::BER::XS::DEFAULT_PROFILE>
357
358			This is the default profile, i.e. the profile that is used when no
359			profile is specified for de-/encoding.
360
361			You cna modify it, but remember that this modifies the defaults for all
362			callers that rely on the defauit profile.
363
364			=item C<$Convert::BER::XS::SNMP_PROFILE>
365
366			A profile with mappings for SNMP-specific application tags added. This is
367			useful when de-/encoding SNMP data.
368
369			Example:
370			$ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;
371
372			=back
373
374			=head2 The Convert::BER::XS::Profile class
375
376			=over
377
378			=item $profile = new Convert::BER::XS::Profile
379
380			Create a new profile. The profile will be identical to the default
381			profile.
382
383			=item $profile->set ($class, $tag, $type)
384
385			Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
386			which must be one of the C<BER_TYPE_*> constants.
387
388			Note that currently, the mapping is stored in a flat array, so large
389			values of C<$tag> will consume large amounts of memory.
390
391			Example:
392			$profile = new Convert::BER::XS::Profile;
393			$profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
394			$ber = ber_decode $data, $profile;
395
396			=item $type = $profile->get ($class, $tag)
397
398			Returns the BER type mapped to the given C<$class>/C<$tag> combination.
399
400			=back
401
402			=head2 BER TYPES
403
404			This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
405			combination to any C<BER_TYPE_*>.
406
407			=over
408
409			=item C<BER_TYPE_BYTES>
410
411			The raw octets of the value. This is the default type for unknown tags and
412			de-/encodes the value as if it were an octet string, i.e. by copying the
413			raw bytes.
414
415			=item C<BER_TYPE_UTF8>
416
417			Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
418			(without validation!) and encodes a perl unicode string into a UTF-8 BER
419			string.
420
421			=item C<BER_TYPE_UCS2>
422
423			Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
424	root	1.14	string.
425	root	1.13
426			=item C<BER_TYPE_UCS4>
427
428			Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
429	root	1.14	string.
430	root	1.13
431			=item C<BER_TYPE_INT>
432
433			Encodes and decodes a BER integer value to a perl integer scalar. This
434			should correctly handle 64 bit signed and unsigned values.
435
436			=item C<BER_TYPE_OID>
437
438			Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
439			dot, e.g. C<1.3.6.1.213>.
440
441			=item C<BER_TYPE_RELOID>
442
443			Same as C<BER_TYPE_OID> but uses relative OID encoding: ASN.1 has this
444			hack of encoding the first two OID components into a single integer in a
445			weird attempt to save an insignificant amount of space in an otherwise
446			wasteful encoding, and relative OIDs are basically OIDs without this
447			hack. The practical difference is that the second component of an OID
448			can only have the values 1..40, while relative OIDs do not have this
449			restriction.
450
451			=item C<BER_TYPE_NULL>
452
453			Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
454			C<ASN_NULL> type, regardless of the perl value.
455
456			=item C<BER_TYPE_BOOL>
457
458			Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
459			boolean value into an C<ASN_BOOLEAN>.
460
461			=item C<BER_TYPE_REAL>
462
463			Decodes/encodes a BER real value. NOT IMPLEMENTED.
464
465			=item C<BER_TYPE_IPADDRESS>
466
467			Decodes/encodes a four byte string into an IOv4 dotted-quad address string
468			in perl. Given ther obsolete nature of this type, this is a low-effort
469			implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
470			so it won't handle all string forms supported by C<inet_aton>.
471
472			=item C<BER_TYPE_CROAK>
473
474			Always croaks when encountered during encoding or decoding - the
475			default behaviour when encountering an unknown type is to treat it as
476			C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
477			error for some types, then CyBER_TYPE_CROAK> is for you.
478
479			=back
480
481			=cut
482
483			our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
484			our $SNMP_PROFILE = new Convert::BER::XS::Profile;
485
486			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
487			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
488			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
489			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
490			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE , BER_TYPE_IPADDRESS);
491			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);
492
493			$DEFAULT_PROFILE->_set_default;
494
495	root	1.1	1;
496
497	root	1.13	=head2 LIMITATIONS
498
499			This module can only en-/decode 64 bit signed and unsigned integers, and
500			only when your perl supports those.
501	root	1.4
502	root	1.13	OBJECT IDENTIFIEERS cannot have unlimited length, although the limit is
503			much larger than e.g. the one imposed by SNMP or other protocols.
504	root	1.4
505	root	1.14	REAL values are not supported and will croak.
506
507			This module has undergone little to no testing so far.
508
509	root	1.1	=head1 AUTHOR
510
511			Marc Lehmann <schmorp@schmorp.de>
512			http://software.schmorp.de/pkg/Convert-BER-XS
513
514			=cut
515