Convert-BER-XS/XS.pm

=head1 NAME

Convert::BER::XS - I<very> low level BER en-/decoding

=head1 SYNOPSIS

 use Convert::BER::XS ':all';

 my $ber = ber_decode $buf
    or die "unable to decode SNMP message";

 # The above results in a data structure consisting of
 # (class, tag, # constructed, data)
 # tuples. Below is such a message, SNMPv1 trap
 # with a Cisco mac change notification.
 # Did you know that Cisco is in the news almost
 # every week because # of some backdoor password
 # or other extremely stupid security bug?

 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
   [
      [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
      [ ASN_UNIVERSAL, 4, 0, "public" ], # community
      [ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
         [
            [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
            [ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1
            [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
            [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
            [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
            [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
               [
                  [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
                     [
                        [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
                        [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
                        ]
                     ]
                  ],
                  ...

 # let's decode it a bit with some helper functions

 my $msg = ber_is_seq $ber
    or die "SNMP message does not start with a sequence";

 ber_is $msg->[0], ASN_UNIVERSAL, ASN_INTEGER32, 0
    or die "SNMP message does not start with snmp version\n";

 # message is SNMP v1 or v2c?
 if ($msg->[0][BER_DATA] == 0 || $msg->[0][BER_DATA] == 1) {

    # message is v1 trap?
    if (ber_is $msg->[2], ASN_CONTEXT, 4, 1) {
       my $trap = $msg->[2][BER_DATA];

       # check whether trap is a cisco mac notification mac changed message
       if (
          (ber_is_oid $trap->[0], "1.3.6.1.4.1.9.9.215.2") # cmnInterfaceObjects
          and (ber_is_i32 $trap->[2], 6)
          and (ber_is_i32 $trap->[3], 1) # mac changed msg
       ) {
          ... and so on

 # finally, let's encode it again and hope it results in the same bit pattern

 my $buf = ber_encode $ber;

=head1 DESCRIPTION

WARNING: Before release 1.0, the API is not considered stable in any way.

This module implements a I<very> low level BER/DER en-/decoder.

If is tuned for low memory and high speed, while still maintaining some
level of user-friendlyness.

=head2 ASN.1/BER/DER/... BASICS

ASN.1 is a strange language that can be used to describe protocols and
data structures. It supports various mappings to JSON, XML, but most
importantly, to a various binary encodings such as BER, that is the topic
of this module, and is used in SNMP or LDAP for example.

While ASN.1 defines a schema that is useful to interpret encoded data,
the BER encoding is actually somewhat self-describing: you might not know
whether something is a string or a number or a sequence or something else,
but you can nevertheless decode the overall structure, even if you end up
with just a binary blob for the actual value.

This works because BER values are tagged with a type and a namespace,
and also have a flag that says whether a value consists of subvalues (is
"constructed") or not (is "primitive").

Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
those - for example, you have 32 bit signed integers and 16(!) different
string types, but there is no unsigned32 type for example. Different
applications work around this in different ways, for example, SNMP defines
application-specific Gauge32, Counter32 and Unsigned32, which are mapped
to two different tags: you can distinguish between Counter32 and the
others, but not between Gause32 and Unsigned32, without the ASN.1 schema.

Ugh.

=head2 DECODED BER REPRESENTATION

This module represents every BER value as a 4-element tuple (actually an
array-reference):

   [CLASS, TAG, CONSTRUCTED, DATA]

To avoid non-descriptive hardcoded array index numbers, this module
defines symbolic constants to access these members: C<BER_CLASS>,
C<BER_TAG>, C<BER_CONSTRUCTED> and C<BER_DATA>.

Also, the first three members are integers with a little caveat: for
performance reasons, these are readonly and shared, so you must not modify
them (increment, assign to them etc.) in any way. You may modify the
I<DATA> member, and you may re-assign the array itself, e.g.:

   $ber = ber_decode $binbuf;

   # the following is NOT legal:
   $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)

   # but all of the following are fine:
   $ber->[BER_DATA] = "string";
   $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
   @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);

I<CLASS> is something like a namespace for I<TAG>s - there is the
C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
implementations, the C<ASN_APPLICATION> namespace which defines tags for
specific applications (for example, the SNMP C<Unsigned32> type is in this
namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g.
for C<CHOICE>) and a private namespace (C<ASN_PRIVATE>).

The meaning of the I<TAG> depends on the namespace, and defines a
(partial) interpretation of the data value. For example, right now, SNMP
application namespace knowledge ix hardcoded into this module, so it
knows that SNMP C<Unsigned32> values need to be decoded into actual perl
integers.

The most common tags in the C<ASN_UNIVERSAL> namespace are
C<ASN_INTEGER32>, C<ASN_BIT_STRING>, C<ASN_NULL>, C<ASN_OCTET_STRING>,
C<ASN_OBJECT_IDENTIFIER>, C<ASN_SEQUENCE>, C<ASN_SET> and
C<ASN_IA5_STRING>.

The most common tags in SNMP's C<ASN_APPLICATION> namespace
are C<SNMP_IPADDRESS>, C<SNMP_COUNTER32>, C<SNMP_UNSIGNED32>,
C<SNMP_TIMETICKS>, C<SNMP_OPAQUE> and C<SNMP_COUNTER64>.

The I<CONSTRUCTED> flag is really just a boolean - if it is false, the
the value is "primitive" and contains no subvalues, kind of like a
non-reference perl scalar. IF it is true, then the value is "constructed"
which just means it contains a list of subvalues which this module will
en-/decode as BER tuples themselves.

The I<DATA> value is either a reference to an array of further tuples (if
the value is I<CONSTRUCTED>), some decoded representation of the value,
if this module knows how to decode it (e.g. for the integer types above)
or a binary string with the raw octets if this module doesn't know how to
interpret the namespace/tag.

Thus, you can always decode a BER data structure and at worst you get a
string in place of some nice decoded value.

See the SYNOPSIS for an example of such an encoded tuple representation.

=head2 DECODING AND ENCODING

=over

=item $tuple = ber_decoded $bindata

Decodes binary BER data in C<$bindata> and returns the resulting BER
tuple. Croaks on any decoding error, so the returned C<$tuple> is always
valid.

=item $bindata = ber_encode $tuple

Encodes the BER tuple into a BER/DER data structure.

=back

=head2 HELPER FUNCTIONS

Working with a 4-tuple for every value can be annoying. Or, rather, I<is>
annoying. To reduce this a bit, this module defines a number of helper
functions, both to match BER tuples and to conmstruct BER tuples:

=head3 MATCH HELPERS

Thse functions accept a BER tuple as first argument and either paertially
or fully match it. They often come in two forms, one which exactly matches
a value, and one which only matches the type and returns the value.

They do check whether valid tuples are passed in and croak otherwise. As
a ease-of-use exception, they usually also accept C<undef> instead of a
tuple reference. in which case they silently fail to match.

=over

=item $bool = ber_is $tuple, $class, $tag, $constructed, $data

This takes a BER C<$tuple> and matches its elements agains the privded
values, all of which are optional - values that are either missing or
C<undef> will be ignored, the others will be matched exactly (e.g. as if
you used C<==> or C<eq> (for C<$data>)).

Some examples:

   ber_is $tuple, ASN_UNIVERSAL, ASN_SEQUENCE, 1
      orf die "tuple is not an ASN SEQUENCE";

   ber_is $tuple, ASN_UNIVERSAL, ASN_NULL
      or die "tuple is not an ASN NULL value";

   ber_is $tuple, ASN_UNIVERSAL, ASN_INTEGER32, 0, 50
      or die "BER integer must be 50";

=item $seq = ber_is_seq $tuple

Returns the sequence members (the array of subvalues) if the C<$tuple> is
an ASN SEQUENCE, i.e. the C<BER_DATA> member. If the C<$tuple> is not a
sequence it returns C<undef>. For example, SNMP version 1/2c/3 packets all
consist of an outer SEQUENCE value:

   my $ber = ber_decode $snmp_data;

   my $snmp = ber_is_seq $ber
      or die "SNMP packet invalid: does not start with SEQUENCE";

   # now we know $snmp is a sequence, so decode the SNMP version

   my $version = ber_is_i32 $snmp->[0]
      or die "SNMP packet invalid: does not start with version number";

=item $bool = ber_is_i32 $tuple, $i32

Returns a true value if the C<$tuple> represents an ASN INTEGER32 with
the value C<$i32>.

=item $i32 = ber_is_i32 $tuple

Returns true (and extracts the integer value) if the C<$tuple> is an ASN
INTEGER32. For C<0>, this function returns a special value that is 0 but
true.

=item $bool = ber_is_oid $tuple, $oid_string

Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
that exactly matches C<$oid_string>. Example:

   ber_is_oid $tuple, "1.3.6.1.4"
      or die "oid must be 1.3.6.1.4";

=item $oid = ber_is_oid $tuple

Returns true (and extracts the OID string) if the C<$tuple> is an ASN
OBJECT IDENTIFIER. Otherwise, it returns C<undef>.

=back

=head3 CONSTRUCTION HELPERS

=over

=item $tuple = ber_i32 $value

Constructs a new C<ASN_INTEGER32> tuple.

=back

=head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1>

This module is I<not> the XS version of L<Convert::BER>, but a different
take at doing the same thing. I imagine this module would be a good base
for speeding up either of these, or write a similar module, or write your
own LDAP or SNMP module for example.

=cut

package Convert::BER::XS;

use common::sense;

use XSLoader ();
use Exporter qw(import);

our $VERSION;

BEGIN {
   $VERSION = 0.7;
   XSLoader::load __PACKAGE__, $VERSION;
}

our %EXPORT_TAGS = (
   const => [qw(
      BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA

      ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
      ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
      ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
      ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
      ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
      ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING

      ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE

      BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
      BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
      BER_TYPE_IPADDRESS BER_TYPE_CROAK
   )],
   const_snmp => [qw(
      SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
   )],
   encode => [qw(
      ber_decode
      ber_is ber_is_seq ber_is_i32 ber_is_oid
   )],
   decode => [qw(
      ber_encode
      ber_i32
   )],
);

our @EXPORT_OK = map @$_, values %EXPORT_TAGS;

$EXPORT_TAGS{all} = \@EXPORT_OK;

=head1 PROFILES

While any BER data can be correctly encoded and decoded out of the box, it
can be inconvenient to have to manually decode some values into a "better"
format: for instance, SNMP TimeTicks values are decoded into the raw octet
strings of their BER representation, which is quite hard to decode. With
profiles, you can change which class/tag combinations map to which decoder
function inside C<ber_decode> (and of course also which encoder functions
are used in C<ber_encode>).

This works by mapping specific class/tag combinations to an internal "ber
type".

The default profile supports the standard ASN.1 types, but no
application-specific ones. This means that class/tag combinations not in
the base set of ASN.1 are decoded into their raw octet strings.

C<Convert::BER::XS> defines two profile variables you can use out of the box:

=over

=item C<$Convert::BER::XS::DEFAULT_PROFILE>

This is the default profile, i.e. the profile that is used when no
profile is specified for de-/encoding.

You can modify it, but remember that this modifies the defaults for all
callers that rely on the default profile.

=item C<$Convert::BER::XS::SNMP_PROFILE>

A profile with mappings for SNMP-specific application tags added. This is
useful when de-/encoding SNMP data.

Example:

   $ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;

=back

=head2 The Convert::BER::XS::Profile class

=over

=item $profile = new Convert::BER::XS::Profile

Create a new profile. The profile will be identical to the default
profile.

=item $profile->set ($class, $tag, $type)

Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
which must be one of the C<BER_TYPE_*> constants.

Note that currently, the mapping is stored in a flat array, so large
values of C<$tag> will consume large amounts of memory.

Example:

   $profile = new Convert::BER::XS::Profile;
   $profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
   $ber = ber_decode $data, $profile;

=item $type = $profile->get ($class, $tag)

Returns the BER type mapped to the given C<$class>/C<$tag> combination.

=back

=head2 BER TYPES

This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
combination to any C<BER_TYPE_*>.

=over

=item C<BER_TYPE_BYTES>

The raw octets of the value. This is the default type for unknown tags and
de-/encodes the value as if it were an octet string, i.e. by copying the
raw bytes.

=item C<BER_TYPE_UTF8>

Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
(without validation!) and encodes a perl unicode string into a UTF-8 BER
string.

=item C<BER_TYPE_UCS2>

Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
string.

=item C<BER_TYPE_UCS4>

Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
string.

=item C<BER_TYPE_INT>

Encodes and decodes a BER integer value to a perl integer scalar. This
should correctly handle 64 bit signed and unsigned values.

=item C<BER_TYPE_OID>

Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
dot, e.g. C<1.3.6.1.213>.

=item C<BER_TYPE_RELOID>

Same as C<BER_TYPE_OID> but uses relative object identifier
encoding: ASN.1 has this hack of encoding the first two OID components
into a single integer in a weird attempt to save an insignificant amount
of space in an otherwise wasteful encoding, and relative OIDs are
basically OIDs without this hack. The practical difference is that the
second component of an OID can only have the values 1..40, while relative
OIDs do not have this restriction.

=item C<BER_TYPE_NULL>

Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
C<ASN_NULL> type, regardless of the perl value.

=item C<BER_TYPE_BOOL>

Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
boolean value into an C<ASN_BOOLEAN>.

=item C<BER_TYPE_REAL>

Decodes/encodes a BER real value. NOT IMPLEMENTED.

=item C<BER_TYPE_IPADDRESS>

Decodes/encodes a four byte string into an IPv4 dotted-quad address string
in Perl. Given the obsolete nature of this type, this is a low-effort
implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
so it won't handle all string forms supported by C<inet_aton> for example.

=item C<BER_TYPE_CROAK>

Always croaks when encountered during encoding or decoding - the
default behaviour when encountering an unknown type is to treat it as
C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
error for some types, then CyBER_TYPE_CROAK> is for you.

=back

=cut

our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
our $SNMP_PROFILE    = new Convert::BER::XS::Profile;

$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE    , BER_TYPE_IPADDRESS);
$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);

$DEFAULT_PROFILE->_set_default;

1;

=head2 LIMITATIONS

This module can only en-/decode 64 bit signed and unsigned integers, and
only when your perl supports those.

OBJECT IDENTIFIEERS cannot have unlimited length, although the limit is
much larger than e.g. the one imposed by SNMP or other protocols.

REAL values are not supported and will croak.

This module has undergone little to no testing so far.

=head1 AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://software.schmorp.de/pkg/Convert-BER-XS

=cut

Revision:	1.15
Committed:	Sat Apr 20 02:07:29 2019 UTC (5 years, 1 month ago) by root
Branch:	MAIN
Changes since 1.14:	+17 -18 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	root	1.1	=head1 NAME
2
3	root	1.4	Convert::BER::XS - I<very> low level BER en-/decoding
4	root	1.1
5			=head1 SYNOPSIS
6
7			use Convert::BER::XS ':all';
8
9			my $ber = ber_decode $buf
10	root	1.6	or die "unable to decode SNMP message";
11	root	1.1
12	root	1.13	# The above results in a data structure consisting of
13			# (class, tag, # constructed, data)
14			# tuples. Below is such a message, SNMPv1 trap
15	root	1.6	# with a Cisco mac change notification.
16	root	1.13	# Did you know that Cisco is in the news almost
17			# every week because # of some backdoor password
18			# or other extremely stupid security bug?
19	root	1.3
20			[ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
21			[
22			[ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
23			[ ASN_UNIVERSAL, 4, 0, "public" ], # community
24	root	1.6	[ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
25	root	1.3	[
26			[ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
27			[ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1
28			[ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
29			[ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
30			[ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
31			[ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
32			[
33			[ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
34			[
35	root	1.8	[ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
36	root	1.3	[ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
37			]
38			]
39			],
40			...
41
42			# let's decode it a bit with some helper functions
43
44	root	1.1	my $msg = ber_is_seq $ber
45			or die "SNMP message does not start with a sequence";
46
47			ber_is $msg->[0], ASN_UNIVERSAL, ASN_INTEGER32, 0
48			or die "SNMP message does not start with snmp version\n";
49
50	root	1.3	# message is SNMP v1 or v2c?
51	root	1.1	if ($msg->[0][BER_DATA] == 0 \|\| $msg->[0][BER_DATA] == 1) {
52
53	root	1.3	# message is v1 trap?
54	root	1.1	if (ber_is $msg->[2], ASN_CONTEXT, 4, 1) {
55			my $trap = $msg->[2][BER_DATA];
56
57			# check whether trap is a cisco mac notification mac changed message
58			if (
59			(ber_is_oid $trap->[0], "1.3.6.1.4.1.9.9.215.2") # cmnInterfaceObjects
60			and (ber_is_i32 $trap->[2], 6)
61			and (ber_is_i32 $trap->[3], 1) # mac changed msg
62			) {
63			... and so on
64
65	root	1.4	# finally, let's encode it again and hope it results in the same bit pattern
66
67			my $buf = ber_encode $ber;
68
69	root	1.1	=head1 DESCRIPTION
70
71	root	1.7	WARNING: Before release 1.0, the API is not considered stable in any way.
72
73	root	1.4	This module implements a I<very> low level BER/DER en-/decoder.
74	root	1.1
75			If is tuned for low memory and high speed, while still maintaining some
76			level of user-friendlyness.
77
78	root	1.4	=head2 ASN.1/BER/DER/... BASICS
79
80	root	1.15	ASN.1 is a strange language that can be used to describe protocols and
81	root	1.4	data structures. It supports various mappings to JSON, XML, but most
82			importantly, to a various binary encodings such as BER, that is the topic
83			of this module, and is used in SNMP or LDAP for example.
84
85			While ASN.1 defines a schema that is useful to interpret encoded data,
86	root	1.12	the BER encoding is actually somewhat self-describing: you might not know
87	root	1.4	whether something is a string or a number or a sequence or something else,
88			but you can nevertheless decode the overall structure, even if you end up
89			with just a binary blob for the actual value.
90
91			This works because BER values are tagged with a type and a namespace,
92	root	1.15	and also have a flag that says whether a value consists of subvalues (is
93	root	1.4	"constructed") or not (is "primitive").
94
95			Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
96			those - for example, you have 32 bit signed integers and 16(!) different
97			string types, but there is no unsigned32 type for example. Different
98			applications work around this in different ways, for example, SNMP defines
99			application-specific Gauge32, Counter32 and Unsigned32, which are mapped
100			to two different tags: you can distinguish between Counter32 and the
101			others, but not between Gause32 and Unsigned32, without the ASN.1 schema.
102
103			Ugh.
104
105			=head2 DECODED BER REPRESENTATION
106
107			This module represents every BER value as a 4-element tuple (actually an
108			array-reference):
109
110			[CLASS, TAG, CONSTRUCTED, DATA]
111
112	root	1.6	To avoid non-descriptive hardcoded array index numbers, this module
113			defines symbolic constants to access these members: C<BER_CLASS>,
114			C<BER_TAG>, C<BER_CONSTRUCTED> and C<BER_DATA>.
115
116			Also, the first three members are integers with a little caveat: for
117			performance reasons, these are readonly and shared, so you must not modify
118			them (increment, assign to them etc.) in any way. You may modify the
119			I<DATA> member, and you may re-assign the array itself, e.g.:
120
121			$ber = ber_decode $binbuf;
122
123			# the following is NOT legal:
124	root	1.10	$ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)
125	root	1.6
126			# but all of the following are fine:
127			$ber->[BER_DATA] = "string";
128			$ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
129	root	1.11	@$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);
130	root	1.6
131	root	1.4	I<CLASS> is something like a namespace for I<TAG>s - there is the
132			C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
133			implementations, the C<ASN_APPLICATION> namespace which defines tags for
134			specific applications (for example, the SNMP C<Unsigned32> type is in this
135			namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g.
136			for C<CHOICE>) and a private namespace (C<ASN_PRIVATE>).
137
138			The meaning of the I<TAG> depends on the namespace, and defines a
139			(partial) interpretation of the data value. For example, right now, SNMP
140			application namespace knowledge ix hardcoded into this module, so it
141			knows that SNMP C<Unsigned32> values need to be decoded into actual perl
142			integers.
143
144			The most common tags in the C<ASN_UNIVERSAL> namespace are
145			C<ASN_INTEGER32>, C<ASN_BIT_STRING>, C<ASN_NULL>, C<ASN_OCTET_STRING>,
146			C<ASN_OBJECT_IDENTIFIER>, C<ASN_SEQUENCE>, C<ASN_SET> and
147			C<ASN_IA5_STRING>.
148
149			The most common tags in SNMP's C<ASN_APPLICATION> namespace
150			are C<SNMP_IPADDRESS>, C<SNMP_COUNTER32>, C<SNMP_UNSIGNED32>,
151			C<SNMP_TIMETICKS>, C<SNMP_OPAQUE> and C<SNMP_COUNTER64>.
152
153			The I<CONSTRUCTED> flag is really just a boolean - if it is false, the
154			the value is "primitive" and contains no subvalues, kind of like a
155			non-reference perl scalar. IF it is true, then the value is "constructed"
156			which just means it contains a list of subvalues which this module will
157			en-/decode as BER tuples themselves.
158
159			The I<DATA> value is either a reference to an array of further tuples (if
160			the value is I<CONSTRUCTED>), some decoded representation of the value,
161			if this module knows how to decode it (e.g. for the integer types above)
162			or a binary string with the raw octets if this module doesn't know how to
163			interpret the namespace/tag.
164
165			Thus, you can always decode a BER data structure and at worst you get a
166			string in place of some nice decoded value.
167
168			See the SYNOPSIS for an example of such an encoded tuple representation.
169
170	root	1.7	=head2 DECODING AND ENCODING
171
172			=over
173
174			=item $tuple = ber_decoded $bindata
175
176			Decodes binary BER data in C<$bindata> and returns the resulting BER
177			tuple. Croaks on any decoding error, so the returned C<$tuple> is always
178			valid.
179
180			=item $bindata = ber_encode $tuple
181
182			Encodes the BER tuple into a BER/DER data structure.
183
184			=back
185
186	root	1.6	=head2 HELPER FUNCTIONS
187
188			Working with a 4-tuple for every value can be annoying. Or, rather, I<is>
189			annoying. To reduce this a bit, this module defines a number of helper
190			functions, both to match BER tuples and to conmstruct BER tuples:
191
192			=head3 MATCH HELPERS
193
194			Thse functions accept a BER tuple as first argument and either paertially
195			or fully match it. They often come in two forms, one which exactly matches
196			a value, and one which only matches the type and returns the value.
197
198			They do check whether valid tuples are passed in and croak otherwise. As
199			a ease-of-use exception, they usually also accept C<undef> instead of a
200			tuple reference. in which case they silently fail to match.
201
202			=over
203
204			=item $bool = ber_is $tuple, $class, $tag, $constructed, $data
205
206			This takes a BER C<$tuple> and matches its elements agains the privded
207			values, all of which are optional - values that are either missing or
208			C<undef> will be ignored, the others will be matched exactly (e.g. as if
209			you used C<==> or C<eq> (for C<$data>)).
210
211			Some examples:
212
213			ber_is $tuple, ASN_UNIVERSAL, ASN_SEQUENCE, 1
214			orf die "tuple is not an ASN SEQUENCE";
215
216			ber_is $tuple, ASN_UNIVERSAL, ASN_NULL
217			or die "tuple is not an ASN NULL value";
218
219			ber_is $tuple, ASN_UNIVERSAL, ASN_INTEGER32, 0, 50
220			or die "BER integer must be 50";
221
222			=item $seq = ber_is_seq $tuple
223
224			Returns the sequence members (the array of subvalues) if the C<$tuple> is
225			an ASN SEQUENCE, i.e. the C<BER_DATA> member. If the C<$tuple> is not a
226			sequence it returns C<undef>. For example, SNMP version 1/2c/3 packets all
227			consist of an outer SEQUENCE value:
228
229			my $ber = ber_decode $snmp_data;
230
231			my $snmp = ber_is_seq $ber
232			or die "SNMP packet invalid: does not start with SEQUENCE";
233
234			# now we know $snmp is a sequence, so decode the SNMP version
235
236			my $version = ber_is_i32 $snmp->[0]
237			or die "SNMP packet invalid: does not start with version number";
238
239			=item $bool = ber_is_i32 $tuple, $i32
240
241			Returns a true value if the C<$tuple> represents an ASN INTEGER32 with
242			the value C<$i32>.
243
244			=item $i32 = ber_is_i32 $tuple
245
246			Returns true (and extracts the integer value) if the C<$tuple> is an ASN
247			INTEGER32. For C<0>, this function returns a special value that is 0 but
248			true.
249
250			=item $bool = ber_is_oid $tuple, $oid_string
251
252			Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
253	root	1.12	that exactly matches C<$oid_string>. Example:
254	root	1.6
255			ber_is_oid $tuple, "1.3.6.1.4"
256			or die "oid must be 1.3.6.1.4";
257
258			=item $oid = ber_is_oid $tuple
259
260			Returns true (and extracts the OID string) if the C<$tuple> is an ASN
261			OBJECT IDENTIFIER. Otherwise, it returns C<undef>.
262
263			=back
264
265			=head3 CONSTRUCTION HELPERS
266
267			=over
268
269			=item $tuple = ber_i32 $value
270
271			Constructs a new C<ASN_INTEGER32> tuple.
272
273			=back
274
275	root	1.2	=head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1>
276
277			This module is I<not> the XS version of L<Convert::BER>, but a different
278			take at doing the same thing. I imagine this module would be a good base
279	root	1.4	for speeding up either of these, or write a similar module, or write your
280	root	1.2	own LDAP or SNMP module for example.
281
282	root	1.1	=cut
283
284			package Convert::BER::XS;
285
286			use common::sense;
287
288			use XSLoader ();
289			use Exporter qw(import);
290
291	root	1.13	our $VERSION;
292	root	1.1
293	root	1.13	BEGIN {
294			$VERSION = 0.7;
295			XSLoader::load __PACKAGE__, $VERSION;
296			}
297	root	1.1
298			our %EXPORT_TAGS = (
299	root	1.4	const => [qw(
300	root	1.1	BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
301	root	1.4
302	root	1.13	ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
303			ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
304			ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
305			ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
306			ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
307			ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
308
309			ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
310
311			BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
312			BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
313			BER_TYPE_IPADDRESS BER_TYPE_CROAK
314			)],
315			const_snmp => [qw(
316	root	1.4	SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
317			)],
318			encode => [qw(
319			ber_decode
320			ber_is ber_is_seq ber_is_i32 ber_is_oid
321			)],
322			decode => [qw(
323			ber_encode
324	root	1.13	ber_i32
325	root	1.1	)],
326			);
327
328			our @EXPORT_OK = map @$_, values %EXPORT_TAGS;
329
330	root	1.4	$EXPORT_TAGS{all} = \@EXPORT_OK;
331
332	root	1.13	=head1 PROFILES
333
334			While any BER data can be correctly encoded and decoded out of the box, it
335			can be inconvenient to have to manually decode some values into a "better"
336			format: for instance, SNMP TimeTicks values are decoded into the raw octet
337			strings of their BER representation, which is quite hard to decode. With
338			profiles, you can change which class/tag combinations map to which decoder
339			function inside C<ber_decode> (and of course also which encoder functions
340			are used in C<ber_encode>).
341
342			This works by mapping specific class/tag combinations to an internal "ber
343			type".
344
345			The default profile supports the standard ASN.1 types, but no
346			application-specific ones. This means that class/tag combinations not in
347			the base set of ASN.1 are decoded into their raw octet strings.
348
349	root	1.15	C<Convert::BER::XS> defines two profile variables you can use out of the box:
350	root	1.13
351			=over
352
353			=item C<$Convert::BER::XS::DEFAULT_PROFILE>
354
355			This is the default profile, i.e. the profile that is used when no
356			profile is specified for de-/encoding.
357
358	root	1.15	You can modify it, but remember that this modifies the defaults for all
359			callers that rely on the default profile.
360	root	1.13
361			=item C<$Convert::BER::XS::SNMP_PROFILE>
362
363			A profile with mappings for SNMP-specific application tags added. This is
364			useful when de-/encoding SNMP data.
365
366			Example:
367	root	1.15
368	root	1.13	$ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;
369
370			=back
371
372			=head2 The Convert::BER::XS::Profile class
373
374			=over
375
376			=item $profile = new Convert::BER::XS::Profile
377
378			Create a new profile. The profile will be identical to the default
379			profile.
380
381			=item $profile->set ($class, $tag, $type)
382
383			Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
384			which must be one of the C<BER_TYPE_*> constants.
385
386			Note that currently, the mapping is stored in a flat array, so large
387			values of C<$tag> will consume large amounts of memory.
388
389			Example:
390	root	1.15
391	root	1.13	$profile = new Convert::BER::XS::Profile;
392			$profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
393			$ber = ber_decode $data, $profile;
394
395			=item $type = $profile->get ($class, $tag)
396
397			Returns the BER type mapped to the given C<$class>/C<$tag> combination.
398
399			=back
400
401			=head2 BER TYPES
402
403			This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
404			combination to any C<BER_TYPE_*>.
405
406			=over
407
408			=item C<BER_TYPE_BYTES>
409
410			The raw octets of the value. This is the default type for unknown tags and
411			de-/encodes the value as if it were an octet string, i.e. by copying the
412			raw bytes.
413
414			=item C<BER_TYPE_UTF8>
415
416			Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
417			(without validation!) and encodes a perl unicode string into a UTF-8 BER
418			string.
419
420			=item C<BER_TYPE_UCS2>
421
422			Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
423	root	1.14	string.
424	root	1.13
425			=item C<BER_TYPE_UCS4>
426
427			Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
428	root	1.14	string.
429	root	1.13
430			=item C<BER_TYPE_INT>
431
432			Encodes and decodes a BER integer value to a perl integer scalar. This
433			should correctly handle 64 bit signed and unsigned values.
434
435			=item C<BER_TYPE_OID>
436
437			Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
438			dot, e.g. C<1.3.6.1.213>.
439
440			=item C<BER_TYPE_RELOID>
441
442	root	1.15	Same as C<BER_TYPE_OID> but uses relative object identifier
443			encoding: ASN.1 has this hack of encoding the first two OID components
444			into a single integer in a weird attempt to save an insignificant amount
445			of space in an otherwise wasteful encoding, and relative OIDs are
446			basically OIDs without this hack. The practical difference is that the
447			second component of an OID can only have the values 1..40, while relative
448			OIDs do not have this restriction.
449	root	1.13
450			=item C<BER_TYPE_NULL>
451
452			Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
453			C<ASN_NULL> type, regardless of the perl value.
454
455			=item C<BER_TYPE_BOOL>
456
457			Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
458			boolean value into an C<ASN_BOOLEAN>.
459
460			=item C<BER_TYPE_REAL>
461
462			Decodes/encodes a BER real value. NOT IMPLEMENTED.
463
464			=item C<BER_TYPE_IPADDRESS>
465
466	root	1.15	Decodes/encodes a four byte string into an IPv4 dotted-quad address string
467			in Perl. Given the obsolete nature of this type, this is a low-effort
468	root	1.13	implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
469	root	1.15	so it won't handle all string forms supported by C<inet_aton> for example.
470	root	1.13
471			=item C<BER_TYPE_CROAK>
472
473			Always croaks when encountered during encoding or decoding - the
474			default behaviour when encountering an unknown type is to treat it as
475			C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
476			error for some types, then CyBER_TYPE_CROAK> is for you.
477
478			=back
479
480			=cut
481
482			our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
483			our $SNMP_PROFILE = new Convert::BER::XS::Profile;
484
485			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
486			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
487			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
488			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
489			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE , BER_TYPE_IPADDRESS);
490			$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);
491
492			$DEFAULT_PROFILE->_set_default;
493
494	root	1.1	1;
495
496	root	1.13	=head2 LIMITATIONS
497
498			This module can only en-/decode 64 bit signed and unsigned integers, and
499			only when your perl supports those.
500	root	1.4
501	root	1.13	OBJECT IDENTIFIEERS cannot have unlimited length, although the limit is
502			much larger than e.g. the one imposed by SNMP or other protocols.
503	root	1.4
504	root	1.14	REAL values are not supported and will croak.
505
506			This module has undergone little to no testing so far.
507
508	root	1.1	=head1 AUTHOR
509
510			Marc Lehmann <schmorp@schmorp.de>
511			http://software.schmorp.de/pkg/Convert-BER-XS
512
513			=cut
514