--- Convert-BER-XS/XS.pm	2019/04/19 16:49:02	1.3
+++ Convert-BER-XS/XS.pm	2019/04/19 19:46:29	1.4
@@ -1,6 +1,6 @@
 =head1 NAME
 
-Convert::BER::XS - I<very> low level BER decoding
+Convert::BER::XS - I<very> low level BER en-/decoding
 
 =head1 SYNOPSIS
 
@@ -58,11 +58,13 @@
        ) {
           ... and so on
 
+ # finally, let's encode it again and hope it results in the same bit pattern
+
+ my $buf = ber_encode $ber;
+
 =head1 DESCRIPTION
 
-This module implements a I<very> low level BER/DER decoder, and in the
-future, probably also an encoder (tell me if you want an encoder, this
-might speed up the process of getting one).
+This module implements a I<very> low level BER/DER en-/decoder.
 
 If is tuned for low memory and high speed, while still maintaining some
 level of user-friendlyness.
@@ -70,11 +72,84 @@
 Currently, not much is documented, as this is an initial release to
 reserve CPAN namespace, stay tuned for a few days.
 
+=head2 ASN.1/BER/DER/... BASICS
+
+ASN.1 is a strange language that can be sed to describe protocols and
+data structures. It supports various mappings to JSON, XML, but most
+importantly, to a various binary encodings such as BER, that is the topic
+of this module, and is used in SNMP or LDAP for example.
+
+While ASN.1 defines a schema that is useful to interpret encoded data,
+the BER encoding is actually somehat self-describing: you might not know
+whether something is a string or a number or a sequence or something else,
+but you can nevertheless decode the overall structure, even if you end up
+with just a binary blob for the actual value.
+
+This works because BER values are tagged with a type and a namespace,
+and also have a flag that says whther a value consists of subvalues (is
+"constructed") or not (is "primitive").
+
+Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
+those - for example, you have 32 bit signed integers and 16(!) different
+string types, but there is no unsigned32 type for example. Different
+applications work around this in different ways, for example, SNMP defines
+application-specific Gauge32, Counter32 and Unsigned32, which are mapped
+to two different tags: you can distinguish between Counter32 and the
+others, but not between Gause32 and Unsigned32, without the ASN.1 schema.
+
+Ugh.
+
+=head2 DECODED BER REPRESENTATION
+
+This module represents every BER value as a 4-element tuple (actually an
+array-reference):
+
+   [CLASS, TAG, CONSTRUCTED, DATA]
+
+I<CLASS> is something like a namespace for I<TAG>s - there is the
+C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
+implementations, the C<ASN_APPLICATION> namespace which defines tags for
+specific applications (for example, the SNMP C<Unsigned32> type is in this
+namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g.
+for C<CHOICE>) and a private namespace (C<ASN_PRIVATE>).
+
+The meaning of the I<TAG> depends on the namespace, and defines a
+(partial) interpretation of the data value. For example, right now, SNMP
+application namespace knowledge ix hardcoded into this module, so it
+knows that SNMP C<Unsigned32> values need to be decoded into actual perl
+integers.
+
+The most common tags in the C<ASN_UNIVERSAL> namespace are
+C<ASN_INTEGER32>, C<ASN_BIT_STRING>, C<ASN_NULL>, C<ASN_OCTET_STRING>,
+C<ASN_OBJECT_IDENTIFIER>, C<ASN_SEQUENCE>, C<ASN_SET> and
+C<ASN_IA5_STRING>.
+
+The most common tags in SNMP's C<ASN_APPLICATION> namespace
+are C<SNMP_IPADDRESS>, C<SNMP_COUNTER32>, C<SNMP_UNSIGNED32>,
+C<SNMP_TIMETICKS>, C<SNMP_OPAQUE> and C<SNMP_COUNTER64>.
+
+The I<CONSTRUCTED> flag is really just a boolean - if it is false, the
+the value is "primitive" and contains no subvalues, kind of like a
+non-reference perl scalar. IF it is true, then the value is "constructed"
+which just means it contains a list of subvalues which this module will
+en-/decode as BER tuples themselves.
+
+The I<DATA> value is either a reference to an array of further tuples (if
+the value is I<CONSTRUCTED>), some decoded representation of the value,
+if this module knows how to decode it (e.g. for the integer types above)
+or a binary string with the raw octets if this module doesn't know how to
+interpret the namespace/tag.
+
+Thus, you can always decode a BER data structure and at worst you get a
+string in place of some nice decoded value.
+
+See the SYNOPSIS for an example of such an encoded tuple representation.
+
 =head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1>
 
 This module is I<not> the XS version of L<Convert::BER>, but a different
 take at doing the same thing. I imagine this module would be a good base
-for speeding up either fo these, or write a similar module, or write your
+for speeding up either of these, or write a similar module, or write your
 own LDAP or SNMP module for example.
 
 =cut
@@ -91,20 +166,37 @@
 XSLoader::load __PACKAGE__, $VERSION;
 
 our %EXPORT_TAGS = (
-   all => [qw(
-      ber_decode
-      ber_is ber_is_seq ber_is_i32 ber_is_oid
+   const => [qw(
       BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
+
       ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER ASN_TAG_BER ASN_TAG_MASK
       ASN_CONSTRUCTED ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE ASN_CLASS_MASK ASN_CLASS_SHIFT
-      ASN_SEQUENCE ASN_IPADDRESS ASN_COUNTER32 ASN_UNSIGNED32 ASN_TIMETICKS ASN_OPAQUE ASN_COUNTER64
+      ASN_SEQUENCE
+      
+      SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
+   )],
+   encode => [qw(
+      ber_decode
+      ber_is ber_is_seq ber_is_i32 ber_is_oid
+   )],
+   decode => [qw(
+      ber_encode
    )],
 );
 
 our @EXPORT_OK = map @$_, values %EXPORT_TAGS;
 
+$EXPORT_TAGS{all} = \@EXPORT_OK;
+
 1;
 
+=head2 BUGS / SHORTCOMINGs
+
+This module does have a number of SNMPisms hardcoded, such as the SNMP
+tags for Unsigned32 and so on. More configurability is needed, and, if
+ever implemented, will come in a form similar to how L<JSON::XS> and
+L<CBOR::XS> respresent things, namely with an object-oriented interface.
+
 =head1 AUTHOR
 
  Marc Lehmann <schmorp@schmorp.de>