ViewVC Help
View File | Revision Log | Show Annotations | Download File
/cvs/Convert-BER-XS/XS.pm
(Generate patch)

Comparing Convert-BER-XS/XS.pm (file contents):
Revision 1.5 by root, Fri Apr 19 19:46:41 2019 UTC vs.
Revision 1.16 by root, Sat Apr 20 02:07:45 2019 UTC

5=head1 SYNOPSIS 5=head1 SYNOPSIS
6 6
7 use Convert::BER::XS ':all'; 7 use Convert::BER::XS ':all';
8 8
9 my $ber = ber_decode $buf 9 my $ber = ber_decode $buf
10 or die "unable to decode SNMP v1/v2c Message"; 10 or die "unable to decode SNMP message";
11 11
12 # the above results in a data structure consisting of (class, tag, 12 # The above results in a data structure consisting of
13 # (class, tag, # constructed, data)
13 # constructed, data) tuples. here is such a message, SNMPv1 trap 14 # tuples. Below is such a message, SNMPv1 trap
14 # with a cisoc mac change notification 15 # with a Cisco mac change notification.
16 # Did you know that Cisco is in the news almost
17 # every week because # of some backdoor password
18 # or other extremely stupid security bug?
15 19
16 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, 20 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
17 [ 21 [
18 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1 22 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
19 [ ASN_UNIVERSAL, 4, 0, "public" ], # community 23 [ ASN_UNIVERSAL, 4, 0, "public" ], # community
20 [ ASN_CONTEXT, 4, 1, # CHOICE, constructed 24 [ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
21 [ 25 [
22 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid 26 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
23 [ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1 27 [ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1
24 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap 28 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
25 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap 29 [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
26 [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks 30 [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
27 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist 31 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
28 [ 32 [
29 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair 33 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
30 [ 34 [
31 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ], # the oid 35 [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
32 [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value 36 [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
33 ] 37 ]
34 ] 38 ]
35 ], 39 ],
36 ... 40 ...
62 66
63 my $buf = ber_encode $ber; 67 my $buf = ber_encode $ber;
64 68
65=head1 DESCRIPTION 69=head1 DESCRIPTION
66 70
71WARNING: Before release 1.0, the API is not considered stable in any way.
72
67This module implements a I<very> low level BER/DER en-/decoder. 73This module implements a I<very> low level BER/DER en-/decoder.
68 74
69If is tuned for low memory and high speed, while still maintaining some 75If is tuned for low memory and high speed, while still maintaining some
70level of user-friendlyness. 76level of user-friendlyness.
71 77
72Currently, not much is documented, as this is an initial release to
73reserve CPAN namespace, stay tuned for a few days.
74
75=head2 ASN.1/BER/DER/... BASICS 78=head2 ASN.1/BER/DER/... BASICS
76 79
77ASN.1 is a strange language that can be sed to describe protocols and 80ASN.1 is a strange language that can be used to describe protocols and
78data structures. It supports various mappings to JSON, XML, but most 81data structures. It supports various mappings to JSON, XML, but most
79importantly, to a various binary encodings such as BER, that is the topic 82importantly, to a various binary encodings such as BER, that is the topic
80of this module, and is used in SNMP or LDAP for example. 83of this module, and is used in SNMP or LDAP for example.
81 84
82While ASN.1 defines a schema that is useful to interpret encoded data, 85While ASN.1 defines a schema that is useful to interpret encoded data,
83the BER encoding is actually somehat self-describing: you might not know 86the BER encoding is actually somewhat self-describing: you might not know
84whether something is a string or a number or a sequence or something else, 87whether something is a string or a number or a sequence or something else,
85but you can nevertheless decode the overall structure, even if you end up 88but you can nevertheless decode the overall structure, even if you end up
86with just a binary blob for the actual value. 89with just a binary blob for the actual value.
87 90
88This works because BER values are tagged with a type and a namespace, 91This works because BER values are tagged with a type and a namespace,
89and also have a flag that says whther a value consists of subvalues (is 92and also have a flag that says whether a value consists of subvalues (is
90"constructed") or not (is "primitive"). 93"constructed") or not (is "primitive").
91 94
92Tags are simple integers, and ASN.1 defines a somewhat weird assortment of 95Tags are simple integers, and ASN.1 defines a somewhat weird assortment of
93those - for example, you have 32 bit signed integers and 16(!) different 96those - for example, you have 32 bit signed integers and 16(!) different
94string types, but there is no unsigned32 type for example. Different 97string types, but there is no unsigned32 type for example. Different
104This module represents every BER value as a 4-element tuple (actually an 107This module represents every BER value as a 4-element tuple (actually an
105array-reference): 108array-reference):
106 109
107 [CLASS, TAG, CONSTRUCTED, DATA] 110 [CLASS, TAG, CONSTRUCTED, DATA]
108 111
112To avoid non-descriptive hardcoded array index numbers, this module
113defines symbolic constants to access these members: C<BER_CLASS>,
114C<BER_TAG>, C<BER_CONSTRUCTED> and C<BER_DATA>.
115
116Also, the first three members are integers with a little caveat: for
117performance reasons, these are readonly and shared, so you must not modify
118them (increment, assign to them etc.) in any way. You may modify the
119I<DATA> member, and you may re-assign the array itself, e.g.:
120
121 $ber = ber_decode $binbuf;
122
123 # the following is NOT legal:
124 $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)
125
126 # but all of the following are fine:
127 $ber->[BER_DATA] = "string";
128 $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
129 @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);
130
109I<CLASS> is something like a namespace for I<TAG>s - there is the 131I<CLASS> is something like a namespace for I<TAG>s - there is the
110C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1 132C<ASN_UNIVERSAL> namespace which defines tags common to all ASN.1
111implementations, the C<ASN_APPLICATION> namespace which defines tags for 133implementations, the C<ASN_APPLICATION> namespace which defines tags for
112specific applications (for example, the SNMP C<Unsigned32> type is in this 134specific applications (for example, the SNMP C<Unsigned32> type is in this
113namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g. 135namespace), a special-purpose context namespace (C<ASN_CONTEXT>, used e.g.
143Thus, you can always decode a BER data structure and at worst you get a 165Thus, you can always decode a BER data structure and at worst you get a
144string in place of some nice decoded value. 166string in place of some nice decoded value.
145 167
146See the SYNOPSIS for an example of such an encoded tuple representation. 168See the SYNOPSIS for an example of such an encoded tuple representation.
147 169
170=head2 DECODING AND ENCODING
171
172=over
173
174=item $tuple = ber_decoded $bindata
175
176Decodes binary BER data in C<$bindata> and returns the resulting BER
177tuple. Croaks on any decoding error, so the returned C<$tuple> is always
178valid.
179
180=item $bindata = ber_encode $tuple
181
182Encodes the BER tuple into a BER/DER data structure.
183
184=back
185
186=head2 HELPER FUNCTIONS
187
188Working with a 4-tuple for every value can be annoying. Or, rather, I<is>
189annoying. To reduce this a bit, this module defines a number of helper
190functions, both to match BER tuples and to conmstruct BER tuples:
191
192=head3 MATCH HELPERS
193
194Thse functions accept a BER tuple as first argument and either paertially
195or fully match it. They often come in two forms, one which exactly matches
196a value, and one which only matches the type and returns the value.
197
198They do check whether valid tuples are passed in and croak otherwise. As
199a ease-of-use exception, they usually also accept C<undef> instead of a
200tuple reference. in which case they silently fail to match.
201
202=over
203
204=item $bool = ber_is $tuple, $class, $tag, $constructed, $data
205
206This takes a BER C<$tuple> and matches its elements agains the privded
207values, all of which are optional - values that are either missing or
208C<undef> will be ignored, the others will be matched exactly (e.g. as if
209you used C<==> or C<eq> (for C<$data>)).
210
211Some examples:
212
213 ber_is $tuple, ASN_UNIVERSAL, ASN_SEQUENCE, 1
214 orf die "tuple is not an ASN SEQUENCE";
215
216 ber_is $tuple, ASN_UNIVERSAL, ASN_NULL
217 or die "tuple is not an ASN NULL value";
218
219 ber_is $tuple, ASN_UNIVERSAL, ASN_INTEGER32, 0, 50
220 or die "BER integer must be 50";
221
222=item $seq = ber_is_seq $tuple
223
224Returns the sequence members (the array of subvalues) if the C<$tuple> is
225an ASN SEQUENCE, i.e. the C<BER_DATA> member. If the C<$tuple> is not a
226sequence it returns C<undef>. For example, SNMP version 1/2c/3 packets all
227consist of an outer SEQUENCE value:
228
229 my $ber = ber_decode $snmp_data;
230
231 my $snmp = ber_is_seq $ber
232 or die "SNMP packet invalid: does not start with SEQUENCE";
233
234 # now we know $snmp is a sequence, so decode the SNMP version
235
236 my $version = ber_is_i32 $snmp->[0]
237 or die "SNMP packet invalid: does not start with version number";
238
239=item $bool = ber_is_i32 $tuple, $i32
240
241Returns a true value if the C<$tuple> represents an ASN INTEGER32 with
242the value C<$i32>.
243
244=item $i32 = ber_is_i32 $tuple
245
246Returns true (and extracts the integer value) if the C<$tuple> is an ASN
247INTEGER32. For C<0>, this function returns a special value that is 0 but
248true.
249
250=item $bool = ber_is_oid $tuple, $oid_string
251
252Returns true if the C<$tuple> represents an ASN_OBJECT_IDENTIFIER
253that exactly matches C<$oid_string>. Example:
254
255 ber_is_oid $tuple, "1.3.6.1.4"
256 or die "oid must be 1.3.6.1.4";
257
258=item $oid = ber_is_oid $tuple
259
260Returns true (and extracts the OID string) if the C<$tuple> is an ASN
261OBJECT IDENTIFIER. Otherwise, it returns C<undef>.
262
263=back
264
265=head3 CONSTRUCTION HELPERS
266
267=over
268
269=item $tuple = ber_i32 $value
270
271Constructs a new C<ASN_INTEGER32> tuple.
272
273=back
274
148=head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1> 275=head2 RELATIONSHIP TO L<Convert::BER> and L<Convert::ASN1>
149 276
150This module is I<not> the XS version of L<Convert::BER>, but a different 277This module is I<not> the XS version of L<Convert::BER>, but a different
151take at doing the same thing. I imagine this module would be a good base 278take at doing the same thing. I imagine this module would be a good base
152for speeding up either of these, or write a similar module, or write your 279for speeding up either of these, or write a similar module, or write your
159use common::sense; 286use common::sense;
160 287
161use XSLoader (); 288use XSLoader ();
162use Exporter qw(import); 289use Exporter qw(import);
163 290
164our $VERSION = 0.1; 291our $VERSION;
165 292
293BEGIN {
294 $VERSION = 0.7;
166XSLoader::load __PACKAGE__, $VERSION; 295 XSLoader::load __PACKAGE__, $VERSION;
296}
167 297
168our %EXPORT_TAGS = ( 298our %EXPORT_TAGS = (
169 const => [qw( 299 const => [qw(
170 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA 300 BER_CLASS BER_TAG BER_CONSTRUCTED BER_DATA
171 301
172 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER ASN_TAG_BER ASN_TAG_MASK 302 ASN_BOOLEAN ASN_INTEGER32 ASN_BIT_STRING ASN_OCTET_STRING ASN_NULL ASN_OBJECT_IDENTIFIER
173 ASN_CONSTRUCTED ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE ASN_CLASS_MASK ASN_CLASS_SHIFT 303 ASN_OBJECT_DESCRIPTOR ASN_OID ASN_EXTERNAL ASN_REAL ASN_SEQUENCE ASN_ENUMERATED
174 ASN_SEQUENCE 304 ASN_EMBEDDED_PDV ASN_UTF8_STRING ASN_RELATIVE_OID ASN_SET ASN_NUMERIC_STRING
175 305 ASN_PRINTABLE_STRING ASN_TELETEX_STRING ASN_T61_STRING ASN_VIDEOTEX_STRING ASN_IA5_STRING
306 ASN_ASCII_STRING ASN_UTC_TIME ASN_GENERALIZED_TIME ASN_GRAPHIC_STRING ASN_VISIBLE_STRING
307 ASN_ISO646_STRING ASN_GENERAL_STRING ASN_UNIVERSAL_STRING ASN_CHARACTER_STRING ASN_BMP_STRING
308
309 ASN_UNIVERSAL ASN_APPLICATION ASN_CONTEXT ASN_PRIVATE
310
311 BER_TYPE_BYTES BER_TYPE_UTF8 BER_TYPE_UCS2 BER_TYPE_UCS4 BER_TYPE_INT
312 BER_TYPE_OID BER_TYPE_RELOID BER_TYPE_NULL BER_TYPE_BOOL BER_TYPE_REAL
313 BER_TYPE_IPADDRESS BER_TYPE_CROAK
314 )],
315 const_snmp => [qw(
176 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64 316 SNMP_IPADDRESS SNMP_COUNTER32 SNMP_UNSIGNED32 SNMP_TIMETICKS SNMP_OPAQUE SNMP_COUNTER64
177 )], 317 )],
178 encode => [qw( 318 encode => [qw(
179 ber_decode 319 ber_decode
180 ber_is ber_is_seq ber_is_i32 ber_is_oid 320 ber_is ber_is_seq ber_is_i32 ber_is_oid
181 )], 321 )],
182 decode => [qw( 322 decode => [qw(
183 ber_encode 323 ber_encode
324 ber_i32
184 )], 325 )],
185); 326);
186 327
187our @EXPORT_OK = map @$_, values %EXPORT_TAGS; 328our @EXPORT_OK = map @$_, values %EXPORT_TAGS;
188 329
189$EXPORT_TAGS{all} = \@EXPORT_OK; 330$EXPORT_TAGS{all} = \@EXPORT_OK;
190 331
332=head1 PROFILES
333
334While any BER data can be correctly encoded and decoded out of the box, it
335can be inconvenient to have to manually decode some values into a "better"
336format: for instance, SNMP TimeTicks values are decoded into the raw octet
337strings of their BER representation, which is quite hard to decode. With
338profiles, you can change which class/tag combinations map to which decoder
339function inside C<ber_decode> (and of course also which encoder functions
340are used in C<ber_encode>).
341
342This works by mapping specific class/tag combinations to an internal "ber
343type".
344
345The default profile supports the standard ASN.1 types, but no
346application-specific ones. This means that class/tag combinations not in
347the base set of ASN.1 are decoded into their raw octet strings.
348
349C<Convert::BER::XS> defines two profile variables you can use out of the box:
350
351=over
352
353=item C<$Convert::BER::XS::DEFAULT_PROFILE>
354
355This is the default profile, i.e. the profile that is used when no
356profile is specified for de-/encoding.
357
358You can modify it, but remember that this modifies the defaults for all
359callers that rely on the default profile.
360
361=item C<$Convert::BER::XS::SNMP_PROFILE>
362
363A profile with mappings for SNMP-specific application tags added. This is
364useful when de-/encoding SNMP data.
365
366Example:
367
368 $ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;
369
370=back
371
372=head2 The Convert::BER::XS::Profile class
373
374=over
375
376=item $profile = new Convert::BER::XS::Profile
377
378Create a new profile. The profile will be identical to the default
379profile.
380
381=item $profile->set ($class, $tag, $type)
382
383Sets the mapping for the given C<$class>/C<$tag> combination to C<$type>,
384which must be one of the C<BER_TYPE_*> constants.
385
386Note that currently, the mapping is stored in a flat array, so large
387values of C<$tag> will consume large amounts of memory.
388
389Example:
390
391 $profile = new Convert::BER::XS::Profile;
392 $profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT);
393 $ber = ber_decode $data, $profile;
394
395=item $type = $profile->get ($class, $tag)
396
397Returns the BER type mapped to the given C<$class>/C<$tag> combination.
398
399=back
400
401=head2 BER TYPES
402
403This lists the predefined BER types - you can map any C<CLASS>/C<TAG>
404combination to any C<BER_TYPE_*>.
405
406=over
407
408=item C<BER_TYPE_BYTES>
409
410The raw octets of the value. This is the default type for unknown tags and
411de-/encodes the value as if it were an octet string, i.e. by copying the
412raw bytes.
413
414=item C<BER_TYPE_UTF8>
415
416Like C<BER_TYPE_BYTES>, but decodes the value as if it were a UTF-8 string
417(without validation!) and encodes a perl unicode string into a UTF-8 BER
418string.
419
420=item C<BER_TYPE_UCS2>
421
422Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-2 encoded
423string.
424
425=item C<BER_TYPE_UCS4>
426
427Similar to C<BER_TYPE_UTF8>, but treats the BER value as UCS-4 encoded
428string.
429
430=item C<BER_TYPE_INT>
431
432Encodes and decodes a BER integer value to a perl integer scalar. This
433should correctly handle 64 bit signed and unsigned values.
434
435=item C<BER_TYPE_OID>
436
437Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading
438dot, e.g. C<1.3.6.1.213>.
439
440=item C<BER_TYPE_RELOID>
441
442Same as C<BER_TYPE_OID> but uses relative object identifier
443encoding: ASN.1 has this hack of encoding the first two OID components
444into a single integer in a weird attempt to save an insignificant amount
445of space in an otherwise wasteful encoding, and relative OIDs are
446basically OIDs without this hack. The practical difference is that the
447second component of an OID can only have the values 1..40, while relative
448OIDs do not have this restriction.
449
450=item C<BER_TYPE_NULL>
451
452Decodes an C<ASN_NULL> value into C<undef>, and always encodes a
453C<ASN_NULL> type, regardless of the perl value.
454
455=item C<BER_TYPE_BOOL>
456
457Decodes an C<ASN_BOOLEAN> value into C<0> or C<1>, and encodes a perl
458boolean value into an C<ASN_BOOLEAN>.
459
460=item C<BER_TYPE_REAL>
461
462Decodes/encodes a BER real value. NOT IMPLEMENTED.
463
464=item C<BER_TYPE_IPADDRESS>
465
466Decodes/encodes a four byte string into an IPv4 dotted-quad address string
467in Perl. Given the obsolete nature of this type, this is a low-effort
468implementation that simply uses C<sprintf> and C<sscanf>-style conversion,
469so it won't handle all string forms supported by C<inet_aton> for example.
470
471=item C<BER_TYPE_CROAK>
472
473Always croaks when encountered during encoding or decoding - the
474default behaviour when encountering an unknown type is to treat it as
475C<BER_TYPE_BYTES>. When you don't want that but instead prefer a hard
476error for some types, then C<BER_TYPE_CROAK> is for you.
477
478=back
479
480=cut
481
482our $DEFAULT_PROFILE = new Convert::BER::XS::Profile;
483our $SNMP_PROFILE = new Convert::BER::XS::Profile;
484
485$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_IPADDRESS , BER_TYPE_IPADDRESS);
486$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER32 , BER_TYPE_INT);
487$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_UNSIGNED32, BER_TYPE_INT);
488$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_TIMETICKS , BER_TYPE_INT);
489$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_OPAQUE , BER_TYPE_IPADDRESS);
490$SNMP_PROFILE->set (ASN_APPLICATION, SNMP_COUNTER64 , BER_TYPE_INT);
491
492$DEFAULT_PROFILE->_set_default;
493
1911; 4941;
192 495
193=head2 BUGS / SHORTCOMINGs 496=head2 LIMITATIONS
194 497
195This module does have a number of SNMPisms hardcoded, such as the SNMP 498This module can only en-/decode 64 bit signed and unsigned integers, and
196tags for Unsigned32 and so on. More configurability is needed, and, if 499only when your perl supports those.
197ever implemented, will come in a form similar to how L<JSON::XS> and 500
198L<CBOR::XS> respresent things, namely with an object-oriented interface. 501OBJECT IDENTIFIEERs cannot have unlimited length, although the limit is
502much larger than e.g. the one imposed by SNMP or other protocols.
503
504REAL values are not supported and will croak.
505
506This module has undergone little to no testing so far.
199 507
200=head1 AUTHOR 508=head1 AUTHOR
201 509
202 Marc Lehmann <schmorp@schmorp.de> 510 Marc Lehmann <schmorp@schmorp.de>
203 http://software.schmorp.de/pkg/Convert-BER-XS 511 http://software.schmorp.de/pkg/Convert-BER-XS

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines