ASN.1 and Certificate Decoder

Open a certificate in a hex viewer and it looks like noise. It is not. It is one of the most regular file formats in daily use, and nearly all of it follows a single rule repeated at every level: say what kind of thing comes next, say how long it is, then give the thing itself.

That rule belongs to ASN.1, Abstract Syntax Notation One, a way of describing data structures that the ITU publishes as Recommendation X.680. The notation is only a description, in the way a form is only a set of blank fields. Turning a filled-in form into bytes is a separate job, covered by a separate document, X.690, which defines the Basic Encoding Rules and two stricter subsets of them.

Tag, length, value

Every element starts with an identifier octet. Its top two bits give a class: universal for the built-in types, application, context-specific, or private. The next bit says whether the contents are a single primitive value or are constructed from further elements. The low five bits are the tag number. Universal number 2 is an INTEGER, 6 is an OBJECT IDENTIFIER, 16 is a SEQUENCE and 17 is a SET. If all five bits are ones, the real number follows in later octets, seven bits at a time.

Then comes the length. A length up to 127 fits in one octet. Anything longer uses the long form, where the first octet has its top bit set and its other bits count how many length octets follow. That is why so many certificates begin with the same two bytes, 30 82. The 30 is a universal, constructed element with number 16, a SEQUENCE. The 82 announces a two-octet length, which covers anything from 256 to 65,535 bytes.

The value fills the rest. For a constructed element the value is simply more elements, each with its own tag and length, and so a file becomes a tree. A reader who knows nothing about certificates can still walk that tree from the first byte to the last, which is exactly what a generic decoder does. It needs no schema to find the boundaries, only to know what they mean.

Some values have rules of their own. An INTEGER is stored in two’s complement with the most significant octet first, in as few octets as possible, so 127 is the single octet 7f while 128 needs two, 00 80, because a leading 80 alone would read as a negative number. A BIT STRING spends its first octet counting the unused bits at the end. A BOOLEAN is one octet, zero for FALSE.

How an identifier is packed

Object identifiers are the names in this world. Each is a path of numbers through a registry tree, written with dots, and each algorithm, attribute and extension has one. The path 1.2.840.113549.1.1.11 means RSA signatures with SHA-256. The path 2.5.29.17 is the certificate extension that lists alternative names.

X.690 packs these compactly. The first two numbers are combined into one as forty times the first plus the second, which works because the first can only be 0, 1 or 2. Every number is then written in base 128, most significant group first, with the top bit set on every octet except the last. The standard gives its own small case: the identifier 2.100.3 becomes the three octets 81 34 03.

The longer example can be checked by hand, and pasted into the decoder above to confirm. Take 1.2.840.113549.1.1.11. Forty times 1 plus 2 is 42, or 2a. Then 840 is six 128s and 72 left over, giving 86 48. Then 113549 is six 16384s, 119 128s and 13, giving 86 f7 0d. The last three numbers are small enough to be single octets: 01 01 0b. With the tag 06 and the length 09 in front, the whole element is 06 09 2a 86 48 86 f7 0d 01 01 0b.

Without a table those dotted paths are just numbers. A decoder that names them carries a list copied from the documents that assigned them. An identifier outside the list should be shown as digits. Guessing is worse.

Inside a certificate

RFC 5280 defines the certificate used on the internet as a SEQUENCE of exactly three things: the part to be signed, the algorithm used to sign it, and the signature as a BIT STRING. All of the content a person cares about is in the first of the three.

That first part holds, in order, a version number, a serial number, the signature algorithm again, the name of the issuer, two dates, the name of the subject, the subject’s public key and, in version 3, a list of extensions. The serial number is a positive integer chosen by the issuer, and the profile limits it to 20 octets. Names are sequences of small sets, each pairing an identifier such as common name or organization with a string, and they are usually printed with the short keys CN, O and C from the LDAP string format.

The two dates are a small lesson in old formats. Through the year 2049 the profile requires a form with a two-digit year, and from 2050 a form with four digits. A two-digit year of 50 or more is read as the 1900s and anything lower as the 2000s. Both must end in Z, for universal time, and both must include seconds.

The public key is wrapped the same way whatever its kind. An algorithm identifier says whether it is RSA, an elliptic curve key on a named curve, or one of the newer Edwards curve keys, and a BIT STRING carries the key. For RSA, the contents of that BIT STRING are themselves an encoded structure of two integers, the modulus and the public exponent. The Baseline Requirements that browsers and certificate authorities work under ask for an RSA modulus of at least 2048 bits.

Extensions repeat the trick of a structure inside a string. Each one is an identifier, an optional flag marking it critical, and an OCTET STRING whose contents are another complete encoding: a list of names, a set of permitted key uses, a flag saying whether the subject is itself an authority, or addresses for revocation data. The same nesting is used for the timestamps that certificate transparency logs return, which travel as an opaque block inside one more extension.

Requests, keys and signatures use the same grammar. A signing request is a subject, a public key and some attributes, signed with the matching key. An RSA private key is nine integers. An elliptic curve signature is two.

PEM, BER and DER

None of that is text, and much of the tooling around certificates wants text. PEM is the wrapper. RFC 7468 describes it as the bytes in Base64, between a line reading -----BEGIN CERTIFICATE----- and a matching line that begins -----END, with the Base64 wrapped at exactly 64 characters per line. The label names what should be inside, such as a certificate, a request or a private key, but it is only a label. Several blocks can sit in one file, one after another.

The basic rules leave the sender choices. A length may be written with more octets than it needs. A constructed element may use an indefinite length, closed later by two zero octets. A string may be split into pieces. TRUE may be any non-zero octet. All of these decode to the same value, and for most data that freedom is harmless.

For signed data it is fatal. RFC 5280 says the signature is computed over the DER encoding of the part to be signed, so signer and checker must arrive at the very same bytes. DER, the Distinguished Encoding Rules, removes every choice: lengths in the fewest octets, never indefinite, strings never split, unused bits zero, the members of a SET OF in sorted order, and TRUE written as 0xFF only. X.690 defines a canonical encoding as one with no implementation-dependent options, a one-to-one mapping between values and encodings, and that is the property a signature needs. A careful decoder reads the looser forms anyway and says which rule each one breaks.

What decoding cannot tell you

A decoder shows what a certificate says. It cannot show whether any of it is true. The validation procedure in RFC 5280 asks different questions. Does the signature verify under the issuer’s key? Does the current time fall inside the validity period? Has the certificate been revoked? Does each name in the chain match the one above it, back to an authority the checker already relies on? None of those answers is in the file. A forged certificate decodes as cleanly as a real one.

There are limits on the reading itself, too. A context-specific tag such as [0] replaces the type it stands for, so without the schema the bytes under it are only bytes. Whether a SET was meant as an ordered-by-tag set or a sorted list cannot be known from the encoding alone. And a string type that allows character set switching may not display the way its author intended.

One kind of file deserves more care than the rest. A private key decodes into the very numbers that must stay secret. The RFC that defines the text format notes that such data often originates from untrusted sources, and a pasted key makes the opposite trip, into a tool its owner has not inspected. Inspecting a public certificate anywhere is fine. A private key belongs only in tools that demonstrably keep it on the machine, and the safest habit is to inspect a throwaway key with the same structure instead.