Design decisions

A few of OpenSSL's quirks make a lot more sense once you know the history. This page collects the ones that come up most often when reading the source.

Why provider-loaded crypto in 3.0?

The pre-3.0 model had every algorithm linked into libcrypto.so and gated by build-time flags (no-rc4, no-md5). That worked but had three problems:

1. FIPS required a forked OpenSSL — a separate fips-openssl source tree compiled to produce a static library that callers had to link in instead of libcrypto. The CMVP validation of one OpenSSL version did not transfer to the next, and there were long stretches with no validated upstream OpenSSL.
2. Algorithm policy (which algorithm to prefer when multiple exist) was hardcoded. There was no way to say "use the same source for AES, but get RSA from this PKCS#11 token".
3. External crypto — open-source community implementations of post-quantum, GOST, SM2 family, hardware tokens — had to ship as Engines, which was a smaller and increasingly creaky pluggability point.

The 3.0 redesign solved all three: providers are real shared modules, the FIPS one is in-tree, the property-based fetch lets policy flow from configuration, and OSSL_OP_* covers the same surface Engines did.

The reorg took two and a half years (Q1 2019 to Q3 2021), making 3.0 the first major release in three years and the largest since 0.9.6.

Why OSSL_PARAM instead of varargs?

The "controller" API in pre-3.0 was EVP_PKEY_CTX_ctrl(ctx, type, op, p1, p2) with a per-algorithm switch interpreting op/p1/p2. That:

• Could not cross the provider/core boundary safely.
• Was hard to extend without ABI breakage.
• Ergonomically forced casts everywhere.

OSSL_PARAM is a simple typed dictionary:

OSSL_PARAM params[] = {
    OSSL_PARAM_int("foo", &i),
    OSSL_PARAM_octet_string("bar", buf, sizeof(buf)),
    OSSL_PARAM_END
};

Both ends of the call know the keys (declared in core_names.h) and types. New parameters can be added without changing the function signature; older receivers ignore unknown keys. The translator at crypto/evp/ctrl_params_translate.c (~111 KB) keeps backward source compatibility for the legacy ctrl path.

Why is the verifier (x509_vfy.c) so big?

X.509 path validation is the most fiddly part of the standard. RFC 5280 alone is 151 pages of conformance text, and on top of it sit:

• The hostname matching rules (RFC 6125, RFC 9525).
• Path length and pathLenConstraint.
• Name constraints, including the dirName subtype that requires a recursive subtree check.
• The certificate policy tree and policy mappings.
• Extended key usage, key usage, basic constraints — and their interplay.
• Multiple chain construction strategies (legacy, trusted-first, partial chain).
• Revocation: CRL and OCSP, including delta CRLs and CRL distribution-point URI fetching.
• Per-purpose checks (S/MIME, server, client, code-signing, time-stamp, OCSP signer).
• Custom verifier callbacks that can rewrite the result.

x509_vfy.c is ~136 KB of mostly straight-line conformance code with comments tying chunks back to RFC sections. The size reflects the spec.

Why two ASN.1 systems?

The tasn_*.c template engine in crypto/asn1/ covers most types. But certain "reused" types (X509_NAME in particular) want canonicalisation behaviour — re-encoded so that two equivalent forms compare equal byte-for-byte — and bypass parts of the templating system. They have hand-rolled encoders in crypto/asn1/x_*.c or crypto/x509/x_*.c. Cleaning this up has been a recurring topic on the issue tracker but the current split has correctness advantages (the canonical form is robustly preserved across re-encodings).

Why BIO and not FILE * / int fd?

BIO predates 1.0; it was already in SSLeay. The reasons:

• Filter chains. A signed-and-encrypted PKCS#7 stream is implemented by chaining a base64 BIO over a cipher BIO over a digest BIO over a file BIO. No buffer copies; the filter stack does what you'd otherwise build by hand.
• Cross-platform. A FILE * is FILE * on POSIX and Windows but Win32 sockets are not file descriptors. BIOs unify the abstractions.
• Async I/O semantics. BIO_should_retry lets non-blocking I/O bubble up cleanly through filters.

The trade-off: BIO is "just another socket abstraction" plus an ad-hoc ctrl() operation that you can't avoid learning. Newer designs (e.g. the Reactor I/O in ssl/rio/) layer over BIO.

Why "library contexts" rather than per-thread state?

A web server with two virtual hosts may want one host using FIPS and another using ChaCha20-Poly1305, in the same process. Per-thread state doesn't help — the threads are interchangeable in a thread pool. Per-call state would mean every API takes a "config" argument.

A library context gives every consumer (call site, library, virtual host) its own world of providers, properties, default DRBG, error stack, namemap, … and isolates them. The cost: every API gained an _ex variant taking the libctx; old callers default to a global context.

Why a separate FIPS module?

CMVP validation is granular: a "validated module" is a precise sequence of bytes. Any change to the bytes invalidates the validation. The FIPS module needs to ship as bytes that don't change while bug fixes and feature work continue elsewhere in libcrypto.

Splitting fips.so out gives:

• A stable artifact that survives libcrypto changes for years.
• An integrity check (the module HMACs itself).
• A separate set of source files (providers/fips.module.sources) whose churn is constrained.

It also lets vendors ship validated modules with different vendor names and metadata — important for ecosystems where a customer needs vendor-specific certification.

Why is apps/openssl so big?

apps/openssl.c is a dispatcher; the real subcommands are in apps/<name>.c. There are 50+ subcommands, each of which may take 10-30 different flags. They duplicate many features (text formatting, verification options, SSL config) which is mostly handled by the helpers in apps/lib/. The CLI predates many of the post-getopt_long argument-parsing libraries and uses a hand-rolled OPTIONS table per command (OPT_* enums, OPT_*_OPTIONS[] arrays).

Why doesn't OpenSSL use C99 / C11 features more?

OpenSSL must build on a wide matrix of compilers, including ones with limited C99 support (XLC variants on AIX, older HP-UX cc, Sun Studio, MS VC). The general rule: stick to C90 plus carefully chosen extensions (stdint.h is fine, _Generic is not). Some C11 atomics are used through wrappers (include/internal/refcount.h) with fallbacks for compilers that lack them.

This is part of why the codebase looks slightly old-fashioned in places — explicit type casts, Hungarian-ish naming, no inline initialisation for compound literals in headers, etc.

Why so many BIOs and not pipe(2)?

Because OpenSSL targets Windows too — and on Windows, sockets are not pipe-compatible. BIO_s_bio (bss_bio.c) is a software pipe pair that works the same on every platform. It's also useful for unit tests that need to run a TLS handshake without a real socket.

Why is QUIC in libssl?

Because it shares the TLS 1.3 handshake. The cleanest factoring kept the TLS state machine where it has always been (ssl/statem/) and made the TLS handshake a guest of the QUIC channel: the QUIC code in ssl/quic/ interposes a record method that delivers TLS messages as QUIC CRYPTO frames. The SSL object becomes polymorphic — a TLS connection, a QUIC connection, a QUIC stream, or a QUIC listener — but the TLS handshake code is unchanged.

Splitting QUIC into a separate library would have meant duplicating the TLS state machine or adding a hard dependency from a hypothetical libquic to libssl. Keeping it together kept the binary smaller and the API symmetric.

Why is apps/'s argv parser hand-rolled?

A long-standing convention. Replacing it would require touching 50 subcommands with regression risk. The current OPT_* table-driven approach is uniform and lints well, and doesn't introduce a getopt_long dependency on platforms where that varies.