CMS / PKCS family

Active contributors: Tomas Mraz, Stephen Henson (historical), Pauli, Matt Caswell, slontis

Purpose

OpenSSL implements the Cryptographic Message Syntax (CMS, RFC 5652 and friends) plus the older PKCS family it generalises:

• CMS (crypto/cms/) — signed-data, enveloped-data, authenticated-enveloped-data, digested-data, encrypted-data, compressed-data. Used by S/MIME, PKCS#7-derived protocols, ASIC-S, RPC-Sec-GSS, …
• PKCS#7 (crypto/pkcs7/) — the predecessor of CMS, kept for compatibility with old S/MIME, code-signing, and applications that still use the legacy PKCS#7 schema.
• PKCS#12 (crypto/pkcs12/) — the password-protected key+cert bundle (.p12 / .pfx).

Each maps onto an ASN.1 schema (see subsystems/asn1) and uses BIO chains for streaming I/O.

Directory layout

crypto/
├── cms/
│   ├── cms_asn1.c       -- the CMS ASN.1 templates
│   ├── cms_lib.c        -- common helpers; BIO chain assembly; MIME I/O
│   ├── cms_sd.c         -- SignedData (sign/verify)
│   ├── cms_env.c        -- EnvelopedData (encrypt/decrypt)
│   ├── cms_enc.c        -- the inner content-encryption helpers
│   ├── cms_dh.c         -- KARI / KAGreement (DH/ECDH key agreement recipient info)
│   ├── cms_ec.c         -- ECDH KARI specifics
│   ├── cms_rsa.c        -- KTRI for RSA, RSA-OAEP
│   ├── cms_kari.c       -- generic KARI driver
│   ├── cms_pwri.c       -- PasswordRecipientInfo
│   ├── cms_smime.c      -- the SMIME read/write entry points
│   ├── cms_io.c         -- BER indefinite streaming I/O
│   ├── cms_att.c        -- attribute helpers
│   ├── cms_ess.c        -- ESS extensions (signing certificate, content hint, signed receipt request)
│   ├── cms_cd.c         -- CompressedData
│   ├── cms_dd.c         -- DigestedData
│   └── cms_local.h, cms_err.c
├── pkcs7/
│   ├── pk7_asn1.c, pk7_lib.c, pk7_smime.c, pk7_doit.c, pk7_attr.c, pk7_mime.c, pk7_local.h, pk7_err.c
├── pkcs12/
│   ├── p12_asn.c        -- PKCS#12 schema
│   ├── p12_init.c, p12_add.c, p12_attr.c, p12_crpt.c, p12_decr.c, p12_kiss.c, p12_mutl.c, p12_npas.c, p12_p8d.c, p12_p8e.c, p12_local.h, p12_err.c, p12_sbag.c, p12_utl.c
└── ess/
    └── ess_lib.c, ess_asn1.c, ess_err.c   -- the ESS structures shared with CMS/PKCS#7

CMS overview

The CMS surface revolves around an CMS_ContentInfo * and a small set of high-level operations:

Operation	Function
Sign	CMS_sign(signer_cert, signer_key, additional_certs, content_bio, flags)
Verify	CMS_verify(cms, certs, store, content_bio, output_bio, flags)
Encrypt	CMS_encrypt(recipient_certs, content_bio, cipher, flags)
Decrypt	CMS_decrypt(cms, recipient_key, recipient_cert, dcont, output_bio, flags)
Compress / decompress	CMS_compress / CMS_uncompress

Flags include CMS_DETACHED, CMS_BINARY, CMS_NOCERTS, CMS_NOCRL, CMS_NOATTR, CMS_TEXT, CMS_STREAM, CMS_USE_KEYID, CMS_PARTIAL, CMS_KEY_PARAM, CMS_DEBUG_DECRYPT, CMS_NO_SIGNING_TIME, CMS_AUTH_DATA_OAEP_LABEL_REQUIRED, …

For composing signed-and-then-enveloped or partial structures, CMS_PARTIAL plus CMS_add1_signer, CMS_add0_recipient_key, CMS_add0_recipient_password, etc. let the caller assemble piece-by-piece, ending with CMS_final to compute the actual signatures and encryptions.

Signed-data

A SignedData has the high-level shape:

SignedData ::= SEQUENCE {
    version            CMSVersion,
    digestAlgorithms   SET OF DigestAlgorithmIdentifier,
    encapContentInfo   EncapsulatedContentInfo,   -- inner content (or a reference to it for detached)
    certificates       [0] IMPLICIT CertificateSet OPTIONAL,
    crls               [1] IMPLICIT RevocationInfoChoices OPTIONAL,
    signerInfos        SET OF SignerInfo
}

OpenSSL's signer flow:

sequenceDiagram
    participant App
    participant SD as cms_sd.c
    participant Inner as Inner content BIO
    participant MD as EVP_MD via provider
    App->>SD: CMS_sign(cert, key, certs, content_bio, flags)
    SD->>SD: build CMS_SignerInfo + signed attributes (signing time, content type, message digest)
    SD->>MD: digest content_bio
    SD->>SD: sign signed attributes (or content for !CMS_NOATTR & detached cases)
    SD-->>App: CMS_ContentInfo *
    App->>SD: SMIME_write_CMS / i2d_CMS_bio_stream

Streaming: with CMS_STREAM, the function returns a CMS_ContentInfo not yet fully serialised; the caller wraps it with BIO_new_CMS (which chains a content-feeding BIO over the output BIO) and writes the content bytes through. The CMS BIO finalises the structure when freed. This avoids buffering the entire content in memory.

Enveloped-data and recipient infos

EnvelopedData supports several recipient-info kinds, each with its own helper file:

RecipientInfo kind	File	Use
KEKRecipientInfo (symmetric KEK)	cms_lib.c, cms_env.c	KEK shared by recipient + sender.
KeyTransRecipientInfo (KTRI)	cms_rsa.c	RSA / RSA-OAEP transport.
KeyAgreeRecipientInfo (KARI)	cms_kari.c, cms_dh.c, cms_ec.c	DH/ECDH agreement.
PasswordRecipientInfo (PWRI)	cms_pwri.c	PBKDF2-derived KEK from a passphrase.
OtherRecipientInfo	passes through	for extensions.

Encrypt:

CMS_ContentInfo *cms = CMS_encrypt(
    sk_X509_recipients,           /* the recipients' certs */
    content_bio,                  /* bytes to encrypt */
    EVP_aes_256_gcm(),            /* content-encryption alg */
    CMS_BINARY | CMS_STREAM);

Per-recipient cipher params (RSA-OAEP, ECDH KDF, …) are configured before CMS_encrypt via CMS_PARTIAL plus CMS_RecipientInfo_* setters, or after by walking CMS_get0_RecipientInfos.

ASIC-S, S/MIME, "MIME"

cms_smime.c reads and writes S/MIME-formatted messages: the multipart/signed and application/pkcs7-mime container formats. The "MIME" here is a tiny in-tree implementation, not a full MIME library — just enough to handle CMS payloads. Signature verification of a multipart/signed bundle requires the verifier to canonicalise CRLF the way the signer did; flags CMS_TEXT and CMS_BINARY control this.

ESS extensions

The signing-certificate-v2 attribute (RFC 5126/5035) and other ESS attributes used by RFC 3161 timestamps live in crypto/ess/. CMS pulls them in through cms_ess.c.

PKCS#7

PKCS#7 is the predecessor of CMS. The schema and operations are conceptually similar but the wire format differs (and PKCS#7 lacks AEAD support). OpenSSL keeps the PKCS#7 implementation in crypto/pkcs7/ mostly for parsing legacy code-signing artifacts, S/MIME v2, and similar; new code should use CMS.

The API mirror: PKCS7_sign, PKCS7_verify, PKCS7_encrypt, PKCS7_decrypt, SMIME_write_PKCS7, SMIME_read_PKCS7. Internally PKCS#7 is simpler — a single recipient-info kind (PKCS7_RECIP_INFO), no AEAD, no compressed-data.

PKCS#12

PKCS#12 (.p12 / .pfx) is the password-protected key+cert bundle used everywhere from S/MIME identity import to TLS server-key bundles.

PKCS12 *p12 = PKCS12_create(
    "passphrase", "Friendly Name",
    private_key, cert, ca_chain,
    /* nid_key */ -1, /* nid_cert */ -1,
    /* iter */ 0, /* mac_iter */ -1, /* keytype */ 0);
i2d_PKCS12_fp(fp, p12);

/* later: */
PKCS12 *p12 = d2i_PKCS12_fp(fp, NULL);
EVP_PKEY *key = NULL; X509 *cert = NULL; STACK_OF(X509) *ca = NULL;
PKCS12_parse(p12, "passphrase", &key, &cert, &ca);

PKCS#12 is layered: a PFX wrapper holds an outer signed/MAC'd AuthSafe of SafeContents, each of which holds bags (key bag, cert bag, secret bag, CRL bag, …). The bag encryption uses PBKDF1 (legacy) or PBKDF2 (modern, post-3.0) plus a content-encryption alg. The MAC is HMAC-SHA1 by default but -macalg chooses SHA-256 etc.

The default MAC iteration count and PBES2 cipher were modernised in 3.0; old applications that need PKCS12 compatible with Windows XP-era importers can fall back via PKCS12_create_ex.

Common pitfalls

• CRLF sensitivity. S/MIME signed text messages are signed after canonicalisation. Ferrying them through systems that mangle line endings breaks signatures.
• Wrong CMS_BINARY / CMS_TEXT. Pass CMS_BINARY when the content is bytes, not text.
• Detached signatures need CMS_DETACHED on both sign and verify.
• PKCS#12 iteration counts. The default is now strong enough; the legacy MAC iterations field is small (1 in some SDKs); consult doc/man3/PKCS12_create.pod.
• PKCS#7 vs CMS. Don't use PKCS#7 for new designs.

Integration points

• CMS is used by apps/openssl cms and apps/openssl smime. Some scripts expect CMS output by default; others want PKCS#7.
• PKCS#12 is loaded by apps/openssl pkcs12 and by OSSL_STORE automatically when given a .p12 file.
• CMP (RFC 4210) uses CMS-style message protection; see features/cmp.
• Timestamps wrap a TimeStampToken (a CMS SignedData) in their RFC 3161 response — see crypto/ts/.

Entry points for modification

• New CMS recipient info kind: extend CMS_RecipientInfo (cms_lib.c), add a per-kind file, register in the dispatcher.
• New content-encryption mode: ensure the EVP cipher you want to support is exposed by a provider; CMS's content-encryption uses EVP_CIPHER directly.
• New PKCS#12 bag type: extend crypto/pkcs12/p12_asn.c and p12_kiss.c.

Documentation

• doc/man3/CMS_sign.pod, CMS_verify.pod, CMS_encrypt.pod, CMS_decrypt.pod, CMS_compress.pod, CMS_uncompress.pod, CMS_get0_SignerInfos.pod, CMS_RecipientInfo_*.
• doc/man3/PKCS12_create.pod, PKCS12_parse.pod, PKCS12_newpass.pod.
• doc/man3/PKCS7_sign.pod, PKCS7_verify.pod, SMIME_*.
• doc/man7/ossl-guide-cms.pod (where present).