Storage

The storage subsystem is what makes data durable. Under src/mongo/db/storage/ lives a thin abstraction layer; under src/third_party/wiredtiger/ lives the WiredTiger storage engine that implements that abstraction in production.

Purpose

The storage layer provides:

• Record stores — append-friendly key/value heaps where keys are RecordIds and values are BSON documents.
• Sorted data interfaces — ordered key/value structures used to back indexes.
• Recovery units — per-operation snapshot/transaction objects.
• Timestamps — replicated visibility points used by replication, transactions, and snapshots.
• Durability and recovery — checkpointing, journaling, and crash recovery.

The abstraction is small enough that alternative storage engines have been plugged in historically (MMAPv1, in-memory). WiredTiger is the only production engine today; the in-memory engine and devnull engine are used for tests.

High-level architecture

graph TD
    subgraph Server
        IndexCatalog[IndexCatalog]
        Coll[Collection]
        IAM[IndexAccessMethod]
    end
    Coll -->|RecordStore| Storage
    IAM -->|SortedDataInterface| Storage
    Storage -->|RecoveryUnit| Engine

    subgraph Engine[StorageEngine]
        WT[WiredTiger wrapper<br/>src/mongo/db/storage/wiredtiger]
        Lib[WiredTiger library<br/>src/third_party/wiredtiger]
    end

    WT --> Lib
    Lib --> Disk[(disk files)]

Key abstractions

Type	Role
StorageEngine (src/mongo/db/storage/storage_engine.h)	Process-wide engine. Owns the database catalog, journaling, checkpointing.
RecordStore	A heap of (RecordId, RecordData) entries. Backs collections.
SortedDataInterface	Ordered key/value structure for indexes.
RecoveryUnit	Per-operation transaction. Owns the storage snapshot and accumulated writes.
WriteUnitOfWork (src/mongo/db/storage/write_unit_of_work.h)	RAII wrapper around a recovery-unit transaction. Commit on success, rollback on exception.
RecordId (src/mongo/db/record_id.h)	The opaque record identifier. Per-engine encoding.
KeyString (src/mongo/db/storage/key_string/)	Lexicographic encoding of compound BSON keys used by indexes.
SnapshotManager	Manages named snapshots used by replication and read concern.

WiredTiger wrapper

The WiredTiger wrapper at src/mongo/db/storage/wiredtiger/ translates the server's abstractions into WiredTiger's C API. Key files:

• wiredtiger_kv_engine.cpp — top-level engine object. Owns the WT_CONNECTION.
• wiredtiger_record_store.cpp — RecordStore implementation.
• wiredtiger_index.cpp — SortedDataInterface implementation.
• wiredtiger_recovery_unit.cpp — RecoveryUnit (transaction).
• wiredtiger_session_cache.cpp — pool of WT sessions, one per ongoing operation.

The wrapper handles:

• Translating RecordId to a WT key (int64_t for collections; KeyString for indexes).
• Mapping the server's read concerns and timestamps to WT's transaction snapshots.
• Coordinating checkpoints with replication's stable timestamp.
• Rate-limiting writes when the cache fills up via the flow control machinery.

Timestamps

MongoDB's read concerns and transactions are built on timestamps the storage engine tracks:

• stableTimestamp — the latest cluster timestamp at or below which the data on disk is consistent across all replicas. Checkpoints take a snapshot at the stable timestamp; recovery rolls forward from there.
• oldestTimestamp — the oldest timestamp the engine still keeps history for. Reads at older timestamps fail.
• allCommittedTimestamp — the cluster's "all committed" point used by majority read concern.

The replication coordinator drives these timestamps as it confirms majority commit. The storage engine uses them to know which snapshots to retain.

Journaling and recovery

WiredTiger has its own journal (journal/). The server enables it by default, and on startup runs:

1. WiredTiger recovery — replay the journal to restore the engine to a consistent on-disk state.
2. Replication recovery — replay the oplog from the stable timestamp forward to bring the data store to the last known durable replicated state.

The interaction is described in src/mongo/db/repl/replication_recovery.cpp and the storage-engine API at src/mongo/db/storage/storage_engine.h.

Lock-free / partial lock paths

The "shard role" API in src/mongo/db/shard_role/ includes a acquireCollectionMaybeLockFree variant for read-only access. It uses the storage engine's snapshot rather than a collection lock, which is important for streaming reads (cursor getMore, change streams).

Collection catalog

The CollectionCatalog (src/mongo/db/catalog/collection_catalog.h) is the in-memory cache of collection metadata: namespace string, UUID, options, the index catalog, and the storage-engine-specific identifier. It's updated atomically with DDL operations and is the central source of truth for "what collections exist on this node?".

The persistent counterpart — the on-disk catalog of collections and indexes — is owned by the storage engine and stored in WiredTiger's metadata file.

Other engines

• devnull engine (src/mongo/db/storage/devnull/) — discards all writes. Used for benchmarks that want to isolate the rest of the server from disk I/O.
• In-memory engine — pre-existing variant that keeps everything in RAM. Used in some test suites.
• The (now removed) MMAPv1 engine lived under src/mongo/db/storage/mmap_v1/ for many years; the API in src/mongo/db/storage/ is shaped by the lessons of supporting both.

Key source files

File	Purpose
src/mongo/db/storage/storage_engine.h	The engine abstraction.
src/mongo/db/storage/record_store.h	RecordStore interface.
src/mongo/db/storage/sorted_data_interface.h	Index storage interface.
src/mongo/db/storage/recovery_unit.h	Transaction abstraction.
src/mongo/db/storage/key_string/	Compound-key encoding for indexes.
src/mongo/db/storage/wiredtiger/	The WiredTiger wrapper.
src/third_party/wiredtiger/	The WiredTiger library.
src/mongo/db/catalog/collection_catalog.h	In-memory collection metadata.

Integration points

• Replication drives the timestamps and consumes the snapshot manager for read concerns.
• Index builds create new SortedDataInterface instances and rely on the storage engine for atomic commit.
• Transactions use the storage transaction (RecoveryUnit) as the unit of atomicity.
• Sharding migrations and resharding write through the same storage interfaces.
• Time-series uses the same storage interfaces — buckets are just regular documents in system.buckets.<coll>.

Entry points for modification

Most storage-related work touches the WiredTiger wrapper. Index changes also touch key_string. Cross-engine changes — extending the abstraction itself — require touching every engine and every consumer; they are rare. Test coverage is in src/mongo/db/storage/*_test.cpp and the dedicated disk resmoke suite.