Index builds

Building an index on a non-trivial collection is one of the most concurrency-sensitive operations in the server. The index-build coordinator under src/mongo/db/index_builds/ drives the two-phase index build protocol: a primary collects keys from the data, replicates checkpoints, and only commits when every voting member has caught up.

Purpose

A two-phase index build:

• Lets a replica set keep accepting writes during the build.
• Coordinates across primary and secondaries so they all arrive at the same final index.
• Survives primary failover — a build that started on the previous primary continues on the new primary.
• Provides explicit abort and commit hooks for ops, fail points, and timeouts.

The single-phase variant (used for in-memory indexes and a few corner cases) builds an index synchronously without coordination.

Two-phase protocol

stateDiagram-v2
    [*] --> Initialize
    Initialize --> CollectionScanning: open scan
    CollectionScanning --> KeyInsertion: process keys
    KeyInsertion --> AwaitingCommitVote: ready to commit
    AwaitingCommitVote --> Committing: voteCommitIndexBuild from majority
    Committing --> Committed: oplog commitIndexBuild applied
    AwaitingCommitVote --> Aborting: abortIndexBuild
    Aborting --> Aborted
    Committed --> [*]
    Aborted --> [*]

Phases:

1. Initialize — register the build with the index-build coordinator, replicate startIndexBuild.
2. Collection scan — read every document and produce keys via the appropriate IndexAccessMethod.
3. Key insertion / sort — sort the keys (using the in-tree disk-based sorter) and insert into the index data structures.
4. Voting — once a member finishes, it sends voteCommitIndexBuild to the primary. The primary waits for a quorum.
5. Commit — the primary issues commitIndexBuild, replicated to all members. Each member finalizes its index and exposes it to readers.

If anything goes wrong (build fails, user aborts, FCV downgrades), the coordinator drives an abort that's also replicated.

Resilience

• Primary failover — index-build state is persisted in config.system.indexBuilds. The new primary takes over and continues coordinating.
• Member crashes — on restart, the storage engine's recovery brings the partial index back to a known state and the coordinator resumes from where it left off.
• Resource pressure — the build respects memory and disk-spill budgets via the sorter and yields to long-running operations.

Key abstractions

Type / file	Role
IndexBuildsCoordinator (src/mongo/db/index_builds/)	Per-process registry and driver of in-flight builds.
IndexBuildEntry (config server)	Cluster-visible build record.
IndexAccessMethod (src/mongo/db/index/)	Per-index-type implementation (BTree, hashed, geo, text, wildcard, time-series).
IndexCatalog (src/mongo/db/catalog/index_catalog.h)	Per-collection index registry; what find consults to pick an index.
Sorter (src/mongo/db/sorter/)	Disk-spilling external sort used to merge keys.
commitIndexBuild / abortIndexBuild oplog entries	Replicated phase transitions.

Failure injection

Many fail points exist for tests:

• hangAfterStartingIndexBuild
• hangBeforeIndexBuildCommit
• voteCommitIndexBuildBeforeCommit
• failIndexBuildWithError

These are documented in docs/fail_points.md and used heavily by the noPassthrough/ and replsets/ tests.

Key source files

File	Purpose
src/mongo/db/index_builds/index_builds_coordinator.cpp	Main coordinator.
src/mongo/db/index_builds/index_builds_coordinator_mongod.cpp	mongod-specific behavior.
src/mongo/db/index_builds/index_build_entry_helpers.cpp	Helpers for the config.system.indexBuilds records.
src/mongo/db/index_builds/primary_driven/	The newer primary-driven variant.
src/mongo/db/index/index_access_method.h	Per-type index implementation.
src/mongo/db/sorter/	External sort used for key insertion.

Integration points

• Replication — phase transitions are replicated as oplog entries.
• Storage — the WiredTiger record store underpins index storage and provides the timestamping needed for replicated builds.
• Sharding — chunked migrations and resharding require coordinated builds on the destination.
• Op observer — fires hooks on the start, commit, and abort of builds.

Entry points for modification

Most index-build work goes through IndexBuildsCoordinator. New index types add an IndexAccessMethod subclass under src/mongo/db/index/. Changes to the protocol need to be paired with corresponding oplog-entry updates and tests across both standalone and replica-set fixtures. The index_builds-flavored resmoke suites and the FSM concurrency suites are the primary regression gates.