Design decisions

A non-exhaustive set of architectural choices that show up across the codebase, with the reasoning behind them.

A single sorted KV map

Every piece of user and system data in CockroachDB lives in one logical, sorted, byte-keyed map. The map is partitioned into ranges of ~512 MiB. The decision predates v1.0 and lets the same machinery (Raft, MVCC, the allocator) handle SQL data, system metadata, time series, and locks uniformly.

Tradeoff: every layer carries the cost of the KV model — even small system records pay the price of full transactional semantics. CockroachDB has, over the years, added optimizations (timestamp-ordered scans, intent metadata elision, separated lock tables) to ease the cost.

MVCC with hybrid logical clocks

pkg/util/hlc/ provides Hybrid Logical Clocks. Every value carries an HLC timestamp. CockroachDB chose HLCs over Spanner-style TrueTime because TrueTime requires specialized hardware. HLCs work on commodity nodes by enforcing a maximum clock offset (--max-offset, default 500 ms) and rejecting nodes that exceed it.

SQL transactions are non-locking by default

CockroachDB's distributed transactions are optimistic and serializable. A read does not lock a row; instead, the timestamp cache (pkg/kv/kvserver/tscache/) tracks the latest read timestamp per key range. A writer to that key range either proceeds or refreshes its read timestamp, depending on whether reads it has done remain valid. This avoids long-held locks but introduces refresh and retry logic that pervades pkg/kv/kvclient/kvcoord/.

SELECT ... FOR UPDATE and replicated locks (pkg/kv/kvserver/concurrency/) are available when an application needs explicit locking.

Range-level Raft

Each range is its own Raft group. This isolates failures: losing a quorum on one range does not block others. The cost is that every node hosts thousands of Raft groups; CockroachDB had to invent quiescence (replica_raft_quiesce.go) and per-store schedulers (pkg/kv/kvserver/scheduler.go) to keep idle ranges cheap.

Symmetric, single-binary topology

There are no special nodes. Any node can serve SQL, KV, gossip, and admin traffic. This simplifies operations (one binary, one deployment) and matches the "scales horizontally" goal. Specialization (e.g., SQL-only pods) is implemented as a runtime mode rather than a separate binary.

Cluster-version-gated change

Backwards-incompatible behavior is always gated on a cluster version (pkg/clusterversion/). The cluster only enables the new behavior after every node has confirmed it understands the new version. This makes rolling upgrades viable but means every contributor must reason about mixed-version semantics.

Bazel + Go

CockroachDB uses Bazel for hermetic, reproducible builds (and enforced visibility between Go packages) on top of Go for productivity and a strong stdlib. The cost is a heavier on-ramp; ./dev exists to mask Bazel's verbosity.

Pebble

CockroachDB switched from RocksDB to Pebble (a native-Go LSM-tree) around 2019–2020. Reasons:

• Avoid the cgo overhead of RocksDB calls.
• Better integration with Go runtime for memory accounting.
• Direct control over the LSM format for new features (range keys, format-major versions).
• Reduced binary size and build complexity (no embedded C++ build).

Job-driven async work

Schema changes, backups, restores, changefeeds, imports, TTL, and migrations all run as jobs (pkg/jobs/). Centralizing the lifecycle (claim, run, pause, resume, cancel) in one framework gives uniform observability and resumability. The cost is a layer of abstraction every long-running feature must adopt.

Span configuration replaces gossiped system config

The legacy "system config" gossip blob held all zone configs and descriptor IDs. Past a few thousand objects, gossiping the entire blob became expensive. pkg/spanconfig/ moved this to a dedicated system table and a rangefeed-driven reconciler — the table-count axis no longer impacts gossip.

Multi-tenancy as runtime decomposition

Rather than fork the binary, CockroachDB runs multi-tenancy as a runtime mode of the same cockroach binary. The host process can spawn many SQL servers; each tenant has its own KV namespace prefix and its own SQL stack. This keeps the operational story simple and lets every tenant inherit improvements without re-deploying separate binaries.

Why Raft was forked

pkg/raft/ started as a copy of etcd/raft. Forking allowed CRDB-specific changes (term cache, store-liveness leadership, learner replicas, joint-consensus learner promotion) to land without coordinating with etcd. The cost is occasional drift from the upstream paper-conformance suite; the fork preserves the original test corpus.

Related pages

• overview/architecture — the layered view.
• systems/cluster-version-and-upgrade — gating contract.
• lore — when these decisions were made.