Design decisions

A non-exhaustive list of the most consequential architectural choices the project has made and the reasoning behind them.

HTTP for peer transport, gRPC for clients

server/etcdserver/api/rafthttp/ uses HTTP/1.1 streams + pipelines, not gRPC. The choice predates gRPC's Go ecosystem maturity and persists because:

• HTTP semantics are simple to debug (cf. tools/local-tester/).
• The peer protocol is a small, fixed set of message types — no need for codegen.
• A separate listener (:2380) and protocol let operators firewall peer traffic independently.

bbolt as the storage engine

server/storage/backend/ wraps bbolt (a fork of BoltDB). Key trade-offs:

• Single-writer is fine because the apply pipeline serializes mutations anyway.
• mmap-based reads give cheap, snapshot-isolated reads for MVCC.
• One bolt file means atomic snapshots are essentially free.
• Drawback: long read transactions block the freelist garbage collector — see systems/backend caveats.

MVCC keyed by revision, not vector clock

Revisions are a single global monotonic counter ((main, sub)), not per-key versions. This is what makes Watch from a revision both correct and simple — you stream every event with revision >= start. Per-key version vectors would have made Watch cheaper at the cost of much more complicated determinism in apply.

Linearizable reads via Raft ReadIndex

Linearizable reads do not require writing a new entry; they confirm leadership using ReadIndex and serve from the local committed state. This is one of the most cited etcd performance wins and is documented as the "consistent read workflow" in Documentation/etcd-internals/diagrams/consistent_read_workflow.png.

Apply pipeline is single-threaded per member

EtcdServer.applyAll processes one entry at a time on each member. Parallel apply has been considered repeatedly but bumps into:

• The need for deterministic outcomes across members.
• bbolt's single-writer model.
• Watch event ordering guarantees.

The current approach favors simplicity and correctness over throughput; you scale the cluster horizontally by adding clusters, not threads.

Two CLIs (etcdctl + etcdutl)

Splitting offline tooling into etcdutl keeps etcdctl's dependency cone clean (etcdctl doesn't link in server/storage/). The cost is two binaries to ship; the win is sharper failure modes — etcdutl works on an offline data dir, etcdctl on a live cluster.

Go workspace, multi-module repository

Decided in v3.5, finalized for v3.6. Each module (api, client/v3, client/pkg, pkg, server, cache, etcdctl, etcdutl, tests, plus tools) tags independently. Costs: more go.mod files, gomodguard maintenance, more involved dependency bumps. Benefits: external clients can update client/v3 without an etcd server bump, and the dependency tree of the public API module is tiny.

Structured logging only

Removed capnslog support in 3.5. Zap is non-optional. This simplifies the code (no logger interface) and forces structured fields, which the dashboards/alerts in contrib/mixin/ depend on.

Feature gates over --experimental-*

Adopted in v3.6. See features/feature-gates. Trades discoverability of --experimental-* for a clean deprecation policy.

Robustness over benchmarks

After the 3.5 inconsistency incident the project deliberately invested in tests/robustness/ and Antithesis runs even at the cost of slower release cadence. v3.6 took almost four years to ship in part because of this.

What etcd consciously isn't

• It is not a multi-tenant database. There is no built-in namespacing beyond key prefixes; if you want hard isolation, run more clusters.
• It is not a transactional database with arbitrary multi-key serializable transactions. Txn is intentionally limited to compare-and-swap-or-else against a single revision.
• It is not a high-throughput store. Every write is durably fsynced; the design point is small-volume, high-criticality data.
• It is not geo-distributed. The Raft consensus assumes low intra-cluster RTT.

Cross-references

• lore.md — when each decision was made.
• systems/ — how each manifests in code.