envoyproxy/envoy

Design decisions

The most opinionated parts of Envoy are deliberate. They were argued over years before the public release in 2016, refined over the next decade, and have aged well enough that very few have been revisited. This page collects the load-bearing decisions and the rationale.

For Matt Klein's first-hand explanations, see his blog series:

Single process, multi-thread, no shared mutable state

Pre-Envoy proxies were either:

Multi-process, with workers as separate processes that share state via shared memory or RPC. Operationally complex, harder to debug.
Multi-thread with locks, suffering high tail latency from contention.

Envoy chose multi-thread within a single process, with no shared mutable state. Each worker has its own dispatcher and owns its connections, listeners, and L7 streams. Configuration and host sets are propagated via "thread-local slots" (Envoy::ThreadLocal) where the main thread publishes a new immutable snapshot and each worker receives it asynchronously. The data plane never locks; the only synchronisation is the slot-update fences.

Consequence: it scales linearly with cores, has predictable tail latency, and is debuggable with a normal C++ debugger. See threading model.

libevent / dispatcher per worker

Each worker runs a libevent-backed event loop. Synchronous dispatcher post + async dispatcher post are the only IPC primitives. There is no thread pool that hops work between workers — connections live and die on the worker that accepted them.

Stats are central, lock-free, named via interned strings

Stats are designed to be cheap on the hot path, cheap in memory at scale, and uniform across export formats. Per-worker counters with a flush-thread merge replace contended atomic increments. The symbol-table interning makes "tens of thousands of cluster metrics" a few megabytes instead of hundreds. See stats.

xDS is the configuration protocol

Configuration is gRPC-based, not REST-based. Envoy was an early adopter of gRPC streaming for live config push. The decision to standardise on Discovery{Request,Response} for every API meant a single gRPC mux can carry every resource type (ADS), and incremental delta xDS layered cleanly on top of the same shape.

The cost: every Envoy deployment needs a control plane. The benefit: every Envoy deployment can react to config changes in milliseconds. See xDS configuration.

BoringSSL, not OpenSSL

Envoy depends on BoringSSL — Google's fork of OpenSSL. BoringSSL has stricter API stability guarantees inside Google but explicitly no stability for outside consumers; Envoy ABI-pins a vendored copy. Trade-off: the choice closed off some operating systems' "use the system OpenSSL" expectations, but opened up modern TLS features (TLS 1.3 0-RTT, early data, post-quantum drafts) years before OpenSSL caught up.

Hot restart

Hot restart was a requirement from day one. The team had operational scars from binaries that couldn't be redeployed without dropping connections. The result: a Unix-domain-socket protocol where a new process inherits sockets and shared-memory stat counters, the old process drains for a configurable window, and the operator sees zero traffic loss. See hot restart.

Filter chain — order matters and is explicit

Envoy doesn't auto-order filters by category. The HCM filter chain is the operator's responsibility, declared in order, with the router typically last. This is a usability cost for newcomers, but it makes the runtime behaviour exactly predictable: every filter sees a known prefix of the chain.

Extensions over options

Envoy expresses every configurable behaviour as a registered factory rather than a set of flags. The 70+ HTTP filters, 30+ network filters, 10+ tracers, 10+ access loggers, 15+ clusters, 10+ load balancing policies, 10+ transport sockets — all are extension factories using the same REGISTER_FACTORY macro. Adding behaviour does not change the core; it ships a new factory.

Routes are a separate dimension

Routes, virtual hosts, and route configurations are conceptually distinct from listeners and clusters. They have their own xDS API (RDS / VHDS / SRDS). This means a single listener can serve thousands of dynamically updated routes without rebuilding the listener, and a single route configuration can be reused by multiple listeners. See HTTP connection manager, router.

Versioning: never break v3

The stability commitment described in API versioning is a deliberate design decision. The cost is technical-debt accumulation in the proto schema; the benefit is that every management server stays compatible forever, which is the reason xDS became the lingua franca of cloud-native networking.

Public interface vs implementation split

The repository split — envoy/ for public interfaces, source/ for implementations, test/mocks/ for fakes — is a 2017 refactor that paid off massively. It enforces "you can write filters and tests against an interface without knowing which implementation backs it" and lets contributors swap implementations (different malloc, different Wasm engine, different cert validator) without churning the rest of the tree. See patterns and conventions.

Why C++17 / C++20

Envoy uses modern C++ for performance and correctness — RAII, smart pointers, std::optional, structured bindings, std::variant, ranges. The language choice was non-trivial in 2016 (Go was the obvious cloud-native default), but the team needed:

Low-latency tail (no GC pauses).
Predictable memory layout (large static binary).
BoringSSL's C ABI.
Mature event-loop libraries (libevent).
A path to QUIC via QUICHE (also C++).

Go and Rust were both considered. Go was rejected for GC pauses and memory overhead; Rust was rejected (in 2016) for ecosystem maturity. The Rust ecosystem has since closed that gap, and Envoy now provides Rust extension authoring through dynamic modules.

Bazel

The build is Bazel. Why?

Hermetic builds across platforms.
Per-target sanitiser configuration.
Deterministic remote caching for distributed CI.
Same toolchain handles C++, Go (mobile bridge), Java/Kotlin (mobile), Swift (mobile), Python (test tools).

Bazel is a learning cost, but for a project this size with this many language surfaces, no other build tool was viable.