Threading model

Envoy's threading model is the architectural decision that everything else is built around. It is described in detail in Matt Klein's 2017 blog post; this page summarises the same model as it exists in the current code.

The shape

graph TB
    Main[Main thread<br/>config + cluster manager primary +<br/>admin + xDS subscriptions + stats flush]
    W1[Worker 1<br/>libevent dispatcher]
    W2[Worker 2<br/>libevent dispatcher]
    WN[Worker N<br/>libevent dispatcher]
    File[File flush thread]
    Stats[Stats flush thread]

    Main -.allocates TLS slot.-> W1
    Main -.allocates TLS slot.-> W2
    Main -.allocates TLS slot.-> WN
    Main -.post.-> W1
    Main -.post.-> W2
    Main -.post.-> WN

    W1 -.post.-> Main
    W2 -.post.-> Main
    WN -.post.-> Main

    W1 --> File
    W2 --> File
    WN --> File
    Main --> Stats

• One main thread. Owns config parsing, the primary copy of the cluster manager state, the admin server, xDS subscriptions, and most cross-cutting infrastructure (runtime, secret manager, init manager, drain manager, overload manager, watchdog control). Its Event::Dispatcher runs the same kind of event loop the workers do.
• N worker threads. One per --concurrency (default: physical CPU count, optionally --cpuset-threads-aware). Each worker owns its slice of the listeners, accepts connections, and runs filter chains. Workers do not communicate directly with each other.
• Helpers. A file flush thread for access logs, a stats flush thread, optional gRPC C-core threads. They exist to keep blocking I/O off the workers.

The rule

Workers never block. Workers never lock. Workers do not share mutable state.

Every architectural decision flows from this rule. Hot-path data is replicated per worker via copy-on-write, accessed through ThreadLocal::SlotImpl lookups.

Worker bootstrap

A worker is a Server::WorkerImpl (source/server/worker_impl.h). It owns:

• An Event::Dispatcher — the libevent-based event loop.
• A Network::ConnectionHandler — listener and connection bookkeeping.
• A WatchDog — feeds the GuardDog with liveness pings.
• A Thread::Thread — the OS thread itself.

class WorkerImpl : public Worker {
  void start(OptRef<GuardDog> guard_dog, const std::function<void()>& cb) override;
  void addListener(...) override;
  void removeListener(...) override;
  void stop() override;
  // ...
};

start() creates the OS thread, which immediately enters threadRoutine():

void WorkerImpl::threadRoutine(OptRef<GuardDog> guard_dog,
                               const std::function<void()>& cb) {
  // 1. Tell libevent we're here.
  // 2. Register this dispatcher with ThreadLocal so the main thread can post slot updates here.
  // 3. (Optionally) register with the watchdog.
  // 4. Run dispatcher until exit.
}

After dispatcher_->run(Event::Dispatcher::RunType::RunUntilExit) returns, the thread shuts down its TLS slots and exits.

ThreadLocal slots

The propagation primitive is ThreadLocal::Slot (envoy/thread_local/thread_local.h):

// Allocate from the main thread.
auto slot = tls.allocateSlot();

// Set the value on every worker. The InitializeCb runs on each worker in turn,
// returning the per-worker shared_ptr that gets stored in that worker's slot.
slot->set([](Event::Dispatcher&) -> ThreadLocalObjectSharedPtr {
  return std::make_shared<MyState>();
});

// On a worker thread, read it without locks:
MyState& state = slot->getTyped<MyState>();

Two patterns:

• Identical state on every worker. The InitializeCb returns a shared MyState. Updates push a new shared MyState (copy-on-write); old workers see the old value until they reach the next event-loop iteration that invokes set().
• Per-worker state. The callback returns a fresh, worker-specific MyState. Each worker has its own copy.

Both patterns avoid locks on the read path. Updates are issued from the main thread via runOnAllThreads, which posts a callback to each worker's dispatcher.

The implementation is in source/common/thread_local/thread_local_impl.cc. The typed wrapper TypedSlot<T> is the API most code should use; Slot is the underlying untyped one.

Crossing threads

Direct rules for code:

Want to do	Use
Read shared, mostly-static config from a worker	TypedSlot<T>::get()
Update config seen by all workers	TypedSlot<T>::runOnAllThreads(updateCb) from main
Run a one-off task on all workers	TypedSlot<T>::runOnAllThreads(updateCb, completionCb)
Post a task to a specific dispatcher	Event::Dispatcher::post([] { ... })
Post from worker to main	Hold a reference to the main dispatcher; main_dispatcher.post(...)
Wait for a future (test only)	Event::SimulatedTimeSystem, Thread::CondVar — never on production code paths

Direct mutex use is allowed but discouraged. When unavoidable, use Thread::MutexBasicLockable from source/common/common/thread.h; the format checker bans bare std::mutex for new code.

Cluster manager: a worked example

The cluster manager is the canonical example of this model:

• Main thread. Owns ClusterManagerImpl (source/common/upstream/cluster_manager_impl.cc). Receives CDS/EDS updates from xDS, mutates the primary cluster set, computes per-worker snapshots.
• Workers. Each holds a ThreadLocalClusterManagerImpl view in a TLS slot. When the main thread updates clusters, it posts new immutable ThreadLocalClusterImpl objects into every worker's slot. Workers iterate hosts, pick load balancers, and check circuit breakers without locks.

The split of ClusterManagerImpl (main) vs ThreadLocalClusterManagerImpl (worker) is a direct consequence of the threading rules.

What workers actually run

sequenceDiagram
    participant L as Listener
    participant W as Worker dispatcher
    participant C as Connection
    participant F as Network filter chain
    participant H as HTTP CM
    participant R as Router
    participant CM as Cluster manager (TLS view)
    participant U as Upstream conn pool

    L->>W: accept
    W->>C: new ConnectionImpl
    C->>F: onData
    F->>H: onData (HCM is a network filter)
    H->>H: codec parse, dispatch decoded headers
    H->>R: decodeHeaders
    R->>CM: getThreadLocalCluster
    CM-->>R: ClusterEntry
    R->>U: newStream
    U-->>R: pool callbacks
    R-->>H: forward to upstream

The whole flow happens on a single worker thread. No locks taken; no main-thread RPCs in the hot path.

What happens at shutdown

Server::InstanceImpl::~InstanceImpl():

1. Drains listeners (via drain manager).
2. Stops accepting new connections on each worker.
3. Calls WorkerImpl::stop() on every worker — which posts an exit to that worker's dispatcher.
4. Joins every worker thread.
5. Tears down the cluster manager, listener manager, runtime, secrets, etc.
6. Calls tls.shutdownGlobalThreading() to cause set() to drain any pending callbacks safely.

Because shared_ptr is the canonical reference, slot data is freed via the worker's dispatcher (the shared_ptr destructor runs on the worker that holds the last reference), avoiding cross-thread destruction.

Key source files

File	Role
envoy/thread_local/thread_local.h	Public TLS slot API
source/common/thread_local/thread_local_impl.cc	TLS slot implementation
envoy/server/worker.h	Worker interface
source/server/worker_impl.h	Worker implementation
envoy/event/dispatcher.h	The event loop interface
source/common/event/dispatcher_impl.cc	The libevent-backed dispatcher
source/server/server.cc	Owns workers, TLS, dispatchers
source/common/common/thread.h	Thread / mutex wrappers

Where to read next

• Server lifecycle — how the workers come into being.
• Cluster manager — the most complex consumer of TLS slots.
• Stats — has a similar main/worker split for fast counter increments.