envoyproxy/envoy

Cluster manager

The cluster manager is Envoy's catalogue of upstream clusters. It owns the live host lists, runs health checks, accepts CDS/EDS updates, and hands routes a ThreadLocalCluster ready to load-balance against. The two pillars are ClusterManagerImpl (main thread) and ThreadLocalClusterManagerImpl (per worker), both in source/common/upstream/cluster_manager_impl.cc.

Purpose

For every cluster configured in Bootstrap.static_resources.clusters or pushed via CDS:

Build a Cluster object of the right type (eds, strict_dns, logical_dns, original_dst, static, redis, dynamic_forward_proxy, …).
Maintain its priority sets and host lists, fed by EDS or DNS resolution.
Run health checks, outlier detection, and load reporting.
Replicate a snapshot of cluster state into every worker via thread-local slots.
Manage per-worker connection pools, async clients, and circuit breakers.

The public interface is envoy/upstream/cluster_manager.h.

Layout

source/common/upstream/
├── cluster_manager_impl.{h,cc}     # Both the primary and the per-worker views (~111k lines)
├── upstream_impl.{h,cc}            # ClusterImplBase, HostImpl, HostSet, PrioritySetImpl (~126k)
├── cluster_factory_impl.{h,cc}     # Resolves cluster.type → factory
├── cluster_discovery_manager.{h,cc} # On-demand CDS (ODCDS)
├── cds_api_impl.{h,cc}             # CDS subscription glue
├── od_cds_api_impl.{h,cc}          # ODCDS subscription glue
├── outlier_detection_impl.{h,cc}   # Per-host failure tracking and ejection (~47k)
├── health_checker_impl.{h,cc}      # Active health check dispatch
├── health_discovery_service.{h,cc} # HDS — Envoy as a health-checker
├── load_stats_reporter_impl.{h,cc} # LRS — load reporting back to control plane
├── transport_socket_match_impl.{h,cc} # Per-host transport socket selection
├── conn_pool_map.h                 # Per-host conn pool keyed by protocol
├── priority_conn_pool_map.h        # Per-priority conn pool map
└── edf_scheduler.h, wrsq_scheduler.h # Weighted scheduling primitives

Two-tier architecture

graph TB
    subgraph Main[Main thread]
        CMI[ClusterManagerImpl]
        CDS[CdsApiImpl]
        EDS[EDS subscriptions per cluster]
        HC[HealthChecker]
        OD[OutlierDetector]
    end

    subgraph Worker1[Worker 1]
        TLCM1[ThreadLocalClusterManagerImpl]
        TLC1[ThreadLocalClusterImpl per cluster]
        CP1[ConnPoolMap per host]
    end

    subgraph WorkerN[Worker N]
        TLCMN[ThreadLocalClusterManagerImpl]
        TLCN[ThreadLocalClusterImpl per cluster]
        CPN[ConnPoolMap per host]
    end

    CDS --> CMI
    EDS --> CMI
    HC --> CMI
    OD --> CMI

    CMI -.TLS update.-> TLCM1
    CMI -.TLS update.-> TLCMN
    TLCM1 --> TLC1
    TLCMN --> TLCN
    TLC1 --> CP1
    TLCN --> CPN

The main thread receives all configuration and host updates. It then builds an immutable ThreadLocalClusterImpl snapshot and posts it to every worker's TLS slot. Workers read clusters from their slot without locking — see threading model.

Key abstractions

Type	File	Purpose
`ClusterManagerImpl`	`cluster_manager_impl.h`	Main-thread cluster catalogue and update dispatcher.
`ThreadLocalClusterManagerImpl`	(same file)	Per-worker view; what filters/router consult.
`ThreadLocalClusterImpl`	(same file)	Per-worker view of a single cluster.
`ClusterImplBase`	`upstream_impl.h`	Common base for all cluster types.
`HostImpl`	`upstream_impl.h`	A single endpoint with address, weight, locality, and health flags.
`PrioritySetImpl`	`upstream_impl.h`	A set of `HostSet`s indexed by priority.
`LoadBalancer`	`envoy/upstream/load_balancer.h`	Per-cluster host picker; concrete in `source/extensions/load_balancing_policies/`.
`OutlierDetectorImpl`	`outlier_detection_impl.h`	Per-host EWMA + ejection.
`HealthCheckerImplBase`	`health_checker_impl.h`	Active-HC base; concrete checkers in `source/extensions/health_checkers/`.
`LoadStatsReporterImpl`	`load_stats_reporter_impl.h`	LRS streaming client.

Cluster types

Cluster type is resolved by ClusterFactory::create and produces a different concrete cluster:

Type	Source	Where hosts come from
`STATIC`	`source/extensions/clusters/static/`	Bootstrap config
`STRICT_DNS`	`source/extensions/clusters/strict_dns/`	DNS resolution; refresh on TTL
`LOGICAL_DNS`	`source/extensions/clusters/logical_dns/`	DNS — picks one IP, lazy refresh
`EDS`	`source/extensions/clusters/eds/`	EDS xDS subscription
`ORIGINAL_DST`	`source/extensions/clusters/original_dst/`	Connection's original destination
`dynamic_forward_proxy`	`source/extensions/clusters/dynamic_forward_proxy/`	DNS + cache for L7 forward proxy
`redis`	`source/extensions/clusters/redis/`	Redis topology discovery
`aggregate`	`source/extensions/clusters/aggregate/`	Composite of other clusters

Each cluster type registers a factory; the manager looks up the factory by name.

Update propagation

When CDS adds or modifies a cluster:

CdsApiImpl::onConfigUpdate deserialises the protobuf and calls ClusterManagerImpl::addOrUpdateCluster.
The manager builds (or rebuilds) the Cluster on the main thread, kicks off DNS / EDS / health-check init.
After the cluster's init manager completes (first endpoint set + first HC pass for active HC clusters), the manager posts a new ThreadLocalClusterImpl to every worker.
Workers replace their slot atomically; in-flight requests on old hosts continue, new requests use the new host set.

When EDS pushes new endpoints to an existing cluster, the manager updates PrioritySetImpl and broadcasts a delta to workers. Workers don't tear down conn pools; they update host lists in place.

Connection pool management

Per-worker, the cluster manager maintains a ConnPoolMap per host: a map keyed by (protocol, transport_socket, options) to a Http::ConnectionPool::Instance or Tcp::ConnectionPool::Instance. When the router asks for a pool, the cluster manager (via ThreadLocalClusterImpl::tcpConnPool or httpConnPool) returns or creates one.

Pool lifetimes are tied to host lifetimes; when a host is removed (EDS update or HC failure), its pools drain and free.

See connection pools for the pool internals.

Health checking

Active health checking (source/extensions/health_checkers/) runs per-host on the main thread and broadcasts health-flag changes to workers. The shipped checkers:

http — HTTP/1 + HTTP/2 health probes.
grpc — grpc.health.v1.Health/Check.
tcp — raw TCP send/receive.
redis, thrift — protocol-specific.

Outlier detection (outlier_detection_impl.cc) is passive — the router reports request outcomes; the detector tracks per-host EWMA and may temporarily eject misbehaving hosts.

Async clients

The cluster manager doubles as a factory for Http::AsyncClient (source/common/http/async_client_impl.cc) and Tcp::AsyncTcpClient (source/common/tcp/async_tcp_client_impl.cc) instances. Filters that need to make outbound calls (e.g. ext_authz, jwt_authn JWKS fetch, ratelimit) get their client from cluster_manager.httpAsyncClientForCluster.

On-demand CDS (ODCDS)

ClusterDiscoveryManager (cluster_discovery_manager.cc) supports lazy cluster discovery: a route can reference a not-yet-known cluster, the manager subscribes on-demand, and the request waits until the cluster lands.

Integration points

xDS / CDS / EDS / SDS / HDS / LRS — every dynamic cluster pulls config from the xDS layer.
Transport sockets — clusters can declare per-match transport sockets (transport_socket_match_impl.cc).
Load balancers — cluster type is independent from LB policy; LB factories live in source/extensions/load_balancing_policies/.
Init manager — clusters register init targets that gate "ready to serve".
Router — every routed request asks the cluster manager for the target cluster.

Entry points for modification

Adding a cluster type: implement Upstream::ClusterFactory in source/extensions/clusters/<name>/ and register it.
Adding a load balancer policy: implement under source/extensions/load_balancing_policies/<name>/. See load balancing.
Modifying outlier detection: outlier_detection_impl.cc.
Modifying health-check semantics: the active checker base is in health_checker_impl.cc; per-protocol checkers are extensions.