envoyproxy/envoy

Connection pools

A connection pool is the per-host machinery that multiplexes upstream requests onto a (small, recycled) set of upstream connections. There is one pool per (host, protocol, transport_socket, options) tuple. Pools live on workers; there is no shared pool state across threads.

Purpose

Establish and reuse upstream connections.
Multiplex streams onto HTTP/2 and HTTP/3 connections.
Issue fresh connections for HTTP/1 (one stream per connection).
Enforce circuit breakers (max connections, max pending, max retries, max requests).
Choose between protocols via the conn pool grid when alternate-protocols are advertised.
Drain on host removal or admin events.

Layout

source/common/conn_pool/
├── conn_pool_base.{h,cc}         # Generic conn pool base (queue, draining, callbacks)

source/common/http/
├── conn_pool_base.{h,cc}         # HTTP-specific base
├── conn_pool_grid.{h,cc}         # H1/H2/H3 selection per attempt-cache logic
├── mixed_conn_pool.{h,cc}        # ALPN-aware H1/H2 pool
├── http1/
│   └── conn_pool.{h,cc}          # HTTP/1 pool (one stream per conn)
├── http2/
│   └── conn_pool.{h,cc}          # HTTP/2 pool (multiplexed)
└── http3/
    └── conn_pool.{h,cc}          # HTTP/3 pool (QUIC streams)

source/common/tcp/
├── conn_pool.{h,cc}              # TCP pool
└── original_conn_pool.{h,cc}     # Legacy TCP pool

source/common/upstream/
├── conn_pool_map.h               # Per-host pool keyed by protocol/options
└── priority_conn_pool_map.h      # Pool map by priority

Hierarchy

classDiagram
    class ConnPoolBase
    class HttpConnPoolBase
    class Http1ConnPool
    class Http2ConnPool
    class Http3ConnPool
    class HttpConnPoolGrid
    class TcpConnPool

    ConnPoolBase <|-- HttpConnPoolBase
    ConnPoolBase <|-- TcpConnPool
    HttpConnPoolBase <|-- Http1ConnPool
    HttpConnPoolBase <|-- Http2ConnPool
    HttpConnPoolBase <|-- Http3ConnPool
    HttpConnPoolBase <|-- HttpConnPoolGrid

HttpConnPoolGrid is a meta-pool: given alternate-protocols cache state (source/common/http/http_server_properties_cache_*.cc), it tries HTTP/3 first, falls back to HTTP/2/HTTP/1.1 if QUIC fails, and remembers the outcome.

Key abstractions

Type	File	Role
`ConnPoolBase`	`source/common/conn_pool/conn_pool_base.h`	Generic queue, drain, circuit-breaker accounting.
`HttpConnPoolBase`	`source/common/http/conn_pool_base.h`	HTTP-specific framing: streams, headers.
`Http::ConnectionPool::Instance` (interface)	`envoy/http/conn_pool.h`	What the router actually calls (`newStream`).
`Http::ConnectionPool::Callbacks`	(same)	Notification interface — pool-ready, pool-failure, stream encoder available.
`Tcp::ConnectionPool::Instance`	`envoy/tcp/conn_pool.h`	TCP equivalent; used by `tcp_proxy`, `redis_proxy`, etc.
`Upstream::ConnPoolMap<K,T>`	`source/common/upstream/conn_pool_map.h`	Owning container for pools keyed by `(protocol, transport socket)`.

How a request acquires a pool

sequenceDiagram
    participant Router as Router::Filter
    participant TLC as ThreadLocalClusterImpl
    participant Map as ConnPoolMap
    participant Pool as Http::ConnectionPool::Instance
    participant Codec as Upstream codec

    Router->>TLC: httpConnPool(host, priority, protocol, options)
    TLC->>Map: getPool(key)
    Map-->>TLC: existing or fresh pool
    TLC-->>Router: pool
    Router->>Pool: newStream(callbacks)
    Pool->>Pool: ensure connection / wait if pending
    Pool-->>Router: onPoolReady(encoder, host)
    Router->>Codec: encodeHeaders + body

The pool decides:

Whether to open a new connection (under max_connections).
Whether to queue the request (under max_pending_requests).
Whether to immediately fail (onPoolFailure) due to circuit breakers.

For HTTP/2 and HTTP/3, "open a new connection" is rare; new streams multiplex on existing ones. For HTTP/1, every concurrent request needs its own connection.

Drain and reset

Pools support multiple drain modes:

Idle drain — close connections after Cluster.common_http_protocol_options.idle_timeout.
Drain pending — when the host or cluster is removed: complete in-flight streams, refuse new ones.
Hard reset — close immediately on protocol violation.

Each pool exposes drainConnections(DrainBehavior) which the cluster manager calls during host removal or admin commands.

Watermarks

When the upstream connection's write buffer crosses high watermark, the pool propagates onAboveWriteBufferHighWatermark to all in-flight streams — they pause encoding from the downstream side. The reverse signal flows on low watermark. The plumbing reaches the HCM filter manager and ultimately the downstream codec.

Circuit breakers

ResourceManagerImpl (source/common/upstream/resource_manager_impl.h) is the per-cluster, per-priority budget for:

max_connections
max_pending_requests
max_requests
max_retries
max_connection_pools

Pools consult these counters before issuing or queuing.

TCP async clients

Filters that need raw TCP outbound (e.g. redis_proxy, generic TCP probing) use Tcp::AsyncTcpClient (source/common/tcp/async_tcp_client_impl.cc) which sits on top of a TcpConnPool.

Integration points

Cluster manager — owns the pool maps.
Router — main consumer of HTTP pools.
tcp_proxy, redis_proxy, mongo_proxy, etc. — consumers of TCP pools.
Transport sockets — the TLS layer (and others) wrap pool-owned sockets via the cluster's TransportSocketMatcher.
Alternate protocols cache — feeds HttpConnPoolGrid decisions.

Entry points for modification

Adding a new protocol pool: implement Http::ConnectionPool::Instance, register a factory consulted in ProdClusterManagerFactory::allocateConnPool.
Modifying retry/idle drain: read HttpConnPoolBase and the per-protocol implementations.
Modifying circuit breakers: ResourceManagerImpl.