xDS server

The agent runs an in-process Envoy xDS server so each registered Connect proxy can fetch its configuration over a long-lived gRPC stream. Code lives entirely under agent/xds/.

Purpose

Envoy proxies don't know about the catalog, intentions, JWT providers, or config entries. They speak Envoy's xDS protocol to ask "give me my Clusters / Endpoints / Listeners / Routes / Secrets." Consul implements this server side: it consumes a proxycfg.ConfigSnapshot from the proxy config manager (see Proxy config), translates it into Envoy proto resources, and streams updates incrementally with delta-xDS.

Directory layout

agent/xds/
├── server.go               # The ADS gRPC service (DiscoveryRequest/Response + Delta variants)
├── delta.go                # Delta-xDS state machine (incremental updates) (39 KB)
├── clusters.go             # Build CDS clusters from a snapshot (84 KB)
├── listeners.go            # Build LDS listeners (106 KB; the largest non-generated file)
├── listeners_apigateway.go # API gateway-specific listener composition
├── listeners_ingress.go    # Ingress gateway listener composition
├── routes.go               # Build RDS routes from discovery chain
├── endpoints.go            # Build EDS endpoints from health
├── secrets.go              # Build SDS secrets (leaf certs + JWT keys)
├── jwt_authn.go            # JWT authentication filter wiring
├── rbac.go                 # RBAC filter from intentions (43 KB)
├── failover_policy.go      # Locality-aware failover policy
├── locality_policy.go      # Same, with the locality stamping
├── protocol_trace.go       # protobuf trace tap helpers
├── resources.go            # The big "build all resources for a snapshot" entrypoint
├── extensionruntime/       # Bridges to the envoyextensions module
├── platform/               # OS-specific bits
├── naming/                 # Resource name composition (cluster names, SNI, etc.)
├── response/               # DiscoveryResponse helpers
├── accesslogs/             # Access log builder
├── config/                 # Bootstrap config / Envoy version handshake
├── configfetcher/          # In-memory snapshot fetcher
├── testcommon/, testdata/  # Golden file fixtures (huge)
└── ...

Key abstractions

Type	File	Purpose
Server	agent/xds/server.go	Implements envoy_service_discovery_v3.AggregatedDiscoveryService
xDSDeltaServer.Process	agent/xds/delta.go	The delta-xDS goroutine per Envoy connection
BuildResource functions	agent/xds/{clusters,listeners,routes,endpoints,secrets}.go	One per Envoy resource family
ResourceGenerator	agent/xds/resources.go	Coordinates generation across resource families for a snapshot
ConfigSource / Snapshot	agent/proxycfg/snapshot.go	The input contract: everything the xDS server needs about a single proxy

How it works

sequenceDiagram
    participant Envoy
    participant XDS as xds.Server
    participant Manager as proxycfg.Manager
    participant Cache as agent caches (catalog, intentions, leaf, root)

    Envoy->>XDS: gRPC StreamAggregatedResources
    XDS->>Manager: Watch(proxyID)
    Manager->>Cache: subscribe to deps
    Cache-->>Manager: events
    Manager-->>XDS: ConfigSnapshot v1
    XDS->>XDS: build CDS / LDS / RDS / EDS / SDS protos
    XDS-->>Envoy: DeltaDiscoveryResponse (typed_url=cluster.v3.Cluster)
    XDS-->>Envoy: DeltaDiscoveryResponse (typed_url=listener.v3.Listener)
    Envoy->>XDS: ack/nack
    Note over Manager,Cache: catalog/intentions/cert change
    Manager-->>XDS: ConfigSnapshot v2
    XDS->>XDS: diff resources, send only what changed
    XDS-->>Envoy: incremental updates

The delta-xDS state machine in delta.go tracks what each Envoy already has, computes additions/removals/updates per resource family, and respects ack/nack semantics — a NACK for one cluster shouldn't roll back the whole snapshot.

Resource generators

Family	Builder file	Highlights
Clusters	clusters.go	One cluster per upstream target; mTLS sockets via SDS; locality-aware load assignments
Listeners	listeners.go	Inbound listener with RBAC + JWT filters; outbound listeners per upstream and per gateway port
Routes	routes.go	Built from the compiled discovery chain (resolver+router+splitter)
Endpoints	endpoints.go	Health-aware target endpoints; supports peering and gateway addressing
Secrets	secrets.go	Leaf cert + root CA bundle delivered via SDS; JWT keys delivered as static secrets

The "compose-all" path is resources.go's ResourceGenerator.AllResourcesFromSnapshot — it walks the snapshot kind switch (connect-proxy, mesh-gateway, ingress-gateway, terminating-gateway, api-gateway) and assembles the per-Envoy resource set.

API gateway path

API gateways are a separate listener composition flow (listeners_apigateway.go) because they need:

• Per-route service-router semantics with HTTP filters.
• TLS termination from inline-certificate or file-system-certificate config entries (secrets.go).
• JWT validation per route (jwt_authn.go).
• Per-route filters that compose with envoy extensions (gw_per_route_filters_ce.go).

agent/consul/gateways/ holds the controller that reconciles api-gateway / route config entries; the xDS server consumes the resulting state via the proxy config manager.

Envoy extensions

envoyextensions/ (a separate Go module) is a plugin framework for in-process Envoy resource modifications. The xDS server's extensionruntime/ bridges its internal data flow to that module so extensions can mutate resources mid-flight (add filters, tweak clusters). Built-in extensions cover Lambda integration, AWS authentication, lua filters, OTEL tracing, property override, and external authorization.

Performance considerations

• The xDS server limits per-stream goroutines and uses a backpressure-aware ack/nack tracker in delta.go.
• agent/log-drop/ is wired in to drop xDS log lines under load to keep stderr from saturating.
• xdscapacity/ (in agent/consul/xdscapacity/) tracks how many proxies each server is feeding so peering and federation can balance the load when there are many servers.

Integration points

• Authentication: Envoy connects with the agent's local TLS or via the dataplane gRPC API. ACL token checks live in the gRPC auth interceptor (agent/grpc-external/).
• Proxy config glue: agent/proxycfg-glue/ adapts in-process data sources (cache, state store, leaf cert manager) to the contracts the xDS server expects, so the same xDS server runs both standalone and embedded in the dataplane stack.
• Test machinery: the xDS server has thousands of golden test cases in agent/xds/testdata/. Run with -update to regenerate when intentional resource shape changes happen.

Entry points for modification

• Add a new resource attribute: modify the corresponding builder file (clusters.go, listeners.go, ...). Update or add golden tests.
• Add a new gateway kind: add a builder branch in resources.go::AllResourcesFromSnapshot and a per-kind builder file.
• Hook into ack/nack: see the state tracking in delta.go (xDSDeltaConnection struct).
• Add an Envoy extension: see envoyextensions/extensions/ for examples; register in the runtime.