Service mesh and Gateway API

Active contributors: sayboras, mhofstetter, jrajahalme, michi-covalent

Purpose

Cilium implements a sidecar-less service mesh: a single Envoy per node provides L7 traffic management, TLS termination, mutual auth, and Gateway API support without injecting sidecars into pods. The mesh layer reuses the same identity, policy, and observability primitives the rest of Cilium uses.

Capabilities

• Kubernetes Gateway API — Gateway, HTTPRoute, GRPCRoute, TLSRoute, TCPRoute are reconciled into Envoy listeners.
• Ingress — the legacy Ingress resource is supported via the same Envoy plumbing.
• Mutual authentication — workloads authenticate to each other using SPIFFE identities derived from their Cilium identity (pkg/auth/).
• Custom Envoy — CiliumEnvoyConfig and CiliumClusterwideEnvoyConfig let users attach arbitrary Envoy resources (filters, listeners, clusters) to selected services.
• Traffic shaping — rate limiting, retries, header rewrites, request mirroring via Envoy's standard filters.
• Observability — Hubble parses Envoy access logs into HTTP-aware flows.

Directory layout

pkg/ciliumenvoyconfig/         # CEC and CCEC reconciler
pkg/envoy/                     # Envoy supervisor + xDS server
pkg/proxy/                     # per-endpoint redirect plumbing
pkg/auth/                      # mutual auth (SPIFFE)
pkg/gateway-api/               # Gateway API translator (where present)
pkg/ingress/                   # Ingress controller
api/v1/envoy/                  # Cilium-specific Envoy filter protos
images/cilium-envoy/           # the cilium-envoy distroless image

Gateway API flow

graph LR
    User[User applies Gateway + HTTPRoute]
    K8s[kube-apiserver]
    Op[cilium-operator<br/>GatewayClass controller]
    Agent[cilium-agent]
    CEC[CiliumEnvoyConfig]
    Envoy[Envoy on each node]

    User --> K8s
    K8s --> Op
    Op -->|translate| CEC
    K8s --> Agent
    Agent -->|reconcile CEC| Envoy
    Envoy -->|xDS| Envoy

The operator owns the Gateway API reconciliation: it watches Gateway and HTTPRoute and emits one or more CiliumEnvoyConfig (or cluster-wide variants) plus a Service of type LoadBalancer for the data path. The agent watches CEC and pushes configuration to its local Envoy via xDS.

Mutual authentication

pkg/auth/ provides L7-style identity-based authentication for L3/L4 connections. When two endpoints have an authentication-required policy:

1. The agent on the source side initiates an authentication handshake with the agent on the destination side using their respective SPIFFE identities (derived from Cilium identity labels).
2. Once authenticated, the agent programs a "auth" entry into the policy map; subsequent packets are forwarded.
3. Auth state has a configurable TTL and is periodically refreshed.

This gives mutual auth without modifying the application or injecting a sidecar; the agent's auth machinery is the trust anchor.

CEC and CCEC

Two CRDs allow direct Envoy programming:

CRD	Scope	Use cases
CiliumEnvoyConfig	Namespaced	Per-service custom filter chains, header policies.
CiliumClusterwideEnvoyConfig	Cluster	Global TLS interception, cluster-wide L7 rate limits.

A CiliumEnvoyConfig selects services via labels and embeds a partial Envoy Bootstrap. The reconciler in pkg/ciliumenvoyconfig/manager.go validates the config, translates it into xDS resources, and publishes them.

Ingress

The legacy Kubernetes Ingress controller (pkg/ingress/) maps Ingress resources onto the same Envoy machinery. cilium ingress subcommands in the CLI manage the lifecycle (cilium-cli/cli/ingress.go).

Integration points

• Envoy: see systems/envoy-l7-proxy.md for the proxy itself.
• Identity: auth and L7 policy bind to the same identities that L3/L4 policy uses.
• Hubble: L7 flows include HTTP method, path, status, latency.
• LB-IPAM: Gateway resources usually create LoadBalancer services that the LB-IPAM controller assigns IPs to.

Entry points for modification

• New Gateway API feature: extend pkg/gateway-api/ and the CEC translator.
• New auth backends: extend pkg/auth/ (today only the SPIFFE-derived backend ships).
• New Cilium Envoy filter: define proto in api/v1/envoy/, implement the C++ filter in the cilium/proxy repo, wire it via pkg/envoy/listener.go.

Key source files

File	Purpose
pkg/ciliumenvoyconfig/manager.go	CEC reconciler.
pkg/envoy/listener.go	Listener generation.
pkg/auth/manager.go	Mutual auth manager.
pkg/ingress/cell.go	Ingress controller cell.
cilium-cli/cli/gateway-api.go	Gateway API CLI.