Load balancing

Load balancing is the per-request decision: given a cluster's set of healthy hosts, which one should serve this request? The cluster manager builds and maintains the host set; the LB policy picks a host. Policies are extensions registered through the load-balancing factory framework in source/extensions/load_balancing_policies/.

Shipped policies

Policy	Path	Behaviour
round_robin	round_robin/	Plain round-robin over healthy hosts.
random	random/	Random pick.
least_request	least_request/	"Power of two choices" — pick N random hosts, take the one with the fewest active requests. Default N=2.
ring_hash	ring_hash/	Consistent hashing on a ring; key from header / source IP / cookie / etc.
maglev	maglev/	Google's Maglev hash; better load distribution than ring_hash.
subset	subset/	Selects the subset of hosts that match request metadata, then defers to a child policy.
cluster_provided	cluster_provided/	The cluster type provides its own LB (original_dst, cluster_header).
wrr_locality	wrr_locality/	Weighted round-robin by locality, then defers to a child policy for in-locality LB.
client_side_weighted_round_robin	client_side_weighted_round_robin/	Weights derived from server-emitted load reports (xDS LRS).
override_host	override_host/	Lets a filter pin a request to a specific host via filter state — useful for sticky sessions.
dynamic_modules	dynamic_modules/	LB implemented as a dynamic module.

How the policy fits in

sequenceDiagram
    participant Router
    participant TLC as ThreadLocalClusterImpl
    participant LB as LoadBalancer
    participant Pool as Conn pool

    Router->>TLC: chooseHost(context)
    TLC->>LB: chooseHost(context)
    LB-->>TLC: host
    TLC-->>Router: host
    Router->>Pool: newStream(host)

LoadBalancerContext (envoy/upstream/load_balancer.h) carries everything the policy might need: hash key, downstream address, request headers/info, metadata match criteria. Policies are stateless across calls (or hold per-thread state in TLS).

Priority and locality

Before a policy runs, the cluster manager computes the priority level to use. Each cluster has up to N priorities; if the highest-priority level is healthy enough, it wins; otherwise traffic spills to lower priorities. Within a priority level, locality-weighted routing optionally weights by locality. The mechanics are in source/common/upstream/load_balancer_factory_base.h and source/common/upstream/upstream_impl.cc.

Subset policy

subset wraps another policy. Matching is by route metadata against host metadata. Common use: route per-customer / per-tenant traffic to dedicated nodes within the same cluster, or do canary routing without separate clusters.

Hash policies

ring_hash and maglev need a hash. The hash policy comes from the route (HashPolicy.header, cookie, source_ip, query_parameter, filter_state) — see source/common/router/router.cc. The first matching hash policy that produces a value wins.

Static and dynamic weights

round_robin honours each host's static weight. client_side_weighted_round_robin derives weights from per-host server load reports (the upstream emits ORCA load metrics and Envoy converts them into weights).

Adding a new policy

1. New directory under source/extensions/load_balancing_policies/<name>/.
2. Implement Upstream::ThreadAwareLoadBalancer and / or Upstream::LoadBalancer (per-worker).
3. Register a Upstream::TypedLoadBalancerFactory.
4. Proto under api/envoy/extensions/load_balancing_policies/<name>/v3/.
5. Register in extensions_build_config.bzl and add tests.

The simplest reference is random or round_robin; subset is a good wrapper example; client_side_weighted_round_robin is the most complex production policy.

See also

• Cluster manager
• Connection pools
• Router — produces the LoadBalancerContext.