pingora-load-balancing

Active contributors: davis, ewang, yuchen

Purpose

Load balancing recipes built on top of pingora-proxy. Provides:

• A LoadBalancer<S> parameterized over a backend selection strategy.
• Backend health checking (TCP, HTTP/1, HTTP/2) running as a background service.
• Service discovery (static, with hooks for dynamic).
• Selection strategies: RoundRobin, Random, Weighted, Consistent (ketama-based), FVN (hash).

Directory layout

pingora-load-balancing/src/
├── lib.rs               LoadBalancer, Backend, prelude
├── background.rs        Background service wrapper
├── discovery.rs         Discovery trait, static impl
├── health_check.rs      TCP/HTTP health checks
└── selection/           Strategy modules: round_robin, random, weighted, consistent, fnv

Key abstractions

Type	File	What it is
LoadBalancer<S>	pingora-load-balancing/src/lib.rs	Holds a set of backends, a selection strategy, optional health check
Backend	pingora-load-balancing/src/lib.rs	Address + weight
Backends	pingora-load-balancing/src/lib.rs	The current healthy/unhealthy set
BackendIter	pingora-load-balancing/src/lib.rs	Iterator returned by select_with
Discovery	pingora-load-balancing/src/discovery.rs	Trait that produces the backend set
HealthCheck	pingora-load-balancing/src/health_check.rs	Trait + TcpHealthCheck and HttpHealthCheck impls
BackendSelection	pingora-load-balancing/src/selection/mod.rs	Strategy trait
RoundRobin, Random, Weighted, Consistent, FVN	selection/*.rs	Strategies

How it works

graph TD
    discovery[Discovery::discover<br/>static or dynamic]
    update[LoadBalancer::update]
    backends[Backends set]
    hc[HealthCheck task]
    healthy[Healthy subset]
    select[LoadBalancer::select<br/>or select_with]
    proxy[ProxyHttp::upstream_peer]

    discovery --> update --> backends
    backends --> hc
    hc --> healthy
    healthy --> select
    select --> proxy

The user creates a LoadBalancer<RoundRobin> from an iterator of socket addresses. To enable health checks, call set_health_check(TcpHealthCheck::new()) and wrap the load balancer in background_service("health check", upstreams) — this starts a background tokio task that periodically rechecks each backend.

The update method is called from the background task to swap in a new healthy set without dropping in-flight requests. The 0.8.0 release fixed a bug where ketama state wasn't preserved across updates. The record_discovery_and_build_durations change (Mar 2026) added timing observability to this update path.

Selection strategies

Strategy	Module	Behavior
RoundRobin	selection/round_robin.rs	Atomic counter, simple
Random	selection/random.rs	Uniform random
Weighted<R>	selection/weighted.rs	Weighted random, generic over RNG/iterator
Consistent	selection/consistent.rs	Ketama hash ring, calls into pingora-ketama
FVN	selection/fnv.rs	Hashing with FVN (used for legacy compatibility)

Health checks

TcpHealthCheck opens a TCP connection (and optionally completes a TLS handshake). HttpHealthCheck issues an HTTP request and asserts on the status code. Health-check frequency is set via LoadBalancer::health_check_frequency.

The record_durations change in LoadBalancer::update exposes discovery_duration and build_duration so operators can alert on slow updates.

Integration points

• Users implement ProxyHttp::upstream_peer and call self.load_balancer.select(...) inside.
• Wraps as a BackgroundService registered with the Server.
• Re-exported under pingora::lb::* when lb cargo feature is on.

Entry points for modification

• New selection strategy → add a module under selection/, implement BackendSelection.
• New discovery source (DNS, k8s, etc.) → implement Discovery.
• New health check protocol → implement HealthCheck.

Key source files

File	Purpose
pingora-load-balancing/src/lib.rs	LoadBalancer, Backend, top-level API
pingora-load-balancing/src/health_check.rs	TCP and HTTP health checks
pingora-load-balancing/src/discovery.rs	Discovery trait
pingora-load-balancing/src/background.rs	Background-service wrapper
pingora-load-balancing/src/selection/	Selection strategies