cloudflare/pingora

Background and design notes

Why Pingora is shaped the way it is.

The "framework, not server" decision

Pingora is a framework, not a configurable server. Compare to nginx (configurable) and Envoy (configurable, with extension points). Pingora makes you write Rust code to express your proxy logic. The README justifies this with "extensive customization" being a primary reason to choose Pingora — internal Cloudflare proxies have customization needs that don't fit a config language.

The trade-off: simple cases need more code than nginx, but complex cases that would require Lua/WASM in nginx are just impl ProxyHttp.

Why phase-based?

The phase model (request_filter, upstream_peer, response_filter, etc.) follows the nginx mental model. Engineers moving from nginx to Pingora map their access_by_lua to request_filter, proxy_pass to upstream_peer, etc. The CHANGELOG and Cloudflare blog posts make this lineage explicit.

The state machine itself is implemented from scratch in Rust — there's no nginx code or even structural translation, but the user-facing surface is deliberately familiar.

Why a custom timer wheel?

pingora-timeout exists because tokio's per-runtime timer wheel registers each timer with O(1) but non-trivial cost. At Cloudflare scale (millions of QPS per box), each request involves several short timeouts. Amortizing timer registration across calls cuts CPU. The fast wheel sacrifices precision (a few ms) for throughput.

Why custom case-preserving headers?

The http crate's HeaderMap lowercases header names. That breaks compatibility with legacy origin servers that case-match Set-Cookie exactly, or sniff for If-Modified-Since (capital I). A proxy can't pre-normalize because the response might be cached and re-served to clients that depend on the original case. So pingora-http keeps a side table of original cases.

Why TinyUFO?

Standard LRU is good enough for many workloads but performs poorly under skewed access — one-off requests evict frequently-used items. TinyLFU and S3-FIFO solve this, and tinyufo is a Rust implementation tuned for the cache shapes Pingora cares about (memory-bounded, async, multi-thread).

Why four TLS backends?

Different reasons to want different backends:

OpenSSL: ubiquitous and mature
BoringSSL: FIPS compliance, matches Cloudflare production
rustls: pure Rust, no native deps (good for some deployment shapes)
s2n-tls: AWS uses it, some users want to align

Maintaining four is more work than maintaining one, but the alternative is forcing every user of Pingora into a TLS choice they may not be allowed to make in their environment.

Why the workspace is "wide and shallow"

21 crates, most of them small, exists for two reasons:

Independent publication. Crates like pingora-error, pingora-pool, pingora-ketama, and tinyufo are usable on their own — you can pull pingora-ketama into your own non-Pingora project and just use ketama hashing.
Compile-time gating. Mutually exclusive TLS backends and optional features (cache, lb, proxy) work better when each is its own crate. The umbrella pingora crate wires them up via cargo features without forcing every dependent of pingora-core to recompile.

The cost is more Cargo.toml files to maintain. The lockstep version policy (every crate has the same version) keeps it manageable.

Why "rolling MSRV every six months"

Pingora targets stable Rust. The MSRV policy is "we'll bump if the new version is at least six months old." This balances:

New language features (let-else, GAT, async-trait removal someday)
Compatibility with downstream users who can't update Rust on demand

The current MSRV (1.84) was bumped from 1.83 in commit 9855feb (Apr 2026).

Pitfalls to know about

Cache APIs are volatile. The README and the cache module's docs both warn. Read the CHANGELOG before upgrading.
CacheKey::default was removed in 0.8.0. Implement cache_key_callback yourself.
atty was removed for security. If you depended on it transitively, audit.
CONNECT proxying is off by default in 0.8.0. Enable explicitly if you need it.
rustls is "highly experimental" per pingora/Cargo.toml. The warning is real.
Graceful upgrade is Linux-only. SCM_RIGHTS doesn't have a portable equivalent.

Internal Cloudflare context

The CONTRIBUTING file is explicit that internal use comes first. Cloudflare's CDN runs on a fork (or, more accurately, the public repo is a periodic snapshot of the internal repo with internal-only changes stripped). Accepted PRs land in main via a batched rebase that interleaves them with internal changes, which is why your commits don't appear under your own merge commit.

This also means that "Pingora-the-public-thing" is a slightly different thing than "Pingora-the-Cloudflare-internal-thing." The public version is fully functional but lags slightly on internal-only optimizations.