pingora-proxy

Active contributors: ewang, mgumport, andrew, davis

Purpose

The HTTP proxy framework. Implements a phase-based reverse-proxy state machine. Users provide a type implementing the ProxyHttp trait; this crate handles HTTP/1 and HTTP/2 wire format, connection pooling, retries, response caching glue, and the user's filters.

Directory layout

pingora-proxy/src/
├── lib.rs               HttpProxy struct, http_proxy/http_proxy_service entry points
├── proxy_trait.rs       ProxyHttp trait — the user-facing surface
├── proxy_h1.rs          HTTP/1 proxy main loop
├── proxy_h2.rs          HTTP/2 proxy main loop
├── proxy_custom.rs      Custom (encapsulated) HTTP transport loop
├── proxy_cache.rs       Cache state machine integration (~2,800 lines)
├── proxy_common.rs      Shared helpers across loops
├── proxy_purge.rs       Cache purge handling (PURGE method)
└── subrequest/          Subrequest framing (cache fill, etc.)

Key abstractions

Type	File	What it is
ProxyHttp	pingora-proxy/src/proxy_trait.rs	The trait users implement; one required method (upstream_peer), many optional
HttpProxy<SV, C>	pingora-proxy/src/lib.rs	Concrete proxy app generic over user impl SV and custom connector C
Session	pingora-proxy/src/lib.rs (re-export)	Per-request handle: downstream session + cache + ctx
FailToProxy	pingora-proxy/src/proxy_trait.rs	Return type from fail_to_proxy()
PurgeStatus	pingora-proxy/src/proxy_purge.rs	Outcome of a PURGE request
MultiRangeInfo, RangeType	pingora-proxy/src/proxy_cache.rs (re-export)	Range-request handling
http_proxy_service, http_proxy_service_with_name	pingora-proxy/src/lib.rs	Constructor functions that wrap a ProxyHttp impl into a Service

How it works

Lifecycle of one request

graph TD
    accept[ServerApp::process_new<br/>session accepted]
    early[early_request_filter]
    rfilter[request_filter]
    cache_lookup[Cache lookup<br/>proxy_cache.rs]
    upstream_peer[upstream_peer]
    connect[Connect to peer<br/>via Connector]
    failconn{Connection<br/>OK?}
    fail_to_connect[fail_to_connect]
    upstream_request[upstream_request_filter]
    send[Send request to upstream]
    recv[Receive response]
    upstream_response[upstream_response_filter]
    response[response_filter / response_body_filter]
    log[logging]

    accept --> early --> rfilter --> cache_lookup
    cache_lookup -->|miss/disabled| upstream_peer
    cache_lookup -->|hit| response
    upstream_peer --> connect --> failconn
    failconn -->|err| fail_to_connect --> upstream_peer
    failconn -->|ok| upstream_request --> send --> recv --> upstream_response --> response --> log

The top-level driver lives in proxy_h1.rs::HttpProxy::proxy_to_h1_upstream, proxy_h2.rs::proxy_to_h2_upstream, and proxy_custom.rs::proxy_to_custom_upstream. They share helpers from proxy_common.rs. Cache-served responses bypass the upstream loop entirely; the cache state machine in proxy_cache.rs decides whether to consult the cache, fill it, revalidate, or stale-while-revalidate.

Three core entry points

// Build a service from a ProxyHttp impl
pub fn http_proxy_service<SV>(conf: &Arc<ServerConf>, sv: SV) -> Service<HttpProxy<SV>>;

// Same, but with a custom name (shows up in metrics/logs)
pub fn http_proxy_service_with_name<SV>(...) -> ...;

// Builder API (added in 0.8.0) — for setting ServerOptions etc.
let svc = HttpProxyServiceBuilder::new(conf, sv)
    .with_options(...)
    .build();

Subrequests

A subrequest is a request made from inside an active proxy session, e.g. for cache fill or a cache purge. The subrequest submodule (pingora-proxy/src/subrequest/) contains the small state machine that frames them. The pipe_subrequest utility (added in 0.8.0) lets a proxy treat a subrequest as a "pipe" — direct send of the request body, with chained outcome handling.

Cache integration

proxy_cache.rs is the largest single file in the workspace. It:

• Decides cache key (calls user's cache_key_callback)
• Looks up the cache via the Storage trait from pingora-cache
• Honors Vary via the VarianceBuilder
• Coordinates write locks so that simultaneous misses don't dogpile
• Implements range requests, byte-range responses, and conditional revalidation
• Streams cache fills back to the downstream while writing them

User callbacks specific to caching include cache_lookup_filter, cache_hit_filter, cache_miss, should_serve_stale, etc. — all defined as default-empty methods on ProxyHttp.

ProxyHttp callback summary

The trait is documented end-to-end in Proxy phases. Highlights:

• Required: type CTX, fn new_ctx, async fn upstream_peer
• Pre-upstream: early_request_filter, request_filter, request_body_filter, proxy_upstream_filter
• Connection: connected_to_upstream, fail_to_connect
• Upstream send: upstream_request_filter, adjust_upstream_modules
• Response: upstream_response_filter, upstream_response_body_filter, upstream_response_trailer_filter, response_filter, response_body_filter, response_trailer_filter
• Cache: ~10 cache callbacks (cache_key_callback, cache_lookup_filter, etc.)
• Errors and termination: error_while_proxy, fail_to_proxy, logging
• Helpers: request_summary, suppress_error_log, init_downstream_modules

Integration points

• Depends on pingora-core for sessions, peers, modules, server lifecycle.
• Depends on pingora-cache (when cache feature on) for the cache state machine.
• Used by pingora-load-balancing examples and end-to-end tests.

Entry points for modification

• Adding a new proxy phase → add the trait method (default-empty) in proxy_trait.rs, then call it at the right point in proxy_h1.rs/proxy_h2.rs. Keep the call site identical across protocol versions.
• Adding a feature flag for a new phase → wire through [features] in this crate's Cargo.toml, propagate up to pingora/Cargo.toml.
• Touching proxy_cache.rs → there's no good substitute for reading the file. Most cache bugs come from misordering interactions with the cache lock.

Key source files

File	Purpose
pingora-proxy/src/lib.rs	HttpProxy struct, entry-point functions, builder
pingora-proxy/src/proxy_trait.rs	The ProxyHttp trait — the public surface
pingora-proxy/src/proxy_h1.rs	HTTP/1 main loop
pingora-proxy/src/proxy_h2.rs	HTTP/2 main loop
pingora-proxy/src/proxy_custom.rs	Custom transport main loop
pingora-proxy/src/proxy_cache.rs	Cache state machine integration
pingora-proxy/src/proxy_purge.rs	PURGE method handling
pingora-proxy/src/proxy_common.rs	Shared helpers
pingora-proxy/src/subrequest/	Subrequest plumbing
pingora-proxy/examples/load_balancer.rs	The README/quickstart example