HTTP caching

pingora-cache is the cache state machine; pingora-proxy/src/proxy_cache.rs is the integration into the proxy loop. This page shows how they fit together.

When to enable

Caching is opt-in at compile time (cache cargo feature) and opt-in at runtime (call enable_cache_filter in your request_filter or set up cache filters on the proxy).

The README warns: cache APIs are "highly volatile." The CHANGELOG confirms this with frequent breaking changes between 0.6/0.7/0.8.

State machine

stateDiagram-v2
    [*] --> Uninit
    Uninit --> Disabled: NoCacheReason
    Uninit --> CacheKey: enable + key set
    CacheKey --> Lookup
    Lookup --> Hit: fresh
    Lookup --> Miss
    Lookup --> Stale: expired
    Stale --> Revalidating: conditional GET
    Revalidating --> Hit: 304
    Revalidating --> Miss: 200 (replace)
    Miss --> Expired: write done
    Hit --> Expired: read done
    Disabled --> [*]
    Expired --> [*]

The phase enum is CachePhase in pingora-cache/src/lib.rs. The transitions are driven by the proxy's calls into HttpCache.

Per-request flow

sequenceDiagram
    participant Proxy as Proxy loop<br/>(proxy_cache.rs)
    participant Cache as HttpCache<br/>(pingora-cache)
    participant Storage as Storage trait
    participant Lock as CacheKeyLock
    participant User as ProxyHttp impl

    Proxy->>User: cache_key_callback
    User-->>Proxy: CacheKey
    Proxy->>Cache: cache_lookup
    Cache->>Storage: lookup(key)
    Storage-->>Cache: Hit / Miss / Stale
    alt Hit
        Cache-->>Proxy: HitHandler
        Proxy->>User: cache_hit_filter
        Proxy-->>Client: stream cached body
    else Miss
        Cache->>Lock: acquire write permit
        Lock-->>Cache: WritePermit or wait
        Proxy->>User: upstream_peer
        Proxy->>Upstream: fetch
        Cache->>Storage: open MissHandler
        loop body chunks
            Proxy->>Storage: write chunk via MissHandler
            Proxy->>Client: forward chunk
        end
    else Stale
        Proxy->>Upstream: conditional GET
        alt 304
            Cache->>Storage: update meta
            Proxy->>Client: serve cached body
        else 200
            Cache->>Storage: replace
            Proxy->>Client: forward upstream body
        end
    end

The lock-coalescing in the middle is what prevents stampedes: the second request for the same missing key sees LockStatus::Pending and either waits for the first to finish or bypasses the cache (depending on cache_lookup_filter's return).

What the user supplies

User code provides:

1. The cache key. Override cache_key_callback. Default is unimplemented! (intentionally — there's no sensible default since 0.8.0 removed CacheKey::default).
2. The cacheability decision. Override cache_lookup_filter to decide whether this request should consult the cache, given the current request and the cache's current state.
3. Variance. Override the part that builds variance via VarianceBuilder if the cached responses depend on additional dimensions beyond Vary.
4. Stale handling. Override should_serve_stale to control whether stale responses can be served.
5. Cache filters. Lots of optional callbacks let you log, adjust headers, or short-circuit at specific cache transitions.

Storage backend

Storage (in pingora-cache/src/storage.rs) is the trait you implement for a custom backend:

• lookup(key) -> Result<Option<(CacheMeta, HitHandler)>>
• purge(key, PurgeType) -> Result<bool>
• get_miss_handler(key, meta, ttl) -> Result<MissHandler>

HitHandler exposes the cached body as a stream of chunks; MissHandler accepts chunks and writes them. The reference impl MemCache (pingora-cache/src/memory.rs) is in-memory; production deployments typically have a custom disk-backed Storage.

Eviction

Out of the box, eviction is handled by the storage backend. pingora-cache/src/eviction/ provides reusable eviction managers (SimpleLRU, Fifo) that a custom Storage can compose with. The 0.7.0 release added "evict when asset count exceeds optional watermark."

Range requests

Range request handling lives in pingora-proxy/src/proxy_cache.rs::range_filter. It supports single-range and multipart-range responses, with a configurable upper bound on the number of multipart ranges (200 in 0.7.0).

A subtle change in 0.7.0: bytes= with an empty/whitespace-only range-set now returns 416, per RFC 9110 14.1.2.

Cache PUT

pingora-cache/src/put.rs implements explicit cache insertion (PUT). Used by Cloudflare-internal flows that want to populate the cache from outside the proxy loop.

Source map

Concern	File
State machine	pingora-cache/src/lib.rs
Storage trait	pingora-cache/src/storage.rs
Cache lock	pingora-cache/src/lock.rs
Cache key	pingora-cache/src/key.rs
Variance	pingora-cache/src/variance.rs
Cache-Control parser	pingora-cache/src/cache_control.rs
Eviction	pingora-cache/src/eviction/
Predictor	pingora-cache/src/predictor.rs
In-memory backend	pingora-cache/src/memory.rs
Cache PUT	pingora-cache/src/put.rs
Proxy integration	pingora-proxy/src/proxy_cache.rs
PURGE method	pingora-proxy/src/proxy_purge.rs

See also

• pingora-cache
• pingora-proxy
• Proxy phases — cache callbacks listed at the bottom