pingora-timeout

Active contributors: yuchen, ewang

Purpose

Faster async timer than tokio::time for sub-second timeouts. Pingora handles a lot of short timeouts (TLS handshake, header read, body chunk) and the cost of registering a timer with tokio's wheel adds up. This crate amortizes timer registration so each timeout is closer to a free atomic load than a syscall.

Directory layout

pingora-timeout/src/
├── lib.rs              Timeout, fast_timeout API
├── fast_timeout.rs     Shared timer wheel implementation
└── timer.rs            Internal timer wheel

Key abstractions

Type	File	What it is
Timeout	pingora-timeout/src/lib.rs	Drop-in replacement for tokio::time::Timeout
fast_timeout	pingora-timeout/src/fast_timeout.rs	Shared timer wheel, fast_sleep, fast_timeout
Timer	pingora-timeout/src/timer.rs	Internal wheel implementation

How it works

A standard tokio::time::sleep registers with tokio's per-runtime timer wheel. For a 100ms sleep that's fine. For tens of thousands of short sleeps per second per worker, the registration overhead dominates.

pingora-timeout keeps its own coarse timer wheel keyed off a wall-clock reference. Sleeps shorter than the wheel resolution wake up immediately on the next tick. The result: O(1) registration, O(1) wakeup, no per-sleep tokio bookkeeping.

fast_timeout is the ergonomic API that mirrors tokio::time::timeout but uses the fast wheel. The cost is reduced precision — wakeups are bounded by the wheel resolution (a few milliseconds typically).

Integration points

• pingora-core/src/server/mod.rs uses pingora_timeout::fast_timeout for the shutdown grace period.
• pingora-core HTTP/1 and HTTP/2 read/write paths use this for header and body timeouts.
• Also re-exported as pingora::time when the time cargo feature is on.

Entry points for modification

• Tuning wheel resolution → fast_timeout.rs.
• Internal wheel data structure → timer.rs.

Key source files

File	Purpose
pingora-timeout/src/lib.rs	Public API
pingora-timeout/src/fast_timeout.rs	Shared wheel + fast_sleep/fast_timeout
pingora-timeout/src/timer.rs	Wheel implementation