pingora-runtime

Active contributors: yuchen, ewang

Purpose

Thin wrapper over tokio's runtime. Provides two flavors used by pingora-core::server: work-stealing (the standard tokio multi-threaded runtime) and "no-steal" (N isolated single-threaded runtimes). Picks one at server bootstrap based on the work_stealing YAML key.

Directory layout

pingora-runtime/src/
└── lib.rs               Runtime, RuntimeBuilder, BlockingPoolOpts

Key abstractions

Type	File	What it is
Runtime	pingora-runtime/src/lib.rs	Wraps tokio's Runtime and an optional pool of single-thread runtimes
RuntimeBuilder	pingora-runtime/src/lib.rs	Configures threads, name, blocking pool, work-stealing
BlockingPoolOpts	pingora-runtime/src/lib.rs	Tokio blocking-thread-pool configuration

How it works

A Runtime is one of two variants:

• Steal: a standard tokio multi-threaded runtime. Threads share work via the work-stealing scheduler.
• NoSteal: an array of N tokio current-thread runtimes, each on its own OS thread. Tasks pinned to a runtime stay there. Less even load distribution, but no cross-thread synchronization. Used for very latency-sensitive paths in production.

The blocking pool config (BlockingPoolOpts) configures tokio's spawn_blocking thread pool: max threads, keep-alive duration, and stack size. Added in commit b994854 (Mar 2026) so callers can size the pool for their workload.

Integration points

• Used by pingora-core::server::Server::run_forever to spawn one runtime per service.
• Used directly by background services that want a runtime distinct from any service's runtime.

Entry points for modification

• New runtime flavor → add a variant to the Runtime enum and a builder method.
• Blocking pool tuning → the BlockingPoolOpts struct.

Key source files

File	Purpose
pingora-runtime/src/lib.rs	The whole crate (~400 lines)