turbo-tasks framework

The incremental computation engine that powers everything in Turbopack. Generic — knows nothing about bundlers, JavaScript, or HTTP. Just async memoized functions, persistent caching, and dependency tracking.

Source: turbopack/crates/turbo-tasks/src/. The biggest file in the crate is manager.rs at 82,204 lines — and the storage backend's mod.rs at 151,584 lines is the single largest Rust file in the entire repository.

Mental model

A turbo-tasks program describes computations as memoized async functions. Each function:

• Has typed inputs.
• Returns a Vc<T> — a "virtual computation" that resolves to a T.
• Is keyed by its inputs; identical inputs return the same Vc.
• Tracks the Vc values it reads as dependencies.
• Re-runs only when one of its dependencies invalidates.

graph TD
    Read[file_contents(path)] --> Vc1[Vc<String>]
    Parse[parse(file_contents)] --> Vc2[Vc<Ast>]
    Vc1 --> Parse
    Compile[compile(parse)] --> Vc3[Vc<Code>]
    Vc2 --> Compile
    Bundle[bundle(compile)] --> Vc4[Vc<Bundle>]
    Vc3 --> Bundle

    Watcher[fs watcher] -. invalidates .-> Vc1
    Vc1 -. propagates .-> Vc2
    Vc2 -. propagates .-> Vc3
    Vc3 -. propagates .-> Vc4

When a file changes, only Vc1 is invalidated, and Vc2/Vc3/Vc4 get marked dirty. The next pull of Vc4 triggers re-evaluation walking back through the graph; any node whose inputs are unchanged returns its cached value without re-running.

The #[turbo_tasks::function] macro

User code declares a memoized function with the #[turbo_tasks::function] attribute:

#[turbo_tasks::function]
async fn parse(source: ResolvedVc<File>) -> Result<Vc<Ast>> {
    let source = source.await?;
    let ast = swc_parse(&source.contents)?;
    Ok(Ast::cell(ast))
}

The macro transforms this into a registered "native function" with a stable id and arg-hashing logic. Implementation lives in turbopack/crates/turbo-tasks-macros/.

Vc and ResolvedVc

• Vc<T> — a strong handle to a memoized computation that produces a T. .await? resolves it.
• ResolvedVc<T> — a Vc<T> whose inner computation has been resolved to a stable cell id, suitable for use as a function input (idempotent across calls).
• RawVc — the underlying type-erased handle.

turbopack/crates/turbo-tasks/src/raw_vc.rs is the lowest-level handle.

The manager

turbopack/crates/turbo-tasks/src/manager.rs (82 KB) is the runtime: it owns the task scheduler, the cell store, the dependency tracker, the invalidation propagator, and the work-stealing executor that drives the tokio runtime. Every task runs through the manager.

Backend

The framework is generic over a "backend" that owns task storage. turbo-tasks-backend (turbopack/crates/turbo-tasks-backend/) is the production backend: a hybrid in-memory + persistent store. The backend's mod.rs is 151 KB — by far the largest file in the repo. It implements:

• Cell allocation per task instance.
• Read/write tracking with fine-grained invalidation.
• Persistence checkpoints to turbo-persistence.
• Garbage collection of stale cells.
• Multi-process coordination (one process owns the persisted store at a time).

storage.rs and storage_schema.rs define the on-disk format.

operation/ holds the per-task-state-transition primitives that compose into the larger backend operations.

Cell types

A computation produces values that get stored in cells. The framework supports several cell strategies:

• Tagged cells — one slot per call-site, replacing the previous value on each call.
• Auto-keyed cells — one slot per content hash, dedupes equal results.
• #[turbo_tasks::value] — the macro that declares a type as cellable.

Implementation: turbopack/crates/turbo-tasks/src/value_type.rs (19 KB).

Effects, completions, collectibles

• Effect (effect.rs) — a side-effecting task that runs once per scope, used for things like writing files.
• Completion (completion.rs) — a unit-typed Vc used for "this is done" signals.
• Collectibles (collectibles.rs) — a way for child tasks to bubble values up to a parent.

Strong / weak references

A task's dependency graph distinguishes strong dependencies (keep the result alive) from weak ones. The framework uses triomphe-style atomic refcounts (triomphe_utils.rs) for the strong / weak split.

Trace and debug

• trace.rs — Tracing subscriber integration for task spans.
• debug/ — Debug formatters for Vc<T> and friends.

Macros

turbopack/crates/turbo-tasks-macros/ defines:

• #[turbo_tasks::function] — declare a memoized function.
• #[turbo_tasks::value] — declare a cellable type.
• #[turbo_tasks::value_impl] — declare value-type impls.
• #[turbo_tasks::value_trait] — declare a trait that can be a Vc<dyn Trait>.

Tested via turbopack/crates/turbo-tasks-macros-tests/.

Filesystem

turbopack/crates/turbo-tasks-fs/ provides the async filesystem: every read goes through a memoized Vc<FileContent>. File watching plumbs invalidations into the framework so a file change propagates without polling.

Network

turbopack/crates/turbo-tasks-fetch/ — memoized HTTP fetching. Two requests for the same URL share a single result cell.

Env

turbopack/crates/turbo-tasks-env/ — memoized env-var reading. Used by language plugins to inline values at compile time.

Hash / bytes

• turbopack/crates/turbo-tasks-hash/ — stable hashing helpers.
• turbopack/crates/turbo-tasks-bytes/ — byte streams suitable for incremental processing.
• turbopack/crates/turbo-tasks-auto-hash-map/ — hash-map keyed by the auto-hashable type.

Allocations

turbopack/crates/turbo-tasks-malloc/ — allocation tracking instrumentation. Used by the trace UI to attribute memory to tasks.

Testing

turbopack/crates/turbo-tasks-testing/ — a deterministic, sequential backend for tests. Same trait surface as the production backend, no parallelism or persistence.

Why this framework

The Turbopack thesis is that incremental bundling should be expressed as a graph of pure(ish) functions that the runtime memoizes and invalidates automatically. Every other Turbopack crate is built on this: turbopack-core defines Asset, Module, Chunk as #[turbo_tasks::value] types; turbopack-ecmascript declares parse, analyze, transform as #[turbo_tasks::function]; etc.

The result is that:

• Bundling a 100-file project takes the same time as bundling a 10,000-file project on the second pass — only the changed graph nodes recompute.
• HMR is the same code path as initial bundling — it just operates on smaller subsets.
• Build cache persistence is automatic — the same engine writes the on-disk store.

Key source files

File	Purpose
turbopack/crates/turbo-tasks/src/lib.rs	Crate entry
turbopack/crates/turbo-tasks/src/manager.rs	The runtime
turbopack/crates/turbo-tasks/src/raw_vc.rs	RawVc handle
turbopack/crates/turbo-tasks/src/value_type.rs	#[turbo_tasks::value] machinery
turbopack/crates/turbo-tasks/src/macro_helpers.rs	Helpers for the proc macros
turbopack/crates/turbo-tasks/src/effect.rs	Effect tasks
turbopack/crates/turbo-tasks/src/invalidation.rs	Invalidation propagation
turbopack/crates/turbo-tasks/src/scope.rs	Task scopes
turbopack/crates/turbo-tasks-backend/src/backend/mod.rs	Production backend (151 KB — largest file in repo)
turbopack/crates/turbo-tasks-macros/	Proc macros