cts_runner

cts_runner (cts_runner/) drives the WebGPU Conformance Test Suite against wgpu via Deno. It's the single most important regression test surface: WebGPU conformance is what tells the project that its implementation matches what every other WebGPU implementation does.

Purpose

• Build a Deno binary linked against deno_webgpu (and therefore against the in-tree wgpu-core).
• Check out the WebGPU CTS at the revision pinned in cts_runner/revision.txt.
• Run the configured set of CTS selectors and report results.

Directory layout

cts_runner/
├── Cargo.toml
├── README.md
├── revision.txt          single-line: pinned CTS commit
├── examples/             test harness examples
├── src/                  Rust glue
├── tests/
├── test.lst              ~42 KB of CTS selectors that must pass
├── fail.lst              ~15 KB of known-failing selectors with pass-rate annotations
└── skip.lst              ~2 KB of selectors with ≥ 90% skips

How to run

The canonical entry point is cargo xtask cts:

cargo xtask cts                                    # default test list
cargo xtask cts 'webgpu:api,operation,buffers:*'    # one suite
cargo xtask cts -- --list 'webgpu:...'             # list, don't run
cargo xtask cts --backend metal                    # specify backend (used by fails-if conditions)

The xtask command:

1. Checks out (or refreshes) the CTS at cts_runner/revision.txt's revision (unless --skip-checkout).
2. Builds cts_runner with the right Deno features.
3. Loads test.lst, optionally filters with --filter <regex>.
4. Runs Deno with the CTS code.
5. Compares results against expectations in test.lst / fail.lst.

xtask/src/cts.rs is the implementation.

test.lst / fail.lst / skip.lst

The three files capture expectations:

• test.lst — selectors that must pass. CI fails if anything here fails.
• fail.lst — selectors that fail. Annotate with // xx% when the suite isn't 100 % failing.
• skip.lst — selectors that are ≥ 90 % skips.

Rules from AGENTS.md:

• Use the broadest wildcard that exclusively matches the right output file.
• Don't use a higher-level wildcard if it would catch tests belonging in a different file.
• Move tests into test.lst only when an entire suite is 100 % passing or skipped — no flakes, no remaining failures.

CTS revision bumps

renovate[bot] periodically opens PRs that bump cts_runner/revision.txt. Recent example: chore(deps): update cts digest to 8640b42 (#9466). After the bump, xtask cts may report new test selectors; expectation files must be updated to match.

What lives in tests/

A small Rust test harness that runs cts_runner itself with a fixed input. Used to make sure the runner's plumbing doesn't regress.

Integration points

• Above: CI (.github/workflows/ci.yml, .github/workflows/cts.yml).
• Below: deno_webgpu (in this repo), deno_core, deno_console, deno_url, deno_web, deno_webidl, deno_features — Deno's runtime crates pinned in the workspace.
• External: https://github.com/gpuweb/cts at the pinned revision.

Entry points for modification

• A new CTS suite expectation — edit test.lst / fail.lst / skip.lst.
• The runner harness — cts_runner/src/. Most changes are about wiring new Deno features or fixing test-discovery quirks.
• The xtask glue — xtask/src/cts.rs.