rust-lang/rust

Testing

Almost every change in this repository ends with a test. The test suite is large, varied, and driven by a single tool: compiletest.

Suites at a glance

The tests/ directory is split into suites; each is a directory with its own conventions:

Suite	What it tests	Driver
`tests/ui/`	Compiler stdout/stderr from real Rust programs	compiletest
`tests/run-make/`	End-to-end builds via a Rust support library	compiletest + `run-make-support`
`tests/run-make-cargo/`	End-to-end builds that need Cargo	compiletest
`tests/codegen-llvm/`	Specific patterns in emitted LLVM IR / asm	`FileCheck`-style annotations
`tests/assembly-llvm/`	Final assembly output	as above
`tests/codegen-units/`	Codegen unit partitioning	compiletest
`tests/incremental/`	Caching correctness across edits	compiletest
`tests/mir-opt/`	MIR after specific passes	snapshot diff
`tests/coverage/`	LLVM source-based coverage	compiletest
`tests/debuginfo/`	LLDB / GDB / CDB sessions	compiletest
`tests/pretty/`	AST/HIR pretty-printing round-trip	compiletest
`tests/rustdoc-html/`	rustdoc HTML output	jsondoclint / html-checker
`tests/rustdoc-json/`	rustdoc JSON output	`jsondocck`
`tests/rustdoc-ui/`	rustdoc diagnostics	compiletest
`tests/rustdoc-gui/`	rustdoc HTML in a real browser	`rustdoc-gui-test`
`tests/crashes/`	Currently-known ICEs (not a regression suite)	compiletest
`tests/ui-fulldeps/`	UI tests that depend on rustc internals (slower)	compiletest
`tests/build-std/`	`-Zbuild-std` testing	compiletest

Running tests

./x test                                # everything (slow!)
./x test tidy                           # repo-wide lint
./x test tests/ui                       # one suite
./x test tests/ui/borrowck              # one directory
./x test tests/ui/borrowck/borrow.rs    # one file
./x test --stage 1 tests/ui             # use a stage-1 toolchain
./x test tests/ui --bless               # update expected output
./x test tests/ui --test-args=...       # pass args to compiletest

For incremental development, run a single test or directory; the full tests/ui/ is several thousand tests.

Writing a UI test

A UI test is a Rust source file with optional directives at the top and an expected .stderr / .stdout file:

//@ check-fail
//@ edition: 2021

fn main() {
    let x: u32 = "hello"; //~ ERROR mismatched types
}

The //@ lines are compiletest directives (exit code, edition, target, feature flags, …). The //~ ERROR lines anchor expected diagnostics to specific source lines. The expected stderr lives in a sibling .stderr file, automatically generated by --bless.

Common directives (the full list is in src/tools/compiletest/src/directive_list.rs):

Directive	Purpose
`//@ check-pass`	Compilation should succeed (no codegen)
`//@ check-fail`	Compilation should fail at type check
`//@ build-pass` / `build-fail`	As above, including codegen
`//@ run-pass` / `run-fail`	Compile and run; success/exit-with-error
`//@ edition: 2021`	Set the edition
`//@ compile-flags: -Cfoo=bar`	Pass through to rustc
`//@ revisions: a b c`	Run multiple compilation passes with different cfgs
`//@ ignore-windows`	Skip on Windows; many `ignore-` and `only-` variants exist
`//@ aux-build: helper.rs`	Build a dependency crate

MIR-opt tests

tests/mir-opt/ snapshots MIR after a given pass. To regenerate:

./x test tests/mir-opt --bless

The expected MIR lives in .mir files alongside the source.

Codegen and assembly tests

tests/codegen-llvm/ and tests/assembly-llvm/ use FileCheck-style // CHECK: comments to assert that specific lines appear in the LLVM IR or asm output. These are sensitive to LLVM version; pin them tightly.

run-make tests

A run-make test is a directory with an rmake.rs file that uses run-make-support to drive rustc/cargo/the linker and assert on outputs. Use these when a UI test isn't expressive enough — e.g., multi-crate builds, weird link configurations, custom file layouts.

Tidy

./x test tidy is a fast lint over the entire repo:

Style violations (line length, trailing whitespace, tab usage)
License header presence
Alphabetical ordering of certain lists (the # tidy-alphabetical-start markers)
File-size limits (huge generated files must opt out)
Banned external dependencies (extdeps.rs)
Several other consistency checks

Run it before pushing — it's the most common reason CI fails on otherwise-fine PRs.

Snapshot vs. semantic tests

Many compiler tests are snapshots: they capture compiler output verbatim and check it byte-for-byte. This is fast and detects regressions, but means diagnostic improvements need --bless. Use UI snapshots whenever feasible; reach for semantic assertions only when the property under test isn't capturable in .stderr.

Tracking flaky tests

If a test is intermittently flaky, file an issue with CI-spurious-test-failure and tag the relevant team. Fixing flakiness is treated as a P-high task because spurious failures break the merge queue.