Testing

Prometheus has a deep test suite — about 184,000 lines of test code across 265 _test.go files at the time of writing. This page maps the test infrastructure so you know where to add new tests and how to run the right subset.

Test categories

Type	Where	How to run
Standard unit tests	*_test.go next to source	go test ./...
TSDB integration tests	tsdb/db_test.go, tsdb/head_test.go, …	go test ./tsdb/...
TSDB without isolation	tsdb/	go test ./tsdb/ -test.tsdb-isolation=false
Build-tag variants	dedupelabels, slicelabels, forcedirectio	go test --tags=dedupelabels ./...
End-to-end server upgrade	cmd/prometheus/main_upgrade_test.go	go test ./cmd/prometheus/ --test.version-upgrade=true -run TestVersionUpgrade
PromQL test files (.test)	promql/promqltest/testdata/*.test	Driven by promql/promql_test.go -> promqltest.RunTest
Benchmarks	Benchmark* functions, *_bench_test.go	go test -count=6 -benchmem -bench .
Fuzz	util/fuzzing/	OSS-Fuzz (see .github/workflows/fuzzing.yml)
UI tests	web/ui/mantine-ui/**/*.test.{ts,tsx} (Vitest/Jest)	make ui-test (or cd web/ui && CI=true npm test)
Lint	.golangci.yml	make lint

The PromQL test format

PromQL has its own DSL for end-to-end tests in promql/promqltest/. Each .test file in promql/promqltest/testdata/ declares a load step, a sequence of eval blocks, and expected results. Example:

load 5m
  http_requests{job="api-server", instance="0"} 0+10x10

eval instant at 50m sum(rate(http_requests[5m]))
  {} 2

eval range from 0 to 50m step 5m sum(rate(http_requests[5m]))
  {} 0 2 2 2 2 2 2 2 2 2 2

Runner: promqltest.RunTest(t, content, engine) from promql/promqltest/. Files are picked up by promql/promql_test.go::TestPromqlAcceptance and similar tests.

TSDB testing

tsdb/db_test.go (about 9,700 lines) exercises end-to-end scenarios over a real on-disk database backed by tsdbutil. tsdb/head_test.go (~8,300 lines) covers in-memory series management. The tests use:

• tsdb/testutil.go — test helpers (OpenTestDB, time-controlled clocks).
• tsdb/testdata/ — fixture files (e.g. 20kseries.json used by bench_tsdb).
• synctest — Go 1.25 synthetic time package, used in newer flake-resistant tests (e.g. TestDelayedCompaction).

Race-detector runs are gated behind specific tags because the WAL packages are race-sensitive only under certain options (forcedirectio, slicelabels).

Scrape testing

scrape/scrape_test.go is over 6,900 lines and uses an in-process HTTP test server plus teststorage (util/teststorage/) as the appendable. New scrape behaviour should add a focused test case here, exercising the scrape loop with mock targets and counting samples that reach the storage.

Web/API testing

web/api/v1/api_test.go contains scenario-driven tests built on api_scenarios_test.go. The API has hundreds of tests covering every route. Start by adding a scenario in the same style — a struct describing the request, expected status code, and JSON body.

web/web_test.go exercises the routing layer, including the agent-mode path-allowlist behaviour.

OpenAPI golden testing

web/api/v1/openapi_golden_test.go regenerates the OpenAPI spec and diffs it against web/api/v1/testdata/openapi.yaml. If you change an API endpoint, run the test with -update (where supported) or update the golden file manually.

UI testing

UI tests use Vitest (Mantine UI) and Jest (legacy react-app):

make ui-test               # runs both
cd web/ui && npm test      # Mantine only, watch mode

Lint:

make ui-lint
cd web/ui/react-app && npm run lint  # legacy app, kept separate due to dep conflicts

Mixins tests

Prometheus ships a Jsonnet "mixin" of alerts and dashboards under documentation/prometheus-mixin/. CI runs mixtool lint and jsonnetfmt -- check on every PR.

Benchmarks

AGENTS.md documents the standard performance-PR workflow:

go test -count=6 -benchmem -bench <package>

Save before/after output and run benchstat to produce the comparison table that goes into the PR body. For the TSDB, the dedicated bench_tsdb make target runs promtool tsdb bench write against the 20k-series fixture and produces SVG profile graphs:

make bench_tsdb TSDB_BENCHMARK_NUM_METRICS=1000

Fuzzing

util/fuzzing/ defines fuzz targets that run on OSS-Fuzz. Local invocation:

go test ./util/fuzzing/... -fuzz=Fuzz<Name> -fuzztime=30s

The .github/workflows/fuzzing.yml workflow periodically pushes the corpus to OSS-Fuzz; the public OSS-Fuzz dashboard is linked from the README badge.

Compliance / RW sender tests

compliance/ is a separate Go module that runs sender-side compliance tests against the Remote Write 1.0 spec. It is invoked via make -C compliance test.

When tests fail in CI

• Flaky synctest tests: check whether the offending test still uses time.Sleep. The push to migrate to synctest is ongoing.
• make protoc/proto regeneration mismatch: run the local equivalent and commit the result.
• Go version mismatch: CI pins quay.io/prometheus/golang-builder:1.26-base; your local Go must match the version in go.mod. scripts/check-go-mod-version.sh checks this.
• make ui-lint failures: ESLint config is at web/ui/mantine-ui/eslint.config.mjs and web/ui/react-app/.eslintrc.json.