Testing

Grype has four nested test layers, each with a distinct purpose.

Unit tests

Live next to the code (*_test.go). Run with:

task unit
# or directly
go test ./...

Coverage gate: 47% (enforced in Taskfile.yaml::unit via .github/scripts/coverage.py). The bar is intentionally modest because much of the engineering value is exercised by integration and CLI tests rather than fine-grained unit tests.

Patterns visible in the codebase:

• Heavy use of stretchr/testify (assert, require).
• Snapshot testing with gkampitakis/go-snaps for presenter outputs.
• Mock providers are in dedicated packages: grype/vulnerability/mock/, grype/matcher/mock/.
• Most stores have hand-written builder helpers (e.g. grype/db/v6/vulnerability_provider_mocks_test.go).

Integration tests

Live in test/integration/. They build real test images via stereoscope-fixture-* Docker images and exercise full scans:

task integration

The task also runs grype db update and grype -race alpine:latest outside go test to exercise the data-race detector against a live workload.

Test fixtures are managed by internal/dbtest/:

task fixture-status              # show fixture state
task regenerate-fixtures         # rebuild from a vunnel data dir
task regenerate-fixtures-dry-run # preview without writing

vunnel is the upstream tool that produces normalized vulnerability JSON; you point the regeneration task at a local ~/vunnel/data cache.

CLI tests

Live in test/cli/. They drive the snapshot binary as an external process:

task cli

The snapshot binary path is ./snapshot/<os>-build_<os>_<arch>/grype. The CLI suite verifies:

• Exit codes (0 success, 1 generic error, 2 --fail-on, 100 DB upgrade available — see cmd/grype/cli/cli.go::SetupConfig::WithMapExitCode).
• Output formats (JSON shape, table layout, SARIF schema validity).
• DB subcommands (db check, db status, db list, db search).

Quality gate

The most distinctive test layer. Lives in test/quality/ and is invoked with:

task quality

It uses yardstick and vulnerability-match-labels to compare Grype's output against a hand-labeled corpus. This is the regression net for matching changes — it catches both missing matches and spurious ones.

The pinned database for the quality gate is at test/quality/test-db; refresh it with:

task update-quality-gate-db

Install test

Runs the production install.sh against multiple OS environments:

task install-test
task install-test-cache-save
task install-test-cache-load
task install-test-ci-mac

This is a guard against shipping a broken installer.

Race detector

The task integration step explicitly invokes go run -race cmd/grype/main.go alpine:latest. The note in Taskfile.yaml calls out that DB updates are intentionally run outside the race detector because doing both simultaneously is "very slow."

Coverage and benchmarks

Coverage profile is dropped at .tmp/unit-coverage-details.txt. There is one benchmark: grype/load_vulnerability_db_bench_test.go.

Where to put new tests

Change	Test type
New helper function	Unit test alongside the code
New matcher behavior	Unit test in the matcher package + integration test in test/integration/
New CLI flag	CLI test in test/cli/ + unit test for the option struct in cmd/grype/cli/options/
New output format	Unit test with snapshot in grype/presenter/<format>/ + CLI test
Database schema change	Unit test against the affected store + drift checker (task check-db-schema-drift)