clickhouse/clickhouse

Testing

ClickHouse has many test layers. They are organized in tests/ and run from CI under ci/jobs/. New behaviour without a corresponding test rarely makes it through review.

Layers

Stateless SQL tests — `tests/queries/0_stateless/`

The largest suite. Each test is a .sql (or .sh) script paired with a .reference file containing the expected output. They are stateless: each run creates and drops its own tables.

Runner: tests/clickhouse-test (a 250 KB Python script).
The naming convention is NNNNN_short_description.sql. New tests should be created with tests/queries/0_stateless/add-test <name> (per AGENTS.md), which assigns the next number prefix and produces both the test and reference files.
Per AGENTS.md: avoid no-* tags (e.g. no-parallel) unless strictly necessary.

Quick local runs:

build/programs/clickhouse-test                       # whole suite
build/programs/clickhouse-test 03000_my_test         # filter by prefix

Stateful SQL tests — `tests/queries/1_stateful/`

These rely on preloaded datasets (the public ClickHouse test datasets). They run after the stateless suite in CI.

Integration tests — `tests/integration/`

There are 750+ integration test directories. Each contains a test.py plus a docker-compose file describing the topology — typically a multi-node ClickHouse cluster, sometimes with Kafka, MySQL, Postgres, MinIO/S3 mocks, etc. Each tests a real-world scenario: replication, distributed queries, backups to S3, Kafka ingestion, MaterializedPostgreSQL, lakehouse readers, etc.

Run them through Praktika:

python -m ci.praktika run "integration" --test <selectors>

Configuration lives in tests/integration/helpers/. For details, see tests/integration/README.md.

Unit tests — `gtest_*.cpp`

Distributed throughout src/**/tests/. Built into a single unit_tests_dbms binary by CMake. Useful for narrowly-scoped components (ActionsDAG, Aggregator, KeyCondition, the parsers, etc.). Run with ctest or by invoking the binary with --gtest_filter=....

Performance tests — `tests/performance/`

XML files describing query workloads. CI runs them on each PR and reports a TSV diff against master. The .claude/tools/fetch_perf_report.py helper (see Tooling) fetches and filters the results.

SQL logic — `tests/sqllogic/`

A port of the SQLite SQL logic test suite. Drives ClickHouse with thousands of generated queries.

Fuzzers

Three flavors:

libFuzzer harnesses scattered under src/**/fuzzers/ and programs/server/fuzzers/ — they target individual parsers, format readers, and codecs.
Functional fuzzer (src/Client/QueryFuzzer.cpp) — mutates queries from a corpus and runs them.
Casa del Dolor (tests/casa_del_dolor/) — a Python-driven chaos/property test that issues random DDL and DML against a running server. Has its own contributor docs.

Jepsen — `tests/jepsen.clickhouse/`

Clojure-based linearizability tests against clickhouse-keeper and ReplicatedMergeTree.

Stress and stateless-stress

tests/stress/ and the stress job in CI run the stateless suite under heavy concurrency, often with sanitizers, to surface race conditions.

Adding a test

Per AGENTS.md, prefer adding a new test rather than appending to an existing one. The numbering convention groups tests by topic, so new tests for the same area get adjacent numbers.

To add an SQL test:

tests/queries/0_stateless/add-test my_new_feature           # creates 03NNN_my_new_feature.sql + .reference
tests/queries/0_stateless/add-test my_new_feature.sh        # for shell-based tests

Run it locally and copy the output into the .reference file (the add-test script handles a lot of this), then commit both files.

Running tests under sanitizers

Build with -DSANITIZE=address (or thread, memory, undefined) and run the same clickhouse-test driver. CI runs every PR through every sanitizer; failing locally before pushing is a good idea for tricky changes.

When debugging integration tests, log files are produced under <build>/test_<name>.log. Per AGENTS.md, redirect each test's output to a unique log file so multiple tests can run in parallel; analyze the log with a subagent or grep rather than eyeballing thousands of lines.

Coverage

Build with -DCMAKE_BUILD_TYPE=RelWithDebInfo -DSANITIZE_COVERAGE=1 to get LLVM source-based coverage. The reports are post-processed in ci/jobs/build_check.py.