Open-Source Wikis

/

Apache Arrow

/

How to contribute

/

Testing

apache/arrow

Testing

Arrow has tests at every layer: unit tests inside each language implementation, cross-language integration tests, fuzz tests, and benchmarks that double as regression checks.

Test frameworks per language

Component Framework Convention Run
C++ core, Parquet, Gandiva, Acero, Dataset, Flight GoogleTest *_test.cc next to source ctest after cmake --build, or run ./build/release/arrow-array-test directly
C++ benchmarks Google Benchmark *_benchmark.cc ./build/release/arrow-buffer-benchmark, etc.
Python pytest python/pyarrow/tests/test_*.py python -m pytest pyarrow
R testthat r/tests/testthat/test-*.R R -e 'devtools::test()'
Ruby test-unit ruby/red-*/test/test-*.rb bundle exec rake test
C-GLib GLib testing c_glib/test/test-*.rb (run as Ruby!) cd c_glib && bundle exec rake test
MATLAB MATLAB unit test framework matlab/test/arrow/+arrow/... runArrowMatlabTests from MATLAB

C++ test conventions

Each *_test.cc lives next to the source it covers. The file is added to cmake_modules-driven targets via the add_arrow_test() macro in cpp/cmake_modules/BuildUtils.cmake. Common patterns:

  • Test fixtures. cpp/src/arrow/testing/ provides reusable helpers: arrow::random for random arrays, arrow::testing::AssertArraysEqual, arrow::TestEnvironment.
  • Death tests. Used sparingly to verify DCHECK and ARROW_LOG(FATAL) paths.
  • Parameterized tests. Heavily used in compute kernels — see cpp/src/arrow/compute/kernels/scalar_arithmetic_test.cc.
  • Type-parameterized tests. Used in array tests to share scaffolding across primitive types — see array_test.cc.

To run a single test binary with a filter:

./cpp/build/release/arrow-array-test --gtest_filter='Int32Array*'

Python test conventions

PyArrow tests live in python/pyarrow/tests/ and use pytest. python/pyarrow/conftest.py configures fixtures shared across the suite. Tests are tagged with markers defined in python/pyarrow/tests/conftest.py so they can be run selectively:

  • pytest -m "not slow" skips slow tests.
  • pytest -m "memory_leak" runs leak tests under tracemalloc.
  • pytest -m "s3" runs S3 tests against MinIO.

The pyarrow.tests.util module has helpers; the bigger fixtures (S3, GCS, Azure) launch local servers via Docker compose services declared in compose.yaml (minio, azurite, gcs-server).

R test conventions

R uses testthat. Tests live in r/tests/testthat/test-*.R. Each tests one feature area (test-Array.R, test-dplyr-*.R, test-Parquet.R). Helpers are in r/tests/testthat/helper-*.R. The dplyr backend has its own test taxonomy:

  • test-dplyr-mutate.R, test-dplyr-summarize.R, etc., test the verb translations.
  • test-dplyr-funcs-*.R test individual function bindings (date/time, math, string, type).

Cross-language integration tests

cpp/src/arrow/integration/ is the C++ runner for cross-language compatibility. The driver lives in dev/archery/archery/integration/. The pipeline:

  1. Each language implementation produces Arrow IPC files for a battery of schemas (arrow-test-style binary).
  2. Each implementation is asked to read every other implementation's output and compare.
  3. Flight integration tests (cpp/src/arrow/flight/integration_tests/) verify that a Flight client in language A can talk to a Flight server in language B.

CI runs the integration matrix in .github/workflows/integration.yml. Tested combinations include C++ ↔ Java ↔ Go ↔ JS ↔ Rust ↔ C# ↔ Nano-arrow.

Fuzz testing

The repo contains LibFuzzer harnesses under:

  • cpp/src/arrow/ipc/file_fuzz.cc, stream_fuzz.cc, tensor_stream_fuzz.cc — fuzz the IPC reader.
  • cpp/src/arrow/csv/fuzz.cc — fuzz the CSV parser.
  • cpp/src/arrow/json/ — JSON parser fuzzers (when enabled).
  • cpp/src/parquet/file_fuzz.cc (referenced by Parquet fuzz target) — fuzz the Parquet reader.

Corpora are generated by generate_fuzz_corpus.cc files alongside each fuzzer. Continuous fuzzing happens on Google's OSS-Fuzz infrastructure.

Benchmarks

Each subsystem has matched *_benchmark.cc files (buffer_test.ccbuilder_benchmark.cc, compute/exec_test.cccompute/function_benchmark.cc, ...). Benchmark binaries follow Google Benchmark conventions:

./build/release/arrow-bit-util-benchmark --benchmark_filter=BitmapAnd

Conbench stitches benchmark history together; dev/conbench_envs/ holds the conda environments.

Coverage

python/.coveragerc configures Python coverage. C++ coverage uses lcov with -DARROW_TEST_COVERAGE=ON. R coverage uses covr. There's no central coverage gate — instead, each subsystem maintains tests close to the code.

Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.

Testing – Apache Arrow wiki | Factory