Debugging

Tips for hunting down failures in the Arrow codebase.

C++ core

GDB pretty-printers

The repo ships cpp/gdb_arrow.py — a 70 KB Python module that teaches GDB how to render Arrow types. Load it once per debugging session:

(gdb) source /path/to/arrow/cpp/gdb_arrow.py
(gdb) p *array
$1 = arrow::Int32Array of length 5, null count 0
[ 1, 2, 3, 4, 5 ]

The CMake install target copies this file to share/doc/arrow so packagers can ship it. cpp/src/arrow/libarrow_gdb.py.in is the template that auto-loads the printer for a specific libarrow.so.

Sanitizers

Build with the appropriate sanitizer flag:

cmake -DARROW_USE_ASAN=ON ...    # AddressSanitizer
cmake -DARROW_USE_TSAN=ON ...    # ThreadSanitizer
cmake -DARROW_USE_UBSAN=ON ...   # UndefinedBehaviorSanitizer

cpp/valgrind.supp provides Arrow-specific suppressions for valgrind users.

Logging

ARROW_LOG(level) << "message" in cpp/src/arrow/util/logging.h is the standard. Levels are DEBUG, INFO, WARNING, ERROR, FATAL. The runtime threshold is set by ARROW_LOG_LEVEL environment variable.

For structured logging, cpp/src/arrow/util/logger.h defines an arrow::util::Logger interface that accepts arrow::util::LogDetails. Most subsystems use the simpler ARROW_LOG macro, but Acero and Flight optionally route through structured loggers.

Tracing

Arrow integrates with OpenTelemetry through cpp/src/arrow/util/tracing.h and tracing_internal.h. Key spans are emitted by Acero (exec_plan.cc), Flight (server_tracing_middleware.cc, client_tracing_middleware.cc), and the dataset scanner. Set OTEL_EXPORTER_OTLP_ENDPOINT to capture spans.

Common failure modes

• Status::Invalid: ... from RETURN_NOT_OK. Look for ARROW_RETURN_NOT_OK(...) in the call chain.
• CRC mismatch from IPC reader. Usually short-buffer reads — see the recent fix GH-49896: [C++] Reject short buffer reads in IPC reader.
• Async deadlock. Use ARROW_FUTURE_DEBUG=1 to dump pending futures. cpp/src/arrow/util/future.cc has detailed diagnostics.
• Thread pool exhaustion. Check arrow::internal::GetCpuThreadPool() saturation; the default is std::thread::hardware_concurrency().

Python

Mixing Python and C++ debugging

Build PyArrow in debug mode (-DCMAKE_BUILD_TYPE=Debug), then run Python under GDB:

gdb --args python -m pytest pyarrow/tests/test_compute.py -x
(gdb) catch throw
(gdb) run

The cpp/gdb_arrow.py printers work just as well from a Python process.

pyarrow.cffi and _C extensions

When something goes wrong at the boundary, check:

• python/pyarrow/lib.pyx — main .pyx file
• python/pyarrow/error.pxi — exception translation
• python/pyarrow/conftest.py — test fixtures

Memory leaks

PyArrow tests can run under tracemalloc with the memory_leak marker:

pytest -m memory_leak pyarrow

python/pyarrow/_pyarrow_cpp_tests.pyx exposes the C++ test infrastructure to Python so leak-checking can be done on the same process.

R

Debugging Rcpp/cpp11 calls

The R package's C++ glue lives in r/src/. Most C++ work happens in r/src/arrowExports.cpp, which is generated by cpp11::cpp_register(). To debug from R:

options(arrow.debug = TRUE)
options(arrow.debug.altrep = TRUE)

Then launch R under gdb (R -d gdb) and trigger the failing code path.

Common failure modes

• Error: Invalid Argument: .... arrow::Status errors are wrapped via arrow::r::status_to_error() in r/src/safe-call-into-r.h. The original C++ status message is preserved.
• Segfaults under multi-threaded R. Check whether the offending compute call is calling back into R (it shouldn't). The safe-call-into-r-impl.cpp infrastructure is designed to keep callbacks on the main thread.

Ruby and C-GLib

The C-GLib bindings are debugged with gdb:

G_DEBUG=fatal-warnings gdb --args ruby -Ilib test/test-array.rb

When a GObject-level error occurs the GLib g_warning becomes fatal, which often surfaces the underlying issue. c_glib/doc/ holds reference docs for the bindings.

Cross-implementation issues

For bugs that involve cross-language compatibility (e.g. C++ writes a file, Java can't read it):

1. Reproduce in cpp/src/arrow/integration/ (integration_command.cc).
2. Examine the produced bytes with arrow::ipc::JsonReader/JsonWriter (defined in the integration tests) or with archery integration --debug.
3. Compare against the JSON form, which is the canonical reference for the integration tests.

The dev/archery/archery/integration/ package has the runner scripts.