Getting started

This page explains how to fetch the source, build the C++ core, and run the language wrappers from a development checkout. The canonical instructions live in the official documentation at https://arrow.apache.org/docs/dev/developers/index.html — this page summarizes the parts most relevant to navigating this repo.

Prerequisites

Component	Requires
C++ core (cpp/)	C++17 toolchain (gcc 9+, clang 10+, MSVC 2019+), CMake ≥ 3.16, Boost (optional), Thrift (for Parquet), gRPC + Protobuf (for Flight), OpenSSL, zlib
Python (python/)	Python ≥ 3.9, Cython, NumPy, scikit-build
R (r/)	R ≥ 4.0, cpp11, a C++17 toolchain
Ruby (ruby/)	Ruby ≥ 3.1, GLib + GObject Introspection, the C-GLib build
MATLAB (matlab/)	MATLAB R2020a+, CMake, the C++ library
C-GLib (c_glib/)	GLib ≥ 2.58, GObject Introspection, vala (optional), the C++ library

The .env file in the repository root captures the recommended versions of language toolchains, dependencies, and base images used by the CI Docker workflows. It is used by compose.yaml to coordinate cross-language testing.

Cloning

git clone https://github.com/apache/arrow.git
cd arrow
git submodule update --init --recursive

Submodules are listed in .gitmodules and include test data fixtures.

Building the C++ library

The canonical build is CMake. A minimal Linux/macOS build:

mkdir -p cpp/build
cd cpp/build
cmake .. \
  -DCMAKE_BUILD_TYPE=Release \
  -DARROW_PARQUET=ON \
  -DARROW_DATASET=ON \
  -DARROW_FLIGHT=ON \
  -DARROW_COMPUTE=ON \
  -DARROW_FILESYSTEM=ON
cmake --build . --parallel
ctest --output-on-failure

Important toggles (cpp/CMakeLists.txt):

Option	Default	Meaning
ARROW_BUILD_SHARED / ARROW_BUILD_STATIC	ON / OFF	Library types
ARROW_PARQUET	OFF	Build Parquet C++
ARROW_DATASET	OFF	Build the dataset framework
ARROW_FLIGHT	OFF	Build Flight RPC
ARROW_FLIGHT_SQL	OFF	Build Flight SQL
ARROW_GANDIVA	OFF	Build Gandiva (requires LLVM)
ARROW_S3 / ARROW_GCS / ARROW_AZURE / ARROW_HDFS	OFF	Cloud filesystems
ARROW_PYTHON	OFF	Build C++ helpers PyArrow needs
ARROW_BUILD_TESTS / ARROW_BUILD_BENCHMARKS	OFF / OFF	Tests, benchmarks

CMake presets are defined in cpp/CMakePresets.json (ninja-debug-minimal, ninja-release-flight, ninja-debug-python, etc.) — a convenient way to pick a build flavor:

cmake --preset ninja-release-python
cmake --build --preset ninja-release-python

A Meson build also exists (cpp/meson.build) with the same component toggles in cpp/meson.options.

Vendored vs system dependencies

Arrow can fetch dependencies via CMake's ExternalProject or use system-installed ones. The *_SOURCE variables (Thrift_SOURCE, Protobuf_SOURCE, gRPC_SOURCE, ...) accept BUNDLED, SYSTEM, or AUTO. The vendored code lives in cpp/src/arrow/vendored/ and cpp/thirdparty/. A vcpkg manifest (cpp/vcpkg.json) and Brewfile (cpp/Brewfile) cover macOS/Windows.

Building PyArrow

PyArrow is a Cython extension on top of libarrow. The standard development flow:

# 1. Build C++ with PyArrow's required components ON.
cmake --preset ninja-debug-python -S cpp -B cpp/build
cmake --build cpp/build --target install

# 2. Build PyArrow itself.
export ARROW_HOME="$PWD/cpp/build/install"
export LD_LIBRARY_PATH="$ARROW_HOME/lib:$LD_LIBRARY_PATH"
cd python
python -m pip install -r requirements-build.txt
python -m pip install -e . --no-build-isolation

# 3. Test.
python -m pytest pyarrow

python/pyproject.toml is the build entry point; python/CMakeLists.txt invokes the C++ part of the build. The _build_backend/ directory contains a custom backend that respects ARROW_HOME. See python/scripts/ for helper scripts used by CI.

Building the R package

cd r
# Source build of libarrow if needed (uses tools/nixlibs.R)
R -e 'devtools::install_deps()'
R CMD INSTALL .

Use R -e 'devtools::test()' to run the test suite. The R package can either bundle its own libarrow build or link against a pre-built one (ARROW_USE_PKG_CONFIG, ARROW_R_DEV). See r/PACKAGING.md and r/configure.

Building C-GLib and Red Arrow

C-GLib uses Meson:

# Build and install libarrow first, then:
cd c_glib
meson setup build -Dprefix="$ARROW_HOME"
meson compile -C build
meson install -C build

Red Arrow then installs as Ruby gems against the GLib introspection data:

cd ruby/red-arrow
bundle install
bundle exec rake test

Building the MATLAB bindings

cd matlab
cmake -S . -B build
cmake --build build

Run the test suite from MATLAB with runArrowMatlabTests. See matlab/README.md.

Running cross-implementation integration tests

The integration test runner lives in dev/archery/. Archery is a Python CLI that orchestrates Arrow developer tooling (release, integration tests, benchmarking, lint). Install it with:

pip install -e dev/archery[all]
archery integration --help

Integration tests verify that data produced by one implementation can be read by every other implementation. The C++ harness lives in cpp/src/arrow/integration/.

Docker-based builds

compose.yaml defines services for every supported build matrix (e.g. conda-cpp, conda-python, ubuntu-r, ubuntu-ruby, ubuntu-c-glib). To reproduce a CI build locally:

docker compose run --rm conda-cpp

This is the recommended path when reproducing CI failures or building on an unsupported host.

Useful entry points for new contributors

• CONTRIBUTING.md — high-level pointers to the official contribution guide.
• dev/release/ — release tooling. dev/release/01-prepare.sh, 02-source.sh, etc. show the release dance.
• dev/archery/archery/ — the developer CLI.
• ci/scripts/ — the scripts CI calls. They are the source of truth for "what does the test matrix actually run?"
• docs/source/developers/ — the official developer documentation that the website builds from.