Open-Source Wikis

/

Apache Arrow

/

How to contribute

/

Patterns and conventions

apache/arrow

Patterns and conventions

Coding conventions you'll encounter throughout the Arrow C++ codebase. Each language wrapper has its own dialect on top of these.

Error handling: Status and Result

arrow::Status (cpp/src/arrow/status.h) is a value type with a StatusCode and an optional error message and detail. It is the universal error type. Functions that can fail return Status directly:

Status WriteRecord(const RecordBatch& batch);

Functions that produce a value on success use arrow::Result<T> (cpp/src/arrow/result.h):

Result<std::shared_ptr<RecordBatch>> ReadNext();

The macros in cpp/src/arrow/result.h and status.h are used everywhere:

  • ARROW_RETURN_NOT_OK(expr) — return early if expr (a Status or Result) is not OK.
  • ARROW_ASSIGN_OR_RAISE(lhs, expr) — assign the contained value of a Result<T> into lhs, or return early.
  • ARROW_RETURN_IF(cond, status) — return status if the condition is true.

Custom StatusDetail types are used for richer error info (e.g., arrow::flight::FlightStatusDetail in cpp/src/arrow/flight/types.h).

Memory ownership: Buffer and shared_ptr

  • Buffers (Buffer, MutableBuffer, ResizableBuffer) are reference-counted. Pass them as std::shared_ptr<Buffer>.
  • Arrays own a std::shared_ptr<ArrayData> which itself owns shared buffers. Slicing an array is O(1) — it produces a new Array pointing into the same ArrayData with a different offset and length.
  • Builders own their work-in-progress buffers and emit std::shared_ptr<Array> on Finish.
  • arrow::MemoryPool is the allocator. Get the default pool via arrow::default_memory_pool() (cpp/src/arrow/memory_pool.h).

Async: Future and AsyncGenerator

Arrow has its own future-based async machinery (it predates std::future reaching parity for the project's needs):

  • arrow::Future<T> (cpp/src/arrow/util/future.h) — completion-driven future with continuations (Then, AddCallback).
  • arrow::AsyncGenerator<T> (cpp/src/arrow/util/async_generator.h) — a std::function<Future<T>()> that yields T on each call until it yields the iteration end token.
  • arrow::internal::ThreadPool and arrow::internal::GetCpuThreadPool() (cpp/src/arrow/util/thread_pool.h) — the default pool that backs Acero, dataset scanning, Flight, and IO.

Acero's ExecNodes emit work as futures and chain them with AsyncGenerator operators. Reading a dataset returns an AsyncGenerator<RecordBatch> that the scanner schedules across the IO and CPU pools.

Type traits and visitors

The C++ Arrow type system is a closed set of types. Two patterns are used everywhere to dispatch on types:

  • Type traits. arrow::TypeTraits<T> and arrow::CTypeTraits<T> (cpp/src/arrow/type_traits.h) map between C++ types and Arrow types.
  • Visitors. VisitTypeInline, VisitArrayInline, VisitDataInline, VisitScalarInline (in cpp/src/arrow/visit_*.h) accept a callable and dispatch to the right overload at compile time.

This pattern is preferred over runtime dynamic_cast chains for performance.

Schema and metadata

  • arrow::Schema is immutable. WithMetadata, WithField return new schemas.
  • arrow::KeyValueMetadata (cpp/src/arrow/util/key_value_metadata.h) is the canonical metadata container; it preserves insertion order.

Naming conventions

  • C++ files: lower_snake_case.{h,cc}.
  • Tests: *_test.cc, benchmarks: *_benchmark.cc, internal helpers: *_internal.h.
  • Public API headers: live alongside source. cpp/src/arrow/api.h is the umbrella include.
  • Namespaces: arrow::, arrow::compute::, arrow::dataset::, arrow::flight::, arrow::ipc::, arrow::io::, arrow::fs::, parquet::, gandiva::, arrow::engine::, arrow::acero::.
  • C++ classes: PascalCase. Functions and methods: PascalCase (consistent with Google C++ style for public API; a few internal-only helpers use lower_snake_case).

Visibility macros

Each shared library defines a visibility macro (ARROW_EXPORT, PARQUET_EXPORT, GANDIVA_EXPORT, ARROW_FLIGHT_EXPORT, etc.). Public symbols carry the macro; internal symbols don't. Symbol maps under cpp/src/arrow/symbols.map, cpp/src/parquet/symbols.map, etc., further restrict the exported ABI.

Internal vs. public headers

A header named *_internal.h is meant for use within the same library only. They are installed but not part of the stable API. Most are aggressively guarded with #ifndef ARROW_VERSION_* checks.

Vendored dependencies

cpp/src/arrow/vendored/ holds copies of small third-party libraries Arrow uses internally. Each subdirectory has its own README and license. Vendoring is preferred over taking a hard build dependency for tiny, infrequently changing libraries.

Benchmarks live next to tests

Every test file has an optional benchmark sibling (buffer_test.cc and builder_benchmark.cc, bitmap_test.cc and bit_util_benchmark.cc). The benchmarks use Google Benchmark and are included via add_arrow_benchmark() in CMake.

Language wrapper conventions

  • PyArrow uses lower_snake_case for Python functions and CamelCase for classes. Cython files mirror C++ method names.
  • R arrow uses R6 classes with Pascal-case names for the classes and snake_case for functions. Method names follow tidyverse conventions where they exist (read_parquet, write_csv_arrow).
  • Ruby (Red Arrow) follows Ruby conventions — Arrow::Array, Arrow::RecordBatch, snake_case methods.
  • C-GLib follows the GLib naming conventions (garrow_array_*, gparquet_arrow_file_reader_*).

Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.

Patterns and conventions – Apache Arrow wiki | Factory