Glossary

Project-specific vocabulary used across the Arrow codebase. Where a term has a standard meaning outside Arrow, the entry highlights how Arrow uses it.

Core data model

• Array. An immutable, columnar sequence of values of a single Arrow type. The base class is arrow::Array in cpp/src/arrow/array/array_base.h. Every concrete subclass (Int32Array, StringArray, ListArray, StructArray, ...) has a corresponding builder.
• ArrayData. The "physical" representation of an array: a DataType, a length, a null count, child ArrayData for nested types, and a vector of Buffers. Defined in cpp/src/arrow/array/data.h.
• Buffer. A reference-counted, contiguous byte range. arrow::Buffer in cpp/src/arrow/buffer.h. Buffers are the unit of memory ownership; arrays hold buffers, not raw pointers.
• ChunkedArray. A logical column made of one or more Array chunks of the same type. cpp/src/arrow/chunked_array.h.
• RecordBatch. A schema plus a collection of equal-length arrays — the columnar equivalent of a row batch. cpp/src/arrow/record_batch.h.
• Table. A schema plus a collection of ChunkedArray columns. The Arrow analogue of a data frame. cpp/src/arrow/table.h.
• Schema. An ordered list of fields plus optional metadata. cpp/src/arrow/type.h.
• Field. A name plus a DataType plus a nullability flag plus optional metadata. cpp/src/arrow/type.h.
• DataType. A logical type: int32, utf8, list<int32>, struct<x: float64, y: float64>, decimal128(10, 2), timestamp[us, tz=UTC], etc. Defined as a class hierarchy in cpp/src/arrow/type.h.
• Scalar. A single value of an Arrow type, used by compute kernels and Acero expressions. cpp/src/arrow/scalar.h.
• Datum. A union over Array, ChunkedArray, RecordBatch, Table, and Scalar. cpp/src/arrow/datum.h. Used as the universal input/output type for compute kernels.
• MemoryPool. The allocator interface backing every Arrow allocation. The default pool wraps jemalloc on Linux and the system allocator elsewhere. cpp/src/arrow/memory_pool.h.

Format

• IPC. Interprocess communication. Arrow's binary serialization of record batches and dictionaries. Two flavors: the stream format (continuous messages) and the file format (random access with a footer). Defined by the FlatBuffers schemas in format/.
• C data interface. A C ABI for zero-copy in-process exchange of arrays between language runtimes. Defined in cpp/src/arrow/c/abi.h. Ships ArrowSchema, ArrowArray, ArrowArrayStream, and ArrowDeviceArray.
• Feather. The legacy on-disk file format. Feather V1 is a custom layout (cpp/src/arrow/ipc/feather.fbs); Feather V2 is the Arrow file format with a fixed extension.
• Flight. The high-throughput RPC framework. Built on gRPC + Arrow IPC. format/Flight.proto.
• Flight SQL. An extension of Flight that adds SQL semantics (statement execution, prepared statements, catalogs). format/FlightSql.proto.

Compute and execution

• Function. A logical compute operation — for example, add, cast, sum. arrow::compute::Function in cpp/src/arrow/compute/function.h.
• Kernel. A specific implementation of a function for a particular combination of input types. arrow::compute::Kernel in cpp/src/arrow/compute/kernel.h.
• Function registry. A global lookup table mapping function names to function objects. cpp/src/arrow/compute/registry.h. Default kernels are registered by cpp/src/arrow/compute/initialize.cc.
• Expression. A compute expression tree (a literal, a field reference, or a call to a function). arrow::compute::Expression in cpp/src/arrow/compute/expression.h.
• Acero. Arrow's streaming, push-based query execution engine. Lives in cpp/src/arrow/acero/. Made of ExecNode types connected into an ExecPlan.
• ExecPlan. A directed acyclic graph of ExecNode instances representing a query. cpp/src/arrow/acero/exec_plan.h.
• ExecNode. A unit of execution in Acero. Examples: SourceNode, FilterNode, ProjectNode, HashJoinNode, HashAggregateNode, OrderByNode, SinkNode.
• Substrait. A cross-engine query plan format. The cpp/src/arrow/engine/substrait/ adapter consumes Substrait plans and produces Acero ExecPlans.

Datasets

• Dataset. A logical collection of one or more files (or in-memory record batches) with a unified schema. arrow::dataset::Dataset in cpp/src/arrow/dataset/dataset.h.
• Fragment. A discrete unit inside a dataset — typically a single file or row group. arrow::dataset::Fragment in the same file.
• Scanner. Reads a dataset, applies projection and filter pushdown, and yields record batches. cpp/src/arrow/dataset/scanner.h.
• Partitioning. A scheme for encoding column values into directory paths or filenames. Hive-style and directory partitioning are both supported. cpp/src/arrow/dataset/partition.h.
• FileFormat. A pluggable adapter that reads and writes a specific file format inside a dataset (FileFormatParquet, FileFormatIpc, FileFormatCsv, FileFormatJson, FileFormatOrc).

Parquet

• Row group. A horizontal partition of a Parquet file containing column chunks. parquet::FileMetaData::row_groups.
• Column chunk. All values for a single column inside a single row group. parquet::ColumnChunkMetaData.
• Page. The unit of compression and encoding inside a column chunk. parquet::DataPage, parquet::DictionaryPage. Defined in cpp/src/parquet/column_page.h.
• Page index. Per-page min/max statistics and offsets that allow row-level skipping. cpp/src/parquet/page_index.h.
• Bloom filter. A probabilistic filter used for column-level skipping in Parquet. cpp/src/parquet/bloom_filter.h.
• Modular encryption. Parquet's column-level encryption. Lives in cpp/src/parquet/encryption/.

Gandiva

• Projector. A Gandiva object that compiles a list of expressions and projects them onto record batches. cpp/src/gandiva/projector.h.
• Filter. A Gandiva object that compiles a single boolean expression into a selection vector. cpp/src/gandiva/filter.h.
• DEX (decomposed expression). Gandiva's intermediate representation. cpp/src/gandiva/dex.h.
• Precompiled bitcode. LLVM bitcode for handwritten C++ helpers (decimal arithmetic, regex, hashing, casting). Compiled at build time and linked into the JIT. cpp/src/gandiva/precompiled/.

Flight / gRPC

• FlightClient / FlightServer. The client and server endpoints. cpp/src/arrow/flight/client.h, server.h.
• FlightDescriptor. Identifies a stream by command bytes or a path. cpp/src/arrow/flight/types.h.
• FlightInfo. Metadata about an available stream: schema, ticket, endpoints, total record count.
• Ticket. An opaque token a client uses to fetch a specific data stream.
• Endpoint. A (Ticket, locations) pair that tells a client where to fetch data.
• Action / DoAction. Custom server-defined operations beyond the standard Flight verbs.

Miscellaneous

• Archery. The Python CLI under dev/archery/ used for release management, integration testing, benchmarking, and linting.
• Crossbow. The CI orchestration tool used to fan out builds across many platforms. Configuration lives in dev/tasks/.
• Docker compose. compose.yaml defines the supported build/test matrix as Docker services. Used both locally and on CI.
• Conbench. The benchmarking infrastructure. dev/conbench_envs/ holds Conda environments used by Conbench runners.