Open-Source Wikis

/

Apache Arrow

/

Implementations

/

R

apache/arrow

R

Active contributors: Nic Crane, Hyukjin Kwon, Bryce Mecum, Sutou Kouhei

The R arrow package wraps the C++ library and exposes R6 classes for every Arrow concept (Array, Schema, Table, RecordBatch, Dataset, ...). It also provides a dplyr backend that translates dplyr verbs into Arrow Acero ExecPlans, letting R users do larger-than-memory analytics with familiar syntax.

Purpose

Make Arrow first-class in R. The package is on CRAN and conda-forge and is the canonical bridge between R, Arrow, Parquet, and the cloud filesystems.

Layout

r/
├── DESCRIPTION              # Package metadata
├── NAMESPACE                # Auto-generated by roxygen
├── NEWS.md                  # Changelog
├── arrow.Rproj
├── R/                       # R source (~80 .R files)
│   ├── arrow-package.R      # Package init
│   ├── arrow-info.R         # Build feature flags and runtime info
│   ├── array.R, chunked-array.R, scalar.R, schema.R, type.R, field.R
│   ├── record-batch.R, table.R
│   ├── record-batch-reader.R, record-batch-writer.R
│   ├── ipc-stream.R         # IPC reader/writer wrappers
│   ├── feather.R            # Feather V1/V2
│   ├── parquet.R            # Parquet reader/writer
│   ├── csv.R, json.R        # CSV / JSON
│   ├── compression.R, io.R, buffer.R, memory-pool.R
│   ├── filesystem.R         # FileSystem abstractions
│   ├── flight.R             # Flight client
│   ├── dataset.R, dataset-factory.R, dataset-format.R, dataset-partition.R, dataset-scan.R, dataset-write.R
│   ├── compute.R, expression.R
│   ├── extension.R          # Extension type framework
│   ├── dictionary.R, metadata.R
│   ├── dplyr.R + 22 dplyr-*.R files  # dplyr backend
│   ├── duckdb.R             # DuckDB integration
│   ├── python.R             # PyArrow interop
│   ├── query-engine.R       # The Acero plan builder
│   ├── udf.R                # User-defined functions
│   ├── arrow-datum.R        # Datum class
│   └── arrowExports.R       # Auto-generated by cpp11::cpp_register()
├── src/                     # C++ glue (~50 .cpp files)
│   ├── arrowExports.cpp     # Auto-generated
│   ├── altrep.cpp           # ALTREP integration
│   ├── array.cpp, array_to_vector.cpp, r_to_arrow.cpp
│   ├── compute.cpp, compute-exec.cpp
│   ├── dataset.cpp, parquet.cpp, csv.cpp, json.cpp, feather.cpp
│   ├── filesystem.cpp, io.cpp
│   ├── safe-call-into-r.h, safe-call-into-r-impl.cpp
│   └── arrow_types.h, arrow_cpp11.h
├── inst/                    # Package data (NOTICE.txt, build configs)
├── man/                     # Roxygen-generated docs
├── tests/testthat/          # testthat suite
├── vignettes/               # Long-form docs
├── pkgdown/                 # Site config for the docs site
├── tools/                   # Build helpers (nixlibs.R, etc.)
├── data-raw/                # Raw data scripts
├── extra-tests/             # Tests that run only in CI
├── cheatsheet/              # arrow-cheatsheet.pdf source
├── configure / configure.win / Makevars / Makefile
├── PACKAGING.md, STYLE.md, README.md, NEWS.md
└── _pkgdown.yml             # Pkgdown site config

R6 class layer

Every Arrow C++ class has an R6 wrapper:

arr <- Array$create(c(1L, 2L, 3L))
arr$type
arr$length()
batch <- record_batch(x = 1:3, y = c("a", "b", "c"))
batch$schema
table <- arrow_table(x = 1:3, y = c("a", "b", "c"))

R6 classes wrap a C++ pointer (held in xptr_ slots) and inherit shared behavior from arrow-object.R. The bridge between the R6 class and the underlying C++ object lives in r/src/arrowExports.cpp, generated by cpp11::cpp_register() from annotations in the *.cpp files.

ALTREP integration

R supports "alternative representations" (ALTREP) for vectors — a way to back an R vector with a custom storage backend without copying data into base R memory. PyArrow does not have an analogue; R's ALTREP framework lets the arrow package expose Arrow arrays directly as R atomic vectors with no data conversion until the user actually pulls a value.

r/src/altrep.cpp (43 KB) implements ALTREP backings for integer, double, character, logical, and timestamp Arrow arrays. This is what makes as.data.frame(arrow_table) essentially free until columns are accessed.

dplyr backend

The dplyr backend is the R package's flagship feature. It lets users run dplyr code over Arrow datasets that don't fit in memory. Verbs translate into Acero ExecPlans:

  • filter()FilterNode
  • select() / rename()ProjectNode
  • mutate() / transmute()ProjectNode with new computed columns
  • arrange()OrderByNode
  • group_by() %>% summarise()HashAggregateNode
  • inner_join() / left_join() / etc. → HashJoinNode
  • union()UnionNode
  • slice_head() / slice_tail()FetchNode
  • distinct() → distinct via aggregate

The dispatch table lives in r/R/dplyr-funcs.R and per-category function bindings are in r/R/dplyr-funcs-{agg,augmented,conditional,datetime,doc,math,simple,string,type}.R. The plan builder is in r/R/query-engine.R. When collect() is called, the plan is executed by Acero and the result is materialized as an R data frame.

DuckDB integration

r/R/duckdb.R exposes to_duckdb() and to_arrow() so users can move tables between Arrow and DuckDB without copying. DuckDB exposes Arrow record batch streams natively, and the R package wires this up through the C data interface.

PyArrow interop

r/R/python.R and r/src/bridge.cpp enable Arrow R users to share data with PyArrow via reticulate. It's again zero-copy thanks to the C data interface.

Build

The R package can build in two modes:

  1. Bundled libarrow. The package downloads or builds its own copy of libarrow at install time. r/configure and r/tools/nixlibs.R orchestrate this. Used on CRAN and on systems without a system libarrow.
  2. Pre-built libarrow. If ARROW_HOME is set or a system libarrow.pc exists, the package links against it.

Macros in r/Makevars.in flip between modes based on the configure outcome. r/PACKAGING.md is the canonical reference for packaging.

Testing

r/tests/testthat/ mirrors the r/R/ structure with one test-*.R per topic. Long-running tests live in r/extra-tests/. CI runs the suite under multiple R versions and on Windows / macOS / Linux via .github/workflows/r.yml and r_extra.yml.

Distribution

CRAN releases are coordinated via dev/release/post-08-r.sh. Conda-forge has its own feedstock. R-universe nightly builds are configured via .github/workflows/r_nightly.yml. The maintenance branch tag r-universe-release (apr 2026) marks the latest snapshot used by R-universe.

Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.

R – Apache Arrow wiki | Factory