Ruby (Red Arrow)

Active contributors: Sutou Kouhei, Raúl Cumplido, Sten Larsson

The ruby/ directory hosts Red Arrow — a set of Ruby gems that wrap the C-GLib bindings. The naming follows the convention used in the red-data-tools ecosystem.

Purpose

Provide an idiomatic Ruby API for Apache Arrow. Red Arrow is used by red-arrow-numo-narray (NumPy-equivalent integration), Charty (visualization), and a number of other Ruby data science gems.

Layout

ruby/
├── Gemfile, Rakefile, README.md
├── red-arrow/               # The base gem (libarrow + arrow-glib)
├── red-arrow-cuda/          # CUDA support
├── red-arrow-dataset/       # Dataset support
├── red-arrow-flight/        # Flight RPC client/server
├── red-arrow-flight-sql/    # Flight SQL
├── red-arrow-format/        # Vendored format definitions
├── red-gandiva/             # Gandiva expression compiler
└── red-parquet/             # Parquet reader/writer

Each gem has its own *.gemspec, Rakefile, lib/, test/, and Gemfile. The pattern is the same:

• lib/arrow.rb — top-level entry point that loads the GLib bindings via gobject-introspection.
• lib/arrow/*.rb — Ruby files that add idiomatic methods on top of the GObject types (Arrow::Table#to_h, Arrow::Array#each, Arrow::Schema#each_field, etc.).
• test/test-*.rb — test-unit tests.

Pattern

When a user does require "arrow", the gem:

1. Loads the gobject-introspection Ruby gem.
2. Locates the Arrow-1.0.typelib file installed alongside libarrow-glib.so.
3. Generates Ruby classes (Arrow::Array, Arrow::Table, Arrow::Schema) that proxy method calls to the C functions described by the typelib.
4. Mixes in the Arrow::*Loader modules that add Ruby-flavored methods (each, to_a, to_h, operators).

This means the Ruby surface is generated dynamically from the GLib introspection data. Adding a new method to arrow-glib makes it instantly available from Ruby; adding Ruby-flavored sugar means editing lib/arrow/*.rb.

red-arrow

ruby/red-arrow/lib/arrow/ has dedicated Ruby files for major concepts:

• array.rb, array-builder.rb, chunked-array.rb
• record-batch.rb, record-batch-reader.rb, record-batch-writer.rb, record-batch-stream-reader.rb
• table.rb, table-loader.rb, table-saver.rb, table-formatter.rb, table-list-formatter.rb
• schema.rb, field.rb, data-type.rb
• csv-loader.rb, csv-read-options.rb, json-loader.rb
• file-system.rb, file-info.rb, s3-file-system.rb, gcs-file-system.rb
• dictionary-array.rb, decimal128.rb, decimal256.rb
• expression.rb, function.rb
• slicer.rb — Pandas-like array slicing.

The Arrow::Table class doubles as a DataFrame-like API:

table = Arrow::Table.new(name: ["alice", "bob"], age: [30, 25])
table.slice([0])
table.group_by(:name).count
table.save("data.parquet")

red-parquet, red-gandiva, ...

The pattern repeats: each gem extends lib/parquet/*.rb (or lib/gandiva/*.rb, etc.) on top of GObject-Introspection-loaded classes from parquet-glib/gandiva-glib. Parquet::ArrowFileReader.new("data.parquet").read_table is the typical entry point.

red-arrow-flight and red-arrow-flight-sql

The Flight gems wrap arrow-flight-glib, exposing Arrow::Flight::Client, Arrow::Flight::Server, and the SQL-specific subclasses. They include enough scaffolding that users can subclass Arrow::Flight::Server and implement the Flight verbs in Ruby — the Ruby methods are dispatched into by the C++ server via the GLib callbacks.

Testing

Each gem ships its own test suite. Run all of them from the Ruby root:

cd ruby
bundle install
bundle exec rake test

CI workflows are in .github/workflows/ruby.yml. Tests run on Ubuntu, Fedora, AlmaLinux, Debian, and macOS in the matrix.

Releases

RubyGems releases happen during the Apache Arrow release cycle. dev/release/post-13-ruby.sh is the publication script. Each gem is published independently.