S3 Select

MinIO implements the S3 Select API: a client can issue a SQL SELECT against a CSV/JSON/Parquet object and receive only the matching rows. This is mostly a feature of internal/s3select/, with thin glue in the request handler.

Pieces

Step	File
HTTP entry	cmd/object-handlers.go (SelectObjectContent)
SQL parser	internal/s3select/sql/
Format readers	internal/s3select/{csv,json,parquet,simdj,jstream}/
Per-record evaluator	internal/s3select/select.go
Output framer	internal/s3select/select.go
Encryption transparency	cmd/encryption-v1.go (decrypts before passing payload)

Lifecycle of a request

sequenceDiagram
    participant C as Client
    participant H as object-handlers
    participant S as internal/s3select
    participant XL as ObjectLayer

    C->>H: POST /bucket/key?select XML
    H->>XL: GetObject (decrypt if SSE)
    H->>S: New(request)
    loop while reader has rows
        S->>XL: read next chunk
        S->>S: parse + filter
        S->>C: emit row(s)
    end
    S->>C: end-of-stream marker

Supported features

• SELECT cols FROM s3object [WHERE ...] against:
  • CSV (with optional header).
  • JSON Lines and JSON documents.
  • Parquet (column projection pushed down).
• Output as CSV or JSON.
• Progress events at a configurable cadence.
• A small set of SQL functions (string ops, casts).

Not supported

• Joins.
• Aggregations beyond COUNT(*).
• Output as Apache Parquet.
• User-defined functions.

Why this matters

Selective reads can dramatically reduce egress and parsing cost on the client when querying large CSV/JSON/Parquet objects. A 10 GB log file with 1% interesting rows returns ~100 MB to the client.

Where to start reading

• internal/s3select/select.go — orchestrates the per-record loop.
• internal/s3select/sql/parser.go — defines the grammar (uses alecthomas/participle).
• internal/s3select/parquet/parquet.go — Parquet path.
• cmd/object-handlers.go — search for SelectObjectContent.

See s3select for the package-level details.