Aggregations

What they are

Aggregations are a tree-shaped analytics computation that runs alongside (or instead of) a query. The two main flavors:

• Bucket aggregations group documents into buckets (terms, date_histogram, range, geo_grid, composite, filter(s), nested, reverse_nested, geo_distance, etc.).
• Metric aggregations compute a value over the matching docs (avg, sum, min, max, cardinality, percentiles, extended_stats, top_hits, geo_centroid, etc.).

Bucket aggregations can have sub-aggregations, producing nested results.

A third class, pipeline aggregations, runs after the bucket tree exists (derivative, moving_avg, moving_function, cumulative_sum, bucket_script, bucket_selector, bucket_sort).

Source layout

server/src/main/java/org/elasticsearch/search/aggregations/
├── Aggregator.java                      Per-shard collector
├── AggregationBuilder.java              JSON parser + factory
├── InternalAggregation.java             Per-shard result; reduced on coordinator
├── bucket/                              Bucket impls (terms, date_histogram, range, ...)
├── metrics/                             Metric impls
├── pipeline/                            Pipeline aggs
├── support/                             ValuesSource, FieldContext, MissingValueResolver
└── AggregationPhase.java                Hooks the tree into the search pipeline

modules/aggregations/                   Sandboxed advanced aggs (e.g. geohex_grid)
x-pack/plugin/analytics/                Commercial aggs (top_metrics, weighted_avg, geoline, t_test, ...)

How a tree runs

graph TD
  Q[QueryPhase per shard] -->|Collector| AGG[AggregationPhase]
  AGG --> ROOT[Root Aggregator]
  ROOT --> B1[Bucket aggregator: date_histogram]
  B1 --> M1[Metric: avg]
  B1 --> M2[Metric: percentiles]
  AGG -->|InternalAggregations| RED[Reduce on coordinator]
  RED --> RES[SearchResponse]

Per shard, the aggregation tree subscribes to the same Collector chain that scores documents. As Lucene visits each matching doc, every aggregator gets a chance to update its bucket(s). At the end of the query phase, each aggregator produces an InternalAggregation snapshot.

The coordinator collects InternalAggregations from all shards and reduces them recursively. The reduce pass merges term buckets, sums counts, combines histograms, etc.

For very large shard counts, the coordinator runs a partial reduce on data nodes to keep coordinator memory bounded. For composite aggregations and search-after-style pagination over buckets, the partial-reduce step also handles ordering.

ValuesSource

Aggregators don't read MappedFieldType directly. Instead, they receive a ValuesSource (e.g. ValuesSource.Numeric, Bytes, GeoPoint) that abstracts whether the underlying data lives in doc values, fielddata, or runtime fields. This is what lets the same terms aggregation work over a keyword field, an ip field, a runtime field, or a numeric field with formatting.

Tier-aware execution and composite

Many aggregations have multiple internal modes:

• terms has breadth_first (default) and depth_first (better for very deep sub-tree).
• date_histogram rounds based on calendar interval, fixed interval, or fields with time_zone.
• composite paginates buckets via after keys for stable iteration.

Numeric stability and approximate results

• cardinality uses HyperLogLog++ (configurable precision).
• percentiles defaults to t-digest (libs/tdigest); can use HDR via ?method=hdr.
• geo_grid has geohash_grid, geotile_grid, and geohex_grid (X-Pack analytics).

Pipeline aggregations

Pipeline aggregations run on the coordinator after the bucket tree is built, walking the tree and emitting derived buckets. Shape: derivative, moving_function, bucket_script, cumulative_sum. They are the "post-processing" half of the system.

ESQL parallel

ES|QL has its own aggregation engine (x-pack/plugin/esql/.../planner/) that does columnar block-based execution rather than the document-by-document model here. The classic aggregation framework remains the only path for _search users.

Where to extend

• New aggregation: implement Aggregator, AggregatorFactory, InternalAggregation, and a parser. Register via SearchPlugin#getAggregations (or, for bucket aggs that need it, getCompositeValueSourceParsers).
• Custom values source: ValuesSourceRegistry accepts new ValuesSourceType registrations.
• New pipeline aggregation: PipelineAggregationBuilder + PipelineAggregator.