Search

Purpose

The search system runs a distributed search across N shards in two phases (query then fetch) and assembles a single SearchResponse. It is also the host for aggregations, suggesters, highlighters, rescoring, and cross-cluster search.

Directory layout

server/src/main/java/org/elasticsearch/search/
├── SearchService.java                Per-node entry point; owns active SearchContexts
├── SearchModule.java                 Registers query parsers, agg builders, suggesters
├── DefaultSearchContext.java         Per-shard search context
├── builder/SearchSourceBuilder.java  The JSON `_search` body
├── query/                            Query phase (collector + scorer)
├── fetch/                            Fetch phase (source, highlights, fields)
├── aggregations/                     Bucket + metric aggregations
├── suggest/                          Suggesters
├── rescore/                          Rescore phase
├── dfs/                              Distributed term frequency phase (for global scoring)
├── slice/                            Sliced scroll
├── crossproject/                     Cross-project federation
├── retriever/                        Retrievers (RRF, k-NN, standard, rule)
├── rank/                             Rank pipelines (RRF, vector hybrid)
├── runtime/                          Runtime fields
└── ...

server/src/main/java/org/elasticsearch/action/search/
├── TransportSearchAction.java        Coordinator-side fan-out + reduction
├── SearchTransportService.java       Per-shard transport plumbing
├── AbstractSearchAsyncAction.java    State machine for the two-phase search
└── ...

Key abstractions

Type	File	Role
SearchService	server/src/main/java/org/elasticsearch/search/SearchService.java	Per-node search executor
TransportSearchAction	server/src/main/java/org/elasticsearch/action/search/TransportSearchAction.java	Coordinating-node action
AbstractSearchAsyncAction	.../action/search/AbstractSearchAsyncAction.java	Async state machine that runs query then fetch
SearchSourceBuilder	server/src/main/java/org/elasticsearch/search/builder/SearchSourceBuilder.java	Strongly-typed _search body
QueryBuilder	server/src/main/java/org/elasticsearch/index/query/QueryBuilder.java	Polymorphic query DSL
Aggregator	server/src/main/java/org/elasticsearch/search/aggregations/Aggregator.java	Per-shard aggregation collector
Retriever	server/src/main/java/org/elasticsearch/search/retriever/Retriever.java	Higher-level abstraction over query + rerank
SearchHit / SearchHits	server/src/main/java/org/elasticsearch/search/SearchHit.java	Result envelope

Query then fetch

sequenceDiagram
  participant C as Coordinator
  participant S1 as Shard 1
  participant S2 as Shard 2
  C->>S1: QuerySearchRequest
  C->>S2: QuerySearchRequest
  S1-->>C: QuerySearchResult (top doc IDs + scores)
  S2-->>C: QuerySearchResult (top doc IDs + scores)
  C->>C: merge, pick global top-N
  C->>S1: ShardFetchRequest (selected docs)
  C->>S2: ShardFetchRequest (selected docs)
  S1-->>C: FetchSearchResult (source, highlights, fields)
  S2-->>C: FetchSearchResult (source, highlights, fields)
  C-->>Client: SearchResponse

The query phase only ships (docId, score) tuples, keeping per-shard work bounded. The fetch phase loads the actual hits for the global top-N. For aggregations, an additional reduce phase runs on the coordinator (and optionally a partial reduce on the data nodes for very large shard counts).

DFS, scroll, search after, point-in-time

• DFS query then fetch — first phase computes global term statistics so that scoring is comparable across shards. Required for accurate scoring on very small shards or relevance benchmarks.
• Scroll — opens a snapshot reader per shard for stable iteration over large result sets. Discouraged for new code; use search_after + PIT.
• Search after — stable cursor-based pagination using sort values.
• Point in time (PIT) — opens a server-side search context that can be re-used across queries; the modern replacement for scroll.

Aggregations

Aggregations are tree-shaped. A bucket aggregation (terms, date_histogram, range, geo_grid) splits the matching documents into buckets and recursively computes sub-aggregations. Metric aggregations (avg, sum, cardinality, percentiles) compute a value per bucket. Pipeline aggregations (derivative, moving_average, bucket_script) operate on the bucket results post-hoc.

Aggregator instances run per shard and return InternalAggregation objects which are reduced on the coordinator. Most aggregations support a streaming "single-pass" mode and a "deferred" mode for accuracy tuning. See Aggregations.

Retrievers and ranking

The retriever layer (newer than the classic query DSL) lets users compose retrievals: a vector retriever, a BM25 retriever, an RRF reranker that fuses them, a rule-based reranker that applies business overrides. A retriever rewrite rewrites itself into the lowest-level QueryBuilder the engine understands. RRF (Reciprocal Rank Fusion) is implemented in x-pack/plugin/rank-rrf for the licensed variant and extended by rank-vectors.

Vector and hybrid search

Dense vector queries use HNSW indexes built by Lucene. The knn retriever / query supports filters, similarity thresholds, and per-segment search. For approximate hybrid scoring, RRF fuses BM25 and k-NN results without forcing the user to pick weights.

Cross-cluster and cross-project search

crossproject/ (server/src/main/java/org/elasticsearch/search/crossproject/) plus RemoteClusterService (transport/) federate a search to peers. Each peer runs its own query phase; the coordinator merges. CCS supports the minimize_roundtrips mode (default) which reduces latency at the cost of some flexibility.

Cancellation and tracking

Search requests are CancellableTasks. The coordinator can cancel via the task management API; data nodes propagate cancellation to the per-shard context so Lucene aborts long collectors. SearchTaskWatchdog (server/src/main/java/org/elasticsearch/action/search/SearchTaskWatchdog.java) terminates pathological queries that ignore cooperative cancellation.

Integration points

• Plugins contribute QueryBuilders, aggregations, suggesters, rescorers, and retrievers via SearchPlugin.
• X-Pack adds: ESQL (its own engine), async search (async-search), rule-based search (search-business-rules), eql/sql, semantic text search.
• Mappings drive query rewriting via MappedFieldType.termQuery/rangeQuery/fuzzyQuery.
• Caches — request cache short-circuits the entire two-phase flow when applicable.

Entry points for modification

• Adding a query? Implement QueryBuilder + parser; register via SearchPlugin#getQueries.
• Adding an aggregation? Aggregator, AggregatorFactory, InternalAggregation, parser; register via SearchPlugin#getAggregations.
• Adding a retriever? Extend Retriever, register via SearchPlugin#getRetrievers.
• Performance tuning? Look at SearchService.QUERY_PHASE_PARALLEL_COLLECTION and the per-shard concurrency knobs.