Stateless and searchable snapshots

Two related X-Pack features that re-imagine the storage layer around object storage:

• Searchable snapshots — mount snapshot blobs as a queryable index, no full local copy required.
• Stateless — push the durable copy of every index continuously to object storage; nodes are caches and can be replaced at will.

Searchable snapshots

x-pack/plugin/searchable-snapshots/
├── SearchableSnapshots.java                Plugin entry point
├── SearchableSnapshotsEngine.java          Engine that reads from repository blobs
├── store/SearchableSnapshotDirectory.java  Lucene Directory backed by a repository + cache
├── action/                                 Mount / clear-cache APIs
└── ...

x-pack/plugin/blob-cache/
├── BlobCacheService.java                   Per-node disk cache for blob ranges
└── ...

x-pack/plugin/snapshot-based-recoveries/    Recover replicas from snapshots instead of primaries

A "mount" of a snapshot creates a regular index whose shards use SearchableSnapshotsEngine instead of InternalEngine. Reads bypass Lucene's normal Directory and go through SearchableSnapshotDirectory, which translates segment-file reads into HTTP GET requests against the snapshot repository (with byte-range support). The BlobCacheService keeps the most recently-read ranges on local disk so hot data is fast.

Two flavors:

• Fully cached ("warm" tier): the cache size is set to the index size, so the index is effectively local.
• Partially cached ("frozen" tier): cache is much smaller than the index; cold queries hit the object store directly.

Stateless (x-pack/plugin/stateless)

x-pack/plugin/stateless/
├── StatelessPlugin.java                                 Entry point (one of the busiest files in the repo)
├── src/main/java/org/elasticsearch/xpack/stateless/
│   ├── engine/                                          Stateless engine + object-store-backed Directory
│   ├── index/                                           StatelessIndexShard, hollow shards
│   ├── recovery/                                        Recovery from object store
│   ├── translog/                                        Stateless translog / WAL on object store
│   ├── search/                                          Per-node search cache integration
│   └── ...
└── ...

x-pack/plugin/stateless-health-shards-availability/      Health indicator
x-pack/plugin/stateless-master-failover/                 Master failover for stateless topology
x-pack/plugin/stateless-no-wait-for-active-shards/       Adapted client-facing semantics
x-pack/plugin/stateless-sigterm/                         Graceful shutdown helper

In stateless mode every index commit lands in object storage (S3 / GCS / Azure Blob); nodes are stateless caches. A node failure does not require a peer-to-peer recovery — a replacement node loads the latest commit straight from the object store. This is the topology behind Elastic Cloud Serverless.

Stateless is one of the most actively-evolving subsystems in the repository (StatelessPlugin.java was edited 46 times in the last 90 days). The shape is still settling; the relevant primitives are:

• Object-store engine — replaces InternalEngine. Writes go to object storage; local Lucene is a read cache.
• Hollow shards — shards with metadata only, no local segment files; used for nodes that should be searchable but not authoritative.
• Stateless translog — write-ahead log lives in object storage with strong consistency primitives.
• Cluster state in object storage — long-term durability beyond the master's local disk.

Snapshot-based recoveries

x-pack/plugin/snapshot-based-recoveries/ lets a new replica recover from the most recent snapshot instead of streaming every segment from the primary. Combined with frequent SLM snapshots, this dramatically cuts primary load when scaling out.

Integration points

• All three features reuse Repository / BlobStoreRepository for the object-store interaction.
• Health indicators (stateless-health-shards-availability) plug into the standard HealthService.
• ILM uses searchable_snapshot actions to migrate indices to the frozen tier.
• The BlobCache is shared between searchable snapshots and stateless when both are present.

Where to extend

• New repository backend: implement Repository (typically by extending BlobStoreRepository); reuse already gives you searchable snapshots.
• Tuning the blob cache: xpack.searchable.snapshot.shared_cache.* settings.
• Stateless internals: read StatelessPlugin (frequently changing) for the latest entry points.