Data models

A pointer page to the most important on-disk and on-the-wire data shapes.

Cluster state

ClusterState (server/src/main/java/org/elasticsearch/cluster/ClusterState.java):

• version — monotonically increasing version on each accepted publication.
• term — coordinator term (Raft-like).
• metadata — Metadata object: indices, templates, ingest pipelines, ILM policies, transforms, role mappings, snapshot lifecycle, etc.
• routing_table — per-shard primary/replica routing.
• nodes — DiscoveryNodes membership.
• blocks — ClusterBlocks controlling read/write/metadata.
• customs — plugin-contributed Custom documents.

State is persisted as a Lucene index by PersistedClusterStateService and replicated via PublicationTransportHandler as compressed diffs.

Index metadata

IndexMetadata:

• index — (name, UUID)
• version and mapping_version and settings_version and aliases_version — independent monotonic counters.
• settings — Settings.
• mapping — MappingMetadata.
• aliases, system, hidden, state (open/close), tier_preference, lifecycle_metadata.

Mapping

A mapping is a recursive tree of FieldMappers plus a few metadata fields (_source, _routing, _id, _seq_no, etc.). At runtime, each field is represented by a MappedFieldType that knows how to build queries against itself.

Routing table

RoutingTable is a per-index list of IndexShardRoutingTable. Each shard table has a list of ShardRouting entries (one primary + zero or more replicas), each with a state (UNASSIGNED, INITIALIZING, STARTED, RELOCATING).

Transport versions

TransportVersion (server/src/main/java/org/elasticsearch/TransportVersion.java) plus TransportVersions (server/src/main/java/org/elasticsearch/TransportVersions.java) define the wire-format constants. server/src/main/resources/transport/upper_bounds/<version>.csv is generated and checked in.

Adding a wire-format change:

1. Add a constant via TransportVersion.fromName("descriptive_name").
2. Run ./gradlew generateTransportVersion to refresh the upper_bounds CSV.
3. Gate new fields with out.getTransportVersion().onOrAfter(MY_NAME).

Index versions

Parallel to transport versions but for on-disk index format (IndexVersion, IndexVersions). Lucene versions also gate compatibility — see IndexMetadataVerifier.

Persistent tasks

PersistentTasksCustomMetadata (server/src/main/java/org/elasticsearch/persistent/PersistentTasksCustomMetadata.java) is a Custom of cluster state that tracks long-running tasks (ML jobs, transforms, follow tasks, ILM-driven async waits). Each task has an id, a task name, parameters, and an assigned node.

Snapshot repository layout

<repo>/
├── index-<gen>                          Latest top-level metadata (`RepositoryData`)
├── index.latest                         Pointer to the latest gen
├── snap-<uuid>.dat, meta-<uuid>.dat     Per-snapshot metadata
└── indices/<index-uuid>/
    └── <shard-id>/
        ├── index-<gen>                  Per-shard metadata (`BlobStoreIndexShardSnapshots`)
        ├── snap-<uuid>.dat              Shard-level snapshot description
        └── __<file-hash>                Lucene segment files (immutable, dedup-able)

Translog

A per-shard write-ahead log. Each translog file (translog-<gen>.tlog) is a sequence of Translog.Operation records: Index, Delete, NoOp. Files are rolled when:

• They reach index.translog.flush_threshold_size.
• A flush completes (the live translog is rolled and the previous gen retained for retention leases).

XContent

The polymorphic content layer. Every API-visible type implements:

XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException;

And exposes a static <T> T fromXContent(XContentParser parser) factory. The XContent layer abstracts JSON / YAML / CBOR / Smile (selected by Content-Type / Accept).