Index lifecycle and snapshot lifecycle

What they are

Two related X-Pack features that automate index management:

• ILM (Index Lifecycle Management): moves indices through hot / warm / cold / frozen / delete phases according to a policy.
• SLM (Snapshot Lifecycle Management): creates and retains snapshots on a schedule.

Both store policies in cluster-state metadata and run a per-master service that drives state machines forward.

ILM

x-pack/plugin/ilm/
├── src/main/java/org/elasticsearch/xpack/ilm/
│   ├── IndexLifecycleService.java         Master-side scheduler
│   ├── IndexLifecycleRunner.java          Drives a single index forward
│   ├── action/                            REST + transport actions
│   └── ...
└── ...

server/src/main/java/org/elasticsearch/cluster/metadata/
└── IndexAbstraction.java, ComposableIndexTemplate.java   Hosts ILM policy refs

server/src/main/java/org/elasticsearch/dlm/                 Data Lifecycle Management (newer alternative)
modules/data-streams/                                       Data stream rollover and retention

A policy is an ordered list of phases (hot, warm, cold, frozen, delete) and per-phase actions (rollover, forcemerge, searchable_snapshot, set_priority, allocate, shrink, migrate, wait_for_snapshot, delete). Each action is implemented by a LifecycleAction/Step pair; IndexLifecycleRunner walks the steps for each managed index.

stateDiagram-v2
  [*] --> Hot
  Hot --> Warm: min_age
  Warm --> Cold: min_age
  Cold --> Frozen: min_age
  Frozen --> Delete: min_age
  Delete --> [*]

Steps may be cluster steps (run synchronously on the master, like updating settings) or async wait steps (poll a condition like "index size below X"). Failed steps stay in the current step until the operator intervenes.

DLM (Data Lifecycle Management)

server/src/main/java/org/elasticsearch/dlm/ plus modules/data-streams/ ship a leaner alternative for data streams: just retention plus optional rollover, configured directly on the data stream rather than as a separate policy. DLM coexists with ILM and is the preferred path for new time-series data streams in 9.x.

Stateless DLM transitions

x-pack/plugin/dlm-frozen-transition/ adds the stateless-tier transition steps (move shards to object-storage backed engines).

SLM

x-pack/plugin/slm/
├── SnapshotLifecycleService.java        Schedules snapshot creation
├── SnapshotRetentionService.java        Retains / deletes old snapshots per policy
└── ...

A policy declares schedule (cron), name (template), repository, retention (expire_after, min_count, max_count), and config (which indices). SnapshotLifecycleService triggers TransportCreateSnapshotAction; SnapshotRetentionService walks existing snapshots and deletes expired ones.

Integration with searchable snapshots

ILM's searchable_snapshot action coordinates with the searchable-snapshots plugin: the action takes a snapshot, mounts it as a searchable-snapshot index in the frozen tier, and removes the original. This is the workhorse for cost-tiering very old data.

Where to extend

• New ILM action: implement LifecycleAction and one or more Steps; register via IndexLifecyclePlugin.
• New ILM step type: Step subclasses live in x-pack/plugin/core/.../ilm/. Async steps extend AsyncWaitStep.
• Tuning: indices.lifecycle.poll_interval and slm.history_index_enabled.