Blocks and compaction

A block is an immutable, on-disk directory containing the chunks, index, tombstones, and metadata for a fixed time range. Blocks are produced by the compactor and consumed by queriers. The block format is shared between Prometheus, Cortex, Mimir, and Thanos.

On-disk layout

data/
├── 01HQ1V0M0...A8B/           # ULID block ID (lexicographically sortable by time)
│   ├── chunks/
│   │   ├── 000001             # chunk segment file
│   │   └── 000002
│   ├── index                  # compacted postings + label index
│   ├── tombstones             # range deletions
│   └── meta.json              # version, mint/maxt, sources, compaction info
├── 01HQ1V8XK1...AAA/          # next block (overlapping if vertical compaction enabled)
└── …

meta.json is the source of truth: it lists MinTime, MaxTime, Stats (series/samples/chunks counts), Compaction (level, sources, parents), and Version. The schema is defined in tsdb/block.go::BlockMeta.

Key types

Type	File	Role
Block	tsdb/block.go	Open block; exposes Querier, ChunkQuerier, Index, Tombstones.
BlockMeta	tsdb/block.go	The meta.json structure (mint/maxt, stats, compaction info).
BlockReader	tsdb/block.go	Read-only interface used by queriers.
BlockWriter	tsdb/blockwriter.go	Standalone writer used by promtool backfill.
LeveledCompactor	tsdb/compact.go	The default compactor. Produces leveled and vertical (overlap) merges.
Compactor	tsdb/compact.go	Compactor interface; allows alternate strategies.
Stats	tsdb/block.go	Series/sample/chunk counts populated during compaction.

Compaction policy

graph LR
    Head -->|2h block| L0
    L0 -->|3 blocks| L1
    L1 -->|3 blocks| L2
    L2 -->|3 blocks| L3
    L3 -.-> Retention[Drop on retention]

tsdb/compact.go::LeveledCompactor.Plan() selects the next compaction:

1. Find the smallest non-empty level that has compactionRanges[level] consecutive blocks of the same range.
2. Merge them into a block whose range is the next level's range (default 2h * 3^level).
3. Stop at the maximum block range (default 31 days, --storage.tsdb.max-block-duration).

Vertical compaction merges overlapping blocks at the same level. It is enabled by default and is required for out-of-order samples and for receiving from downsampled long-term sources.

Delayed compaction (3.x) randomly delays compaction to spread load across replicas — see --storage.tsdb.compaction-delay-max-percent.

Retention

tsdb.DB.cleanupBlocksGreaterThan evaluates retention every --storage.tsdb.retention.time interval:

• retention.time — drop blocks older than this duration.
• retention.size — drop oldest blocks until total bytes <= limit.
• retention.percentage (3.11, #18080) — drop oldest blocks until used disk percent <= limit.

Retention runs after compaction; the deleted block directories are renamed to <ULID>.tmp-for-deletion first (atomic-ish on Linux) and then removed.

Tombstones

A tombstone is a range deletion record (labels matcher, mint, maxt) stored in the block's tombstones file. The /api/v1/admin/tsdb/delete_series endpoint records them; they are honoured at read time and physically removed during the next compaction. /api/v1/admin/tsdb/clean_tombstones triggers an immediate compaction over tombstoned blocks.

Snapshots

/api/v1/admin/tsdb/snapshot hard-links every block (and optionally the head) into <data dir>/snapshots/<ULID>/. Implementation: tsdb.DB.Snapshot() in tsdb/db.go. Useful for backup and forklift migrations.

Block writer (backfill)

tsdb/blockwriter.go lets promtool produce blocks without standing up a Prometheus server. The path:

1. Open a BlockWriter with a temp directory.
2. Append samples in any order via the appender API.
3. Flush() produces a single block per 2-hour bucket.
4. The blocks can be moved into a real Prometheus data dir.

This is how promtool tsdb create-blocks-from openmetrics and create-blocks-from rules work.

Self-metrics

• prometheus_tsdb_compactions_total / _failed_total / _triggered_total
• prometheus_tsdb_compaction_duration_seconds
• prometheus_tsdb_compaction_chunk_* — chunk size/range distributions
• prometheus_tsdb_compaction_populating_block (gauge — 1 while a block is being written)
• prometheus_tsdb_storage_blocks_bytes — total on-disk block size
• prometheus_tsdb_lowest_timestamp / _highest_timestamp — overall data range
• prometheus_tsdb_size_retentions_total / _time_retentions_total — retention triggers

Entry points for modification

• Custom compaction selector: implement the Compactor interface in tsdb/compact.go. The leveled compactor is the default; alternative implementations have lived in the codebase historically (vertical-only, etc.).
• Block format change: bump tsdb/block.go::version and document the upgrade in tsdb/CHANGELOG.md. The reader must support the previous format for at least one release.
• Retention policy: tsdb.DB.cleanupBlocksGreaterThan and cleanupBlockSizeGreaterThan; the metric-driven percentage policy is at cleanupBlocksByPercentage.

See Index and Chunks for what's inside a block.