Index and postings

The block index turns label-name/value combinations into ordered series IDs. It is what makes up{job="api-server", env="prod"} cheap to evaluate over a 31-day block.

Files

tsdb/index/
├── index.go         # Index reader/writer (~1,500 lines).
├── postings.go      # In-memory and on-disk posting list operations.
├── postingsstats.go # Stats about postings cardinality (used by /api/v1/status/tsdb).
└── *_test.go

Block index format (overview)

The on-disk index file (<block>/index) starts with a header (magic + version) followed by:

1. Symbol table — deduplicated string table for label names, label values, and metric names.
2. Series section — for each series: ref, label set encoded as symbol indices, list of chunks {minTime, maxTime, ref}.
3. Label index — for each label name, the sorted list of values seen.
4. Postings section — for each (label name, label value), a sorted list of series refs.
5. Postings offset table — mapping from (label, value) to byte offset of its postings list.
6. Table-of-contents — fixed-size footer with absolute offsets to each section.

The format is documented inline in tsdb/index/index.go and matches the spec used by Cortex/Thanos/Mimir for compatibility.

In-memory postings

MemPostings (in tsdb/index/postings.go) is the head's posting structure. It maintains:

• m map[string]map[string][]storage.SeriesRef — per label name, per value, the sorted refs.
• lvs map[string][]string — per label name, the sorted unique values.

It supports the same postings operations as the on-disk variant but is mutable — insertions happen during series creation in the head.

Postings operations

The query planner operates on iterators of series refs. Key combinators:

Function	What it does
Intersect(a, b, …)	Series present in all input postings (galloping merge).
Merge(a, b, …)	Union of postings.
Without(p, exclude)	Postings in p but not in exclude.
Sort(p)	Order postings by ref (only used for un-ordered inputs).
EmptyPostings()	Iterator over the empty set.
ErrPostings(err)	Iterator that surfaces an error on first call.

Performance optimisations in tsdb/index/postings.go:

• Galloping search for intersections of mismatched-cardinality lists.
• Per-call buffer pooling (util/zeropool).
• Fast-paths for single-list and two-list intersections.
• The 3.10 LabelValues intersection optimisation (#18069) avoids a quadratic blow-up when a matcher hits many label values.

Label values intersection

When a query has multiple matchers, the planner asks LabelValues(name, matchers) per matcher and intersects. The 3.10 PR #18069 rewrote this to compute the intersection in a single pass instead of nesting per-matcher loops.

Reading the index

Index.LookupLabelValues, Index.Postings, and Index.Series are the three primitives a querier uses. The block querier in tsdb/querier.go chains them:

1. Resolve postings for each matcher.
2. Intersect the postings.
3. For each surviving ref, fetch the series record (labels + chunk refs).
4. Open chunks via ChunkReader.

Index-related metrics

• prometheus_tsdb_postings_for_matchers_cache_hits_total / _misses_total
• prometheus_tsdb_postings_for_matchers_cache_*_seconds
• prometheus_tsdb_storage_blocks_postings_count (per-block)

A postings-for-matchers LRU cache lives in tsdb/postings_for_matchers_cache.go; it stores the resolved series-ref list for a given (block, matcher set) to avoid recomputing on repeated identical queries (common in dashboards).

Custom postings decoder

Some operators wrap the index reader to skip postings decoding when their plan can prove a postings list is empty or trivially derivable. The hook is the PostingsDecoderFactory field on tsdb.Options. The default factory uses the standard format.

Entry points for modification

• New posting operator: add to postings.go and reuse the Postings interface; cover with property-based tests in postings_test.go.
• Index format extension: must be backward compatible. Bump the version constant in tsdb/index/index.go and document it in tsdb/CHANGELOG.md.

See Blocks for how the index ties into a block's lifecycle and PromQL for how query planning uses postings.