Data models

The on-disk and wire formats Prometheus uses. Documenting them here makes it easier to reason about cross-version compatibility.

On-disk formats

TSDB block

<data dir>/<ULID block ID>/
├── chunks/
│   ├── 000001       # variable-length chunks, packed sequentially
│   └── 000002
├── index            # postings + label index
├── tombstones       # range deletion records
└── meta.json        # mint/maxt, stats, source/parent refs, version

• Block ID: ULID (oklog/ulid/v2). Lexicographically sortable by encoded time.
• meta.json schema: tsdb.BlockMeta in tsdb/block.go.
• Chunk format: typed bytes; types are XOR, XOR2, Histogram, FloatHistogram (see tsdb/chunkenc/).
• Index format: symbol table + series records + posting lists + offset table; documented inline in tsdb/index/index.go.
• Tombstones: sequence of (ref, mint, maxt) triples (tsdb/tombstones/).

Compatibility: a block written by version N must be readable by version N, N+1, and (typically) N-1. Format changes are tracked in tsdb/CHANGELOG.md.

TSDB head (in-memory)

data/
├── chunks_head/
│   ├── 000000       # mmapped completed head chunks
│   └── 000001
├── wal/
│   ├── 00000003     # write-ahead log segments (~128 MB each)
│   └── 00000004
└── chunks_head/snapshot.NNNNNNN/   # optional, with --enable-feature=memory-snapshot-on-shutdown

WAL records (tsdb/record/record.go):

Type	Carries
Series	Series ref + label set.
Samples	Float sample batch.
Tombstones	Range deletions.
Exemplars	Exemplars attached to a series.
HistogramSamples	Native integer histogram samples.
FloatHistogramSamples	Native float histogram samples.
CustomBucketsHistogramSamples	NHCB samples.
Metadata	Type/unit/help metadata.
ST	Start timestamps (3.11, behind feature flag).

Each record is preceded by a one-byte type tag.

Agent WAL

Same wlog package, no compaction. Truncation is age-based.

Wire formats

Remote write v1

POST /api/v1/write
Content-Type: application/x-protobuf
Content-Encoding: snappy

Payload: prompb.WriteRequest (prompb/remote.proto):

WriteRequest {
  repeated TimeSeries timeseries
  repeated MetricMetadata metadata
}

TimeSeries {
  repeated Label labels
  repeated Sample samples
  repeated Exemplar exemplars
  repeated Histogram histograms
}

Histograms in v1 use a flat layout in prompb.Histogram.

Remote write v2

Content-Type: application/x-protobuf;proto=io.prometheus.write.v2.Request

Payload: io.prometheus.write.v2.Request (prompb/io/prometheus/write/v2/types.proto):

Request {
  repeated string symbols
  repeated TimeSeries timeseries
}

TimeSeries {
  repeated uint32 labels_refs        // refs into symbols[]
  repeated Sample samples
  repeated Exemplar exemplars
  Metadata metadata
  int64    created_timestamp         // ST
  repeated Histogram histograms
}

The shared symbol table dramatically reduces payload size. Metadata is per-series, not separate.

OTLP

The receiver accepts standard OTLP ExportMetricsServiceRequest (protobuf or JSON) at /api/v1/otlp/v1/metrics. Conversion is handled by the upstream github.com/prometheus/otlptranslator library.

Federation / OpenMetrics

/federate?match[]=... returns OpenMetrics text. The encoder is in web/federate.go. Each series carries the local external_labels.

Scrape protocols

Negotiated via Accept header during a scrape. Supported MIME types:

• application/openmetrics-text;version=1.0.0;charset=utf-8
• application/openmetrics-text;version=0.0.1;charset=utf-8
• text/plain;version=0.0.4;charset=utf-8
• application/vnd.google.protobuf;proto=io.prometheus.client.MetricFamily;encoding=delimited

Parsers live in model/textparse/.

Internal types

Labels

labels.Labels is the immutable, sorted label set. Three storage representations behind build tags (slicelabels, stringlabels, dedupelabels). All implement the same accessor interface; the choice affects memory and CPU.

labels.SymbolTable interns label values per scrape to avoid allocating identical strings repeatedly.

Native histogram

histogram.Histogram (integer counts) and histogram.FloatHistogram (float counts after rate/binop). Sparse exponential layout via BucketSpan. Custom-bucket variants store CustomValues instead of Schema.

Sample

(timestamp_ms int64, value float64). Histograms attach via H *histogram.Histogram or FH *histogram.FloatHistogram on the same value carriers.

Exemplar

(labels labels.Labels, value float64, ts int64, hasTs bool). Total label string length capped at 128 bytes.

Metadata

Per-series (MetricType, Unit, Help). MetricType is Counter, Gauge, Histogram, Summary, GaugeHistogram, Info, Stateset, Unknown.

Stale markers

model/value.StaleNaN is a specific NaN bit-pattern that flags a series as stale. Scrape paths emit it when a target stops returning a series. PromQL last_over_time(... [...]offset 5m) and friends honour the marker.

Tombstone records

Range deletions are written through /api/v1/admin/tsdb/delete_series. Each tombstone is (matcher, mint, maxt). They are honoured at read time and physically applied during the next compaction over the affected blocks.

Cross-version stability

• TSDB block format: backward compatible across at least one minor.
• WAL record format: same. New record types added as new tags.
• Remote write v1: API version 0.1.0; compatible with all 2.x and 3.x.
• Remote write v2: API version 2.0.0; introduced in 2.x, default for new senders in 3.x.
• HTTP API v1: stable since v2.0.0 (Nov 2017).

See TSDB and Remote for implementation detail.