Native histograms

Native histograms are a sparse, exponential-bucket histogram representation that fit one observation into a single sample. They replace the classical _bucket / _count / _sum triple-series scheme and dramatically reduce cardinality for high-resolution histograms.

Touchpoints

• Type: model/histogram/histogram.go (integer counts) and float_histogram.go (float counts).
• Encoding: tsdb/chunkenc/histogram.go and float_histogram.go.
• Wire format: prompb/types.proto (RW1 + RW2) and prompb/io/prometheus/write/v2/types.proto.
• Scrape: model/textparse/protobufparse.go is the recommended path (Prometheus protobuf format). OpenMetrics 1.0 also supports them via _bucket -> NHCB conversion.
• Engine: promql/quantile.go, promql/info.go, promql/engine.go — most operators handle *histogram.Histogram and *histogram.FloatHistogram.
• API: web/api/v1/api.go returns histograms in the value/values field as [ts, hist] with the histogram serialised in the JSON codec.

Anatomy of a native histogram sample

Histogram {
  Count:           total observations
  Sum:             sum of observations
  Schema:          int that controls bucket boundary spacing
  ZeroThreshold:   |value| below this counted in ZeroCount
  ZeroCount:       count for the zero bucket
  PositiveSpans:   sparse bucket layout
  PositiveBuckets: per-span counts (delta-encoded)
  NegativeSpans:   ...
  NegativeBuckets: ...
}

Schema = 0 corresponds to ratio 2; schema = 8 to ratio 2^(2^-8) ≈ 1.0027 — finer schemas have more buckets but more compression headroom.

NHCB (custom buckets)

NHCB stores classical histogram boundaries (le="...") using the native histogram's encoding. The translator in util/convertnhcb/ consumes a stream of classical histogram samples per series and emits NHCB samples without buckets. The wire format is unchanged.

Operators

PromQL operators that act on histograms:

• histogram_quantile(q, hist) — promql/quantile.go.
• histogram_count(hist), histogram_sum(hist), histogram_avg(hist), histogram_stddev(hist), histogram_stdvar(hist).
• histogram_fraction(lower, upper, hist).
• Arithmetic: +, -, *, / between histograms (where defined).
• 3.11 added </ and >/ (bucket trimming) and histogram_quantiles(...) for variadic quantile.

Schema migrations

A series can change schema mid-stream (a target's --enable-feature=native-histograms flag flips, for example). The chunk encoder handles this by closing the current chunk and opening a new one with the new schema.

Storage size

Native histograms use one chunk per series, the same as float series. A typical scrape interval of 15s keeps about 240 samples per chunk. Memory consumption is roughly (n_buckets * 2 bytes) + overhead per sample.

Entry points for modification

• New operator: add to promql/functions.go (registered via parser/functions.go); make sure both Histogram and FloatHistogram paths are covered.
• Schema change: must be additive — older Prometheus must still read the encoded chunk. Bump tsdb/CHANGELOG.md.
• Compactor/queries: check tsdb/chunkenc/histogram_meta.go for layout validation.

See Chunk encodings and PromQL engine.