neondatabase/neon

Storage layers

The pageserver's on-disk data structure is a layered, log-structured store of WAL records and base page images. This page describes the layer model and where each piece lives in the code. The canonical long-form description is docs/pageserver-storage.md.

Two kinds of layers

A layer covers a key range and an LSN range. Each layer comes in one of two flavors:

Image layer — a snapshot. For every key in [key_start, key_end), holds the page image at exactly lsn. Equivalent to a full base backup of that key range at that LSN.
Delta layer — a slice of WAL. For every key in [key_start, key_end) with at least one update in [lsn_start, lsn_end), holds the WAL records that touched it.

To read a page at a given LSN, the pageserver:

Finds the most recent image layer that contains the key at LSN ≤ requested LSN.
Collects all delta layers containing the key with image_lsn < lsn_end ≤ requested_lsn.
Replays the deltas on top of the image via WAL-redo.

If no image layer exists yet for a key (early in a timeline's life), the chain starts from the timeline's ancestor or from a synthetic empty page.

In-memory and on-disk

Beyond the two file formats, there's a third kind of layer that exists only in RAM:

In-memory layer (a.k.a. ephemeral layer, file pageserver/src/tenant/ephemeral_file.rs). Holds WAL just after it arrives from the safekeeper, before it's written out as an L0 delta. Backed by a temporary file so that it survives memory pressure but is discarded on restart.

When the in-memory layer accumulates more than checkpoint_distance bytes of WAL (default 256 MB; see pageserver/src/config.rs), it is "frozen" and written out as an L0 delta file in one shot.

L0 vs L1

L0 and L1 distinguish layers by their shape, not their content:

L0 — covers the whole key space, a narrow LSN range. Output of the in-memory layer flush.
L1 — covers a narrow key range, the whole LSN range (since the last image layer cut). Output of compaction.

A reader looking up a page might have to scan many L0 layers (because each contains all keys), but only one or two L1 layers (because each covers a narrow key range). Compaction's job is to convert the L0 backlog into L1 layers, and to re-cut image layers when the redo chain gets too deep. See Compaction.

File format

Each layer file is a self-contained binary file. The on-disk layout (simplified):

[header]
[blocks ... (4 KiB each)]
[summary block]

Header — magic, version, key/lsn ranges, format flags.
Body blocks — for image layers, a B-tree mapping key → (offset, length) where the page image lives; for delta layers, a B-tree mapping (key, lsn) → (offset, length).
Blob storage — the actual page images or WAL records, stored in compressed blocks.
Summary block — at a fixed offset; metadata used to validate the file.

The B-tree implementation is a custom on-disk format in pageserver/src/tenant/disk_btree.rs (≈40 KB of Rust). It's a simple, append-only, sorted block layout optimized for reading sequential key ranges. There's no in-place mutation; the file is built once and only read afterward.

For the actual byte layout look at:

pageserver/src/tenant/storage_layer.rs — trait and shared types.
pageserver/src/tenant/storage_layer/image_layer.rs — image-specific format.
pageserver/src/tenant/storage_layer/delta_layer.rs — delta-specific format.

Filenames encode the layer's key range and LSN range, e.g. 000000000000000000000000000000000000-FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF__000000016B5A50-000000016F9A38. This makes layer-map reconstruction on startup a simple directory scan with filename parsing — no separate index file is required.

Layer map

pageserver/src/tenant/layer_map.rs (~80 KB of Rust) is the in-memory data structure that answers "which layer should I look at for key K at LSN L?" It's an interval tree keyed by (key_range, lsn_range) with O(log n) point lookups.

Operations:

search() — find the layer covering a (key, lsn) point. Used on every GetPage@LSN.
range_search() — find layers covering a (key_range, lsn_range). Used by compaction and basebackup.
insert_historic() / replace_historic() — update the map after compaction or layer download.

The layer map is per-timeline. It does not hold layer data, only metadata (range + filename + on-disk vs not-on-disk flag).

Remote storage

The on-disk layer files are mirrored to cloud object storage by pageserver/src/tenant/remote_timeline_client.rs (~3.3k lines). The remote layout per timeline is:

<prefix>/<tenant_id>/timelines/<timeline_id>/
  index_part.json                    -- the manifest
  000000...-FFFFFFFF...__lsn-lsn     -- layer files (same filenames as local)
  ...

index_part.json lists every layer that should exist plus a disk_consistent_lsn. Pageservers reconcile their local view against index_part.json on startup and on attach.

A layer can be:

Local + remote: cached locally for fast access.
Remote only: evicted from local disk; downloaded on demand.
Local only: scheduled for upload; will become "local + remote" once the upload completes.

The RemoteTimelineClient runs an upload queue with strict ordering (layer must be uploaded before index_part.json references it; deletions go through a multi-step protocol via pageserver/src/deletion_queue.rs to avoid losing data on crash).

Eviction

Local disk is bounded. pageserver/src/disk_usage_eviction_task.rs is a background task that, when local usage crosses a threshold, evicts the least-recently-used layer files (ones still safely backed by remote storage) until usage drops back below the watermark. Per-timeline accounting lives in pageserver/src/tenant/timeline/eviction_task.rs.

Sharding

When a tenant is sharded, each shard owns a strict subset of the key space (selected by a stable hash of the relation key prefix). Layers are partitioned by shard: every shard has its own set of layer files in remote storage. Cross-shard reads happen at the compute level — the compute's pageserver client (pageserver/client/, pageserver/client_grpc/) routes each GetPage@LSN to the correct shard.

See libs/pageserver_api/src/shard.rs for the ShardIdentity and the key-to-shard mapping.

Key source files

File	Purpose
`pageserver/src/tenant/storage_layer.rs`	Layer trait, image/delta enum, shared serialization.
`pageserver/src/tenant/storage_layer/image_layer.rs`	Image layer reader/writer.
`pageserver/src/tenant/storage_layer/delta_layer.rs`	Delta layer reader/writer.
`pageserver/src/tenant/layer_map.rs`	In-memory interval-tree index of layers.
`pageserver/src/tenant/disk_btree.rs`	The on-disk B-tree used inside layer files.
`pageserver/src/tenant/ephemeral_file.rs`	In-memory layer backing file.
`pageserver/src/tenant/remote_timeline_client.rs`	Upload/download orchestration.
`pageserver/src/tenant/upload_queue.rs`	Ordered upload queue.
`pageserver/src/deletion_queue.rs`	Multi-step deletion protocol.
`pageserver/src/disk_usage_eviction_task.rs`	Disk-pressure eviction.