Architecture

Neon decouples Postgres compute from Postgres storage. A compute node runs a slightly patched Postgres process whose storage manager has been replaced; pages and WAL no longer live on local disk. Instead:

• The compute streams WAL to a quorum of safekeepers, which durably persist it.
• Safekeepers stream WAL to the pageserver, which decodes records, applies them via WAL-redo, and writes immutable layer files to local disk and cloud object storage.
• When the compute needs a page, it issues a GetPage@LSN request to the pageserver, which reconstructs that page version from the layered storage.
• A storage broker lets pageservers and safekeepers find each other; a storage controller assigns tenants to pageserver shards and orchestrates migration. A storage scrubber reaps abandoned objects in S3.

This split is what makes Neon "serverless": the compute is stateless, can be stopped, started, or scaled independently of storage, and any number of read replicas can attach to the same timeline at different LSNs.

High-level data flow

graph LR
    Client[Postgres Client] -->|TLS / WS / HTTP| Proxy[proxy]
    Proxy -->|psql wire protocol| Compute[Compute Node<br/>patched Postgres + neon ext]
    Compute -->|WAL stream| SK1[safekeeper 1]
    Compute -->|WAL stream| SK2[safekeeper 2]
    Compute -->|WAL stream| SK3[safekeeper 3]
    SK1 & SK2 & SK3 -->|published via| Broker[storage_broker]
    Broker -->|subscribe| PS[pageserver]
    SK1 -->|WAL fetch| PS
    PS -->|GetPage@LSN| Compute
    PS -->|layer uploads| S3[(Cloud Object Storage)]
    StorCon[storage_controller] -->|shard placement| PS
    Scrubber[storage_scrubber] -.->|garbage collect| S3

Components and where they live

Component	Directory	Role
Pageserver	pageserver/	Stores layer files, ingests WAL, serves GetPage@LSN, runs WAL-redo. See pageserver/src/lib.rs.
Safekeeper	safekeeper/	Durable WAL acceptor, Paxos-style consensus over WAL. Entry point safekeeper/src/bin/safekeeper.rs.
Storage broker	storage_broker/	Pub/sub mesh that lets pageservers discover the latest WAL position from safekeepers. Tonic-based gRPC service.
Storage controller	storage_controller/	Manages tenant placement and sharding across pageservers. Backed by a Postgres metadata DB (see storage_controller/migrations/).
Storage scrubber	storage_scrubber/	Offline tool that scans cloud storage and removes leaked or unreachable objects.
Endpoint storage	endpoint_storage/	Per-endpoint object store for things like uploaded extension files.
Compute ctl	compute_tools/	Supervises a Postgres process on a compute node: configures it, applies the spec, downloads extensions, hot-reloads on changes. Binary in compute_tools/src/bin/compute_ctl.rs.
Compute image	compute/	Dockerfile and VM image specs for the production compute node.
Postgres fork	vendor/postgres-v{14,15,16,17}	Upstream Postgres with Neon's modifications (see docs/core_changes.md).
Neon extension	pgxn/neon/	Postgres extension that replaces the smgr with a network-backed implementation talking to the pageserver. Includes the walproposer C library.
Proxy	proxy/	TLS-terminating Postgres protocol proxy with auth, pooling, websocket tunneling, and SQL-over-HTTP.
Local control plane	control_plane/	The neon_local CLI (cargo neon) used in dev and integration tests.
Test runner	test_runner/	Python + pytest integration suite.
Shared libraries	libs/*	25+ Rust crates (utils, metrics, postgres_ffi, pq_proto, pageserver_api, safekeeper_api, walproposer, …).

Pageserver internals

The pageserver is the largest single subsystem. It manages many tenants; each tenant has many timelines (branches); each timeline owns a set of immutable on-disk layer files plus an in-memory layer where freshly received WAL accumulates.

graph TD
    subgraph Pageserver
        WalReceiver[WAL receiver<br/>per timeline] --> InMem[In-memory layer]
        InMem -->|checkpoint| L0[L0 delta layer files]
        L0 -->|compaction| L1[L1 delta layers + image layers]
        L1 -->|upload| Remote[Remote storage<br/>libs/remote_storage]
        Remote -->|on-demand download| L1
        PageService[Page service<br/>page_service.rs] -->|read| L1 & InMem
        PageService -->|reconstruct| WalRedo[WAL-redo process<br/>walredo.rs]
        WalRedo -->|invokes| RedoProc[(postgres --wal-redo)]
    end
    Compute -->|GetPage@LSN| PageService
    SK[safekeeper] -->|stream WAL| WalReceiver

Layer files come in two shapes (see pageserver/src/tenant/storage_layer.rs):

• Image layers hold a snapshot of a key range at a single LSN.
• Delta layers hold WAL records for a key range across a range of LSNs.

To answer GetPage@LSN, the pageserver finds the most recent image layer for the key, then replays delta records up to the requested LSN through the WAL-redo Postgres subprocess. See Pageserver overview and Storage layers.

Safekeeper consensus

Safekeepers run a custom Paxos variant over the WAL stream. The compute embeds a walproposer library (pgxn/neon/ C code, also exposed via libs/walproposer) that proposes WAL records to a quorum of safekeepers. Once a quorum acknowledges, the LSN is durable. Pageservers fetch already-durable WAL from any safekeeper. See Safekeeper and docs/safekeeper-protocol.md.

Sharding and orchestration

A single tenant can be sharded across multiple pageserver nodes for horizontal scale. Sharding is keyed by relation block ranges. Placement is managed by the storage controller (storage_controller/), which keeps shard-to-pageserver assignments in a Postgres metadata DB and exposes a control API. Compute nodes route GetPage@LSN to the right shard via the pageserver client (pageserver/client/, pageserver/client_grpc/). See Storage controller.

Connection edge

The proxy accepts connections in three flavors:

• Plain Postgres-over-TCP with SNI-based project routing.
• Postgres-over-WebSockets for environments that cannot open arbitrary TCP.
• SQL-over-HTTP (/sql endpoint) and a REST broker mode that authorizes JWTs and forwards rows as JSON.

Auth backends include console (the production control plane), postgres (a local Postgres used for development), and link / web (passwordless email link). See Proxy.

Cloud storage

libs/remote_storage abstracts S3, Azure Blob, GCS, and a local-filesystem backend (LocalFs) used in tests. Pageservers, safekeepers, and the scrubber all use this crate. Configuration is documented in docs/settings.md.

Observability

All Rust services use tracing with a JSON formatter (libs/tracing-utils), Prometheus metrics via the metrics crate (libs/metrics), and OpenTelemetry/OTLP traces. See How to monitor.