WAL redo

To answer GetPage@LSN, the pageserver often has to take a base page image and apply a series of WAL records on top of it. Rather than re-implementing every Postgres WAL operator in Rust, Neon runs a sandboxed Postgres process in a special WAL redo mode and asks it to do the redo.

Why a subprocess

Postgres has hundreds of WAL record types and the redo logic for each type lives across dozens of *_redo functions in the Postgres source tree. Reimplementing them in Rust would be a large surface to maintain across Postgres version upgrades. Neon takes the pragmatic path: link the existing Postgres redo code into a tiny harness binary and feed it (page, [wal records]) over a pipe.

The harness lives in pgxn/neon_walredo/ (a Postgres extension/library) and a small launcher in pageserver/src/walredo.rs.

Lifecycle

sequenceDiagram
    participant PS as Page service
    participant Mgr as WalRedoManager
    participant Proc as postgres --wal-redo
    PS->>Mgr: request_redo(key, base_page, wal_records, lsn)
    alt no process running
        Mgr->>Proc: spawn (one per tenant)
    end
    Mgr->>Proc: write base page + records (pipe)
    Proc->>Proc: apply each WAL record via Postgres redo handlers
    Proc->>Mgr: write resulting page (pipe)
    Mgr->>PS: Vec<u8>

One redo subprocess is started per tenant on demand (pageserver/src/walredo.rs::WalRedoManager::request_redo). The process is reused across many redo calls until either:

• The tenant is detached (process is killed).
• The tenant's redo subprocess panics or exits unexpectedly (the manager logs and respawns).
• A failpoint or shutdown signal kills it.

Per-tenant isolation matters: a malformed WAL record from one tenant shouldn't be able to corrupt another tenant's redo state.

Sandbox

The redo subprocess runs Postgres with extra restrictions:

• On Linux, libseccomp filters limit the subprocess to a minimal syscall set (see Makefile's --with-libseccomp configure flag and the seccomp setup in pgxn/neon_walredo/).
• The process has no network access, no on-disk Postgres data directory beyond what it absolutely needs, and reads/writes only via the pipes.

This matches the threat model "untrusted tenant data should not be able to escape the redo process and reach pageserver state."

Protocol over the pipe

pageserver/src/walredo.rs defines the framing. Each redo request is encoded as:

[u32 length][BeginRedoForBlock]
[u32 length][PushPage(base_page)]    -- the starting image
[u32 length][ApplyRecord(wal_record)]  -- repeated per WAL record
[u32 length][GetPage]

The harness reads each frame, applies it, and on GetPage writes the materialized page back. On error it writes a tagged error frame; the manager catches the error, kills the process (since redo state may be inconsistent), respawns it, and propagates the error to the caller.

Performance considerations

• Process creation cost. Spawning a Postgres process is not cheap. Reusing the long-lived subprocess amortizes the cost across many requests.
• Per-tenant isolation tax. A pageserver with thousands of tenants would have thousands of redo subprocesses. Pageservers in production handle this via memory limits and aggressive idle-timeout shutdowns; see the metrics under pageserver_walredo_*.
• Batched redo. When a request needs many records on top of one image, all records are sent in one round-trip rather than once per record.

Failure modes

If the redo subprocess returns an error or exits unexpectedly:

1. The manager logs the failure with the relevant tenant_id and the WAL records that were sent.
2. It increments pageserver_walredo_failed_total.
3. The pending request is failed with WalRedoError.
4. The next redo request for that tenant respawns the process.

Pathological WAL records (e.g. corrupt LSNs, unexpected record types) are the most common cause of redo failures during upgrades. The error messages include enough context to file an issue against the upstream vendor/postgres-v* fork.

Useful in-repo references

File	Purpose
pageserver/src/walredo.rs	Manager, process launcher, pipe protocol, retry logic.
pgxn/neon_walredo/	Postgres-side harness library that runs the redo.
docs/pageserver-walredo.md	Older but still-accurate prose description.
libs/wal_decoder/	Shared crate that decodes raw WAL into the per-page records sent to redo.

Why this is interesting

The WAL redo subprocess is the most explicit place where Neon's "we are not reimplementing Postgres" rule is visible: instead of reading WAL records and applying them in Rust, we shell out to the real thing. The cost is process management complexity. The benefit is automatic correctness whenever Postgres adds a new WAL record type — you only need to vendor the new Postgres minor version into vendor/.

See also

• Pageserver for the surrounding system.
• Storage layers for what gets fed into redo.
• Page service for how requests trigger redo.