Safekeeper API

The safekeeper exposes two API surfaces:

1. The WAL replication protocols (data plane) over libpq.
2. The HTTP management API (control plane).

WAL replication

Compute → safekeeper

The compute (running the walproposer inside the neon Postgres extension) opens a libpq connection to the safekeeper and uses Postgres's COPY-mode framing to ship the bidirectional protocol. Server side: safekeeper/src/handler.rs and safekeeper/src/receive_wal.rs.

The protocol messages (defined in libs/safekeeper_api/):

From walproposer	Purpose
ProposerGreeting { server_id, system_id, term, …}	Identify the proposer and its current term.
VoteRequest { term }	Election: ask for a vote in this term.
ProposerElected { term, start_streaming_at }	Election complete; begin streaming from this LSN.
AppendRequest { records, term, start_streaming_at, end_lsn }	Append a batch of WAL records.

From safekeeper	Purpose
AcceptorGreeting { server_id, term }	Reply to greeting.
VoteResponse { term, vote_given, last_log_term, flush_lsn }	Election: vote / refuse and report state.
AppendResponse { term, flush_lsn, commit_lsn, hs_feedback }	Acknowledge or refuse an append.

Once a quorum acknowledges an AppendRequest, the LSN is committed and the proposer can release commit waits.

docs/safekeeper-protocol.md documents the protocol in long form along with its TLA+ model in safekeeper/spec/.

Safekeeper → pageserver

The pageserver's WAL receiver opens an ordinary Postgres physical replication connection to the safekeeper. Server side: safekeeper/src/send_wal.rs. The pageserver requests a streaming START_REPLICATION at its current disk_consistent_lsn and consumes WAL records from there.

There's also a newer interpreted WAL stream (safekeeper/src/send_interpreted_wal.rs) where the safekeeper decodes records into per-page tuples before sending. The pageserver opts into this on connect.

HTTP management API

Spec: safekeeper/src/http/openapi.yaml. Mounted under safekeeper/src/http/.

Sample routes:

Method	Path	Purpose
GET	/v1/status	Liveness.
GET	/v1/timeline/{tenant_id}/{timeline_id}	Timeline state (term, flush/commit LSNs, peers).
POST	/v1/timeline	Create a timeline.
DELETE	/v1/timeline/{tenant_id}/{timeline_id}	Delete a timeline.
POST	/v1/pull_timeline	Bootstrap from another safekeeper.
POST	/v1/copy_timeline	Copy from another safekeeper without joining quorum.
POST	/v1/timeline/{tenant_id}/{timeline_id}/membership	Reconfigure quorum membership.
GET	/v1/debug_dump	Diagnostic dump of timeline state.
GET	/metrics	Prometheus metrics.

The storage controller calls the safekeeper management API to drive failover, quorum reconfiguration, and bulk operations.

Authentication

JWT Bearer; scopes include safekeeperdata for the management API and the same scope for replication. The signing keys are configured at startup.

Clients

Crate	Path	Notes
safekeeper_client	safekeeper/client/	Rust HTTP client for the management API.
safekeeper_api	libs/safekeeper_api/	Shared message types + URL helpers.

The compute side does not use these crates; it uses the C walproposer.c from pgxn/neon/.

Recovery and pull-timeline

When a safekeeper joins or recovers, it can pull_timeline from another safekeeper:

POST /v1/pull_timeline
{
  "tenant_id": "...",
  "timeline_id": "...",
  "http_hosts": ["http://other-sk-1:7676", "http://other-sk-2:7676"]
}

The pulling safekeeper streams missing WAL segments and the control file from a peer until its flush_lsn matches.

See also

• Safekeeper — the service.
• docs/safekeeper-protocol.md — protocol writeup with TLA+ links.
• safekeeper/spec/ — the TLA+ specification of the consensus protocol.
• libs/safekeeper_api/ — shared types.