neondatabase/neon

The neon Postgres extension

pgxn/neon/ is the Postgres extension that turns a vanilla(ish) Postgres into a stateless Neon compute. It replaces the storage manager with a network-backed implementation, embeds the walproposer that streams WAL to safekeepers, runs an in-process page cache, and exposes a handful of internal SQL functions for diagnostics.

It's the most consequential C code in the repository — about 25k lines spread across pgxn/neon/*.c.

Sibling extensions

pgxn/ actually contains four extensions plus a Rust support crate:

Directory	Purpose
`pgxn/neon/`	The main extension. Replaces smgr, runs walproposer, manages the page cache. Loaded into every compute.
`pgxn/neon_rmgr/`	Custom WAL resource managers for Neon-specific WAL records. Loaded into both the compute and the WAL-redo subprocess.
`pgxn/neon_test_utils/`	Internal SQL functions used by tests. Built only with `--features=testing`.
`pgxn/neon_walredo/`	Library used by the pageserver's `postgres --wal-redo` subprocess. Provides the harness that takes (page, [WAL records]) over a pipe and returns the redone page.
`pgxn/neon/communicator/`	Rust crate (Cargo workspace member) implementing an async pageserver client used by `pgxn/neon` via FFI.

What `pgxn/neon` does

Replaces the storage manager (`pagestore_smgr.c`)

Postgres calls into smgr_* functions whenever it wants to read or write a relation page. In a Neon compute these calls are intercepted by pagestore_smgr.c (~75 KB of C) and translated into:

Reads → GetPage@LSN requests to the pageserver.
Writes → no-op locally; the WAL records produced by the dirty buffer will reach the pageserver via the safekeepers.
smgrnblocks → cached relation size, refreshed via the pageserver.

The pageserver client used here is libpagestore.c (~48 KB), which speaks the libpq replication protocol (and increasingly gRPC, via pgxn/neon/communicator/). It pools connections, tracks pending requests, and integrates with Postgres's interrupt handling so cancellable queries actually cancel.

Runs walproposer (`walproposer.c`)

walproposer.c (~92 KB) is the C implementation of the safekeeper consensus protocol — running inside the Postgres process. It hooks the WAL insertion path, batches new WAL, and proposes batches to a quorum of safekeepers. Once a majority acknowledges, the LSN is reported as durable to Postgres, which can then release commit waits.

walproposer_pg.c (~67 KB) is the Postgres glue: it sets up signal handlers, opens connections to safekeepers, and integrates with Postgres's WalSenderMain / WalReceiverMain machinery. walproposer_compat.c provides per-Postgres-version shims.

The same walproposer.c also exists as a Rust-friendly static library in libs/walproposer/, which is how the safekeeper's simulation tests can drive the protocol without spawning a Postgres process.

Caches relation sizes (`relsize_cache.c`)

A cache of RelFileNode → blocks mappings so most RelationGetNumberOfBlocks calls are O(1). Refreshed when WAL records that change relation size are observed.

Local file cache (`file_cache.c`)

Optional disk-backed cache of pages local to the compute, sitting between Postgres's shared_buffers and the network. Reduces tail latency for repeated cold reads. ~58 KB of C.

Last-written-LSN cache (`neon_lwlsncache.c`)

Tracks the last WAL LSN that wrote to a particular page. Used to lower the LSN sent in GetPage@LSN requests — if a page hasn't been written in a while, we can request it at a lower LSN where the pageserver may have an image cached.

Performance counters (`neon_perf_counters.c`)

Exposes Neon-specific counters (page request latencies, file cache hits/misses, etc.) via a SQL view in the neon schema.

DDL handler (`neon_ddl_handler.c`)

Hooks DDL events to keep auxiliary state in sync — e.g. when a table is dropped, invalidate cached metadata.

Logical replication monitor (`logical_replication_monitor.c`)

Tracks logical replication slots and reports state to compute_ctl.

Versioning

The extension is neon with a version chain encoded in the neon--*.sql files:

neon--1.0.sql
neon--1.0--1.1.sql
neon--1.1--1.2.sql
...
neon--1.5--1.6.sql

Each upgrade step adds or changes SQL-callable functions. compute_ctl ensures the right version is installed at compute start.

SQL surface

A small set of functions is callable from SQL after CREATE EXTENSION neon. Examples:

neon.get_perf_counters() — Neon-specific perf counters.
neon.local_cache_pages() — local file cache statistics.
neon.backpressure_lsns() — LSN positions used in backpressure calculation.

The exact list is the union of definitions across neon--*.sql files. pgxn/neon_test_utils/ adds more for tests.

How it's loaded

The compute_ctl writes shared_preload_libraries = 'neon' (and a few other Neon libs) into postgresql.conf before starting Postgres. Postgres loads the .so, which calls _PG_init, registers hooks, and starts background workers (e.g. the walproposer worker).

Communicator (`pgxn/neon/communicator/`)

A Cargo workspace member crate (Cargo crate name communicator) that provides an async, multiplexed pageserver client implemented in Rust. It's compiled into a static library and FFI-linked from the C extension. The motivation is simple: the libpq client in C does not multiplex well, and the gRPC client only exists in Rust. The communicator hides both.

Per-Postgres-version compatibility

Postgres APIs change slightly between major versions (MyProcNumber, PGIOAlignedBlock, hook signatures, etc.). neon_pgversioncompat.h and neon_pgversioncompat.c provide compile-time shims. Recent commits like Add compatibility macros for MyProcNumber and PGIOAlignedBlock (#12715) and Fix postgres version compatibility macros (#12658) show the pattern in motion.

Key source files

File	Purpose
`pgxn/neon/neon.c`	`_PG_init`, GUC registration, top-level glue.
`pgxn/neon/pagestore_smgr.c`	smgr replacement. Hot path.
`pgxn/neon/libpagestore.c`	libpq pageserver client.
`pgxn/neon/communicator.c` + `communicator/`	Rust async pageserver client behind FFI.
`pgxn/neon/walproposer.c`	Safekeeper proposer protocol implementation.
`pgxn/neon/walproposer_pg.c`	Postgres glue for walproposer.
`pgxn/neon/file_cache.c`	Local on-disk cache.
`pgxn/neon/relsize_cache.c`	Relation size cache.
`pgxn/neon/neon_lwlsncache.c`	Last-written-LSN cache.
`pgxn/neon/neon_ddl_handler.c`	DDL-event hooks.
`pgxn/neon/neon--*.sql`	Version chain of installed SQL objects.
`pgxn/neon_walredo/`	Library used in the pageserver's redo subprocess.
`docs/core_changes.md`	Documents the upstream Postgres changes that pair with this extension.