Replication

PostgreSQL replicates data using its own WAL stream as the transport. Two flavors are supported: physical (a.k.a. streaming) replication that ships raw WAL bytes, and logical replication that decodes WAL into row changes and reapplies them on the subscriber. Both are implemented in src/backend/replication/.

Source layout

src/backend/replication/
├── README                  # design notes
├── libpqwalreceiver/       # libpq-based wal-receiver implementation
├── logical/                # logical decoding + apply worker
│   ├── decode.c
│   ├── launcher.c
│   ├── logical.c
│   ├── logicalfuncs.c
│   ├── origin.c
│   ├── proto.c
│   ├── reorderbuffer.c
│   ├── snapbuild.c
│   ├── tablesync.c
│   ├── worker.c
│   └── ...
├── pgoutput/               # built-in logical decoding output plugin
├── pgrepack/               # REPACK CONCURRENTLY support
├── repl_gram.y             # bison grammar for replication commands
├── repl_scanner.l          # flex scanner
├── slot.c                  # replication slot management
├── slotfuncs.c             # SQL functions: pg_create_*_slot, etc.
├── syncrep.c               # synchronous replication waits
├── walreceiver.c           # standby-side WAL receiver
├── walreceiverfuncs.c      # WAL receiver SQL functions
└── walsender.c             # primary-side WAL sender

Streaming replication

Streaming (physical) replication ships exact byte copies of WAL records from a primary to one or more standbys.

graph LR
    subgraph "Primary"
        Backends -->|XLogInsert| WAL[(pg_wal/)]
        WalSender["walsender.c<br/>(per replica)"] -->|read WAL records| WAL
    end
    subgraph "Standby"
        WalReceiver["walreceiver.c"] -->|write WAL| StandbyWAL[(pg_wal/)]
        Startup["startup.c<br/>(redo loop)"] -->|read & replay| StandbyWAL
        Startup -->|update buffers| Buffers["shared_buffers"]
    end
    WalSender ==>|streaming protocol| WalReceiver
    Standby -.->|hot standby query| Client[Client]

Walsender

walsender.c runs in a process forked by the postmaster when an incoming connection requests replication mode (replication=true in the startup packet). The walsender:

1. Authenticates the connection (using pg_hba.conf rules with the special replication keyword).
2. Reads replication commands using the small grammar in repl_gram.y: IDENTIFY_SYSTEM, START_REPLICATION, START_REPLICATION SLOT ..., CREATE_REPLICATION_SLOT, READ_REPLICATION_SLOT, etc.
3. For START_REPLICATION: sits in a loop reading WAL from pg_wal/ and shipping it over the protocol. Tracks the receiver's reported flush LSN for synchronous replication and slot bookkeeping.

Walreceiver

walreceiver.c runs as an auxiliary process on the standby. Started by the startup process when it transitions to streaming after replaying archive WAL. It:

1. Connects to the primary using libpqwalreceiver (a loadable shared library that wraps libpq).
2. Issues IDENTIFY_SYSTEM, then START_REPLICATION from the appropriate LSN.
3. Reads WAL bytes and writes them into the standby's pg_wal/.
4. Notifies the startup process via shared memory; the startup process's redo loop wakes and applies the new records.
5. Periodically sends feedback to the primary (LSNs of received, written, flushed, applied positions).

Hot standby

A standby can serve read-only queries while replaying WAL. The startup process keeps the proc array up to date with running-XACTs information from the WAL stream. User queries see only XIDs that are visible at the applied LSN; if a redo operation conflicts with a query (e.g., wants to drop a buffer the query has pinned), the conflict mechanism kicks in.

The conflict types include: pinned buffer needed, snapshot too old, conflict with RECOVERY_CONFLICT_LOCK, conflict with RECOVERY_CONFLICT_DATABASE (DROP DATABASE was replayed). See src/backend/storage/ipc/standby.c.

hot_standby_feedback lets the standby send its xmin upstream so the primary doesn't vacuum away rows the standby still wants. Tradeoff: bloat on the primary.

Synchronous replication

synchronous_commit = on, remote_apply, remote_write controls when the primary considers a transaction durable: only after the WAL has reached one or more synchronous standbys. synchronous_standby_names selects which standbys count. Source: src/backend/replication/syncrep.c.

Replication slots

Source: src/backend/replication/slot.c. A slot is server-side state that records "this consumer has confirmed up to LSN N." It causes the primary to refuse to recycle WAL beyond that point and to retain dead row versions newer than the consumer needs. Two flavors:

• Physical slots — for streaming replicas. Optional but standard.
• Logical slots — for logical decoding. Required.

Slots persist on disk and survive restart. The cost is that an unused slot will pin WAL forever (max_slot_wal_keep_size was added to limit that risk).

Logical replication

Logical replication ships row changes, not raw WAL bytes. The pipeline:

graph LR
    PWAL[(Primary WAL)] --> Decoder["logical/decode.c<br/>+ ReorderBuffer"]
    Decoder --> Plugin["Output plugin<br/>(pgoutput, ...)"]
    Plugin --> Walsender["walsender.c"]
    Walsender ==>|logical protocol| ApplyWorker["worker.c<br/>(apply worker)"]
    ApplyWorker --> SubscriberTables[(subscriber tables)]

Logical decoding

Source: src/backend/replication/logical/. The decoder reads WAL via xlogreader.c, classifies records by transaction, and feeds them into a reorder buffer (reorderbuffer.c) that buffers per-transaction changes until the transaction commits. At commit time, the buffered changes are streamed to the output plugin in commit order.

Key files:

File	Role
decode.c	Driver: pull records, dispatch to per-rmgr decode callbacks.
reorderbuffer.c	Per-transaction buffering, spill-to-disk, two-phase decode.
snapbuild.c	Build a "historic snapshot" — what the world looked like at any LSN.
origin.c	Replication origin tracking (avoid loops).
tablesync.c	Initial table sync via COPY before streaming changes.
worker.c	Apply worker: receives changes, applies them.
proto.c	Wire protocol used between walsender and apply worker.
launcher.c	Logical replication launcher: starts apply workers per subscription.

Output plugins are loadable libraries that turn ReorderBufferTXN into bytes. The built-in plugin is pgoutput (src/backend/replication/pgoutput/). Third-party plugins (wal2json, decoderbufs, etc.) are popular for CDC.

Publications and subscriptions

The SQL-level model:

• CREATE PUBLICATION pub FOR TABLE t1, t2; — declares which tables to publish.
• CREATE SUBSCRIPTION sub CONNECTION '...' PUBLICATION pub; — on the subscriber, creates a logical slot on the publisher and starts an apply worker.

pg_publication, pg_publication_rel, pg_subscription, pg_subscription_rel are the catalog tables.

Apply workers (logical/worker.c) are dynamic background workers spawned by the logical replication launcher (launcher.c). The launcher monitors pg_subscription; for each enabled subscription, it ensures one apply worker (and possibly per-table sync workers) is running.

Initial sync

When a subscription is created (or a new table is added to its publication), the apply worker spawns a table sync worker per table. Each runs COPY of the table's data, then catches up via streaming changes, then merges into the main apply worker's stream. Source: tablesync.c.

Conflicts

Logical replication can fail on the apply side if a row already exists, has been modified concurrently, or violates a constraint. The apply worker errors out and waits for the user to resolve (via ALTER SUBSCRIPTION ... SKIP or by adjusting the data). Newer versions add row-filter and column-list features to reduce conflicts.

Two-phase logical decoding

Logical decoding can stream a transaction's changes before commit, so a long transaction doesn't have to wait for COMMIT to start replicating. Used with streaming = on on the subscription. Source: reorderbuffer.c::ReorderBufferStreamTXN.

Replication commands

The replication protocol shares wire format with the regular protocol but uses a small command set defined in repl_gram.y. The commands are not SQL; they are a separate language. Examples:

IDENTIFY_SYSTEM
TIMELINE_HISTORY <tli>
SHOW <name>
START_REPLICATION [SLOT <name>] [PHYSICAL] <lsn> [TIMELINE <tli>]
START_REPLICATION SLOT <name> LOGICAL <lsn> [(option=value, ...)]
CREATE_REPLICATION_SLOT <name> { PHYSICAL | LOGICAL <plugin> } ...
DROP_REPLICATION_SLOT <name>
READ_REPLICATION_SLOT <name>
BASE_BACKUP [...]                  -- used by pg_basebackup

pg_basebackup itself uses these commands; it is essentially a libpq client that requests BASE_BACKUP and writes the resulting tar/plain stream to disk.

Failover

PostgreSQL ships the primitives for failover but not a full automated failover daemon (third-party tools like Patroni, repmgr, pg_auto_failover do that):

• pg_promote() — promote a standby to primary.
• recovery_target_timeline — recover up to / pick a particular timeline at fork points.
• pg_rewind — re-align an old primary to a new one, avoiding a full base backup.

Promotion bumps the timeline ID. The history file in pg_wal/ records the LSN where the timeline split.

Entry points for modification

• New replication command: repl_gram.y + walsender.c.
• New output plugin: implement the OutputPluginCallbacks API and ship as a shared library; load with shared_preload_libraries.
• New apply behavior (e.g., conflict resolution): logical/worker.c. Be prepared for the patch to take multiple commitfests.
• New replication slot kind: rare; touches slot.c and the WAL reservation mechanics in xlog.c.

For the WAL substrate underneath, see WAL and recovery. For the replication-side tooling (pg_basebackup, pg_receivewal), see Apps.