Replicated MergeTree

ReplicatedMergeTree (and its variants: ReplicatedSummingMergeTree, ReplicatedAggregatingMergeTree, etc.) extends MergeTree with multi-replica replication coordinated through clickhouse-keeper or ZooKeeper. Source: src/Storages/StorageReplicatedMergeTree.cpp (~520 KB) and src/Storages/MergeTree/Replicated*.

How replication works

Each replicated table has a Keeper path of the form /clickhouse/tables/{shard}/{database}/{table}/. Under it Keeper stores:

.../
├── log/                  # the global operation log
├── replicas/<replica>/   # per-replica state (queue, last-known log entry)
├── blocks/               # deduplication of recent inserts
├── columns/              # current column set
├── metadata/             # current schema digest
├── leader_election/      # ephemeral leader markers
├── nonincrement_block_numbers/  # for MOVE PARTITION
├── quorum/               # in-flight quorum inserts
└── ...

graph TD
    Insert[INSERT] --> Sink[ReplicatedMergeTreeSink]
    Sink --> Block[Write part to local disk]
    Sink --> Dedup{Block ID seen<br/>in /blocks?}
    Dedup -->|yes| Drop[Drop, return]
    Dedup -->|no| Log[Add LogEntry to /log/]
    Log --> Replicas[All replicas]
    Replicas --> Queue[Pull entry into /replicas/x/queue/]
    Queue --> Exec{Local part?}
    Exec -->|yes| Done[Activate the part]
    Exec -->|no| Fetch[Fetch via DataPartsExchange.cpp]
    Fetch --> Done
    Done --> Merge[Future merges go through the same log]

Key classes

Class / file	Purpose
StorageReplicatedMergeTree.cpp	The engine itself. Owns the Keeper handle, queue thread, restart thread, leader election, and the entire replication state machine.
ReplicatedMergeTreeQueue.cpp	The per-replica queue of pending log entries.
ReplicatedMergeTreeLogEntry.cpp	Operation kinds: GET_PART, MERGE_PARTS, MUTATE_PART, DROP_RANGE, ATTACH_PART, CLEAR_COLUMN, REPLACE_RANGE, SYNC_PINNED_PART_UUIDS, ALTER_METADATA, MOVE_PART.
ReplicatedMergeTreeSink.cpp	Writes new parts and registers them in /log/. ~60 KB of careful synchronization.
ReplicatedMergeTreePartCheckThread.cpp	Periodic verification of local parts against Keeper.
ReplicatedMergeTreeRestartingThread.cpp	Reconnect and re-initialize on Keeper session loss.
ReplicatedMergeTreeAttachThread.cpp	Synchronous startup attach, brings a new replica up to date.
ReplicatedMergeTreeMergeStrategyPicker.cpp	Picks which replica should run a given merge.
ReplicatedMergeTreePartHeader.cpp	Compact metadata header used in /blocks/.
ReplicatedMergeTreeQuorumEntry.cpp	Quorum-insert tracking.
ReplicatedMergeTreeTableMetadata.cpp	The replicated metadata digest used for schema consistency checks.
DataPartsExchange.cpp	The HTTP service that ships parts between replicas.
EphemeralLockInZooKeeper.cpp	Helper that takes ephemeral exclusive locks for partitioning operations.

Operations

Insert

ReplicatedMergeTreeSink::commitPart:

1. Compute the block ID (hash(rows) mod a window for deduplication).
2. Take /blocks/<id> exclusively. If present, the block was inserted before — drop and return success.
3. Write the part locally.
4. Add a GET_PART log entry to /log/.
5. Wait for quorum if requested (insert_quorum).
6. Other replicas pull the entry, fetch the part from this replica via the exchange HTTP service, and activate it.

Merge

A replica that wins leader election (or that is selected by MergeStrategyPicker) selects parts to merge, writes a MERGE_PARTS log entry naming the resulting part, and runs the merge. All other replicas either:

• Run the merge locally too (if they have the source parts and the policy allows).
• Or fetch the resulting part from a peer.

The first to finish wins; redundant work is cancelled.

Mutation

A MUTATE_PART log entry is added per affected part. Replicas execute or fetch.

ALTER

Most ALTERs are recorded as ALTER_METADATA entries that bump the table's metadata version in Keeper. Subsequent local operations check the version before proceeding. Schema-changing alters that rewrite data go through mutations.

Failure handling

Lost Keeper sessions trigger ReplicatedMergeTreeRestartingThread which detaches the table from Keeper, releases ephemerals, drops cached state, and re-attaches once the session is back. Stuck queue entries are surfaced in system.replication_queue. system.replicas shows replica-level health (is_session_expired, future_parts, parts_to_check, queue_size, inserts_in_queue, etc.).

SYSTEM RESTART REPLICA <table> and SYSTEM SYNC REPLICA <table> are the operator escape hatches.

Quorum inserts

insert_quorum=N (or auto for majority) waits until N replicas have acknowledged the part. Without quorum, an insert is durable on at least the originating replica and is replicated asynchronously.

Parallel replicas (read path)

A separate feature, parallel replicas, splits a single SELECT across replicas of the same shard. Implemented in ParallelReplicasReadingCoordinator.cpp and RequestResponse.cpp. The leader announces a marker plan; followers pull mark ranges. See Distributed queries.

Reading

The read path is identical to plain MergeTree. The replicated layer adds:

• Choice of the freshest replica when Distributed queries it.
• The parallel-replicas coordinator described above.

Settings

Replication-specific settings (in addition to all MergeTree ones):

• replicated_* — backoff, fetch concurrency, parallel fetch, deduplication window.
• insert_quorum, insert_quorum_timeout.
• select_sequential_consistency — wait for the latest log entry before reading.
• replicated_max_ratio_of_wrong_parts — bring-up tolerance.
• zookeeper_session_expiration_check_period, replica_can_become_leader.

Common diagnostics

• system.replicas — per-replica health.
• system.replication_queue — pending operations.
• system.zookeeper_log — every Keeper request issued by this server.
• system.parts filtered by replica_path.
• SYSTEM RESTART REPLICA <table>.
• SYSTEM SYNC REPLICA <table> — block until the queue is drained.
• SYSTEM DROP REPLICA <name> — force-remove a dead replica from Keeper.

Entry points for modification

• New replication operation → extend ReplicatedMergeTreeLogEntry with a new type and handle it in StorageReplicatedMergeTree::executeLogEntry.
• New replication setting → MergeTreeSettings.cpp.
• New restart/repair logic → ReplicatedMergeTreeRestartingThread.cpp and ReplicatedMergeTreePartCheckThread.cpp.

Related pages

• MergeTree
• Coordination
• Replication feature page
• Backups