Storage engine

The storage subsystem lives under src/Storages/. It is a registry of table engines — concrete subclasses of IStorage — plus the orchestration code (mutations, alters, partitioning, projections, TTL, virtual columns) shared between them.

The IStorage abstraction

IStorage (src/Storages/IStorage.h, ~35 KB) defines the contract every table engine implements. The big methods are:

• read(...) — produces a QueryPlan step that reads rows.
• write(...) — accepts a Block stream and writes data.
• alter(...) — applies an ALTER (column add/drop, mutation, settings change).
• mutate(...) / mutationEntry(...) — schedules an asynchronous mutation.
• optimize(...) / truncate(...) / dropPartition(...) — operational primitives.
• getInMemoryMetadataPtr() — the table schema, indices, projections, TTLs, settings.
• Virtual column declarations.

StorageFactory (src/Storages/StorageFactory.cpp) maps the engine name in CREATE TABLE ... ENGINE = X(...) to a constructor. Engines register themselves through registerStorageX() functions invoked from src/Storages/registerStorages.cpp.

Catalog of engines

Family	Engines	Source
MergeTree	MergeTree, ReplicatedMergeTree, Replacing, Aggregating, Summing, Collapsing, VersionedCollapsing, Graphite (and replicated variants of all)	src/Storages/MergeTree/, see MergeTree, Replicated MergeTree
Log family	Log, TinyLog, StripeLog	StorageLog.cpp, StorageStripeLog.cpp
Memory	Memory, Set, Join, Buffer, Null, Merge, View, MaterializedView, WindowView, LiveView	StorageMemory.cpp, etc.
Object storage	S3, Azure, HDFS, Iceberg, DeltaLake, Hudi, Hive, file-cluster variants	src/Storages/ObjectStorage/, src/Storages/Hive/
File systems	File, URL, FileLog	StorageFile.cpp, StorageURL.cpp, src/Storages/FileLog/
Streaming	Kafka, RabbitMQ, NATS, S3Queue/AzureQueue	src/Storages/Kafka/, src/Storages/NATS/, src/Storages/RabbitMQ/, src/Storages/ObjectStorageQueue/
External DBs	MySQL, PostgreSQL, MongoDB, Redis, SQLite, XDBC (ODBC bridge), KeeperMap, ArrowFlight	StorageMySQL.cpp, StoragePostgreSQL.cpp, StorageMongoDB.cpp, StorageRedis.cpp, StorageSQLite.cpp, StorageXDBC.cpp, StorageKeeperMap.cpp, StorageArrowFlight.cpp
Misc	Dictionary, GenerateRandom, Executable, URL, TimeSeries, RocksDB	StorageDictionary.cpp, StorageGenerateRandom.cpp, StorageExecutable.cpp, StorageTimeSeries.cpp, src/Storages/RocksDB/

This catalog is open-ended; new engines slot in by adding a registerStorageFoo() to registerStorages.cpp.

Shared infrastructure

The flat .cpp/.h files at the root of src/Storages/ are the cross-cutting pieces every engine uses:

• StorageInMemoryMetadata.cpp — the table's columns, indices, projections, TTLs, settings, sampling key.
• ColumnsDescription.cpp — column definitions, defaults, codecs, comments.
• IndicesDescription.cpp, ProjectionsDescription.cpp, ConstraintsDescription.cpp, TTLDescription.cpp, KeyDescription.cpp.
• AlterCommands.cpp, MutationCommands.cpp, PartitionCommands.cpp — ALTER/mutation/partition DSL.
• VirtualColumnUtils.cpp, VirtualColumnsDescription.cpp — virtual columns (_part, _partition_id, _path, _file, _shard_num, _offset).
• KVStorageUtils.cpp — pushdown for KV-style engines (Redis, KeeperMap, MongoDB shard-key reads).
• SelectQueryInfo.h — the bag of context passed into IStorage::read.
• StorageSnapshot.cpp — a stable snapshot of metadata pinned for a query.
• IPartitionStrategy.cpp, HivePartitioningUtils.cpp — Hive-style partitioning for object-storage tables.

Reading

graph LR
    Q[QueryPlan step ReadFromX] --> RD[storage.read]
    RD --> Pipe[Pipeline of source processors]
    Pipe --> Filter[Filter pushdown]
    Filter --> Out[Chunk stream]

Engines return either a Pipe (a small set of source processors) or a QueryPlan step. MergeTree returns ReadFromMergeTree, an explicit step that performs primary-key analysis, granule selection, and parallel reading. Other engines typically return a simpler pipe.

Writing

graph LR
    Q[INSERT] --> Sink[storage.write]
    Sink --> SinkProc[A sink processor]
    SinkProc --> Engine[Engine-specific commit]
    Engine --> Disk[(IDisk / external system)]

The interpreter creates a sink processor by calling IStorage::write. For MergeTree, the sink writes a new part to a temp directory and atomically renames it into place. For external engines, it forwards rows over the wire.

Mutations

ALTER UPDATE/ALTER DELETE are mutations: asynchronous rewrites that the engine schedules. MergeTree records a MergeTreeMutationEntry and processes it through a merge that rewrites only the affected ranges. Other engines either implement mutations natively (Memory) or refuse them.

Lightweight delete (DELETE FROM ...) is special: it appends a _row_exists mask column and is later folded into the data on merges.

TTL

TTLDescription describes a column- or row-level TTL expression. During a merge, parts of the data outside the TTL window are dropped or moved to another disk per the TO DISK/TO VOLUME clause. MergeTreePartsMover.cpp is the executor.

Projections

Projections are alternative materialized layouts of a table stored alongside the main parts. ProjectionsDescription.cpp and MergeTreeData.cpp handle storage; the optimizer selects them in Planner/OptimizeProjections.cpp. See also Projections and TTL.

Where to read next

• MergeTree — the workhorse engine.
• Replicated MergeTree — replication via Keeper.
• Other engines — Distributed, object storage, streaming, external DBs.
• Replication, Distributed queries, Materialized views, Object storage tables.