Compression

src/Compression/ implements ClickHouse's column codecs. The result is a chain of fast lossless compressors that understand the shape of column data, not just byte streams.

Codec catalog

Codec	Strength	File / class
LZ4	Default. Very fast.	CompressionCodecLZ4.cpp
LZ4HC(level)	LZ4 with higher compression at the cost of speed.	CompressionCodecLZ4.cpp
ZSTD(level)	Default for cold data. Higher ratio at moderate CPU cost.	CompressionCodecZSTD.cpp
ZSTD_QAT(level)	ZSTD with Intel QuickAssist hardware acceleration.	CompressionCodecZSTDQAT.cpp
Delta(N)	Encodes the per-element delta. Pairs well with LZ4/ZSTD.	CompressionCodecDelta.cpp
DoubleDelta	Two-level delta, ideal for monotonically-increasing timestamps.	CompressionCodecDoubleDelta.cpp
T64	Bit-packing for integers in narrow ranges.	CompressionCodecT64.cpp
Gorilla	The Facebook Gorilla codec for floats. Excellent for time-series doubles.	CompressionCodecGorilla.cpp
FPC	High-throughput floating-point codec.	CompressionCodecFPC.cpp
GCD	Divides by GCD then encodes the residue.	CompressionCodecGCD.cpp
NONE	No compression.	CompressionCodecNone.cpp
Multiple	Chains several codecs.	CompressionCodecMultiple.cpp
Encrypted(AES_*)	AES-128/AES-256 GCM encryption. Used in <encryption_codecs>.	CompressionCodecEncrypted.cpp
DEFLATE_QPL	Intel QPL hardware DEFLATE.	CompressionCodecDeflateQpl.cpp
LZSSE2, LZSSE4, LZSSE8	Older fast codecs (legacy).	CompressionCodecLZSSE*.cpp

CompressionFactory.cpp is the registry. Codecs declare themselves via registerCodec*. Per-column codecs are configured in CREATE TABLE:

CREATE TABLE events (
    timestamp DateTime CODEC(DoubleDelta, ZSTD(3)),
    user_id   UInt64   CODEC(T64, LZ4),
    payload   String   CODEC(ZSTD(6))
) ENGINE = MergeTree ORDER BY (user_id, timestamp);

The chain runs left-to-right on write and right-to-left on read.

Block layout

The on-disk column file is a sequence of compressed blocks. Each block has a small header:

| 1 byte codec id | 4 bytes uncompressed size | 4 bytes compressed size | <compressed data> |

*.mrk2 / *.mrk3 mark files index into this stream so a reader can seek to a granule without decompressing earlier blocks. The header layout is implemented in src/Compression/CompressionCodec*.cpp and CompressedReadBuffer.cpp/CompressedWriteBuffer.cpp (in src/IO/).

Hardware acceleration

Some codecs have hardware-accelerated variants:

• QAT — Intel QuickAssist for ZSTD and DEFLATE. Detected at runtime in CompressionCodecZSTDQAT.cpp / CompressionCodecDeflateQpl.cpp.
• AVX-512 / SSE — most codecs use auto-vectorized inner loops; T64, Gorilla, and FPC ship explicit SIMD paths.

Selection guidance

The codec selector (src/Storages/CompressionCodecSelector.h) picks defaults based on type and size. Manually-overridden settings:

• network_compression_method, network_zstd_compression_level — for the wire protocol.
• compression, min_compress_block_size, max_compress_block_size — server-side defaults for new parts.
• <compression> blocks in config.xml — per-column-pattern policies (e.g. "for any column matching event_time, use DoubleDelta + ZSTD").

Wire compression

The native TCP protocol compresses each block. network_compression_method defaults to LZ4 for low-latency exchanges; setting it to ZSTD gives better ratios at the cost of CPU.

Encrypted codecs

<encryption_codecs> in config.xml configures one or more named keys; columns can then use CODEC(AES_128_GCM_SIV, ZSTD) or similar to encrypt at rest. CompressionCodecEncrypted.cpp derives a per-block IV from a counter to keep AES-GCM safe under high write rates.

Adding a codec

1. Subclass ICompressionCodec.
2. Implement doCompressData, doDecompressData, getMaxCompressedDataSize.
3. Pick a unique 1-byte id (must be stable for backward compatibility).
4. Register via void registerCodecFoo(CompressionCodecFactory & factory) and call from registerCompressionCodecs.cpp.

Related pages

• IO and disks — CompressedReadBuffer / CompressedWriteBuffer.
• MergeTree — primary consumer.
• Native TCP — wire compression.