IO and disks

ClickHouse has its own I/O abstraction layered above raw filesystem and HTTP/cloud APIs. Two directories carry it:

• src/IO/ — buffered streams, compression wrappers, network buffers, S3/Azure HTTP clients.
• src/Disks/ — virtual disks (IDisk) and object-storage adapters (IObjectStorage).

src/IO/ — buffered streams

The base classes are ReadBuffer and WriteBuffer. Both manage a fixed-size contiguous buffer and a virtual nextImpl() that fills/flushes it. Layered wrappers compose into rich pipelines.

Class	Purpose
ReadBuffer, WriteBuffer	Base.
ReadBufferFromFile, WriteBufferFromFile	Local file I/O via pread/pwrite/io_uring.
ReadBufferFromHTTP, WriteBufferFromHTTP	HTTP body.
ReadBufferFromS3, WriteBufferFromS3	S3-specific (uses AWS SDK in contrib/aws*).
ReadBufferFromAzureBlobStorage, WriteBufferFromAzureBlobStorage	Azure-specific.
ReadBufferFromHDFS, WriteBufferFromHDFS	HDFS-specific.
CompressedReadBuffer, CompressedWriteBuffer	Wraps a buffer with column-codec compression.
LimitReadBuffer, BoundedReadBuffer, LimitedSeekableReadBuffer	Slicing and seek emulation.
BrotliReadBuffer, LZMAInflatingReadBuffer, ZstdInflatingReadBuffer, Bzip2ReadBuffer, XzInflatingReadBuffer, ZlibInflatingReadBuffer, LZ4InflatingReadBuffer, Lz4DeflatingWriteBuffer, ZstdDeflatingWriteBuffer	Format-level compression.
AsynchronousReadBufferFromFileWithDescriptorsCache, ParallelReadBuffer	Async / parallel reads.
MMapReadBufferFromFile	mmap-backed read.
EmptyReadBuffer, NullWriteBuffer, LimitWriteBuffer	Trivial endpoints.
BufferBase, Progress.h, WithFileSize.h	Shared helpers.
S3/, Azure/, HDFS/ subdirs	Vendor-specific clients.
Resource/	A read/write rate limiter and resource-tracking layer.

Read scenarios compose pipelines like:

auto file_buf = std::make_unique<ReadBufferFromS3>(s3_client, bucket, key, ...);
auto cache_buf = std::make_unique<CachedOnDiskReadBufferFromFile>(std::move(file_buf), cache, ...);
auto compressed_buf = std::make_unique<CompressedReadBuffer>(*cache_buf);

Each layer adds buffering, caching, decompression, or progress accounting without copying data.

src/Disks/ — virtual disks

IDisk (Disks/IDisk.h) is a logical disk handle. It exposes filesystem-like operations: exists, isFile, listFiles, readFile, writeFile, removeFile, removeDirectory, moveFile, getFileSize, …, plus getName(), getReservation(...) (for storage-policy capacity tracking), and supportsZeroCopyReplication().

Concrete disks:

Disk	Backing	File
DiskLocal	local filesystem	DiskLocal.cpp
DiskEncrypted	wraps another disk with AES	DiskEncrypted.cpp
DiskCacheWrapper	LRU read cache around another disk	DiskCacheWrapper.cpp
DiskWebServer	read-only HTTP-served directory	DiskWebServer.cpp
DiskObjectStorage	wraps an IObjectStorage (S3/Azure/HDFS)	ObjectStorages/DiskObjectStorage.cpp

The "virtual disk" pattern means MergeTree, BACKUP, and the rest of the engine use the same API regardless of where data lives.

IObjectStorage

Disks/ObjectStorages/IObjectStorage.h is the lower-level abstraction for S3/Azure/HDFS. It exposes methods like listObjects, getObject, putObject, removeObject, plus stream readers/writers. Implementations:

• S3ObjectStorage (S3, MinIO, R2, GCS via S3 API).
• AzureObjectStorage.
• HDFSObjectStorage.
• LocalObjectStorage — a "fake" object store backed by a local directory; used in tests.
• WebObjectStorage — read-only HTTP.

A DiskObjectStorage is an IDisk that delegates to one of these. The metadata (the IDisk filesystem-style tree) is stored either locally or in another IObjectStorage.

Caching

Three caches matter:

1. Page cache (UncompressedCache, MarkCache) — in-memory caches for hot uncompressed blocks and mark files. Live in src/Storages/MergeTree/.
2. FilesystemCache (src/Interpreters/Cache/) — a disk-backed LRU cache that sits in front of remote disks. Used by DiskCacheWrapper / CachedOnDiskReadBufferFromFile. Has its own settings, eviction policy, and system.filesystem_cache table.
3. Query result cache — caches the final output of a query (Interpreters/Cache/QueryCache.cpp).

The FilesystemCache is what makes "data on S3" fast. A typical setup pins frequently-accessed parts on local NVMe while the master copy lives in object storage.

Storage policies

A storage policy is a named ladder of Volumes, each of which is a list of IDisks. New parts land on the first volume; TTL TO DISK / TO VOLUME rules move parts later. Configured under <storage_configuration><policies> of config.xml. See Configuration.

Resource throttling

src/IO/Resource/ provides token-bucket throttlers wired into the read/write path. Settings like max_remote_read_network_bandwidth, max_local_write_disk_bandwidth, etc. flow through them.

File reading method

ClickHouse picks one of:

• read (synchronous pread).
• pread_threadpool (off the I/O thread pool).
• io_uring (modern Linux, opt-in).
• mmap (for hot small files).
• nmap and others.

The choice is per-disk via local_disk_check_period_ms and read_method.

Entry points for modification

• New disk → subclass IDisk and register a factory in DiskFactory.cpp.
• New object storage → subclass IObjectStorage and register in ObjectStorageFactory.cpp.
• New buffer wrapper → derive from ReadBuffer / WriteBuffer.
• New compression for streamed reads → see Compression.

Related pages

• Compression
• MergeTree — primary user of disks and caches.
• Backups — also uses IDisk.
• Object storage tables