Open-Source Wikis

/

ClickHouse

/

Systems

/

Data types and columns

clickhouse/clickhouse

Data types and columns

ClickHouse's type system is split across three directories:

  • src/Core/ — fundamental types: Field, Block, BlockInfo, ColumnsWithTypeAndName, Settings, ServerSettings, MySQLProtocol, Joins.h, iostream_debug_helpers.h, ProtocolDefines.h.
  • src/DataTypes/ — typed metadata classes (DataTypeUInt32, DataTypeArray, DataTypeDateTime, …) plus serialization.
  • src/Columns/ — typed columnar containers (ColumnVector<T>, ColumnString, ColumnArray, ColumnNullable, ColumnLowCardinality, …).

Conceptual model

A column has both a runtime container (IColumn) and a static description (IDataType). The two layers serve different purposes:

  • IDataType knows the type's meaning: how to format it, how to (de)serialize it, what subcolumns it has, the bit-width and signedness.
  • IColumn knows the storage: a contiguous array of values, with operations to insert, sort, compare, hash, filter, and replicate.

Field is the boxed scalar form (a std::variant-style holder) used in interpretation, parameter binding, and Block::getByName(...).

Block is (name, type, column) triples — the internal exchange format between operators in the legacy interpreter.

Chunk (from src/Processors/) is (columns, row count) — the lighter exchange format used by the processor graph. It often lacks names; processors use a separate Block header to describe schema.

Core types

src/Core/Field.h defines the variant. Notable types it carries: integers, floats, strings, arrays, tuples, maps, decimals, UUIDs, dates, times, booleans, JSON, IPv4/IPv6, intervals, AggregateFunctionStateData, custom types.

DataTypes

Group Examples Files
Numbers UInt8, Int64, Float32, Decimal(P,S), BFloat16 DataTypeNumber.cpp, DataTypeDecimalBase.cpp
Strings String, FixedString(N) DataTypeString.cpp, DataTypeFixedString.cpp
Date/Time Date, Date32, DateTime, DateTime64, Time, Time64, Interval DataTypeDate.cpp, DataTypeDateTime.cpp, DataTypeInterval.cpp
Composite Array(T), Tuple(...), Map(K, V), Nested(...) DataTypeArray.cpp, DataTypeTuple.cpp, DataTypeMap.cpp, DataTypeNested.cpp
Modifiers Nullable(T), LowCardinality(T), Variant(T1, T2, ...), Dynamic DataTypeNullable.cpp, DataTypeLowCardinality.cpp, DataTypeVariant.cpp, DataTypeDynamic.cpp
Specials UUID, IPv4, IPv6, Object(JSON), JSON, Enum8, Enum16, Bool DataTypeUUID.cpp, DataTypeIPv4andIPv6.cpp, DataTypeObject*.cpp, DataTypeEnum.cpp
Storage helpers AggregateFunction(name, args), SimpleAggregateFunction(...), Custom* DataTypeAggregateFunction.cpp, DataTypeCustomFixedName.cpp

Each IDataType implements getDefaultSerialization(), returning an ISerialization strategy that handles binary/text/JSON/CSV/TSV/etc. For composite types the serialization recursively uses sub-serializations.

Subcolumns

Composite types (Array, Tuple, Map, Nullable, LowCardinality, Variant, Dynamic, JSON) expose subcolumns. For example, Tuple(a UInt32, b String) has .a and .b. Array(Int) has .size0 and .array_elements. JSON has dynamically-discovered key paths. The MergeTree storage uses subcolumns to read only the parts of a column that a query touches — e.g. SELECT obj.user.id FROM t WHERE ... reads only the obj.user.id subcolumn from disk.

SerializationInfo.cpp records per-column serialization metadata in MergeTree parts so subcolumns survive across versions and merges.

Columns

IColumn (src/Columns/IColumn.h) is the abstract base. Important concrete columns:

Column Storage Notes
ColumnVector<T> PaddedPODArray<T> The integer/float column.
ColumnString offsets + chars buffer Variable-length strings.
ColumnFixedString one big buffer of N-byte slots
ColumnArray offsets + nested column Variable-length arrays.
ColumnTuple n nested columns
ColumnMap array of tuples (under the hood)
ColumnNullable nested column + null map
ColumnLowCardinality dictionary index + dictionary Decoded only at the edges of the pipeline.
ColumnVariant list of nested + discriminators
ColumnDynamic variant of run-time-discovered types
ColumnObject dynamic key/value structure for JSON
ColumnConst wrapper holding a single value broadcast to N rows
ColumnSparse sparse column with a default value Saves space and CPU when most values are equal to the default.
ColumnAggregateFunction array of pointers to states Stores intermediate aggregate states.
ColumnDecimal<T> similar to ColumnVector Carries the scale and precision.

PaddedPODArray (in src/Common/PODArray.h) is the workhorse buffer: aligned, padded, and SIMD-friendly.

ColumnsDescription and Block

ColumnsDescription (src/Storages/ColumnsDescription.cpp) holds the per-table column metadata — name, type, default expression, codec, comment, materialized/aliased status. Block::getByName() looks up by name; Block::insert() appends. The Block header (column names + types but no rows) is the schema contract between operators.

Field, casting, and conversion

src/Functions/CastOverloadResolver.cpp is the meta-function that converts between types. accurateCast, accurateCastOrNull, accurateCastOrDefault are public variants. Internally each pair (source, target) has a convertOne... helper specialized for the column types involved.

Settings

src/Core/Settings.cpp defines hundreds of runtime settings via X-macros. src/Core/ServerSettings.cpp does the same for server-wide settings. src/Core/SettingsEnums.cpp carries enum types. They are surfaced through system.settings and system.server_settings.

Performance notes

  • Inner loops should operate on raw T* from getData().data(), not via [].
  • Use IColumn::insertRangeFrom / insertManyFrom / replicate rather than per-row inserts.
  • LowCardinality is decoded only when needed — operators that can stay in dictionary form often do.
  • ColumnSparse is dispatched explicitly — most operators have a fast path for sparse columns.

Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.

Data types and columns – ClickHouse wiki | Factory