clickhouse/clickhouse
Data types and columns
ClickHouse's type system is split across three directories:
src/Core/— fundamental types:Field,Block,BlockInfo,ColumnsWithTypeAndName,Settings,ServerSettings,MySQLProtocol,Joins.h,iostream_debug_helpers.h,ProtocolDefines.h.src/DataTypes/— typed metadata classes (DataTypeUInt32,DataTypeArray,DataTypeDateTime, …) plus serialization.src/Columns/— typed columnar containers (ColumnVector<T>,ColumnString,ColumnArray,ColumnNullable,ColumnLowCardinality, …).
Conceptual model
A column has both a runtime container (IColumn) and a static description (IDataType). The two layers serve different purposes:
IDataTypeknows the type's meaning: how to format it, how to (de)serialize it, what subcolumns it has, the bit-width and signedness.IColumnknows the storage: a contiguous array of values, with operations to insert, sort, compare, hash, filter, and replicate.
Field is the boxed scalar form (a std::variant-style holder) used in interpretation, parameter binding, and Block::getByName(...).
Block is (name, type, column) triples — the internal exchange format between operators in the legacy interpreter.
Chunk (from src/Processors/) is (columns, row count) — the lighter exchange format used by the processor graph. It often lacks names; processors use a separate Block header to describe schema.
Core types
src/Core/Field.h defines the variant. Notable types it carries: integers, floats, strings, arrays, tuples, maps, decimals, UUIDs, dates, times, booleans, JSON, IPv4/IPv6, intervals, AggregateFunctionStateData, custom types.
DataTypes
| Group | Examples | Files |
|---|---|---|
| Numbers | UInt8, Int64, Float32, Decimal(P,S), BFloat16 |
DataTypeNumber.cpp, DataTypeDecimalBase.cpp |
| Strings | String, FixedString(N) |
DataTypeString.cpp, DataTypeFixedString.cpp |
| Date/Time | Date, Date32, DateTime, DateTime64, Time, Time64, Interval |
DataTypeDate.cpp, DataTypeDateTime.cpp, DataTypeInterval.cpp |
| Composite | Array(T), Tuple(...), Map(K, V), Nested(...) |
DataTypeArray.cpp, DataTypeTuple.cpp, DataTypeMap.cpp, DataTypeNested.cpp |
| Modifiers | Nullable(T), LowCardinality(T), Variant(T1, T2, ...), Dynamic |
DataTypeNullable.cpp, DataTypeLowCardinality.cpp, DataTypeVariant.cpp, DataTypeDynamic.cpp |
| Specials | UUID, IPv4, IPv6, Object(JSON), JSON, Enum8, Enum16, Bool |
DataTypeUUID.cpp, DataTypeIPv4andIPv6.cpp, DataTypeObject*.cpp, DataTypeEnum.cpp |
| Storage helpers | AggregateFunction(name, args), SimpleAggregateFunction(...), Custom* |
DataTypeAggregateFunction.cpp, DataTypeCustomFixedName.cpp |
Each IDataType implements getDefaultSerialization(), returning an ISerialization strategy that handles binary/text/JSON/CSV/TSV/etc. For composite types the serialization recursively uses sub-serializations.
Subcolumns
Composite types (Array, Tuple, Map, Nullable, LowCardinality, Variant, Dynamic, JSON) expose subcolumns. For example, Tuple(a UInt32, b String) has .a and .b. Array(Int) has .size0 and .array_elements. JSON has dynamically-discovered key paths. The MergeTree storage uses subcolumns to read only the parts of a column that a query touches — e.g. SELECT obj.user.id FROM t WHERE ... reads only the obj.user.id subcolumn from disk.
SerializationInfo.cpp records per-column serialization metadata in MergeTree parts so subcolumns survive across versions and merges.
Columns
IColumn (src/Columns/IColumn.h) is the abstract base. Important concrete columns:
| Column | Storage | Notes |
|---|---|---|
ColumnVector<T> |
PaddedPODArray<T> |
The integer/float column. |
ColumnString |
offsets + chars buffer | Variable-length strings. |
ColumnFixedString |
one big buffer of N-byte slots | |
ColumnArray |
offsets + nested column | Variable-length arrays. |
ColumnTuple |
n nested columns | |
ColumnMap |
array of tuples (under the hood) | |
ColumnNullable |
nested column + null map | |
ColumnLowCardinality |
dictionary index + dictionary | Decoded only at the edges of the pipeline. |
ColumnVariant |
list of nested + discriminators | |
ColumnDynamic |
variant of run-time-discovered types | |
ColumnObject |
dynamic key/value structure for JSON | |
ColumnConst |
wrapper holding a single value broadcast to N rows | |
ColumnSparse |
sparse column with a default value | Saves space and CPU when most values are equal to the default. |
ColumnAggregateFunction |
array of pointers to states | Stores intermediate aggregate states. |
ColumnDecimal<T> |
similar to ColumnVector |
Carries the scale and precision. |
PaddedPODArray (in src/Common/PODArray.h) is the workhorse buffer: aligned, padded, and SIMD-friendly.
ColumnsDescription and Block
ColumnsDescription (src/Storages/ColumnsDescription.cpp) holds the per-table column metadata — name, type, default expression, codec, comment, materialized/aliased status. Block::getByName() looks up by name; Block::insert() appends. The Block header (column names + types but no rows) is the schema contract between operators.
Field, casting, and conversion
src/Functions/CastOverloadResolver.cpp is the meta-function that converts between types. accurateCast, accurateCastOrNull, accurateCastOrDefault are public variants. Internally each pair (source, target) has a convertOne... helper specialized for the column types involved.
Settings
src/Core/Settings.cpp defines hundreds of runtime settings via X-macros. src/Core/ServerSettings.cpp does the same for server-wide settings. src/Core/SettingsEnums.cpp carries enum types. They are surfaced through system.settings and system.server_settings.
Performance notes
- Inner loops should operate on raw
T*fromgetData().data(), not via[]. - Use
IColumn::insertRangeFrom/insertManyFrom/replicaterather than per-row inserts. LowCardinalityis decoded only when needed — operators that can stay in dictionary form often do.ColumnSparseis dispatched explicitly — most operators have a fast path for sparse columns.
Related pages
- Functions, Aggregate functions
- Compression — column codecs.
- Formats — readers/writers built on top of
ISerialization. - Reference → data models — full type catalogue.
Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.