Formats

ClickHouse can read and write dozens of file/wire formats. The format infrastructure lives in two places:

• src/Formats/ — central registry, schema inference, format settings, common helpers.
• src/Processors/Formats/ — the actual IInputFormat / IOutputFormat implementations (each format is a processor).

Catalog (selection)

Format	Read	Write	File
TabSeparated (TSV), CSV, Values	✅	✅	TabSeparated*, CSV*, Values*
JSON, JSONEachRow, JSONCompact, JSONAsObject, JSONColumns	✅	✅	JSON*
Native (the wire format), RowBinary, RowBinaryWithNamesAndTypes	✅	✅	Native*, RowBinary*
Pretty, PrettyCompact, PrettySpace (TTY-friendly)	—	✅	Pretty*
Parquet, Arrow, ArrowStream, ORC (via contrib/arrow)	✅	✅	Impl/ParquetBlockInputFormat, Impl/ArrowBlockInputFormat, Impl/ORCBlockInputFormat
Avro, AvroConfluent, Protobuf, ProtobufList, Cap'n Proto, MsgPack	✅	✅	Impl/Avro*, Impl/Protobuf*, Impl/CapnProto*, Impl/MsgPack*
XML	—	✅	Impl/XMLRowOutputFormat.cpp
Markdown, Vertical, Null, LineAsString, Regexp, Template, RawBLOB, MySQLDump	mixed	mixed	Impl/...
Form (HTML form-encoded), BSONEachRow, Numpy, MySQLBulk	mixed	mixed	Impl/...
One	✅	—	Impl/OneBlockInputFormat.cpp (returns a single empty block; useful as a generator).

The full catalogue is registered through FormatFactory::registerFormat*. There are over 40 formats today.

How formats are wired

FormatFactory (src/Formats/FormatFactory.cpp) maps name → input/output factories. A format may register either or both. It also stores schema-inference hooks, format-specific settings, and metadata about whether the format supports parallel parsing or random access.

Inputs derive from IInputFormat (src/Processors/Formats/IInputFormat.h); outputs from IOutputFormat (IOutputFormat.h). Inputs are sources in the processor graph; outputs are sinks.

IRowInputFormat/IRowOutputFormat are convenience bases that operate row-by-row; column-oriented formats like Native, Parquet, Arrow work on whole Chunks for performance.

Schema inference

Many formats can guess a schema from the first few rows. FormatFactory::checkAndGetCommonSchema() calls each format's getSchema(...) hook (Impl/<Format>SchemaReader.cpp). Used by table functions like s3('...'), file('...'), url('...') to make CREATE TABLE ... AS s3('...') work.

src/Formats/EscapingRuleUtils.cpp and Impl/JSONCompactEachRowFormatReader.cpp carry the heuristics for inferring types from JSON-like streams (String vs UInt* vs Date).

Settings

FormatSettings.cpp declares hundreds of per-format settings (format_csv_delimiter, input_format_skip_unknown_fields, output_format_pretty_max_value_width, format_schema, …). Set on the query, the user profile, or the server.

Parallel parsing

Some text formats support parallel parsing: the input is split into row chunks and parsed in worker threads. ParallelParsingInputFormat.cpp orchestrates that. Random-access formats (Parquet, ORC, Arrow) support parallel reading of file groups instead.

Schema / Protobuf / Cap'n Proto

For formats that need an external schema:

• format_schema = 'file:Message' for Protobuf / Cap'n Proto.
• format_schema_path server setting.
• The schema cache lives under src/Formats/CapnProtoSerializer.cpp and Impl/ProtobufSerializer.cpp.

Native format

Native is ClickHouse's own binary columnar format: a sequence of Blocks with column types and names. It is the format used on the wire for the native TCP protocol and is the most efficient way to push data into a server.

Adding a format

1. Implement IInputFormat and/or IOutputFormat (or the row-based variants).
2. Optionally add a schema reader implementing ISchemaReader (Impl/<Format>SchemaReader.cpp).
3. Add a void registerInputFormatFoo(FormatFactory & factory) (and matching output) and call it from registerFormats.cpp.
4. Add tests under tests/queries/0_stateless/.

Related pages

• Processors — formats are processors.
• Data types and columns — ISerialization underpins per-cell formatting.
• Apps → Client — picks an output format.