Planner

The planner (src/Planner/) walks a resolved QueryTree and emits a QueryPlan — a tree of logical steps that describe how to read, filter, aggregate, and order data. From there, each step builds the physical processor graph.

Plan vs pipeline

graph LR
    QT[QueryTree] --> Planner[src/Planner/Planner.cpp]
    Planner --> QP[QueryPlan<br/>tree of QueryPlanStep]
    QP --> Optimize[QueryPlanOptimizations<br/>filter pushdown, projections, ORDER BY removal]
    Optimize --> Build[QueryPlanStep::buildPipeline]
    Build --> Pipeline[QueryPipeline<br/>processor graph]

A QueryPlan is logical and easy to optimize. A QueryPipeline is physical and easy to execute.

Key files

File	Purpose
src/Planner/Planner.cpp	Top-level entry. Builds a QueryPlan from a QueryTree.
src/Planner/PlannerActionsVisitor.cpp	Translates expression nodes into an ActionsDAG.
src/Planner/PlannerJoinTree.cpp	Materializes joins, sub-queries, and table sources.
src/Planner/PlannerJoins.cpp	Picks join algorithm (hash, merge, direct, partial).
src/Planner/PlannerAggregation.cpp	Wires Aggregator config from QueryTree.
src/Planner/PlannerSorting.cpp	Sort, limit, and LIMIT BY planning.
src/Planner/PlannerWindowFunctions.cpp	Window function planning.
src/Planner/PlannerExpressionAnalysis.cpp	Expression-level analysis the planner needs.
src/Planner/PlannerContext.cpp	Per-query context for the planner.
src/Planner/CollectColumnIdentifiers.cpp	Column-usage tracking.
src/Planner/CollectSets.cpp, CollectTableExpressionData.cpp	Bookkeeping for IN sets and table sources.
src/Planner/Utils.cpp	Helpers.

QueryPlan steps

src/Processors/QueryPlan/IQueryPlanStep.h defines the base step. The directory src/Processors/QueryPlan/ contains dozens of concrete steps:

Step	Purpose
ReadFromMergeTree	Reads a MergeTree table with mark-range planning.
ReadFromRemote	Issues a sub-query to remote shards.
ReadFromSystemNumbersStep, ReadFromStorageStep	Generic readers.
FilterStep	A filter (over an ActionsDAG).
ExpressionStep	Project / compute.
AggregatingStep / MergingAggregatedStep	Group-by aggregation in two phases.
JoinStep, FilledJoinStep, MergeJoinStep	Joins.
SortingStep, MergeSortingStep, PartialSortingStep, FinishSortingStep	Sort variants.
LimitStep, OffsetStep, LimitByStep	Limits.
WindowStep	Window functions.
DistinctStep	DISTINCT.
TotalsHavingStep	WITH TOTALS/HAVING.
FillingStep	WITH FILL.
UnionStep, IntersectOrExceptStep	Set operations.
ArrayJoinStep	ARRAY JOIN.
RollupStep, CubeStep	Group-by extensions.
DelayedCreatingSetsStep	Lazily materialized IN-sets.

Each step exposes transformPipeline(QueryPipelineBuilder &, BuildQueryPipelineSettings) to attach its physical processors.

Optimizations

src/Processors/QueryPlan/Optimizations/ is a small framework that walks the QueryPlan and applies rule-based rewrites. Examples:

• Filter pushdown — push FilterSteps through joins, aggregations, and sort.
• PREWHERE lifting — split filter expressions and push them into ReadFromMergeTree.
• Projection use — replace a ReadFromMergeTree with a projection-backed read when feasible.
• ORDER BY removal — drop ORDER BY if a downstream step doesn't need ordering.
• Limit pushdown — push LimitStep past steps that preserve order.
• Aggregator splitting — turn one heavy AggregatingStep into two phases when running across shards.
• Avoid distinct in some cases — when an upstream step guarantees uniqueness.

The optimizer is rule-based, not cost-based.

Choosing a join algorithm

PlannerJoins.cpp picks among:

• Hash join — default; build a hash table on the smaller side.
• Parallel hash join — partitioned variant for large RHS.
• Merge join — sort-merge for already-sorted inputs.
• Direct join — when the right side is a Dictionary/Join storage.
• Partial merge join — out-of-memory friendly merge join.
• Grace-hash join — grace-hash variant.

Selection is influenced by join_algorithm, table sizes, and the available indexes on each side. The chosen algorithm becomes a concrete JoinStep subclass (MergeJoinStep, FilledJoinStep, …) plus an IJoin instance from src/Interpreters/HashJoin/ or src/Interpreters/MergeJoin.cpp.

EXPLAIN

The planner is the layer behind EXPLAIN:

• EXPLAIN QUERY TREE — print the analyzed QueryTree.
• EXPLAIN PLAN — print the QueryPlan.
• EXPLAIN PIPELINE — print the resulting processor graph.
• EXPLAIN ESTIMATE — show row/byte estimates.

Each is implemented in src/Interpreters/InterpreterExplainQuery.cpp.

Entry points for modification

• New plan step → add it under src/Processors/QueryPlan/ and have a planner method emit it.
• New optimization → add a rule under src/Processors/QueryPlan/Optimizations/ and register it.
• New join algorithm → implement an IJoin subclass and pick it from PlannerJoins.cpp.

Related pages

• Analyzer — the upstream stage.
• Processors — the downstream stage.
• Interpreters — joins, aggregator, hash tables, the legacy interpreter.