Analyzer

The analyzer (src/Analyzer/) converts a parser AST into a typed, semantically-resolved QueryTree. It was introduced in July 2022 and became the default in 2024 (enable_analyzer=1). The legacy interpreter still exists for a handful of cases.

What problem does it solve?

The pre-analyzer path went straight from ASTSelectQuery to execution inside InterpreterSelectQuery. Name resolution, type inference, and rewrites were sprinkled across the interpreter, the ExpressionAnalyzer, and a chain of "visitor" passes (TranslateQualifiedNamesVisitor, ExecuteScalarSubqueriesVisitor, …). It worked for typical queries but produced surprising results around:

• Nested subqueries and lateral references.
• Common sub-expression detection.
• Joins with complex ON clauses.
• View expansion into other views.
• Window functions.
• CTE (WITH ... AS) reuse.

The analyzer concentrates all that work in one well-defined pass and produces a QueryTree that downstream code (the Planner) can rely on.

QueryTree

src/Analyzer/IQueryTreeNode.h defines the base class. The hierarchy includes:

Node	Purpose
QueryNode	A SELECT. Holds projection, WHERE, GROUP BY, HAVING, ORDER BY, LIMIT, joins, settings, sort.
UnionNode	UNION of multiple queries.
IdentifierNode	An unresolved name. Replaced by ColumnNode after resolution.
ColumnNode	A resolved column reference.
ConstantNode	Literal.
FunctionNode	Function call (scalar or aggregate).
LambdaNode	Lambda expression.
JoinNode, ArrayJoinNode, CrossJoinNode	Join structure.
TableNode, TableFunctionNode, QueryNode (as a subquery)	Table sources.
WindowNode, SortNode, InterpolateNode, MatcherNode	Modifier nodes.

Every node has a resolved type after analysis. The tree is mutable across analyzer passes.

How it works

graph TD
    AST[ASTSelectQuery] --> Builder[QueryTreeBuilder<br/>src/Analyzer/QueryTreeBuilder.cpp]
    Builder --> Tree[Raw QueryTree]
    Tree --> Passes[QueryTreePassManager<br/>src/Analyzer/QueryTreePassManager.cpp]
    Passes --> Resolve[QueryAnalyzer<br/>name resolution, types]
    Resolve --> RewritePasses[Many passes:<br/>JoinNormalize, FunctionResolution,<br/>OrderBy, Sets, Subquery flattening, ...]
    RewritePasses --> Final[Resolved + optimized QueryTree]

src/Analyzer/QueryTreeBuilder.cpp walks the AST and produces the initial QueryTree. QueryAnalyzer (src/Analyzer/Resolve/QueryAnalyzer.cpp) does the heavy lifting:

• Resolve identifiers against scopes (CTEs, with-clauses, table aliases, lateral views).
• Substitute table functions with their results (a TableFunctionNode becomes a sub-query for view(), etc.).
• Expand * and apply matchers using the table's columns.
• Determine each expression's DataType.
• Bind aggregate functions to their state types.
• Validate that aggregates and window functions are used in legal positions.
• Convert JOIN ON predicates into normalized form.

QueryTreePassManager then runs a series of optimizing/rewriting passes from src/Analyzer/Passes/. Examples:

Pass	What it does
LogicalExpressionOptimizerPass	Boolean simplification.
OrderByLimitByDuplicateEliminationPass	Strips redundant order/limit.
RewriteSumFunctionWithSumAndCountPass	sum(x + 1) → sum(x) + count() * 1.
IfChainToMultiIfPass	Folds nested ifs into multiIf.
FilterPushdownPass	Filters into joins and subqueries.
OptimizeRedundantFunctionsInOrderByPass	Strips constant folding from ORDER BY.
ConvertOrLikeChainPass	x LIKE 'a%' OR x LIKE 'b%' → multiSearchAny(...).
OptimizeGroupByFunctionKeysPass	Removes deterministic functions from GROUP BY.
JoinUsingNormalizePass	Normalizes JOIN USING.
MergeTreeWhereOptimizerPass	Pushes the right predicates into PREWHERE.
MultiIfToIfPass	The reverse of the above when only two branches remain.

The full list lives in src/Analyzer/Passes/PassesRegistration.cpp. Many were ported from the legacy interpreter.

Settings that influence the analyzer

• enable_analyzer (master switch).
• allow_experimental_analyzer (legacy alias).
• analyzer_compatibility_join_using_top_level_identifier, enable_analyzer_explain_subqueries, etc.
• Any setting that toggles a specific optimization (optimize_move_to_prewhere, optimize_redundant_functions_in_order_by, …).

Subqueries and CTEs

CTEs (WITH x AS (SELECT ...) ...) are represented as QueryNode children pinned to their CTE scope. The analyzer either inlines them or registers them as scalar/list dependencies depending on usage.

Joins

JoinNode carries the join kind, strictness, the ON expression as a separate ExpressionNode, and the left/right sides. The analyzer normalizes:

• JOIN USING (a, b) into the equivalent ON form.
• Cross joins with predicates into inner joins (CrossToInnerJoinVisitor was ported here as a pass).
• Multi-key equi-joins into the canonical (L1 = R1) AND (L2 = R2) form.

The planner consumes this normalized tree to choose hash/merge/direct join.

Views

When a table is a view, the analyzer expands its definition in place by recursively analyzing the view's stored query and substituting it.

Where to read next

• Planner — what consumes the QueryTree.
• Interpreters — for the legacy path and shared services (joins, aggregations).
• Functions — function lookup and resolution.

Entry points for modification

• New optimizer pass → add it under src/Analyzer/Passes/, register it in PassesRegistration.cpp.
• New node type → subclass IQueryTreeNode, update visitors.
• New name-resolution rule → src/Analyzer/Resolve/.

Related pages

• Parsers — produces the ASTSelectQuery the analyzer consumes.
• Planner
• Interpreters