SQL

pkg/sql/ is the SQL layer: it turns PostgreSQL-wire-protocol traffic into transactional KV operations. It is by far the largest subsystem in the repo (~3,300 Go files including tests, generated code, and optimizer rules) and is owned by several teams (sql-foundations, sql-queries, obs-prs, disaster-recovery, …).

Purpose

A SQL connection arrives at a pgwire listener. After authentication, it is handed to a connExecutor which drives a transaction state machine, parses statements, plans them via the cost-based optimizer, executes the resulting plan, and streams rows back. Schema changes, jobs, distributed flows, and CDC processors all live inside this package.

High-level flow

graph LR
  PG["pgwire (pkg/sql/pgwire)"] --> CE["connExecutor (conn_executor.go)"]
  CE --> Parser["parser (pkg/sql/parser)"]
  Parser --> Opt["optimizer (pkg/sql/opt)"]
  Opt --> ExecBuilder["execbuilder (pkg/sql/opt/exec/execbuilder)"]
  ExecBuilder --> RowEngine["row engine (planNode tree)"]
  ExecBuilder --> VecEngine["vectorized engine (colexec, colflow)"]
  RowEngine --> KV["kv.Txn"]
  VecEngine --> KV
  CE --> SchemaChanger["schemachanger (declarative & legacy)"]
  CE --> SessionVars["vars.go / sessiondata"]

Sub-areas

Parser (pkg/sql/parser/)

A goyacc grammar (sql.y) compiled into a Go parser. Produces tree.Statement AST nodes (defined in pkg/sql/sem/tree/). The grammar implements PostgreSQL syntax plus CRDB extensions (SELECT ... FROM <table>@<index>, ALTER TABLE ... LOCALITY ..., BACKUP/RESTORE, …).

AST and semantic types (pkg/sql/sem/)

• pkg/sql/sem/tree/ — AST node definitions and walking/transforming utilities.
• pkg/sql/sem/builtins/ — every SQL built-in function. ~30k lines total.
• pkg/sql/sem/plpgsqltree/ — PL/pgSQL procedural language AST.
• pkg/sql/types/ — typed value system covering all PostgreSQL types plus CRDB extensions (UUID, INET, TIMESTAMPTZ, ENUM, GEOMETRY, JSONB, …).

Optimizer (pkg/sql/opt/)

A Cascades-style cost-based optimizer. Architecture:

• pkg/sql/opt/optgen/ — DSL compiler. Reads .opt rule files and emits Go.
• pkg/sql/opt/norm/rules/ — normalization rules (always applied).
• pkg/sql/opt/xform/rules/ — exploration rules (search-driven).
• pkg/sql/opt/memo/ — memoization data structure (the Memo).
• pkg/sql/opt/exec/execbuilder/ — lowers an optimized memo into the row engine.
• pkg/sql/opt/cat/ — the optimizer's catalog interface.
• pkg/sql/opt/props/ — logical and physical properties.
• pkg/sql/opt/constraint/ — span-constraint primitives.

The fallback "old planner" path still exists in pkg/sql/ and is used for some statements not yet handled by the optimizer.

Execution engines

Two engines coexist:

• Row engine — plan nodes in pkg/sql/*.go (e.g., scan_node.go, join.go, update.go). Each implements planNode. Row-at-a-time, easier to reason about, used for most OLTP.
• Vectorized engine — columnar batch execution in pkg/sql/colexec/, pkg/sql/colflow/, pkg/sql/colmem/. Faster for analytical queries with large scans.

DistSQL (pkg/sql/distsql_*.go, pkg/sql/flowinfra/) splits a physical plan across nodes — each node runs a "flow" of "processors" (pkg/sql/execinfra/).

pgwire (pkg/sql/pgwire/)

The PostgreSQL v3 wire protocol. Handles startup, authentication (delegates to pkg/security/), prepared statements, COPY, extended-query protocol, error and notice formatting. auth.go is co-owned by sql-foundations and security teams.

Schema changer

Two implementations live side-by-side:

• Legacy schema changer — pkg/sql/alter_*.go, pkg/sql/create_*.go, pkg/sql/drop_*.go, pkg/sql/schemachange/, pkg/sql/schema_changer.go. Pre-existing path; gradually being replaced.
• Declarative schema changer — pkg/sql/schemachanger/. A graph-rewrite engine where each schema-change "operation" (e.g., add column, drop index) is a deterministic plan of stages. Many DDL statements now run through this path.

The pkg/cli/declarative_*.go tools dump the rules used by the declarative schema changer for documentation and review.

Catalog (pkg/sql/catalog/)

The persistent SQL schema, including:

• Descriptor types (table, schema, database, type, function, schedule, …).
• pkg/sql/catalog/descs/ — the Collection that brokers descriptor reads and caches.
• pkg/sql/catalog/bootstrap/ — bootstrap data for new clusters.
• pkg/sql/catalog/multiregion/ — regional/global table descriptor logic.
• pkg/sql/catalog/lease/ — leases for descriptors.

Session and execution context

• pkg/sql/conn_executor.go and pkg/sql/conn_executor_*.go — the SQL session driver. Owns the transaction state machine.
• pkg/sql/exec_util.go (~150 KB) and pkg/sql/internal.go — shared helpers and the in-process internal SQL executor used by jobs and migrations.
• pkg/sql/vars.go (~210 KB) — every session variable's definition and getter/setter.
• pkg/sql/sessiondata/ — the typed session state.
• pkg/sql/sqlliveness/ — the per-instance "session" abstraction used by jobs and the schema changer.

Job-driven SQL

Several SQL features are implemented as jobs (see systems/jobs):

• TTL deletion (pkg/sql/ttl/)
• Schema change (pkg/sql/schema_changer.go, schemachanger jobs)
• Inspect (pkg/sql/inspect/)
• SQL stats updater (pkg/sql/sql_activity_update_job.go)
• Tenant cluster settings reconciliation (in pkg/sql/tenant_settings.go)

Observability and stats

• pkg/sql/sqlstats/ — per-statement and per-transaction stats persisted in system tables and surfaced via DB Console.
• pkg/sql/contention/ — contention event collection.
• pkg/sql/idxusage/ — index usage stats.
• pkg/sql/execstats/ — per-execution stats.
• pkg/sql/scheduledlogging/ — periodic structured logs.

Other features under SQL

• Vector index — pkg/sql/vecindex/ (multi-level k-means based ANN index, integrated with the VECTOR data type).
• Geo / spatial — pkg/geo/ and pkg/sql/sem/builtins/geo_builtins.go.
• Inverted indexes — pkg/sql/inverted/, pkg/sql/colencoding/.
• Importer — pkg/sql/importer/ (CSV/AVRO/PGDUMP/MYSQLDUMP imports).
• PL/pgSQL — pkg/sql/plpgsql/ and pkg/sql/sem/plpgsqltree/.
• Trigger system — pkg/sql/show_trigger.go and triggers wired into the schema changer.
• Advisory locks — pkg/sql/advisorylock/.
• crdb_internal and pg_catalog — pkg/sql/crdb_internal.go and pkg/sql/pg_catalog.go build PostgreSQL-compatible introspection plus CockroachDB-specific virtual tables.

Key types

Type	File	Description
connExecutor	pkg/sql/conn_executor.go	Per-connection driver and txn state machine
planner	pkg/sql/planner.go	Per-statement context (auth, descriptors, internal executor)
Optimizer	pkg/sql/opt/xform/optimizer.go	Cascades-style optimizer
Memo	pkg/sql/opt/memo/memo.go	Plan memoization
ExecBuilder	pkg/sql/opt/exec/execbuilder/builder.go	Memo → planNode tree
FlowSpec / Flow	pkg/sql/execinfrapb/, pkg/sql/flowinfra/	DistSQL physical plan and runtime
pgwire.Server	pkg/sql/pgwire/server.go	Listener + per-conn dispatcher
*schemaChangeJob	pkg/sql/schemachanger/scjob/	Declarative schema-change job

Entry points for modification

• A new SQL syntax: edit pkg/sql/parser/sql.y, add an AST node in pkg/sql/sem/tree/, run ./dev gen.
• A new optimizer rule: add an .opt file under pkg/sql/opt/{norm,xform}/rules/ and a Go support function in the same directory.
• A new built-in function: register in pkg/sql/sem/builtins/ and run ./dev gen go to regenerate pg_catalog/information_schema views.
• A new session variable: register in pkg/sql/vars.go.
• A new DDL statement: extend the schema changer (preferred) or add a pkg/sql/<verb>_*.go file.

Related pages

• KV client — what SQL hands work to.
• Jobs — long-running SQL operations.
• Cluster version — gating SQL behavior.
• features/changefeed, features/backup-restore, features/multi-region, features/multi-tenant.