Architecture

TiDB is a stateless SQL frontend that translates MySQL-protocol requests into transactions and coprocessor requests against a distributed key-value store. This page walks through how a query flows through the server in this repository, names the subsystems involved, and points at the files that own each step.

Cluster topology

graph LR
  Client[MySQL client / app] -->|MySQL protocol| TiDB[TiDB server]
  TiDB -->|TSO + meta| PD[PD - Placement Driver]
  TiDB -->|2PC, coprocessor| TiKV[TiKV - row store]
  TiDB -->|MPP / coprocessor| TiFlash[TiFlash - column store]
  TiKV -.->|Raft log| TiKV
  TiKV -.->|Multi-Raft Learner| TiFlash

TiDB is stateless: any cluster metadata that needs to survive restart lives in TiKV (system tables) or PD (cluster topology, region routing, TSO). Multiple TiDB servers in front of the same TiKV/PD form a horizontally scalable SQL tier; clients can be load-balanced across them.

The TiDB server side of that topology lives entirely in this repository. The TiKV and PD components live in github.com/tikv/tikv and github.com/tikv/pd respectively, and TiDB talks to them via the Go client github.com/tikv/client-go/v2 (pinned in go.mod).

Inside a TiDB server

graph TD
  subgraph net[Network]
    proto[Server / MySQL protocol<br/>pkg/server/]
  end
  subgraph sql[SQL layer]
    sess[Session<br/>pkg/session/]
    parser[Parser<br/>pkg/parser/]
    plan[Planner<br/>pkg/planner/]
    exec[Executor<br/>pkg/executor/]
    expr[Expression<br/>pkg/expression/]
  end
  subgraph schema[Schema and metadata]
    domain[Domain<br/>pkg/domain/]
    info[Infoschema<br/>pkg/infoschema/]
    ddl[DDL<br/>pkg/ddl/]
    meta[Meta<br/>pkg/meta/]
    stats[Statistics<br/>pkg/statistics/]
  end
  subgraph storage[Storage client]
    kv[KV interface<br/>pkg/kv/]
    distsql[DistSQL<br/>pkg/distsql/]
    store[Store / TiKV driver<br/>pkg/store/]
  end

  proto --> sess
  sess --> parser
  parser --> plan
  plan --> exec
  plan -.->|stats lookup| stats
  plan -.->|schema lookup| info
  exec --> expr
  exec --> distsql
  distsql --> store
  store --> kv
  ddl --> meta
  ddl --> info
  domain --> info
  domain --> stats
  exec -.->|DDL job| ddl

Lifecycle of a SELECT

1. Accept the connection. pkg/server/server.go (Server.Run) listens on the configured TCP/Unix socket; each connection gets a *clientConn defined in pkg/server/conn.go. This is also where MySQL-protocol authentication, packet framing, and capabilities negotiation happen.
2. Establish a session. A connection drives a session.session (pkg/session/session.go). Sessions carry the transaction context (pkg/session/txn.go), the system-variable view (pkg/sessionctx/variable/), and access to the current Domain (the per-instance schema and metadata cache from pkg/domain/domain.go).
3. Parse. The session asks the parser (pkg/parser/) to turn the SQL text into an AST defined in pkg/parser/ast/. The grammar lives in pkg/parser/parser.y and is generated into the (very large) pkg/parser/parser.go. A separate hint grammar lives in pkg/parser/hintparser.y.
4. Compile to a plan. pkg/executor/compiler.go and pkg/planner/optimize.go build an initial logical plan (pkg/planner/core/logical_plan_builder.go), apply rewrite rules (pkg/planner/core/rule_*.go), and pick physical operators by cost (pkg/planner/core/find_best_task.go and pkg/planner/core/plan_cost_ver1.go / plan_cost_ver2.go). For prepared statements the plan can be served from pkg/planner/core/plan_cache*.go. A newer Cascades-style optimizer is being developed in parallel in pkg/planner/cascades/.
5. Execute. pkg/executor/adapter.go adapts the chosen plan into runtime executors built by pkg/executor/builder.go. Operators (joins, aggregations, projections, sorts, etc.) live next to it (pkg/executor/join/, pkg/executor/aggregate/, pkg/executor/sortexec/, …). Expressions are evaluated by pkg/expression/ using vectorized chunks from pkg/util/chunk/.
6. Read data. Most reads go through pkg/distsql/ and pkg/store/copr/coprocessor.go, which pushes down filters, projections, aggregations, and TopN to TiKV/TiFlash via the coprocessor protocol. Point-gets and direct KV access use pkg/kv/ and the TiKV driver in pkg/store/driver/.
7. Return results. Result chunks flow back up through the executor tree, get encoded into the MySQL result-set format by pkg/server/, and are written to the connection.

Lifecycle of a write and DDL

• DML. Writes (INSERT, UPDATE, DELETE) go through pkg/executor/insert*.go, pkg/executor/update.go, pkg/executor/delete.go. They buffer KV mutations in a transactional memory buffer; when the SQL transaction commits, pkg/session/txn.go runs the two-phase commit via the TiKV client.
• DDL. CREATE TABLE, ALTER TABLE, and similar statements are submitted as DDL jobs by pkg/ddl/job_submitter.go. A scheduler (pkg/ddl/job_scheduler.go) picks up jobs across the cluster, and a worker (pkg/ddl/job_worker.go) drives them through state machines (e.g., pkg/ddl/column.go, pkg/ddl/index.go, pkg/ddl/partition.go). Online schema change is staged through schema versions (pkg/ddl/schema_version.go) and the infoschema cache in each TiDB instance reloads accordingly. The DDL framework documentation source-of-truth is docs/agents/ddl/README.md.

Background and cross-cutting

• Domain (pkg/domain/domain.go) owns long-running per-instance work: schema reloads, statistics handle, plan cache eviction, slow-query top-N, the SQL bind manager, runaway query controller, and the TTL job scheduler.
• Statistics (pkg/statistics/) maintains histograms, count-min sketches, and TopN sketches used by the optimizer; pkg/executor/analyze*.go drives ANALYZE TABLE and pkg/statistics/handle/ persists and refreshes them.
• DXF (pkg/dxf/) is the Distributed eXecution Framework used by heavy background tasks like distributed ADD INDEX, IMPORT INTO, and statistics gathering. Tasks are partitioned across TiDB nodes and tracked in cluster system tables.
• TTL (pkg/ttl/), bindings (pkg/bindinfo/), timer (pkg/timer/), plugin (pkg/plugin/), resource group (pkg/resourcegroup/, pkg/resourcemanager/), and telemetry (pkg/telemetry/) all hook into Domain.
• Server-side HTTP and metrics: pkg/server/http_status.go exposes /status, Prometheus metrics from pkg/metrics/, slow log ingestion, profiling, and admin endpoints documented in docs/tidb_http_api.md.

Companion tools in this repo

• BR (br/) — distributed full and incremental backup/restore, CDC log backup/restore. Talks directly to TiKV via gRPC plus a TiDB "glue" for SQL state.
• Lightning (lightning/, pkg/lightning/) — high-throughput data importer with a "physical" mode (writes SST files for TiKV ingestion) and a "logical" mode (tidb backend that writes via SQL).
• Dumpling (dumpling/) — logical export to SQL/CSV; designed as a mysqldump/mydumper replacement that understands TiDB.

These three tools share infrastructure under br/pkg/ (storage backends, encryption, checksum, region tracking, retry/backoff). Lightning and BR also share the pkg/lightning/backend/ import pipeline.

Where to read next

• Systems overview lists each major package with a one-line description and a link to its dedicated page.
• Applications overview covers tidb-server, BR, Lightning, and Dumpling as deployable binaries.
• The repo's own docs/agents/architecture-index.md is the source of truth for path-to-subsystem mapping that contributors and agents follow.