Lore

A timeline view of how pingcap/tidb evolved. Dates are derived from git history (commit timestamps and tag dates). When the "why" behind a change isn't clearly stated in commits, the prose hedges accordingly.

Era 1 — Bootstrap (Sep 2015 – mid 2017)

• Sep 2015: First commit (Add files, 2015-09-06). The earliest commits set up a parser, a basic KV abstraction, and a server skeleton. The early tree had parser/, ast/, kv/, executor/, session/, and store/ at the top level rather than nested under pkg/.
• The earliest persistent design choices visible in code: yacc-driven SQL parser, MySQL-compatible types, and an interface-based KV layer. All three are still in place today.
• v1.0.0 was tagged in 2017 (date in git tag --sort=committerdate). It marked the first GA release.

Era 2 — Productionization and HTAP (2017 – 2020)

• TiDB 2.x and 3.x added pessimistic transactions, statistics improvements, and the early TiFlash integration. The TiFlash MPP path lives mostly in pkg/store/copr/mpp.go and pkg/executor/mpp_gather.go.
• 2019: Dumpling appeared as a mysqldump replacement (dumpling/). Its design RFC is cited in dumpling/README.md.
• TiDB 4.x introduced clustered indexes, partitioned tables, and major optimizer improvements. The legacy planner (pkg/planner/core/) was massively expanded; the rule list in core.logicalOptimize shows the scars of accumulated rules.

Era 3 — Repository restructure (2020 – 2022)

• The most visible reorganization in the history is the move from top-level Go directories (session/, executor/, kv/, …) to the pkg/<name>/ layout. After the move, pkg/ became the home of the SQL server, with cmd/ reserved for binaries, br/, dumpling/, and lightning/ for companion tools, and tools/ for build helpers. The historical git log still shows commits touching the old paths.
• TiDB 5.x added MPP via TiFlash for analytical workloads, the cost model v1, and the first stable Lightning physical-mode import. SST-ingest code in pkg/lightning/backend/local/ dates to this era.
• BR went GA and learned about S3 / GCS / Azure storage backends (br/pkg/utils/, br/pkg/encryption/).

Era 4 — Distributed framework and faster DDL (2022 – 2023)

• TiDB 6.x and 7.x introduced the Distributed eXecution Framework (DXF) under pkg/dxf/. DXF re-paints heavy operations — distributed ADD INDEX, IMPORT INTO, large ANALYZE jobs — as cluster-wide tasks split across TiDB nodes. Files like pkg/ddl/backfilling_dist_*.go and pkg/executor/import_into.go are the SQL-layer hooks.
• The plan cache matured (pkg/planner/core/plan_cache*.go) into a per-instance LRU plus a session-level cache, with rebuild logic for type promotion (plan_cache_rebuild.go).
• The cost model gained a v2 (pkg/planner/core/plan_cost_ver2.go) selectable via tidb_cost_model_version. v1 lives on as the default for some operators and as a fallback.
• Resource control with Request Units (RUs) shipped: pkg/resourcegroup/, pkg/resourcemanager/, pkg/sessionctx/variable/sysvar.go (tidb_resource_control_strict_mode and friends).

Era 5 — Cascades and online schema scale (2024 – present)

• A new Cascades-style optimizer lives in parallel with the legacy planner under pkg/planner/cascades/. It is not yet the default for all queries; the migration is incremental, with pkg/planner/cascades/old/ retained from the earlier prototype.
• Infoschema v2 (pkg/infoschema/infoschema_v2.go) was added so clusters with very large numbers of tables can avoid materialising the entire schema in memory on each diff.
• The standby path (pkg/standby/) was added so a TiDB process can warm up without serving traffic and be promoted on demand.
• Vector search builtins appeared under pkg/expression/builtin_vec.* and pkg/expression/builtin_vec_vec.*. Full-text search builtins are in pkg/expression/builtin_fts.go / fts_helper.go.
• The first half of 2026 (visible in git log --since="90 days ago", 489 commits) shows the project still iterating heavily on the planner, statistics, BR's snap_client, and the executor's MPP digest propagation.

Longest-standing features

• The MySQL-compatible parser (pkg/parser/parser.y) has been the single SQL grammar from day one. It has grown to ~421k lines of yacc but remains the same architectural choice as in 2015.
• The interface-based KV abstraction (pkg/kv/kv.go) is also original. It allowed the project to swap in the unistore mock and the real TiKV driver behind the same kv.Storage.
• The DDL job model (a job written to a TiKV system table, drained by a single elected owner) appeared early and survived multiple optimizations; the state machine in pkg/ddl/job_worker.go evolved without changing the basic shape.

Major rewrites

• The package layout move to pkg/... is the single biggest tree-wide rewrite in history. After it, almost every import path changed.
• The planner has been "rewritten" twice — first to introduce the cost-based optimizer in the late 2010s, then to introduce v2 cost model in 2022, and is being rewritten a third time as the Cascades optimizer rolls out in pkg/planner/cascades/. The classic optimizer in pkg/planner/core/ continues to do most of the work in the meantime.
• Online ADD INDEX was rewritten to use the Lightning ingest path (pkg/ddl/backfilling_*.go), turning what used to be a slow row-by-row backfill into an SST-ingest that scales with TiKV peers.

Deprecated / removed

• Telemetry collection (pkg/telemetry/) was retained as an empty/disabled subsystem after the company stopped collecting telemetry. The package is still referenced by upgrade scripts so removing it isn't trivial; it is effectively dormant.
• The earlier memo-based optimizer prototype under pkg/planner/memo/ and pkg/planner/cascades/old/ has been superseded by the active Cascades implementation; the old/ directory is retained while migration completes.
• A mock SST coprocessor for the old mock store was deprecated in favour of unistore (pkg/store/mockstore/unistore/); the mockcopr code under pkg/store/mockstore/mockcopr/ is kept as a thin compatibility layer.

Growth signals

• 1,058 unique git authors over 10½ years, with around 75 humans active in the last 90 days.
• The repo passed 28,000 commits in early 2026.
• Tags follow a vX.Y.Z pattern with frequent vX.Y.Z-YYYYMMDD-<sha> tags for nightly/release-cut builds; the most recent tag at the time of this snapshot is v8.5.5-20260430-34227a6.