Infoschema and meta

Two packages own the schema metadata side of TiDB. pkg/meta/ writes/reads the persisted form (in TiKV system keys); pkg/infoschema/ exposes an in-memory snapshot used by the planner and executor.

Purpose

• pkg/meta/ is the persistent source of truth for schema. It encodes/decodes table/column definitions, partition definitions, indexes, and TiDB internal counters into the structured key-value layout used inside TiKV.
• pkg/infoschema/ is a thread-safe, immutable snapshot of all schema objects at a particular schema version. It is the read-side API that the planner uses for name resolution and the executor uses for INFORMATION_SCHEMA views.

Directory layout

pkg/meta/
├── meta.go              # Meta struct: schema/table/index getters and setters
├── autoid/              # Auto-increment / auto-random / sequence ID allocators
├── infoschema/          # Persisted infoschema diff records
├── metabuild/           # Helpers to build table info from CREATE TABLE
├── model/               # Persistent model types (DBInfo, TableInfo, ColumnInfo, …)
└── ...

pkg/infoschema/
├── infoschema.go        # InfoSchema interface + builder
├── infoschema_v2.go     # The v2 (cache-backed) infoschema implementation
├── builder.go           # Applies DDL diffs to construct a new InfoSchema
├── tables.go            # Definitions of every memory-backed information_schema/cluster table
├── metric_table_def.go  # Metric tables for INFORMATION_SCHEMA.METRICS_*
├── perfschema/          # PERFORMANCE_SCHEMA tables
├── infoschemav2_test.go # Cache behaviour tests
└── ...

Key abstractions

Type	File	Purpose
meta.Meta	pkg/meta/meta.go	Read/write schema in a TiKV transaction.
meta.NewMeta(txn)	same	Construct a Meta on top of a kv.Transaction.
model.DBInfo, model.TableInfo, model.ColumnInfo, model.IndexInfo	pkg/meta/model/	Persistent schema records; serialised as JSON.
autoid.Allocator	pkg/meta/autoid/	Auto-increment ID allocator.
infoschema.InfoSchema	pkg/infoschema/infoschema.go	Read-only interface: TableByName, SchemaByName, AllSchemas, SchemaMetaVersion, etc.
infoschema.SchemaVersion	same	Monotonic schema version coordinated by DDL.
infoschema.Builder	pkg/infoschema/builder.go	Applies a list of model.SchemaDiff to a previous InfoSchema to produce a new one.
infoschema.MemoryInfoSchema	pkg/infoschema/tables.go	The in-memory tables that back INFORMATION_SCHEMA views.

How a new schema version is published

sequenceDiagram
  participant DDL as pkg/ddl worker
  participant Meta as pkg/meta
  participant Etcd as etcd via pkg/owner
  participant Domain as pkg/domain on every TiDB
  participant IS as pkg/infoschema

  DDL->>Meta: write new TableInfo, bump schema version
  DDL->>Etcd: publish schemaVersion (KV put)
  Domain->>Etcd: watch schemaVersion
  Etcd-->>Domain: notify version N+1
  Domain->>Meta: load diffs N..N+1
  Domain->>IS: Builder.ApplyDiff → new InfoSchema
  Domain->>Domain: swap currentInfoSchema atomically

This is the "online schema change" backbone. Each TiDB instance lazily reloads, but the DDL worker on the owner waits for everyone to converge before progressing certain DDL states (see DDL).

InfoSchema v1 and v2

The repo carries two implementations:

• v1 (legacy): the entire schema is materialised in memory as one infoschema object. Cheap reads but expensive to rebuild on each diff for very large clusters.
• v2 (infoschema_v2.go): a cache-backed schema that lazily fetches table info on demand and applies diffs incrementally. v2 is selected for clusters with many tables. Switching is gated by configuration.

Both implementations satisfy the same InfoSchema interface, so callers don't care.

INFORMATION_SCHEMA / PERFORMANCE_SCHEMA / METRICS_SCHEMA

pkg/infoschema/tables.go declares the schema of every memory-backed table; the runtime fetchers live in pkg/executor/infoschema_reader.go and pkg/executor/memtable_reader.go. Examples:

• INFORMATION_SCHEMA.TABLES, COLUMNS, STATISTICS, KEY_COLUMN_USAGE — derived from the live infoschema.
• INFORMATION_SCHEMA.SLOW_QUERY, STATEMENTS_SUMMARY — from in-memory rolling buffers.
• INFORMATION_SCHEMA.CLUSTER_* — assembled from RPCs to peer TiDB nodes (pkg/server/rpc_server.go).
• INFORMATION_SCHEMA.METRICS_* — defined in metric_table_def.go, served by pkg/executor/metrics_reader.go against Prometheus.
• PERFORMANCE_SCHEMA.* — pkg/infoschema/perfschema/ contains the schema; the runtime tables are stubs because TiDB does not implement performance-schema instruments.

Auto-ID

pkg/meta/autoid/ implements:

• Auto-increment counters (with auto_id_cache for batch allocation).
• Auto-random IDs (used for AUTO_RANDOM columns to avoid hotspots).
• Sequences (CREATE SEQUENCE).

For very large clusters, an external autoid_service lives in pkg/autoid_service/; clients consult it to avoid TiDB-instance-local lock contention.

Integration points

• DDL writes via meta.Meta and triggers infoschema reloads.
• Domain (pkg/domain/domain.go) owns the long-running infoschema reload loop and exposes the current snapshot via Domain.InfoSchema().
• Planner consumes infoschema.InfoSchema for table/column resolution.
• Statistics keys its caches by (physicalID, schemaVersion) from infoschema.
• Executors for INFORMATION_SCHEMA views use infoschema.MemoryInfoSchema plus the runtime fetchers.

Entry points for modification

• New persistent schema field → extend model.TableInfo or its sub-structs (back-compat: keep JSON tags stable; bump a version field if breaking) and update pkg/meta/metabuild/ if it's set during CREATE TABLE.
• New INFORMATION_SCHEMA table → declare schema in tables.go, register the runtime fetcher in pkg/executor/infoschema_reader.go, optionally add a predicate extractor in pkg/planner/core/memtable_predicate_extractor.go.
• New auto-ID strategy → pkg/meta/autoid/.
• Metric table changes → metric_table_def.go plus a Grafana panel under pkg/metrics/grafana/ if appropriate.