Storage client

The "storage layer" inside the TiDB server is the client to TiKV/TiFlash. The actual KV server is in tikv/tikv, and TiDB connects to it via the github.com/tikv/client-go/v2 library plus the in-tree wrappers and helpers under pkg/store/, pkg/kv/, pkg/distsql/, and pkg/tablecodec/.

Purpose

These packages collectively:

• Define the abstract KV interface used by the rest of TiDB (pkg/kv/).
• Encode rows and indexes into the TiKV key-value layout (pkg/tablecodec/).
• Build coprocessor / batch-coprocessor / MPP requests and dispatch them across TiKV and TiFlash (pkg/distsql/, pkg/store/copr/).
• Provide the production driver that connects to a real TiKV+PD cluster (pkg/store/driver/).
• Provide an in-process mock store (pkg/store/mockstore/unistore/) for tests and --store=unistore mode.
• Add small server-side helpers like the GC worker and helper utilities (pkg/store/gcworker, pkg/store/helper).

Directory layout

pkg/kv/                 # Abstract KV / Transaction / Snapshot interfaces
├── kv.go               # Top-level: Storage, Transaction, Snapshot, Iterator
├── union_store.go      # MVCC: in-memory buffer + snapshot
├── txn.go, txn_drop_protect.go
├── checker.go, key_ranges.go
├── option.go, range.go, error.go
└── ...

pkg/tablecodec/         # Encode rows, indexes, handles into TiKV keys
├── tablecodec.go       # All key/value layouts
├── rowindexcodec/

pkg/distsql/            # Distributed-SQL request layer
├── distsql.go          # SelectResult: chunks streamed from coprocessor
├── request_builder.go  # Build coprocessor requests from physical plans
├── select_result.go
└── stream.go

pkg/store/
├── store.go            # Public Open/Register entry points
├── etcd.go             # etcd client glue
├── copr/               # Coprocessor + MPP client
│   ├── coprocessor.go         # Coprocessor request scheduling and result streaming
│   ├── batch_coprocessor.go   # Batch coprocessor (TiFlash row-format batching)
│   ├── batch_request_sender.go
│   ├── mpp.go, mpp_probe.go   # MPP requests against TiFlash
│   ├── region_cache.go        # Region routing cache
│   ├── coprocessor_cache.go   # Coprocessor result cache
│   └── key_ranges.go
├── driver/             # Real TiKV driver
│   ├── tikv_driver.go  # The Driver implementation
│   ├── txn/            # Wraps client-go's transaction
│   ├── error/          # KV error mapping
│   └── ...
├── mockstore/          # In-memory mocks
│   ├── mockstore.go    # Selects between mockstore variants
│   ├── unistore/       # Embedded "unified store" mock TiKV
│   ├── mockcopr/       # Mock coprocessor for unistore
│   └── teststore/
├── gcworker/           # Background GC worker
├── helper/             # Misc storage helpers used by HTTP API
└── pdtypes/            # Local PD type definitions

Key abstractions

Type	Where	Purpose
kv.Storage	pkg/kv/kv.go	The cluster-wide handle. Created by store.Open(...).
kv.Transaction	same	A read/write transaction. Buffers writes and commits via 2PC.
kv.Snapshot	same	Read-only view at a particular start TS.
kv.Iter	same	Forward/reverse iterators over a key range.
tablecodec.EncodeRowKey, EncodeIndexSeekKey, DecodeIndexKV	pkg/tablecodec/tablecodec.go	Translate logical (table, handle, index) ↔ physical KV keys.
distsql.SelectResult	pkg/distsql/distsql.go	Streamed coprocessor result; produces chunks.
distsql.RequestBuilder	pkg/distsql/request_builder.go	Builds a TiKV coprocessor request from a physical plan node.
copr.CopClient	pkg/store/copr/coprocessor.go	Coprocessor RPC client; handles region splits, retries, ordering.
copr.MPPClient	pkg/store/copr/mpp.go	MPP RPC client (for TiFlash).
copr.RegionCache	pkg/store/copr/region_cache.go	Wraps client-go's region cache; backs region routing.
tikvDriver	pkg/store/driver/tikv_driver.go	kv.Driver implementation; produced by --store=tikv.
mockstore.NewMockStore	pkg/store/mockstore/mockstore.go	Builds an in-process mock cluster. Used by --store=unistore and most tests.

Read path

graph LR
  exec[pkg/executor TableReader / IndexReader] --> distsql[pkg/distsql<br/>RequestBuilder]
  distsql --> copr[pkg/store/copr<br/>CopClient]
  copr --> rc[Region cache<br/>region_cache.go]
  copr --> tikv[(TiKV)]
  copr --> tiflash[(TiFlash)]
  tikv --> chunks[Coprocessor chunks]
  tiflash --> chunks
  chunks --> exec

1. The executor's TableReader builds a coprocessor Request through RequestBuilder. The request includes the key ranges, the pushed-down dag.Request (filters, projections, aggregations from pkg/expression/expr_to_pb.go), and execution flags.
2. CopClient.Send consults the region cache to fan out the request to the right TiKV peers.
3. Responses stream back as protobuf chunks; distsql.SelectResult decodes them into chunk.Chunk and feeds the executor.
4. Region splits, leader changes, and stale-read errors trigger automatic retries with refreshed routing.

Write path

Writes never go through coprocessor. The executor builds row mutations and stuffs them into the active kv.Transaction.MemBuffer. At commit time, the session calls Transaction.Commit, which delegates to client-go's two-phase commit (prewrite → commit). Pessimistic transactions also acquire pessimistic locks on each SELECT FOR UPDATE / DML statement.

Coprocessor cache

coprocessor_cache.go provides a per-instance cache of coprocessor results keyed by request hash. It is used to avoid re-running expensive scans for repeated identical queries (most useful for prepared statements that hit the same region with the same filter).

MPP execution

For TiFlash queries, the planner produces an MPP plan. pkg/store/copr/mpp.go and pkg/executor/mpp_gather.go cooperate:

1. The executor's MPPGather enumerates the participating TiFlash peers via copr.MPPClient.
2. A coordinator (registered in pkg/executor/mppcoordmanager/) tracks the query.
3. Each TiFlash worker runs its assigned plan fragment and exchanges intermediate results with peers via MPP exchange operators.
4. Final results stream back through the gather operator.

mpp_probe.go is the health-check used by the planner to decide whether TiFlash is reachable for a given query.

Mock storage (unistore)

pkg/store/mockstore/unistore/ is an in-process implementation of the TiKV API. It is what --store=unistore selects. It supports MVCC, transactions, coprocessor execution, and even a subset of MPP for TiFlash testing. Tests in pkg/testkit/ build TiDB on top of unistore and run real SQL through the entire stack without external processes.

GC worker

pkg/store/gcworker/ runs the TiDB-side coordination of distributed garbage collection: pick a safe-point, advertise it through PD, and let TiKV reclaim old MVCC versions.

Integration points

• Executor: read paths use pkg/distsql/ + pkg/store/copr/; write paths use kv.Transaction.
• Session: holds the active kv.Transaction; manages start TS and commit TS.
• DDL: uses kv.Transaction to mutate metadata via pkg/meta.
• Statistics: writes histograms via kv.Transaction (using mysql.stats_* tables).
• BR / Lightning: bypass pkg/store/ entirely and talk to TiKV directly via gRPC, but they still use pkg/tablecodec/ to compute correct key layouts.

Entry points for modification

• New pushdown predicate kinds → pkg/expression/infer_pushdown.go plus pkg/expression/expr_to_pb.go.
• Coprocessor protocol field → coordinate with the TiKV side; update pkg/store/copr/coprocessor.go and pkg/distsql/.
• New region routing optimisation → pkg/store/copr/region_cache.go.
• New mock-store behaviour for tests → pkg/store/mockstore/unistore/.
• Per AGENTS.md -> Task -> Validation Matrix, storage changes need targeted unit tests; behaviour that depends on real TiKV semantics needs a tests/realtikvtest/ test.