Dumpling

Dumpling is a logical exporter for MySQL-compatible databases. It is designed to replace mysqldump and mydumper when the source is a TiDB cluster, but it works against MySQL too.

Source

• CLI: dumpling/cmd/.
• Library: dumpling/.
• Build: make build_dumpling → bin/dumpling.
• Top-level docs: dumpling/README.md.

Directory layout

dumpling/
├── cmd/dumpling/         # CLI entry
├── cli/                  # Flag wiring
├── context/              # Run context
├── export/               # Core export pipeline (largest sub-package)
├── log/                  # Logging
├── tests/                # Shell-driven integration tests
└── tidb-lightning.toml   # (from related Lightning config example)

The dumpling/export/ package holds the majority of the implementation: connection pooling, parallel table splitting, row-stream writers, and the various output formats.

What it does

Dumpling produces a directory of dump files such that:

• *-schema.sql files contain CREATE TABLE statements.
• *.sql (or *.csv) files contain row data, split into multiple files for parallel restore.
• Optional metadata files capture the dump's binlog/TS position so a downstream replication tool (e.g., DM, TiCDC) can pick up where Dumpling left off.

Notable features (from dumpling/README.md):

• Multi-table parallel export.
• Multiple output formats (SQL, CSV).
• Native writes to S3 and GCS without a local staging directory.
• Advanced table filtering (allowlist/denylist syntax).
• Consistency strategies: auto, flush, snapshot, lock, none.

How an export runs

graph TD
  cli[dumpling --host ...] --> ctx[Build context]
  ctx --> conn[Open connection pool]
  conn --> consist[Pick consistency strategy]
  consist --> split[Split tables into chunks]
  split --> workers[Parallel workers]
  workers --> rows[Stream rows from source]
  rows --> writer[SQL / CSV writer]
  writer --> store[Local FS / S3 / GCS]
  workers --> meta[Write metadata]

1. Connect: Dumpling opens a pool of MySQL-protocol connections to the source.
2. Consistency: depending on the chosen strategy, it issues FLUSH TABLES WITH READ LOCK, SET TRANSACTION SNAPSHOT, takes a TSO from PD, or just runs without coordination.
3. Split: tables are split into chunks (by --rows, by primary key range, or by partition). Each chunk becomes one output file.
4. Stream: workers pull from the source and write to the destination. Compression and CSV escaping are handled at write time.
5. Metadata: a metadata file records the source position (binlog file/pos for MySQL, snapshot TSO for TiDB).

Building and running locally

make build_dumpling
bin/dumpling \
  -h 127.0.0.1 -P 4000 -u root \
  -t 8 \
  --output /tmp/dump \
  --filetype sql

The integration tests under dumpling/tests/ require a local MySQL on 127.0.0.1:3306 plus a few helper binaries (sync_diff_inspector, tidb-server, tidb-lightning, minio); the full instructions are in dumpling/README.md.

Integration with this repo

• Dumpling speaks the MySQL wire protocol both as client (to fetch data) and through pkg/parser/ to round-trip schema definitions for TiDB-specific syntax.
• The dumpling/export/ package depends on pkg/parser/format/ to render TiDB-flavored DDL correctly when the source is a TiDB cluster.
• It does not link into the SQL server itself — it is a standalone client.