Changefeed (CDC)

CockroachDB's change data capture exposes every committed row mutation as a stream of events. Implementation lives mostly in pkg/ccl/changefeedccl/, with rangefeed plumbing in pkg/kv/kvserver/rangefeed/ and pkg/kv/kvclient/rangefeed/.

Data flow

graph LR
  KV["KV writes"] --> RF["per-replica rangefeed processor"]
  RF -->|events| Aggregator["changefeedccl aggregators"]
  Aggregator --> Encoder["row encoder (Avro / JSON / Parquet / Topic)"]
  Encoder --> Sink["sink (Kafka, Pub/Sub, webhook, S3, …)"]
  Aggregator --> Resolved["resolved-timestamp checkpoints"]
  Resolved -->|persist| Job["jobs registry"]

Components

• Rangefeed processor — pkg/kv/kvserver/rangefeed/. Reads from MVCC and a closed-timestamp source, emits values and checkpoints over a gRPC stream.
• Rangefeed client — pkg/kv/kvclient/rangefeed/. The consumer side; handles backfills and reconnects.
• Aggregator — pkg/ccl/changefeedccl/changefeed_processors.go. A DistSQL processor on each gateway that collects events from many ranges, deduplicates, and orders by resolved timestamp.
• Encoders — pkg/ccl/changefeedccl/encoder.go and friends. Serializes rows to JSON, Avro, CSV, or Parquet; supports schema-registry integration for Avro.
• Sinks — pkg/ccl/changefeedccl/sink_*.go. One file per destination: Kafka (v2 native producer + an older sink), Google Pub/Sub, webhooks, S3-compatible buckets, Pulsar, Azure Event Hubs, Snowflake.
• Schema feed — pkg/ccl/changefeedccl/schemafeed/. Watches descriptor changes so that schema-version-aware encoders can re-resolve types.

Job lifecycle

A CREATE CHANGEFEED statement creates a JobType_CHANGEFEED job (pkg/jobs/jobspb/). The resumer (pkg/ccl/changefeedccl/changefeed_dist.go) plans a DistSQL flow with one aggregator per gateway and one rangefeed source per range owned by a node. Progress is recorded as a "resolved high-water timestamp" in the job's progress; on restart, the changefeed resumes from that timestamp.

Guarantees

• At-least-once delivery is the default. Exactly-once is delegated to the sink (Kafka transactional, table-based dedup).
• Per-key order is preserved.
• Resolved-timestamp checkpoints are emitted on a configurable cadence and indicate "no new events at or below this timestamp will appear".

Performance

Changefeeds are tuned by:

• changefeed.event_consumer_workers — fan-out per aggregator.
• changefeed.frontier_checkpoint_frequency and _max_bytes — checkpoint cadence.
• kv.rangefeed.enabled — globally enable/disable rangefeeds.
• The closed-timestamp interval in kv.closed_timestamp.target_duration controls minimum end-to-end latency.

Related code

File / package	Purpose
pkg/ccl/changefeedccl/changefeedbase/	Shared types and options
pkg/ccl/changefeedccl/changefeed_dist.go	Job planner
pkg/ccl/changefeedccl/kvfeed/	Reads rangefeed events into the change pipeline
pkg/ccl/changefeedccl/cdcevent/	Decoded change events
pkg/ccl/changefeedccl/schemafeed/	Schema/descriptor watcher
pkg/util/parquet/	Parquet encoder used by some sinks

Related pages

• KV server / Rangefeed — the source.
• Jobs — execution model.
• SQL — CREATE CHANGEFEED planning.