Bucket replication flow

A walk-through of what happens between "client PUTs an object" and "the same object lands on the replication target".

sequenceDiagram
    participant Client
    participant Source as Source MinIO
    participant Q as Replication queue
    participant W as Worker
    participant Target as Target S3 endpoint
    participant Scanner as Data scanner

    Client->>Source: PUT object
    Source->>Source: Write to xl.meta with replication=PENDING
    Source-->>Client: 200 OK
    Source->>Q: Enqueue replicateObjectInfo
    Q->>W: Pick up
    W->>Source: Read object payload + metadata
    W->>Target: PUT (minio-go)
    alt success
        Target-->>W: 200
        W->>Source: Mark COMPLETED in xl.meta
    else failure
        Target-->>W: 4xx/5xx
        W->>Source: Mark FAILED + retry
        Scanner->>Q: Re-drive FAILED entries
    end

Pieces involved

Step	File
Source-side write	cmd/object-handlers.go, cmd/object-multipart-handlers.go
Replication state in xl.meta	cmd/xl-storage-format-v2.go, cmd/bucket-replication-utils.go
Target registry	cmd/bucket-targets.go
Worker pool	cmd/bucket-replication.go
Per-bucket rules	internal/bucket/replication/
Re-drive of stuck entries	cmd/data-scanner.go
Stats and metrics	cmd/bucket-replication-stats.go, cmd/metrics-v3-bucket-replication.go

Replication of metadata-only changes

Tag updates, ACL changes, lifecycle transitions, and DELETE markers all replicate too. Each emits its own queue entry; the worker translates it into the appropriate target API call.

Multipart and large objects

For multipart uploads the worker preserves part boundaries on the target. The source xl.meta tracks per-part replication state so a partial failure resumes from the failed part instead of restarting.

Reliability

• The queue is per-server but persisted in xl.meta; restarts don't lose pending entries.
• Failed entries flip the bucket's replication state to FAILED for that object until success.
• The scanner periodically re-drives PENDING and FAILED entries (cmd/data-scanner.go) so stuck entries eventually leave.
• Bandwidth caps per target prevent one slow target from starving others (internal/bucket/bandwidth/).

Read proxying

When a client reads an object whose replication is still PENDING, MinIO can transparently proxy the read to the target so the client sees a consistent view. This is opt-in per bucket and lives in cmd/bucket-replication.go.

Knobs

• MINIO_API_REPLICATION_WORKERS — worker pool size.
• MINIO_API_REPLICATION_PRIORITY — tune queue priorities (multipart vs metadata).
• Per-target throttle in internal/bucket/bandwidth/.

Test entry points

• make test-replication-2site, make test-replication-3site.
• make test-delete-replication, make test-delete-marker-proxying.
• make test-sio-error (encryption + replication interaction).

Where to start reading

• cmd/bucket-replication.go (the worker)
• cmd/bucket-targets.go (where targets and credentials live)
• internal/bucket/replication/ (the XML model)

See Replication for the runtime view, and Multi-site clusters for the cluster-level analogue.