Multi-site clusters

A MinIO deployment can span multiple physically-separate clusters ("sites") that stay in step on every bucket, IAM principal, and lifecycle rule. Operators don't pre-script the sync — they run mc admin replicate add once and let MinIO handle it.

Promise

• Every bucket, IAM user, group, policy, lifecycle rule, encryption setting, and tagging change made on any site appears on every other site within seconds.
• Object data flows through the same per-bucket replication machinery used in single-cluster setups.
• A site can rejoin after an outage by running mc admin replicate resync.

What "in step" covers

Resource	Replicated?
Bucket creation/deletion	✓
Bucket policies, lifecycle, replication, encryption, tagging	✓
Quotas	✓
Local IAM users	✓
Groups	✓
Policies	✓
Service accounts	✓
LDAP/OIDC mappings	✓
Object data	✓ (via standard bucket replication, configured automatically)
KMS keys	✗ (KES is expected to be the same KES across all sites)
Server pool topology	✗
Per-site monitoring	✗

Pieces involved

graph TD
    OP[Operator] --> SR[mc admin replicate]
    SR --> SRH[admin-handlers-site-replication.go]
    SRH --> SRSYS[SiteReplicationSys]
    SRSYS --> IAM[IAMSys]
    SRSYS --> BMS[bucket metadata]
    SRSYS --> RT[bucket replication targets]
    BMS -->|sync| PEERS[Peer sites]
    IAM -->|sync| PEERS
    RT --> BR[Per-bucket replication workers]
    BR --> PEERS

Component	File
CLI	mc admin replicate ... (out of tree)
HTTP entry	cmd/admin-handlers-site-replication.go
Cluster coordinator	cmd/site-replication.go
IAM hooks	cmd/iam-store.go (notifies on every change)
Bucket metadata hooks	cmd/bucket-metadata-sys.go, cmd/bucket-handlers.go
Object data sync	cmd/bucket-replication.go (configured to all sites)
Status & metrics	cmd/site-replication-metrics.go

Bootstrapping

mc admin replicate add a b c

1. The CLI talks to one of the sites (call it A).
2. A pre-flight checks: every site responds, root creds match (or --use-cluster-replication provides them), no conflicting IDs.
3. A pushes existing IAM (users, groups, policies, service accounts) to B and C.
4. A pushes existing buckets and their config (lifecycle, replication, encryption, tagging, quota).
5. For every bucket on every site, A configures bucket-replication targets pointing at the other sites.
6. From this point on, all subsequent IAM and bucket-config changes propagate live; object writes flow through bucket replication.

Steady state

• Every IAM mutation in cmd/iam-store.go calls SiteReplicationSys.SyncLocalUser/... which fans out to peer sites.
• Every bucket-config change in cmd/bucket-handlers.go and friends does the same.
• Failures get logged and surfaced via mc admin replicate status.

Resync

If a site falls behind for any reason (long outage, accidental drift), mc admin replicate resync re-pushes the chosen "source of truth" site's IAM + bucket config to the laggard. Object data catches up via the standard replication queue.

Identity provider considerations

• With LDAP or OpenID as the identity source, the IDP is the source of truth for user existence; site replication still syncs policy mappings and service accounts.
• Multi-site test harnesses verify all three modes: native MinIO IDP, LDAP, OIDC.

Test entry points

• make test-site-replication-minio
• make test-site-replication-ldap
• make test-site-replication-oidc

The harnesses live in docs/site-replication/.

Where to start reading

• cmd/site-replication.go
• cmd/admin-handlers-site-replication.go
• docs/site-replication/run-multi-site-minio-idp.sh

See Site replication for the system-level view.