Design decisions

Notes on the architectural choices that show up everywhere in the code.

Single binary, no daemon split

MinIO ships as one binary. No separate gateway, no separate metadata server, no sidecars. Pros: deployment simplicity. Cons: every node carries every subsystem's code, so the binary is large and a memory leak in one subsystem affects all of them.

cmd/ is intentionally flat

cmd/ has 470+ files but no sub-packages. The convention is "one concept per top-level file." Reasons:

• Most files reach across many concerns; sub-packages would require many cross-imports.
• It keeps goimports deterministic.
• It makes git grep the primary discovery tool.

The trade-off is a large per-file surface area — cmd/object-handlers.go is 123 KB.

Reusable bits live in internal/

internal/ is the place to factor out anything that has a clean module boundary, doesn't depend on cmd/, and would benefit from independent unit tests. The compiler enforces "internal" visibility — nothing outside this module can import these packages.

ObjectLayer is the single backend interface

All code that needs to read or write objects calls into the ObjectLayer interface (cmd/object-api-interface.go). Today the only production implementation is erasureServerPools. Historically there were filesystem and gateway backends; they were removed in favour of the unified erasure path.

Server pools allow capacity expansion

A pool is the unit of expansion: you add a new pool of nodes/drives to a running cluster instead of growing existing erasure sets. Sets stay the same shape forever, which keeps healing simple and predictable. The cost is that two pools can drift in performance and fill rate; the rebalance subsystem (cmd/erasure-server-pool-rebalance.go) addresses that.

xl.meta is a single MessagePack file

The current on-disk metadata is one msgp file per object (xl.meta). Earlier designs split it into multiple JSON files (v1). The single-file design:

• Is faster to read/write (one syscall per drive).
• Survives partial writes more cleanly.
• Encodes versions, parts, free-version state, encryption metadata, replication state, and inline tiny data in a single blob.

Cons: schema migration is harder; the v2-legacy file (cmd/xl-storage-format-v2-legacy.go) keeps the upgrade path alive.

Locks live above the storage layer

Distributed locking (cmd/namespace-lock.go → internal/dsync/) is a separate concern from the storage backend. This keeps the backend simple and lets us swap lock implementations (in-process for single-node, DSync for distributed) without touching xl-storage.

Replication is async and per-bucket

Bucket replication runs as a worker pool fed by xl.meta state changes. Sync replication (block on the target ack) was rejected because:

• Targets can be slow or unreachable; sync replication makes the source slow too.
• The xl.meta state machine cleanly recovers from arbitrary failure points.
• The data scanner re-drives stuck entries.

Site replication builds on this: it configures bucket replication targets between every pair of sites at bootstrap time, then keeps IAM and bucket-config in step via the peer REST.

Generated MessagePack codecs

The repo uses github.com/tinylib/msgp for *_gen.go codecs instead of encoding/json or reflection-heavy alternatives. Reasons:

• Predictable allocation count.
• Compact wire format.
• Code is reviewable: codecs are committed and you can grep them.

Cons: schema changes require regen and migration logic.

Two metrics endpoints

v2 (one giant collector) and v3 (one collector per concern) coexist. v2 is kept around because some operators have dashboards and alerts wired to it. New metrics go to v3.

Embedded console

The web UI is the github.com/minio/console Go module, embedded directly into the server binary. Reasons:

• Operators don't run a separate UI process.
• The UI is versioned with the server.

Cons: the UI is a substantial chunk of dependency surface; turning it off requires a custom build.