Lifecycle and tiering

Bucket Information Lifecycle Management (ILM) controls when objects expire, when versions are reclaimed, and when objects are transitioned to a colder tier. The tiering layer (cmd/tier.go) defines the warm backends; lifecycle (cmd/bucket-lifecycle.go) decides what moves and when.

Purpose

• Implement the AWS bucket lifecycle XML spec (Expiration, Transition, NoncurrentVersionExpiration, ...).
• Move cold objects to a remote backend (Azure Blob, Google Cloud Storage, AWS S3, or another MinIO).
• Enforce the rules in the background via the data scanner.

Layout

cmd/
├── bucket-lifecycle.go               # Apply lifecycle rules, transition logic
├── bucket-lifecycle-handlers.go      # PUT/GET/DELETE bucket lifecycle XML
├── bucket-lifecycle-audit.go         # Audit events for lifecycle actions
├── ilm-config.go                     # MINIO_ILM_* tunables
├── lceventsrc_string.go              # stringer enum
├── tier.go                           # Tier definitions and resolver
├── tier-handlers.go
├── tier-last-day-stats.go
├── tier-sweeper.go                   # Cleanup of orphan objects on tier
├── warm-backend.go                   # Common interface
├── warm-backend-azure.go             # Azure Blob Storage
├── warm-backend-gcs.go               # Google Cloud Storage
├── warm-backend-s3.go                # AWS S3 / S3-compatible
└── warm-backend-minio.go             # Another MinIO cluster
internal/bucket/lifecycle/
├── lifecycle.go                      # XML model + per-object decisions
├── rule.go, filter.go, expiration.go
├── transition.go
└── ...
internal/config/ilm/
└── ilm.go                            # ILM config knobs

Key abstractions

Symbol	File	What it is
lifecycle.Lifecycle	internal/bucket/lifecycle/lifecycle.go	The compiled rules for one bucket.
lifecycle.Action	same	The decision (Expire, Transition, NoOp).
TierConfigMgr	cmd/tier.go	The cluster's tier registry.
WarmBackend	cmd/warm-backend.go	The interface every tier backend implements.

How it works

graph TD
    SCAN[Data scanner] -->|per object| EVAL[lifecycle.Eval]
    EVAL --> ACT{Action}
    ACT -- Expire --> DEL[Delete or convert to free version]
    ACT -- Transition --> WARM[Read object]
    WARM --> WRITE[Write to warm backend]
    WRITE --> META[Update xl.meta with tier handle]
    ACT -- NoOp --> SKIP[Skip]

• The scanner walks every object every cycle. For each object version it evaluates the bucket lifecycle config.
• Expirations honour versioning: a versioned bucket converts to a delete marker (or free version), an unversioned bucket deletes outright.
• Transitions stream the object payload through the warm backend's Put and replace the local payload with a tier handle in xl.meta. Subsequent reads are proxied via cmd/object-handlers.go's tier-fetch path.
• The "tier sweeper" (cmd/tier-sweeper.go) removes orphan objects on the tier when the source is gone.

Tiers

A tier is defined as JSON via mc admin tier add and persisted under .minio.sys/buckets/.config/tier/. Each tier has:

• A name (e.g. WARMS3).
• A type: s3, azure, gcs, or minio.
• Endpoint, region, and credentials.
• Optional storage class hint.

The runtime registers a WarmBackend per tier and uses it for transitions. The interface includes Put, Get, Remove, InUse.

Audit and metrics

• Lifecycle actions emit audit events via cmd/bucket-lifecycle-audit.go (and the regular audit pipeline in internal/logger/audit/).
• ILM metrics surface in cmd/metrics-v3-ilm.go.
• Tier-side stats live in cmd/tier-last-day-stats.go.

Testing

• make test-ilm — bucket lifecycle expirations + replication interactions.
• make test-ilm-transition — transitions to warm backends + healing.

Both shell harnesses live under docs/bucket/lifecycle/ and docs/bucket/replication/.

Integration points

• Driven by the data scanner.
• Tier definitions persist in IAM-style config (internal/config/ilm/).
• Reads from a tiered object are forwarded through cmd/object-handlers.go GetObject path.
• Replication interacts with ILM: a replicated object respects the destination's lifecycle, but versioned source-side delete markers must propagate even after expiration.

Entry points for modification

• New tier backend. Implement WarmBackend in a new cmd/warm-backend-<name>.go, register the type in cmd/tier.go.
• New rule predicate. Extend internal/bucket/lifecycle/ to parse the new XML element and produce the new action.
• Tune scanner cadence. Settings live in internal/config/ilm/ and internal/config/scanner/.