Distributed locking

Every object operation that mutates state acquires a cluster-wide lock first. The lock layer has three pieces: a high-level namespace API in cmd/namespace-lock.go, the cluster-wide DSync algorithm in internal/dsync/, and an in-process locker for single-node mode in cmd/local-locker.go. Locks are exposed across the cluster via the lock-rest server (cmd/lock-rest-server.go).

Purpose

Serialise concurrent writes and read-modify-write sequences across nodes so that erasure operations always see a consistent view of xl.meta, multipart state, and bucket metadata.

Layout

cmd/
├── namespace-lock.go            # Public API: NewNSLock, GetLock, GetRLock
├── local-locker.go              # In-process lock implementation
├── lock-rest-client.go          # Client wrapper for cluster locks
├── lock-rest-server.go          # /minio/lock/v1/* endpoints
├── lock-rest-server-common.go   # Shared types
└── shared-lock.go               # Refresh-loop helpers for long-held locks
internal/dsync/
├── dsync.go                     # DSync algorithm
├── drwmutex.go                  # Distributed RW mutex
└── ...
internal/lock/
├── lock_*.go                    # Filesystem-level fcntl/flock primitives
internal/lsync/
└── lsync.go                     # Lightweight in-process sync helpers

Key abstractions

Symbol	File	What it does
RWLocker	cmd/namespace-lock.go	Public interface returned by NewNSLock.
nsLockMap	cmd/namespace-lock.go	Per-server table of acquired locks.
localLocker	cmd/local-locker.go	Single-process implementation.
lockRESTClient	cmd/lock-rest-client.go	RPC wrapper used in distributed mode.
DRWMutex	internal/dsync/drwmutex.go	The DRWMutex algorithm.

How it works

sequenceDiagram
    participant H as Handler
    participant NS as namespace-lock
    participant DS as DSync (cluster)
    participant LL as localLocker

    H->>NS: NewNSLock(bucket, object)
    NS->>NS: globalIsErasure?
    alt single node
        NS->>LL: acquire write lock
    else distributed
        NS->>DS: acquire DRWMutex (quorum of nodes)
        DS->>DS: contact lock-rest peers in parallel
        DS->>DS: collect quorum of OKs
    end
    H->>H: do work
    H->>NS: Unlock(lkctx)
    NS->>LL: release
    NS->>DS: release

DSync

internal/dsync/ implements the DRWMutex algorithm: a write-lock is granted only if a strict majority of the cluster's lock servers accept the request. Each lock-rest peer maintains an in-memory ledger; if the locker process restarts, leases are re-acquired with refreshed UIDs. The leases are time-bounded; long-running operations refresh them periodically (cmd/shared-lock.go).

lock-rest

cmd/lock-rest-server.go exposes:

• POST /minio/lock/v1/lock
• POST /minio/lock/v1/rlock
• POST /minio/lock/v1/unlock
• POST /minio/lock/v1/runlock
• POST /minio/lock/v1/refresh
• POST /minio/lock/v1/expired
• POST /minio/lock/v1/force-unlock

The server side dispatches to the local locker; the client side (cmd/lock-rest-client.go) is what DSync calls into.

Lock keys

• Bucket-level locks use bucket as the key.
• Object-level locks use bucket/object.
• Multipart upload locks use bucket/object/uploadID.
• Background workers use synthetic keys (e.g. data-scanner).

The key namespace is flat; collisions across categories are avoided by prefixing.

Refreshing and expiry

Long-held locks (replication batch jobs, healing scans) get a refresh goroutine via cmd/shared-lock.go. If the holder dies, peers expire the lease using the timestamp written when the lock was granted.

Force unlock

The admin API exposes a force-unlock endpoint for stuck locks (cmd/admin-handlers.go). mc admin lock is the user-facing command. Use sparingly — bypassing a held lock can corrupt xl.meta if the original holder is alive.

Integration points

• Used by every ObjectLayer method in cmd/erasure-server-pool.go, cmd/erasure-object.go, cmd/erasure-multipart.go.
• Used by IAM (cmd/iam-store.go) when reloading policies.
• Used by replication and lifecycle workers.

Entry points for modification

• Add a new lock category. Use objectAPI.NewNSLock(...) with a unique key. Don't reach past namespace-lock.go to DSync directly.
• Tune timeouts. Default operation timeout is set in cmd/globals.go (globalOperationTimeout); shouldn't be changed without measuring impact.
• Add metrics. Lock metrics flow through cmd/metrics-v2.go and the v3 collectors.