Storage backends

Vault never reads or writes raw bytes directly. Every persistence call goes through the AES-GCM barrier (vault/barrier_aes_gcm.go) into a physical.Backend that knows how to talk to a specific KV store. The repository ships 21+ implementations under physical/, plus the inmem test backend in sdk/physical/inmem/.

Purpose

• Provide a uniform Get, Put, Delete, List, Transaction interface across radically different storage systems.
• Optionally provide HA primitives (HABackend) for backends that can supply distributed locking.
• Optionally provide transactional semantics (Transactional) for backends that support multi-key atomic writes.

Directory layout

physical/                       # primary implementations
├── aerospike/
├── alicloudoss/
├── azure/
├── cassandra/
├── cockroachdb/
├── consul/                     # HA-capable
├── couchdb/
├── dynamodb/                   # HA-capable
├── etcd/                       # HA-capable
├── foundationdb/
├── gcs/                        # HA-capable
├── manta/
├── mssql/
├── mysql/
├── oci/                        # HA-capable
├── postgresql/
├── raft/                       # integrated storage; HA via embedded raft
├── s3/
├── spanner/
├── swift/
└── zookeeper/                  # HA-capable

sdk/physical/
├── physical.go                 # Backend, HABackend, Transactional interfaces
├── physical_view.go            # Per-mount prefix view
├── transactions.go             # Helpers for backends without native transactions
├── cache.go                    # Optional read cache
├── encoding.go                 # Encode/decode helpers
├── error.go, path_error.go, latency.go
├── file/                       # local filesystem (dev-only)
├── inmem/                      # in-memory; HA-capable via channels
└── testing.go                  # backend conformance harness

Key abstractions

Symbol	File	Description
Backend	sdk/physical/physical.go	Get, Put, Delete, List.
HABackend	sdk/physical/physical.go	LockWith, HAEnabled.
Transactional	sdk/physical/physical.go	Transaction(ctx, []TxnEntry).
MountTableLimitingBackend	sdk/physical/physical.go	Wraps another backend with a configurable size cap on a single value.
View	sdk/physical/physical_view.go	Per-mount prefix wrapper.
Cache	sdk/physical/cache.go	Read-through LRU.
Factory	sdk/physical/physical.go	func(map[string]string, hclog.Logger) (Backend, error) — every backend registers one.

Each backend implements the interfaces it supports; the server feature-detects (if ha, ok := b.(HABackend); ok) and configures HA accordingly.

Picking a backend

graph TD
    Need[I need to store Vault data] --> Q1{Need integrated HA?}
    Q1 -->|Yes, no external dep| Raft[raft]
    Q1 -->|Yes, already have Consul| Consul[consul]
    Q1 -->|Yes, on Kubernetes| K8s[Use Raft + Kubernetes service registration]
    Q1 -->|No, single node ok| Q2{Cloud-managed object store ok?}
    Q2 -->|S3/GCS/Azure Blob| Cloud[s3 / gcs / azure]
    Q2 -->|Postgres/MySQL| SQL[postgresql / mysql]
    Q2 -->|Filesystem dev only| File[file]

The official recommendation since Vault 1.4 is integrated Raft for new deployments. See Raft and HA.

Transactions and the barrier

The barrier (vault/barrier_aes_gcm.go) batches up to four small writes into a single transaction when the backend supports it (Postgres, etcd, raft, dynamodb). Backends without transactions get one Put at a time and rely on idempotency for crash safety.

HA semantics

HABackend.LockWith(key, value) returns a Lock whose Lock(stopCh) blocks until the leader lock is acquired. Unlock releases it. Vault's standby loop (vault/ha.go) uses this exact API to perform leader election. With Raft the lock is "whoever the Raft leader is"; with Consul/etcd it's a session-backed key.

Service registration

Independent of storage, serviceregistration/ (with consul/ and kubernetes/ implementations) lets the server publish its address and active/standby state to a service registry. This is purely for discovery; it doesn't affect storage.

Integration points

• Created by command/server.go from the storage block in the config.
• Wrapped by the encryption barrier in Core.
• The router gives each mount a View so its keys are prefixed with the mount UUID.
• Quotas, identity store, mount tables, and audit table all sit on the same backend.

Entry points for modification

• Add a new backend: create physical/<name>/, implement Backend (and HABackend / Transactional if applicable), register a factory in command/commands.go.
• Wrap an existing backend with caching / latency: see sdk/physical/cache.go and sdk/physical/latency.go.
• The conformance harness in sdk/physical/testing.go exercises every method; new backends should pass it.