Open-Source Wikis

/

Kubernetes

/

Systems

/

API registry and storage

kubernetes/kubernetes

API registry and storage

This is the layer that turns "kubectl apply -f deploy.yaml" into a row in etcd. It is the single most replicated pattern in the project: every built-in resource is wired the same way.

Pieces

graph TD
    Client -->|HTTP| Endpoints[REST endpoints<br/>staging/src/k8s.io/apiserver/pkg/endpoints]
    Endpoints --> Strategy[Per-resource strategy<br/>pkg/registry/&lt;group&gt;/&lt;resource&gt;/strategy.go]
    Strategy --> GenericStore[Generic registry store<br/>staging/src/k8s.io/apiserver/pkg/registry/generic/registry]
    GenericStore --> Cacher[Watch cacher<br/>staging/src/k8s.io/apiserver/pkg/storage/cacher]
    Cacher --> Etcd3[etcd3 storage<br/>staging/src/k8s.io/apiserver/pkg/storage/etcd3]
    Etcd3 --> ETCD[(etcd)]

Endpoints (HTTP layer)

staging/src/k8s.io/apiserver/pkg/endpoints/installer.go walks the registered REST handlers and installs HTTP routes. The naming convention is {prefix}/{group}/{version}/{namespaces/{ns}/}{resource}/{name}/{subresource}.

Each verb (Get, List, Create, Update, Patch, Delete, Watch, Connect, DeleteCollection) is built generically; the per-resource code only has to implement the right Storage interface (rest.Getter, rest.Lister, …).

Handlers are wrapped by the standard filter chain (panic-recovery, request-info, authentication, audit, authorization, max-in-flight, P&F, content-type negotiation). See components/kube-apiserver/auth-chain.md.

Generic registry store

staging/src/k8s.io/apiserver/pkg/registry/generic/registry/store.go is the concrete REST implementation. It is configured per-resource with:

  • NewFunc / NewListFunc — typed object constructors
  • KeyFunc / KeyRootFunc — etcd key shape
  • PredicateFunc — turns selectors into a storage.SelectionPredicate
  • Strategy — validation, defaulting, status preservation
  • TableConvertorkubectl get -o wide rendering
  • AfterCreate / AfterUpdate / AfterDelete — optional hooks

The store delegates to storage.Interface. The default storage is the watch cacher wrapping etcd3.

Strategy

The per-resource strategy is the place where validation and defaulting live. A typical strategy:

type strategy struct {
    runtime.ObjectTyper
    names.NameGenerator
}

func (strategy) NamespaceScoped() bool                { return true }
func (strategy) PrepareForCreate(ctx, obj) {
    deployment := obj.(*apps.Deployment)
    deployment.Status = apps.DeploymentStatus{}
    deployment.Generation = 1
    pod.DropDisabledFields(&deployment.Spec, nil)
}
func (strategy) Validate(ctx, obj) field.ErrorList    { return validation.ValidateDeployment(obj.(*apps.Deployment)) }
func (strategy) PrepareForUpdate(ctx, new, old) {
    new := new.(*apps.Deployment); old := old.(*apps.Deployment)
    new.Status = old.Status
    if !apiequality.Semantic.DeepEqual(old.Spec, new.Spec) {
        new.Generation = old.Generation + 1
    }
    pod.DropDisabledFields(&new.Spec, &old.Spec)
}

Most resources have a separate status strategy registered for the /status subresource. Status writes from controllers go through the status-strategy, which preserves spec and bumps metadata.resourceVersion only.

Watch cacher

staging/src/k8s.io/apiserver/pkg/storage/cacher/ wraps an etcd-backed Storage with an in-memory ring buffer. Reads and watches are served from the ring; writes still go to etcd. The cacher:

  • Serves LIST with resourceVersion=0 from cache (default for client-go informers).
  • Serves WATCH by replaying ring events from a given resourceVersion.
  • Bookmarks watch streams periodically so clients don't have to do a full re-list when the cache wraps.
  • Implements WatchList (a streaming list that emits Bookmark events as separator points).
  • Optionally serves consistent reads (etcd quorum read revision pins the cache snapshot).

The cacher is the single biggest reason a 5,000-node cluster doesn't melt etcd.

etcd3 storage

staging/src/k8s.io/apiserver/pkg/storage/etcd3/store.go is the concrete storage. It speaks the etcd3 client protocol, encodes objects via the registered codec (proto for built-ins, JSON for CRDs by default), and applies optional encryption-at-rest transformers.

Encryption providers are pluggable:

  • aescbc, aesgcm — symmetric encryption via in-process keys
  • kms — delegate to a KMS plugin (staging/src/k8s.io/kms)
  • secretbox — NaCl secretbox

Server-side apply

Apply is implemented as a special-case PATCH content-type. The merge logic lives in staging/src/k8s.io/apimachinery/pkg/util/managedfields/. Each field carries a manager annotation (e.g. kubectl-client-side-apply, kube-controller-manager); when two managers want to set the same field, the conflict is reported as a 409 unless --force is set.

Apply configurations (typed builders) live in staging/src/k8s.io/client-go/applyconfigurations/ — these are codegen'd from the API types and let typed controllers emit apply patches without hand-rolled JSON.

Discovery

/api, /apis, and /apis/<group> endpoints are served by staging/src/k8s.io/apiserver/pkg/endpoints/discovery/. There are two flavours:

  • Legacy discovery — returns a flat list per group/version.
  • Aggregated discovery — a single endpoint returns the full discovery document, keyed by group/version, in a structured form. Vastly cheaper for kubectl and client-go.

Aggregated discovery is on by default since 1.27.

OpenAPI

/openapi/v2 and /openapi/v3 endpoints serve the schema for every registered resource. Generation is in pkg/generated/openapi/zz_generated.openapi.go (74,000+ lines). The OpenAPI handler is in staging/src/k8s.io/apiserver/pkg/endpoints/openapi.

Subresources

Many resources have one or more subresources, served at /{name}/{subresource}:

  • /status — separate write path used by controllers
  • /scale — for Deployment, ReplicaSet, StatefulSet, the Scale subresource is the contract HPA uses
  • /exec, /attach, /portforward, /log — Pod streaming endpoints
  • /eviction — Pod eviction (respecting PDB)
  • /binding — Pod binding (used historically by the scheduler)
  • /resize — in-place resource resize (alpha)
  • /rotate-server-certificate — Node subresource for certificate rotation

Each subresource has its own storage type and may use an entirely different verb set.

Adding a resource: end-to-end

See components/kube-apiserver/rest-registry.md for the full checklist.

Key source files

File Purpose
staging/src/k8s.io/apiserver/pkg/endpoints/installer.go Endpoint discovery + HTTP route installation
staging/src/k8s.io/apiserver/pkg/registry/generic/registry/store.go Generic store
staging/src/k8s.io/apiserver/pkg/storage/cacher/cacher.go Watch cacher
staging/src/k8s.io/apiserver/pkg/storage/etcd3/store.go etcd3 storage
staging/src/k8s.io/apimachinery/pkg/util/managedfields/ Server-side apply merge
pkg/registry/<group>/<resource>/strategy.go Per-resource strategy
pkg/registry/<group>/<resource>/storage/storage.go Per-resource store wiring
pkg/controlplane/instance.go Wiring all strategies into the apiserver

Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.

API registry and storage – Kubernetes wiki | Factory