pkg/k8s

Active contributors: aanm, christarazi, sayboras, mhofstetter, gandro

Purpose

pkg/k8s/ is Cilium's Kubernetes client + watcher framework. Every binary that talks to a kube-apiserver depends on it: agent, operator, clustermesh-apiserver, and the CLI. It also defines all Cilium CRD Go types and the generated typed client + listers + informers for them.

Directory layout

pkg/k8s/
├── apis/cilium.io/                 # CRD types (v2, v2alpha1)
│   ├── v2/                         # GA CRDs
│   ├── v2alpha1/                   # alpha CRDs
│   ├── client/                     # generated typed client + listers + informers
│   └── client/crds/                # rendered CRD YAML
├── client/                         # Cilium's wrapper around client-go
├── watchers/                       # subscribed Kubernetes informers
├── resource/                       # generic Resource[T] cell wrapping informers
├── statedb/                        # statedb integration for k8s objects
├── synced/                         # sync state across watchers
├── slim/                           # trimmed copies of k8s types (memory-tuned)
├── types/                          # cross-package shared types
├── utils/                          # helpers (parsing labels, names, namespaces)
├── testutils/                      # fakes for tests
└── ...

Resource[T]

pkg/k8s/resource/ exposes a generic Resource[T] Hive cell that wraps a Kubernetes informer for any typed resource. Consumers cell.In a resource.Resource[*v1.Pod] and call Events(ctx) to receive Upsert and Delete events. This is the modern alternative to ad-hoc informers.

Example:

type params struct {
    cell.In
    Pods    resource.Resource[*slim_v1.Pod]
    Nodes   resource.Resource[*v2.CiliumNode]
}

func register(p params, lc cell.Lifecycle) {
    lc.Append(cell.Hook{
        OnStart: func(ctx cell.HookContext) error {
            for ev := range p.Pods.Events(ctx) {
                // handle upsert/delete
                ev.Done(nil)
            }
            return nil
        },
    })
}

Resource events are deduplicated, deliverable-once, and integrate with statedb where helpful.

Slim types

The agent watches many Pod resources; storing the full corev1.Pod consumes a lot of memory. pkg/k8s/slim/ defines a slimmed-down API tree (slim_v1.Pod, etc.) that drops unused fields. Watchers cast incoming objects via the slim conversion at consumption time.

CRD types

All CRD Go types live under pkg/k8s/apis/cilium.io/v2/:

• types_cnp.go / types_ccnp.go — policies.
• types_identity.go — CiliumIdentity.
• types_endpoint.go — CiliumEndpoint.
• types_node.go — CiliumNode.
• types_envoy_config.go — CiliumEnvoyConfig.
• types_cidr_group.go — CiliumCIDRGroup.
• ... and many more.

Alpha types live in pkg/k8s/apis/cilium.io/v2alpha1/. After adding or modifying a type, run make generate-k8s-api to regenerate deepcopy methods, the YAML schema (under client/crds/), and the typed client.

The client wrapper

pkg/k8s/client/ produces a single client.Clientset that wraps:

• The standard Kubernetes typed client.
• The Cilium typed client (for CRDs).
• The slim client (memory-tuned views).
• An apiextensions client (for CRD installation).

The wrapper is created by a Hive cell and injected wherever needed.

Watchers (legacy)

pkg/k8s/watchers/ contains the older watch-based pattern, where each subsystem registers an informer with its own callbacks. The modern Resource[T] pattern is replacing these, but watchers remain in place for compatibility.

Integration points

• Operator + agent: both depend on this package; their behaviour around CRDs is identical.
• Cluster Mesh: uses CRD types as the schema for cross-cluster export.
• Generated code: make generate-k8s-api regenerates everything mechanical.

Entry points for modification

• New CRD: add a Go type under pkg/k8s/apis/cilium.io/v2/ (or v2alpha1 for alpha) with +kubebuilder: annotations. Run make generate-k8s-api. Add docs to Documentation/crdlist.rst.
• New Resource consumer: cell.In a resource.Resource[*MyType].
• New watcher in legacy style: under pkg/k8s/watchers/.

Key source files

File	Purpose
pkg/k8s/client/client.go	Combined clientset.
pkg/k8s/resource/resource.go	Generic Resource[T].
pkg/k8s/apis/cilium.io/v2/types_cnp.go	CNP CRD.
pkg/k8s/apis/cilium.io/v2/types_node.go	CiliumNode CRD.
pkg/k8s/slim/k8s/api/core/v1/types.go	Slimmed core types.
pkg/k8s/synced/synced.go	Sync state coordination.