pkg/endpoint and pkg/endpointmanager

Active contributors: aanm, christarazi, joestringer, joamaki

Purpose

pkg/endpoint/ is the per-pod state machine. Each pod the agent manages on its node is one *endpoint.Endpoint with its own identity, policy, BPF programs, and conntrack maps. pkg/endpointmanager/ is the cluster-of-endpoints view: discovery, lookup, garbage collection, watchers.

Directory layout

pkg/endpoint/
├── endpoint.go                # the Endpoint type
├── endpoint_status.go         # status bookkeeping
├── policy.go                  # apply policy to endpoint
├── bpf.go                     # write per-endpoint BPF maps
├── proxy.go                   # L7 redirect setup
├── fqdn.go                    # FQDN policy refresh
├── restore.go                 # restore from disk after agent restart
├── endpoint_build_queue.go    # bounded queue serialising regeneration
├── regenerator/regenerator.go # the regenerate goroutine
├── log.go                     # endpoint-scoped slog logger
├── id/                        # endpoint id allocation
├── api/                       # REST handlers
├── creator/                   # CNI-driven endpoint creation
├── cell/                      # Hive cell wiring
├── metadata/                  # k8s metadata cache
├── regeneration/              # regen state machine constants
└── ...

pkg/endpointmanager/
├── manager.go                 # Endpoint registry per agent
├── lookup.go                  # find endpoint by id, label, IP, ...
├── observer.go                # change events
├── gc.go                      # endpoint garbage collection
└── ...

Endpoint lifecycle

stateDiagram-v2
    [*] --> WaitingForIdentity
    WaitingForIdentity --> Regenerating: identity allocated
    Regenerating --> Ready: bpf compiled + maps written
    Ready --> Regenerating: policy or labels changed
    Ready --> Disconnecting: pod deleted
    Disconnecting --> [*]
    Regenerating --> Disconnecting: cancellation

States are defined in pkg/endpoint/regeneration/. The regenerator (pkg/endpoint/regenerator/) drives the transitions:

1. Pull the latest SelectorPolicy from pkg/policy/.
2. Compute the policy map deltas.
3. If layout changed (new identity, new feature flag), call the loader to recompile and reattach.
4. Write the policy / nat / ct maps.
5. Notify observers (Hubble, REST API).

Key abstractions

Type	File	Role
Endpoint	pkg/endpoint/endpoint.go	The big state struct.
EndpointAPI	pkg/endpoint/api/	REST handlers (/endpoint/{id}/...).
Manager	pkg/endpointmanager/manager.go	Holds the local set of endpoints, exposes lookups.
BuildQueue	pkg/endpoint/endpoint_build_queue.go	Bounded queue that serialises expensive regeneration.
Regenerator	pkg/endpoint/regenerator/regenerator.go	Goroutine that consumes the queue and drives state transitions.
Restore	pkg/endpoint/restore.go	Rebuilds endpoints after agent restart from /var/run/cilium/state/<id>/.

Restore

After an agent restart, every endpoint's metadata is read from disk and the regenerator schedules a regeneration without dropping running traffic. The pre-existing BPF maps remain pinned, so packets continue to flow. Once the new agent has caught up with the API server, it replaces the policy map atomically.

This is one reason the agent runs as a host network DaemonSet with hostPID: false and uses bpffs pins extensively.

Concurrency

• One goroutine per endpoint owns its mutable state; external callers go through (*Endpoint).LockAlive().
• The build queue caps concurrent regenerations (endpointMaxRegenerationsInParallel) to limit CPU usage during policy storms.
• Writes to BPF maps are serialised through pkg/bpf/'s UpdateLock.

Integration points

• Policy: consumes SelectorPolicy from pkg/policy/.
• Datapath: invokes pkg/datapath/loader/ to compile/attach BPF programs.
• Identity: allocates and refreshes identities via pkg/identity/cache/.
• Hubble: publishes endpoint events for flow annotation.
• CNI: new endpoints come from plugins/cilium-cni/ via pkg/endpoint/creator/.

Entry points for modification

• New per-endpoint capability: add a method on Endpoint and call it during regeneration.
• New endpoint state: update pkg/endpoint/regeneration/state.go and the regenerator state machine.
• New restore data: extend the on-disk format under pkg/endpoint/restore.go and bump the version.

Key source files

File	Purpose
pkg/endpoint/endpoint.go	Core type.
pkg/endpoint/regenerator/regenerator.go	Regen loop.
pkg/endpoint/restore.go	Restart restore.
pkg/endpoint/policy.go	Policy application.
pkg/endpoint/bpf.go	BPF map write.
pkg/endpointmanager/manager.go	Manager.
plugins/cilium-cni/cmd/main.go	CNI client that creates endpoints.