Workloadmeta and tagger

Purpose

The Datadog Agent collects telemetry from a moving target: containers, pods, and tasks come and go, often with new identifiers each time. The agent has two cooperating subsystems that resolve the chaos into something usable:

• Workloadmeta (comp/core/workloadmeta/) — the canonical local store of containers, pods, ECS tasks, processes, and other workload entities. Sources of truth (kubelet, Docker, containerd, ECS, …) write into it; subscribers read from it.
• Tagger (comp/core/tagger/) — the canonical mapping from an entity (container, pod, host, process) to its set of tags. The tagger reads from workloadmeta and from other sources, normalizes the result, and exposes a small read API.

Together these two subsystems are the reason DogStatsD samples, check metrics, log lines, and traces all end up with consistent tags.

Workloadmeta

Workloadmeta is a publish-subscribe store keyed by entity ID. Entities include:

Entity	Source
Container	Docker, containerd, CRI
KubernetesPod	Kubelet API
KubernetesNode	Kubelet / Cluster Agent
KubernetesDeployment, etc.	Cluster Agent (orchestrator)
ECSTask	ECS metadata service
Process	Process Agent
Host	Hostname resolver
ContainerImageMetadata	Image SBOM scanner

Subscribers receive a stream of events: Set, Unset. A subscriber registers a filter (e.g., "all kubernetes pods on this node") and receives only the matching events.

The store is split into collectors (write side) and subscribers (read side). Collectors run as goroutines that watch their source and emit events when state changes. Subscribers register at startup and react.

graph LR
    subgraph collectors[Workloadmeta collectors]
        DOCKER_C[Docker]
        KUBELET_C[Kubelet]
        ECS_C[ECS]
        CONTAINERD_C[containerd]
        SBOM_C[SBOM]
    end
    collectors --> WM[Workloadmeta store]
    WM -->|filter-aware events| AD[Autodiscovery]
    WM -->|filter-aware events| TAGGER[Tagger]
    WM -->|filter-aware events| PROCESS[Process Agent]
    WM -->|filter-aware events| LOGS[Logs pipeline]
    WM -->|filter-aware events| METRICS[Container metrics check]

This avoids each subscriber re-implementing kubelet/docker/ECS reading.

Tagger

The tagger is the canonical "given an entity, what are its tags?" service. It composes its answer from multiple sources:

• Workloadmeta for the structural tags (pod_name, container_id, image_name, kube_namespace, kube_deployment, …).
• Custom tag collectors for sources that don't go through workloadmeta (gke_node_name, env-var-set tags, tags: in datadog.yaml).
• External Data — tags computed by the Cluster Agent and pushed via gRPC.

It also implements cardinality: every consumer asks for tags at a specific cardinality (low, orchestrator, or high), and the tagger returns only the tags appropriate for that level. Low cardinality is for metrics; high cardinality (with PII-like fields) is for events and traces.

The tagger exposes a gRPC API (over the core Agent's API socket) that other binaries connect to. This is why the Process Agent and Trace Agent can produce consistently tagged output without re-implementing kubelet parsing.

Entity IDs

The tagger and workloadmeta agree on a canonical ID format:

container_id://abcdef...
kubernetes_pod_uid://...
ecs_task://...
host://...
process://12345

When a metric arrives at DogStatsD with c:abcdef… origin tagging, the parser converts that to container_id://abcdef…, looks up the tags via the tagger, and applies them.

Subscribers and filters

Workloadmeta lets subscribers filter by:

• Entity kind (e.g., only KubernetesPod).
• Entity source (e.g., only events from the kubelet collector).
• Event type (Set vs Unset).

This keeps subscribers from being woken up for events they don't care about — important when there are many pods and many subscribers.

Cluster Agent integration

In Kubernetes deployments the Cluster Agent is the source of cluster-level workloadmeta — Kubernetes services, endpoints, deployments. Node Agents subscribe to the Cluster Agent's gRPC stream and merge those entities into their local workloadmeta store. This is also where namespace-level labels (computed once per cluster) come from.

Tagger as a stable API

The tagger's external surface is intentionally small:

Tags(entity, cardinality) ([]string, error)
GlobalTags(cardinality) ([]string, error)
Subscribe(filter) (...)

Callers don't see workloadmeta at all — they ask the tagger by entity ID and get back a tag slice. This is what allows the rest of the Agent (DogStatsD, the aggregator, the logs pipeline, the trace agent) to remain blissfully ignorant of which container runtime is in use.

Key abstractions

Type / package	Location	Description
Component (workloadmeta)	comp/core/workloadmeta/def/component.go	Top-level interface
Collector	comp/core/workloadmeta/collectors/	Source-of-truth implementations
Subscriber	comp/core/workloadmeta/scheduler/	Subscriber registration
Component (tagger)	comp/core/tagger/def/	Top-level tagger interface
EntityID	pkg/util/types/	Canonical entity ID
Filter	comp/core/workloadmeta/filter.go	Subscriber filter

Entry points for modification

• New entity type in workloadmeta: define the type in comp/core/workloadmeta/types/, add it to the store, write at least one collector and one consumer.
• New collector: implement Collector and register it in the bundle.
• New tagger source: extend comp/core/tagger/ with a new tag collector or a new source that subscribes to workloadmeta.
• Cluster-level state: extend the Cluster Agent's gRPC stream and the corresponding node-side subscriber.

Related pages

• Systems: Autodiscovery — heavy workloadmeta consumer.
• Apps: Process Agent — connects to the core Agent's tagger over gRPC.
• Apps: Cluster Agent — source of cluster-level entities.