Lore

The Datadog Agent has been in active development since at least early 2016 in this repository (commits go back to 2016-02-09), with prior history in older repos. This page tracks the major eras and shifts visible in the git log and the surviving artifacts.

Eras

The Agent v6 era (2016–2018)

The repository was bootstrapped around the rewrite of the Datadog Agent from Python (Agent v5) to Go (Agent v6). The first commits in this repo lay down the Go module structure, the Python check loader (rtloader/), and the initial pkg/aggregator, pkg/forwarder, and pkg/collector packages.

Key milestones:

• 2016: Repository initialized. Initial Go-based Agent core. rtloader/ C++ runtime introduced so the existing Python check ecosystem could keep running inside the new Go process.
• 2017–2018: Rapid expansion of pkg/ with the foundational subsystems still in use today: aggregator, demultiplexer, sender, serializer, forwarder, autodiscovery.

Agent v7 and Python 3 (2019–2020)

Agent v7 dropped Python 2 support (which v6 had carried for backwards compatibility). The split is still visible in rtloader/two/ (Python 2) versus rtloader/three/ (Python 3) and in build tags like python vs python3.

Significant additions in this era:

• Trace Agent merge: pkg/trace/ was migrated into this repo from a standalone trace-agent repository, consolidating the APM data path.
• Cluster Agent (cmd/cluster-agent/, pkg/clusteragent/): grew to support Kubernetes cluster-level features — External Metrics Provider, admission controller, leader election.
• Process Agent matured into a separate binary with its own checks and lifecycle.

The eBPF and System Probe era (2019–2022)

The system probe and eBPF subsystems became major investments.

• pkg/ebpf/ and pkg/network/tracer/ brought NPM (Network Performance Monitoring) into the agent.
• CWS (pkg/security/) was added for runtime threat detection. SECL — the rule DSL — and the pkg/security/secl/ parser/evaluator come from this period.
• USM (pkg/network/usm/) added eBPF-based application protocol decoding (HTTP, HTTP/2, MySQL, Postgres, Kafka, …).

This is why pkg/security/ is the heaviest-churn directory in the entire repo: the CWS team has shipped continuously since CWS first launched. Lifetime commit counts in pkg/security/ exceed 1000 by the top contributor alone.

The component framework migration (2021–present)

A large chunk of recent work is about migrating from the legacy pkg/-based architecture to the component framework in comp/.

• The framework is built on go.uber.org/fx. Each piece of functionality is exposed behind an interface (comp/<name>/def/) with implementations (comp/<name>/impl/) and Fx wiring (comp/<name>/fx/).
• Bundles like comp/core/, comp/forwarder/, comp/serializer/, comp/dogstatsd/, comp/logs/, comp/trace/, and dozens more provide drop-in replacements for older pkg/ code.
• Migration is incremental — there are still many pkg/ packages that haven't been converted, and comp/ components often delegate to pkg/ implementations underneath.

Roughly half the lines of code in comp/ were added in the last 2–3 years. See Components framework for the technical detail.

The Bazel migration (2023–present)

The repository has been migrating from the legacy Omnibus + Invoke build to Bazel. This shows up in:

• A growing number of BUILD.bazel files alongside Go packages.
• bazel/rules/ebpf/ providing macros (ebpf_prog, cgo_godefs, runtime_compilation_bundle) that replaced ad-hoc Make scripts and go generate invocations.
• Frequent commits with messages like "migrate another batch of packages to bazel" — these have appeared on most weeks of the last year.
• packages/ as the new home of distribution definitions, replacing parts of omnibus/ for deb/rpm/MSI builds.

The migration is unfinished. Both build systems are expected to coexist for some time. New code is generally expected to come with a Bazel target.

The OpenTelemetry convergence (2023–present)

cmd/otel-agent/ and the OTel collector flavor were added. The comp/otelcol/ bundle wires OTel collector pipelines into the same Fx framework as the rest of the Agent. The Trace Agent gained a gRPC OTLP receiver in pkg/trace/otel/. OTel collector flavor is now a first-class Agent variant.

The AI-assisted development era (2024–2026)

A more recent shift visible in the git log is the integration of AI assistants into the development workflow:

• AGENTS.md, CLAUDE.md, and per-package AGENTS.md files document codebase context for AI tools. The repo has clearly chosen to treat AI context as a first-class artifact, with a dedicated section in the root AGENTS.md telling agents to keep these files up to date.
• .claude/skills/ and .claude-plugins/ define agent skills and plugin definitions.
• Many recent commits include Co-authored-by: Claude … trailers (Sonnet, Opus, "Opus 4.6 (1M context)") on substantive feature work and refactors.
• New dyninst (dynamic instrumentation) work in pkg/dyninst/ is one of the most active areas, with weekly commits.

Longest-standing features

Things that have been in the codebase since the early Agent v6 days and are still active:

• Aggregator pipeline (pkg/aggregator/). The TimeSampler / CheckSampler / Demultiplexer architecture is the same conceptual shape as in 2016, though the implementation has been heavily refactored. Batching and tag handling have evolved repeatedly.
• Python check runtime (rtloader/). The CPython embedding has survived the v5 → v6 → v7 transition. The Python 2 path is still compiled in for some legacy installations.
• DogStatsD UDP/UDS server. The protocol and listener model in comp/dogstatsd/listeners/ predates the component framework and was migrated into comp/ largely without changes.
• Forwarder retry queue (comp/forwarder/defaultforwarder/). HTTP-with-disk-backing has been the design since early days; the current home is in comp/, but the retry strategy, exponential backoff, and disk-buffered queue trace back to the v6 era.
• docs/dev/ developer notes. Several files in docs/dev/ (legal, caveats, agent_api) were among the first non-code commits and are still referenced from CONTRIBUTING.md.

Deprecated and removed features

A few notable things that have been removed or are being deprecated:

• APM Event extractor (pkg/trace/event/). Originally used for "Trace Search," it remains in the codebase in a deprecated state — the README in pkg/trace/ calls this sampler deprecated.
• Python 2 in v7. Agent v7 dropped Python 2 support, but the rtloader/two/ directory still exists for v6 builds.
• Old Make-based eBPF builds. Replaced by Bazel macros under bazel/rules/ebpf/.
• The "old" component model. Earlier versions of the codebase had a less formal component pattern; the Fx-based framework in comp/ replaced it. Some legacy patterns remain in pkg/.
• Omnibus, gradually. Still in use for some packaging, but packages/ + Bazel is the migration target.

Major rewrites and migrations

When	What	Where
2016	Python (v5) → Go (v6) Agent	The reason this repo exists
2017–2018	Trace Agent moved into this repo	pkg/trace/, cmd/trace-agent/
2019	Python 2 → Python 3 (v7)	rtloader/three/
2020+	NPM and CWS expansion	pkg/network/, pkg/security/
2021–	Component framework	comp/
2023–	Omnibus → Bazel	BUILD.bazel, packages/, bazel/
2023–	OpenTelemetry first-class	cmd/otel-agent/, comp/otelcol/
2024–	AI-assisted dev workflows	AGENTS.md, .claude/, dyninst/

Growth trajectory

The codebase has grown roughly linearly in commit count, but several inflection points are visible from git log:

• 2018–2019: addition of the Trace Agent and Cluster Agent introduces large bursts of new directories.
• 2020–2021: System Probe and CWS make pkg/network/ and pkg/security/ two of the largest directories.
• 2022–present: comp/ accumulates faster than pkg/ shrinks — net codebase growth is dominated by component-framework wiring.
• 2023–present: heavy BUILD.bazel churn as the build system migration progresses.

339 unique committers landed at least one commit in the last year, distributed across dozens of internal Datadog teams. CODEOWNERS in .github/CODEOWNERS lists the formal team-to-path mapping; see Maintainers for the per-area version.