Network monitoring

Purpose

Datadog's Network Performance Monitoring (NPM) and Universal Service Monitoring (USM) products both come from the System Probe. NPM tracks TCP/UDP flows, conntrack, DNS resolution, and gateway lookups; USM decodes application protocols (HTTP, HTTP/2, gRPC, MySQL, Postgres, Kafka, Redis, …) inside the kernel via eBPF.

This page is the high-level story. For the binary internals, see Apps: System Probe. For the eBPF foundation, see eBPF / system probe.

NPM

graph LR
    subgraph kernel[Linux kernel]
        KP[kprobes / fentry] --> EBPF[eBPF NPM programs]
        EBPF --> MAPS[connection map<br/>DNS map<br/>conntrack map]
    end
    subgraph user[User space]
        TRACER[Network Tracer<br/>pkg/network/tracer]
        ENC[Encoder<br/>pkg/network/encoding]
    end
    NETLINK[netlink / proc] --> TRACER
    MAPS <-- periodic dump --> TRACER
    TRACER --> ENC
    ENC -->|HTTP /network_tracer/connections| PROC[Process Agent]
    PROC --> INTAKE[Datadog intake<br/>NPM payload]

The eBPF programs hook into the kernel's network stack to record connection lifecycle events, sequence numbers, and counters. Periodically the user-space tracer dumps the eBPF maps, augments them with conntrack and netlink data (for routing and NAT), and produces a structured payload.

Programs and Go bindings are in pkg/network/ebpf/. The tracer (pkg/network/tracer/tracer.go) is the central abstraction; on Windows it is replaced by the WFP-driver-backed tracer_windows.go.

USM

USM lives under pkg/network/usm/. It uses eBPF programs that hook syscall/socket boundaries and parse application protocols. Currently supported:

• HTTP/1.1 — full request/response decoding.
• HTTP/2 + gRPC — frame-level decoding.
• MySQL, Postgres — query parsing.
• Kafka — topic-level metrics.
• Redis — command extraction.
• TLS-aware decoding via uprobes on OpenSSL/GoTLS.

USM produces per-(client, server, endpoint) RED metrics — request count, error count, response time distribution — without requiring application instrumentation.

A typical USM event journey:

1. Kernel program records (request, response) pairs in eBPF maps.
2. User-space monitor dumps the maps periodically.
3. Telemetry is encoded and forwarded through the same socket the NPM tracer uses.

Each protocol has its own monitor under pkg/network/usm/<protocol>/.

DNS resolution

pkg/network/dns/ records every DNS query/response observed on the host. NPM uses this to display destination hostnames instead of bare IP addresses; the data also feeds reverse-DNS lookups elsewhere in the Agent (pkg/rdnsquerier/).

Conntrack

pkg/network/netlink/ and the eBPF conntrack module track NAT translations so flows from inside containers can be reported with the destination's "real" address. Conntrack entries are pulled either from netlink (default) or from an eBPF-based conntracker for higher fidelity in some kernels.

Process correlation

pkg/network/tracer/process_cache.go correlates connections to processes (and therefore containers and pods) by walking /proc and the eBPF maps. This is what makes NPM data render with service, container_name, and pod_name tags in the Datadog UI.

Connection rollup

Raw flows can explode in cardinality. The tracer rolls up connections by:

• Coalescing (client port, server, server port) tuples that look like the same logical connection.
• Aggregating short-lived connections by destination.
• Folding ephemeral client ports.

Gateway lookup

pkg/network/gateway_lookup_*.go resolves the next-hop gateway for outgoing traffic. This is what populates the "via" field in NPM's network map.

Configuration

NPM and USM settings live in system-probe.yaml and datadog.yaml:

network_config:
  enabled: true
  collect_dns_stats: true
service_monitoring_config:
  enabled: true
  http2_monitoring_enabled: true
  http_idle_connection_ttl_in_s: 30
  protocols:
    redis:
      enabled: true
    postgres:
      enabled: true

Multiple platforms

Platform	NPM	USM
Linux (eBPF)	Full	Full
Linux (no eBPF)	Limited (netlink/proc-based)	Not available
Windows	Full (WFP driver)	Limited
macOS	Not supported	Not supported

Windows code lives in pkg/network/driver/, pkg/windowsdriver/, and *_windows.go files. The user-space surface is identical across platforms; the kernel side differs entirely.

Internal observability

• system-probe diagnose — runs internal NPM/USM checks.
• system-probe modules list — lists active modules.
• expvar and Agent telemetry expose network_tracer.*, usm.* counters.
• agent status shows NPM/USM sections when the system probe is configured.

Key abstractions

Type / package	Location	Description
Tracer	pkg/network/tracer/tracer.go	NPM tracker
Monitor (USM)	pkg/network/usm/monitor.go	USM dispatcher
Encoder	pkg/network/encoding/	Wire-format encoder
ProcessCache	pkg/network/tracer/process_cache.go	Connection-to-process correlation
Conntracker	pkg/network/netlink/ and pkg/network/tracer/ebpf_conntracker.go	NAT/conntrack
GatewayLookup	pkg/network/gateway_lookup_*.go	Next-hop resolution

Entry points for modification

• New USM protocol: add a monitor under pkg/network/usm/<protocol>/, an eBPF parser if needed, and an encoder entry.
• New NPM dimension: extend pkg/network/encoding/ and the corresponding wire format.
• Better Windows parity: most Linux features have a Windows driver counterpart waiting to be filled in.

Related pages

• Apps: System Probe — the binary that runs all this.
• Features: eBPF / system probe — eBPF infrastructure.
• Apps: Process Agent — consumes NPM data and forwards it.