Metrics, logging, tracing

Active contributors: Cyrus Leung, Robert Shaw, Nick Hill.

Purpose

vLLM exposes operational signals through three channels: structured logs, Prometheus metrics, and OpenTelemetry traces. They share a single in-engine pipeline (StatLoggerManager) so plugins can hook in without forking the engine.

Directory layout

vllm/v1/metrics/
├── loggers.py          # StatLoggerManager + built-in loggers (~52 KB)
├── perf.py             # PerfStats, ModelMetrics (~46 KB)
├── prometheus.py       # Prometheus client setup
├── ray_wrappers.py     # Ray-aware stat forwarding
├── reader.py           # Read-side helpers for metric snapshots
├── stats.py            # Stats dataclasses (PrefixCacheStats, IterationStats, ...)
└── utils.py

vllm/logger.py                # init_logger, custom formatter
vllm/logging_utils/            # message dedup, dump_input, etc.
vllm/tracing.py                # OpenTelemetry helpers (instrument decorator, init_tracer)
vllm/usage/usage_lib.py        # Anonymized usage telemetry (opt-in)
vllm/profiler/                 # Torch profiler integration

Logs

Initialized via vllm.logger.init_logger(__name__). Output goes to stderr by default. Configuration:

Variable	Effect
VLLM_LOGGING_LEVEL	Global level (DEBUG, INFO, WARNING, ERROR)
VLLM_LOGGING_CONFIG_PATH	JSON logging dict-config
VLLM_DEBUG_LOG_API_SERVER_RESPONSE	Verbose HTTP responses
VLLM_TRACE_FUNCTION	Lightweight per-function entry/exit logging

vllm/logging_utils/dump_input.py::dump_engine_exception is used to dump the offending EngineCoreRequest and surrounding scheduler state when EngineCore catches an exception. The dump location is ObservabilityConfig.dump_engine_exception_path (or its env equivalent).

Metrics

Built-in stat loggers in vllm/v1/metrics/loggers.py:

• PrometheusStatLogger — exposes per-step counters/gauges via the /metrics endpoint of the API server.
• LoggingStatLogger — pretty-prints throughput / latency / KV / request lifecycle events to logs at a configurable interval.
• RayPrometheusStatLogger — Ray-cluster-aware variant that forwards to a single Prometheus exporter.

Stats published per step (in IterationStats, SchedulerStats, PerfStats):

Metric	Source
Running / waiting / preempted requests	Scheduler / RequestQueue
GPU prefix-cache hits / misses	PrefixCacheStats
KV cache utilization	KVCacheMetricsCollector
Spec-decode acceptance rates	SpecDecodingStats
Per-request E2E / TTFT / TPOT	IterationStats
Per-step compute / overhead breakdown	PerfStats, ModelMetrics
LoRA hits / evictions	LoRAModelManager reports
Forward-pass time, sampling time	PerfStats
KV connector saves / loads / failures	KVConnectorStats

Plug-in stat loggers register through the vllm.stat_loggers entry point group; see vllm/v1/metrics/loggers.py::load_stat_logger_plugin_factories.

Tracing

vllm/tracing.py wraps OpenTelemetry. When --otlp-traces-endpoint (or ObservabilityConfig.otlp_traces_endpoint) is set, init_tracer("vllm.llm_engine", endpoint) configures the OTLP exporter. The @instrument(span_name="...") decorator is used liberally on hot paths:

• EngineCore.__init__ / step
• Scheduler.schedule / update_from_output
• Executor.__init__ / execute_model
• KV connector operations
• Worker forward passes (when VLLM_TRACE_WORKER is on)

Spans carry attributes like request IDs, scheduler stats, and per-request logprobs (configurable). maybe_init_worker_tracer is the equivalent for worker-side spans.

Usage telemetry

vllm/usage/usage_lib.py posts an anonymized usage report (model name, parallelism, GPU model, vLLM version) to a vLLM endpoint. It is opt-out via VLLM_DO_NOT_TRACK=1 or DO_NOT_TRACK=1. The report is generated once per process at startup and once on shutdown.

Profiling

Two profilers ship in tree:

• Torch profiler — LLM.start_profile() / LLM.stop_profile(). Output lands in VLLM_TORCH_PROFILER_DIR.
• NVTX hooks — vllm/utils/nvtx_pytorch_hooks.py wraps key ops with NVTX ranges for Nsight Systems.

vllm/profiler/ contains higher-level helpers used by vllm bench to gather timeline traces alongside throughput numbers.

Key source files

File	Purpose
vllm/logger.py	Logger setup
vllm/logging_utils/dump_input.py	Exception dumping
vllm/v1/metrics/loggers.py	StatLoggerManager + built-in loggers
vllm/v1/metrics/perf.py	Performance stats
vllm/v1/metrics/stats.py	Stat dataclasses
vllm/v1/metrics/prometheus.py	Prometheus setup
vllm/tracing.py	OTLP integration
vllm/usage/usage_lib.py	Anonymized usage report
vllm/utils/nvtx_pytorch_hooks.py	NVTX ranges

Entry points for modification

• New stat logger: implement StatLoggerFactory and register via the vllm.stat_loggers entry point.
• Custom Prometheus metric: add it in vllm/v1/metrics/prometheus.py (or a plugin); make sure it's defined once per process for multiprocess mode.
• Custom span: decorate the function with @instrument(span_name="...").
• Disable usage telemetry: set VLLM_DO_NOT_TRACK=1 (per-process) or DO_NOT_TRACK=1 (per-user).