Configuration

Active contributors: Harry Mellor, Cyrus Leung, Nick Hill, Robert Shaw.

Purpose

Every component reads from one big config object — VllmConfig — that aggregates ~25 typed sub-configs. Centralization keeps options discoverable (vllm serve --help=ModelConfig), makes serialization/forking easy, and removes per-module config drift.

Directory layout

vllm/config/
├── __init__.py             # Re-exports every sub-config
├── vllm.py                 # The aggregate VllmConfig (~88 KB)
├── model.py                # ModelConfig, dtype, runner kind, tokenizer (~89 KB)
├── parallel.py             # ParallelConfig, EPLBConfig (~40 KB)
├── compilation.py          # CompilationConfig, CUDAGraphMode (~65 KB)
├── speculative.py          # SpeculativeConfig (~46 KB)
├── cache.py                # CacheConfig
├── scheduler.py            # SchedulerConfig
├── lora.py                 # LoRAConfig
├── multimodal.py           # MultiModalConfig
├── observability.py        # ObservabilityConfig (tracing, KV metrics)
├── kv_transfer.py          # KVTransferConfig
├── ec_transfer.py          # ECTransferConfig
├── kv_events.py            # KVEventsConfig
├── kernel.py               # KernelConfig (per-kernel feature flags)
├── load.py                 # LoadConfig (weight loader selection)
├── attention.py            # AttentionConfig
├── mamba.py                # MambaConfig
├── pooler.py               # PoolerConfig
├── profiler.py             # ProfilerConfig
├── quantization.py         # OnlineQuantizationConfigArgs
├── reasoning.py            # ReasoningConfig
├── structured_outputs.py   # StructuredOutputsConfig
├── speech_to_text.py       # SpeechToTextConfig
├── offload.py              # OffloadConfig + variants
├── weight_transfer.py      # WeightTransferConfig
├── device.py               # DeviceConfig
├── model_arch.py           # Architecture-specific defaults
└── utils.py                # @config decorator, get_attr_docs, replace, update_config

Key abstractions

Abstraction	File	Role
VllmConfig	vllm/config/vllm.py	Aggregate dataclass holding every sub-config
set_current_vllm_config / get_current_vllm_config	vllm.py	Thread-local active config (read by layers/kernels)
ModelConfig	vllm/config/model.py	Model id, dtype, max_model_len, tokenizer_mode, runner
ParallelConfig	vllm/config/parallel.py	TP/PP/DP/EP sizes, executor backend, EPLB, batch invariance
SchedulerConfig	vllm/config/scheduler.py	Max running seqs, batched tokens, async scheduling, policy
CacheConfig	vllm/config/cache.py	Block size, GPU/CPU memory split, prefix caching
CompilationConfig	vllm/config/compilation.py	torch.compile mode, CUDA graph mode, FX passes
SpeculativeConfig	vllm/config/speculative.py	Spec decode method + draft model
LoRAConfig	vllm/config/lora.py	Max LoRAs, ranks, lora-specific switches
KernelConfig	vllm/config/kernel.py	Per-kernel feature flags (deepgemm, flashinfer cutlass moe)
KVTransferConfig	vllm/config/kv_transfer.py	KV connector selection
ObservabilityConfig	vllm/config/observability.py	OTLP tracing, KV cache metrics
EngineArgs / AsyncEngineArgs	vllm/engine/arg_utils.py	The argparse-generating wrapper that builds VllmConfig from CLI args

How configs get built

graph TD
    CLI[CLI flags<br/>vllm serve ...]
    KW[Python kwargs<br/>LLM(model=..., tensor_parallel_size=...)]
    EA[EngineArgs / AsyncEngineArgs]
    HF[HF model config<br/>(architectures, max_position_embeddings, ...)]
    Plat[Platform defaults]
    VC[VllmConfig.__post_init__:<br/>cross-config validation, KV size auto-fit]
    Active[set_current_vllm_config()<br/>(thread-local)]

    CLI --> EA
    KW --> EA
    EA --> VC
    HF --> VC
    Plat --> VC
    VC --> Active

EngineArgs.create_engine_config(...) is the main constructor. It fetches the HF config (via vllm/transformers_utils/), overlays platform defaults (e.g., default block_size, gpu_memory_utilization), and runs VllmConfig.__post_init__ for cross-config validation (e.g., "if enable_prefix_caching then cache_config.cache_dtype must be ...").

set_current_vllm_config(config) puts the resolved config in a context manager that layers and kernels read via get_current_vllm_config(). Multiple engines in the same process get isolated state.

--help views

vllm serve --help would be unreadable as one mega-list, so vllm/utils/argparse_utils.py::FlexibleArgumentParser slices the output by config group:

vllm serve --help=ModelConfig
vllm serve --help=ParallelConfig
vllm serve --help=CompilationConfig
vllm serve --help=all

The grouping comes from the @config decorator (vllm/config/utils.py) which walks the dataclass at import time, harvests inline docstrings (get_attr_docs), and emits argparse arguments named after the field.

Validation utilities

• validate_config pre-commit hook ensures every field has a docstring and a sane default.
• vllm/config/utils.py::is_init_field distinguishes user-set fields from derived ones for to_dict round-tripping.
• vllm/config/utils.py::replace/update_config produce modified copies without mutating the original.

Common mistakes when adding config

• Forgetting the docstring. The argparse generator uses it for --help text.
• Storing mutable default. Use field(default_factory=...) — the @config decorator enforces this in pre-commit.
• Leaking config out-of-band. Pass the config object; don't os.environ your way around it.
• Cross-config invariants in __post_init__ of the wrong sub-config. Put them in VllmConfig.__post_init__ so they run after every sub-config is finalized.

Key source files

File	Purpose
vllm/config/vllm.py	Aggregate VllmConfig
vllm/config/utils.py	@config decorator, doc harvesting
vllm/engine/arg_utils.py	CLI arg generation + create_engine_config
vllm/utils/argparse_utils.py	FlexibleArgumentParser (group-aware help)
vllm/envs.py	All environment overrides (~2,500 lines)

Entry points for modification

• Add a new field to an existing sub-config: declare it as a typed dataclass field with a docstring; rebuild via pre-commit run validate-config.
• Add a new sub-config: create vllm/config/<thing>.py, decorate the class with @config, expose it via vllm/config/__init__.py, add it as a field on VllmConfig.
• Wire to CLI: nothing extra — the argparse generator picks it up automatically.
• Add cross-config invariants: VllmConfig.__post_init__.