Patterns and conventions

The codebase is large and old enough to have settled into a handful of strong patterns. Following them makes diffs smaller and reviewers happier.

Configuration is centralized

All runtime configuration goes through VllmConfig (vllm/config/vllm.py). Sub-configs are typed dataclasses under vllm/config/. New options should be added to the appropriate sub-config and wired into EngineArgs (vllm/engine/arg_utils.py) so they show up in vllm serve --help.

The argparse-generation framework (vllm/utils/argparse_utils.py::FlexibleArgumentParser and vllm/config/utils.py::get_attr_docs) introspects field docstrings and emits --help=<ConfigGroup> views automatically. Document config fields with inline docstrings, not external prose, so the help text stays consistent.

Plugins and registries

Several subsystems are pluggable through Python entry points or in-tree registries:

• General plugins: vllm.general_plugins entry-point group. Loaded by vllm.plugins.load_general_plugins() very early in EngineCore startup. Used to register attention backends, model architectures, custom passes, etc.
• Model registry: vllm/model_executor/models/registry.py — every architecture string maps to its implementation here.
• Attention backend registry: vllm/v1/attention/backends/registry.py::AttentionBackendEnum plus register_backend() for runtime overrides.
• Quantization registry: vllm/model_executor/layers/quantization/__init__.py.
• Tool parsers: vllm/tool_parsers/ + ToolParserManager.register_module.
• Reasoning parsers: vllm/reasoning/ + ReasoningParserManager.
• KV connectors: vllm/distributed/kv_transfer/kv_connector/factory.py::KVConnectorFactory.
• EC connectors: vllm/distributed/ec_transfer/ec_connector/factory.py.
• Stat loggers: vllm/v1/metrics/loggers.py::load_stat_logger_plugin_factories.
• Multi-modal: vllm/multimodal/registry.py::MULTIMODAL_REGISTRY.

When extending a subsystem, prefer registering through these registries instead of patching the dispatch tables directly.

Custom ops

Kernels are exposed to Python via torch.library registrations in vllm/_custom_ops.py, vllm/_aiter_ops.py (ROCm AITER), and vllm/_xpu_ops.py. Higher-level layers wrap those ops in CustomOp subclasses (vllm/model_executor/custom_op.py) which provide:

• Dispatch between native, Triton, and pure-PyTorch implementations
• Compatibility with torch.compile (pattern matching, donated buffers, fused passes)
• A consistent forward_native, forward_cuda, forward_rocm, forward_xpu, etc. surface

Adding a new kernel typically means: write the kernel under csrc/, expose it via _custom_ops.py, wrap it in a CustomOp, and use the wrapper in the layer file.

Forward context

Per-step state that has to be visible to many layers (active LoRAs, KV connector metadata, attention metadata, ubatch info) is pushed into a context manager in vllm/forward_context.py. Inside the model, layers read this with get_forward_context(). Avoid threading new arguments through every layer — reach for forward context first.

Error handling

• User-facing validation errors go through vllm.exceptions.VLLMValidationError and hit the FastAPI error handlers in vllm/entrypoints/openai/server_utils.py.
• Engine-internal failures bubble up as EngineDeadError / EngineGenerateError (vllm/v1/engine/exceptions.py). The frontend converts them into HTTP 500/4xx as appropriate.
• Worker-side asserts should be informative — workers don't get to print stack traces in user-visible logs unless wrapped in the dump_engine_exception machinery.

Logging

from vllm.logger import init_logger
logger = init_logger(__name__)

Use logger.info_once(...) / logger.warning_once(...) (vllm/logging_utils/) for messages that would otherwise repeat once per request.

Concurrency

• The frontend is asyncio. Anything that blocks the event loop must be wrapped (asyncio.to_thread or vllm.utils.async_utils).
• The EngineCore loop is synchronous, with a thread for input dispatch.
• The boundary between the two is ZMQ + msgpack (vllm/v1/serial_utils.py).
• Workers run their own loops; communication is collective_rpc from the executor.

Coding style

Configured in pyproject.toml:

• Ruff with E, F, UP, B, ISC, SIM, I, G rule families enabled. Notable ignores: F405/F403 (star imports), E731 (lambda), B905 (zip without strict=), B007, UP032.
• Ruff format with docstring-code-format = true.
• Mypy with the pydantic.mypy plugin and check_untyped_defs = true. follow_imports = "silent" keeps the run fast.
• Typos with a curated allowlist of project-specific words (CUDA mnemonics, MoE algorithm names, Intel CPU features).
• Markdownlint with config in .markdownlint.yaml.

Run pre-commit run --all-files to apply everything.

Type checking

The project keeps mypy clean on touched files. New code should:

• Annotate public function signatures.
• Use from __future__ import annotations when convenient (already pervasive).
• Prefer dataclass or msgspec.Struct over ad-hoc dicts.

Pydantic v2 is used for request/response schemas. msgspec is preferred for hot-path internal messages because it serializes faster.

File and import organization

• vllm/__init__.py uses MODULE_ATTRS lazy loading (a __getattr__ that imports submodules on demand). When adding a new public symbol, register it there too.
• Import order is enforced by ruff's I rules. Group: stdlib, third-party, vllm-internal.
• Avoid top-level imports of heavy modules in CLI entry points (vllm/entrypoints/cli/main.py lazy-imports everything inside main() for fast vllm --help).
• tools/check_init_lazy_imports.py (run by pre-commit) enforces this.

Don't

• Don't use the system python3 or pip directly. Always go through uv and .venv/bin/python (per AGENTS.md).
• Don't add a new top-level dependency without updating requirements/.
• Don't reach into vllm/v1/ from vllm/engine/. The legacy engine/ module is a compatibility shim only — V1 is the source of truth.
• Don't mark a config field as plain Optional[...] if it's a typed sub-config; keep dataclasses nested for --help= discoverability.