Systems

vLLM ships as a single Python package, so its internal structure is best understood as a stack of cooperating systems rather than separate apps or workspace packages. This section walks the stack top-down, from the front-end and engine loop through the scheduler, KV cache, executors/workers, model implementations, attention backends, kernels, and platforms.

The stack

graph TD
    A[Front-ends — vllm/entrypoints/]
    B[Engine clients — vllm/v1/engine/]
    C[EngineCore — vllm/v1/engine/core.py]
    D[Scheduler — vllm/v1/core/sched/]
    E[KV cache — vllm/v1/core/, vllm/v1/kv_offload/]
    F[Executors — vllm/v1/executor/]
    G[Workers + ModelRunner — vllm/v1/worker/]
    H[Model implementations — vllm/model_executor/models/]
    I[Layers — vllm/model_executor/layers/]
    J[Attention backends — vllm/v1/attention/backends/]
    K[Kernels — csrc/, vllm/_custom_ops.py]
    L[Distributed — vllm/distributed/]
    M[Multimodal — vllm/multimodal/]
    N[Platforms — vllm/platforms/]

    A --> B --> C --> D --> F --> G --> H --> I --> J --> K
    C --> E
    G --> L
    H --> M
    K --> N

Pages in this section

• Front-ends and entry points
• Engine core (V1)
• Scheduler
• KV cache and prefix caching
• Executors and workers
• Attention backends
• Model executor and the model zoo
• Distributed: parallelism, KV transfer, EPLB
• Multi-modal
• Compilation, CUDA graphs, and custom ops
• Platforms and hardware backends
• Sampling, structured outputs, speculative decoding
• Configuration
• Metrics, logging, tracing

For higher-level features that span several systems (LoRA, quantization, prefix caching, disaggregated prefill, etc.), see the features section.