Structured outputs

Active contributors: Aaron Pham, Russell Bryant, Cyrus Leung.

Purpose

vLLM can constrain sampling so that the output matches a JSON schema, regex, fixed choice list, or context-free grammar. Constraints are enforced inside the sampler via a bitmask, so the cost is small and CUDA-graph-friendly.

Backends

vllm/v1/structured_output/__init__.py::StructuredOutputManager selects between four backends. Selection happens in vllm/sampling_params.py::SamplingParams._validate_structured_output and is recorded in StructuredOutputsParams._backend:

Backend	File	Notes
xgrammar (default)	vllm/v1/structured_output/backend_xgrammar.py	CUDA-graph-friendly bitmask; co-developed with vLLM
guidance	vllm/v1/structured_output/backend_guidance.py	LLGuidance integration
outlines	vllm/v1/structured_output/backend_outlines.py	Outlines compiler
lm-format-enforcer	vllm/v1/structured_output/backend_lm_format_enforcer.py	Vendor-specific enforcer

StructuredOutputsConfig.backend (vllm/config/structured_outputs.py) overrides auto-selection.

Public API

StructuredOutputsParams on SamplingParams:

SamplingParams(
    structured_outputs=StructuredOutputsParams(
        json={"type": "object", "properties": {...}},
        # OR:
        regex=r"^\d{3}-\d{4}$",
        # OR:
        choice=["yes", "no"],
        # OR:
        grammar="<lark or gbnf string>",
        # OR:
        json_object=True,           # any JSON object
        structural_tag="<...>",     # for tag-style enforcement
        disable_any_whitespace=False,
        disable_additional_properties=False,
        whitespace_pattern=None,
    ),
)

These options are mutually exclusive with one another (validated in __post_init__).

In the OpenAI server, the same options are reachable via response_format (with type: "json_schema") and via the tools array (function calling enforces structured arguments).

How the constraint reaches the sampler

graph LR
    Req[StructuredOutputsParams on request]
    SOM[StructuredOutputManager]
    Be[Backend compiler<br/>xgrammar / guidance / outlines / lmfe]
    GS[Per-request grammar state<br/>vllm/v1/structured_output/request.py]
    GO[GrammarOutput<br/>(advance grammar this step)]
    Smp[Sampler — apply token mask]

    Req --> SOM --> Be --> GS
    GS -->|each step| GO --> Smp

The grammar state advances as tokens are produced. At sampling time, the sampler applies a bitmask zeroing out tokens that would violate the grammar; the model only ever picks legal tokens.

Async grammar compilation

Some grammars (large JSON schemas) take real wall-time to compile. The structured-output manager runs compilation asynchronously and the scheduler may keep a request waiting until compilation finishes. GrammarOutput includes a is_compiling flag the sampler honors.

Reasoning interaction

When a model emits reasoning content (e.g., <think> blocks), the grammar should not constrain the reasoning span. The sampler honors reasoning_ended on the request — it only applies the structured-output mask after the reasoning parser declares the answer span has started.

Key source files

File	Purpose
vllm/sampling_params.py (StructuredOutputsParams)	Public dataclass
vllm/v1/structured_output/__init__.py	StructuredOutputManager
vllm/v1/structured_output/backend_*.py	One file per backend
vllm/v1/structured_output/request.py	Per-request grammar state
vllm/v1/structured_output/utils.py	Schema/grammar helpers
vllm/config/structured_outputs.py	StructuredOutputsConfig
vllm/v1/sample/sampler.py	Where the mask is applied

Entry points for modification

• New backend: subclass the backend interface, register in StructuredOutputManager, list in StructuredOutputsConfig.backend.
• New constraint shape: extend StructuredOutputsParams, add a path in _validate_structured_output, support it in each backend.
• Tighter integration with reasoning: use the reasoning_ended field; the sampler already supports gating the mask by it.

For the sampler internals, see Sampling, structured outputs, speculative decoding.