Pooling tasks

Active contributors: wang.yuqi, Roger Wang, Cyrus Leung, Maximilien de Bayser.

Purpose

vLLM is not just a generative engine. The same infrastructure runs embedding, classification, scoring, reranking, and reward-model workloads. These are collectively called pooling tasks — they consume token-level hidden states and emit a fixed-shape output (a vector, a scalar, a class, or a small list) instead of streaming tokens.

Where it lives

vllm/v1/pool/                       # V1 engine support for pooling
vllm/entrypoints/pooling/           # Frontend helpers (factories, IO processors)
├── factories.py                    # init_pooling_io_processors
├── scoring/                        # Score/rerank IO processors
└── typing.py                       # OfflineInputsContext, OfflineOutputsContext
vllm/model_executor/layers/pooler/  # CLS / mean / last / attention pooling heads
vllm/pooling_params.py              # PoolingParams dataclass

The OpenAI server endpoints these power:

• POST /v1/embeddings
• POST /v1/score and POST /v1/rerank
• POST /v1/classify
• POST /v1/responses (when the response is non-generative)

Plus the offline LLM API methods: embed, encode, score, classify, pool.

Key abstractions

Abstraction	File	Role
PoolingParams	vllm/pooling_params.py	Per-request pooling options (normalize, return_softmax, ...)
Pooler	vllm/model_executor/layers/pooler/	Hidden-state → output transform (CLS, mean, last, attention pooling)
PoolerConfig	vllm/config/pooler.py	Engine-level pooler defaults
IOProcessor	vllm/entrypoints/pooling/	Converts user inputs ↔ tensor inputs
SupportsPooling, SupportsScoring, SupportsClassification	vllm/model_executor/models/interfaces.py	Capability mixins on model classes

Models

Many models can serve in pooling mode if they declare the right capability mixin. Examples in vllm/model_executor/models/:

• bert.py, bert_with_rope.py, roberta.py, modernbert.py, colmodernvbert.py — encoder-only backbones used as embedders.
• colbert.py, colpali.py, colqwen3.py, colqwen3_5.py, voyage.py — late-interaction retrieval.
• gritlm.py, qwen2_rm.py, jina.py, jina_vl.py — embedding / reward variants.
• Causal LMs (Llama, Qwen, etc.) can be pooled too via the universal pooler.

Tasks

vllm/tasks.py::SupportedTask enumerates the runtime task kinds. The pooling-relevant ones:

• embed — return per-request fixed-size embedding
• classify — return softmax over a label set
• score, rerank — return a relevance score given a (query, doc) pair
• reward — reward-model inference

Pipeline

graph LR
    Req[OpenAI /v1/embeddings request]
    IO[IOProcessor:<br/>parse, tokenize, build PoolingParams]
    EngReq[EngineCoreRequest with PoolingParams]
    Sched[Scheduler — runs prefill only, no decode]
    Worker[Worker forward → hidden states]
    Pool[Pooler — CLS / mean / last / etc.]
    PostIO[IOProcessor: postprocess (normalize, softmax, ...)]
    Resp[OpenAI response]

    Req --> IO --> EngReq --> Sched --> Worker --> Pool --> PostIO --> Resp

Key differences from generation:

• The scheduler runs prefill only — no token-by-token decoding. Once hidden states are produced, the request is finished.
• The output side returns PoolingRequestOutput, EmbeddingRequestOutput, ClassificationRequestOutput, ScoringRequestOutput (vllm/outputs.py).
• --runner-option pooling (or --runner pool) is the high-level switch to put the engine into pooling mode.

Multi-modal embeddings

Vision-language embedding models (colpali, colqwen3, colmodernvbert, paddleocr_vl, etc.) work the same way: the multi-modal pipeline runs the encoder, the pooler aggregates per-token features into a vector or set of late-interaction tokens.

Key source files

File	Purpose
vllm/pooling_params.py	PoolingParams
vllm/config/pooler.py	PoolerConfig
vllm/v1/pool/	V1 engine pooling adapters
vllm/entrypoints/pooling/factories.py	IO processor factories
vllm/entrypoints/pooling/scoring/	Score / rerank IO processors
vllm/model_executor/layers/pooler/	Pooling heads
vllm/model_executor/models/interfaces.py	Capability mixins
vllm/outputs.py	Output dataclasses

Entry points for modification

• New pooling head: add a class under vllm/model_executor/layers/pooler/ and reference it from the model class's pooler attribute.
• New IO processor: subclass the relevant base in vllm/entrypoints/pooling/, register through factories.py.
• New pooling task: extend SupportedTask in vllm/tasks.py, add a runner branch in LLM (vllm/entrypoints/llm.py).