Anthropic, gRPC, MCP, SageMaker, pooling frontends

The OpenAI server is the primary HTTP frontend, but vLLM ships several alternative shells. They all sit on top of the same EngineProtocol (V1's AsyncLLM) and share parsers, tokenizers, and chat utilities.

Anthropic Messages

vllm/entrypoints/anthropic/:

• protocol.py — Pydantic request/response types matching the Anthropic Messages API
• serving.py (~36 KB) — handler that translates Anthropic-shaped requests into SamplingParams + StructuredOutputsParams + tool definitions
• api_router.py — FastAPI router mounted next to the OpenAI router

Use case: clients written for anthropic.Anthropic.messages.create(...) can target a vLLM-served model. Tools, vision, and streaming are supported. Reasoning content is mapped to Anthropic's thinking block when the model emits it.

Mounted by vllm/entrypoints/openai/api_server.py when the relevant routes are enabled.

gRPC server

vllm/entrypoints/grpc_server.py:

• A small server intended for sidecar / mesh deployments where gRPC reduces overhead vs HTTP.
• Reuses the same AsyncLLM engine, so behavior matches the OpenAI server semantically.
• Configuration via --api-server-count, --grpc-address, etc.

MCP (Model Context Protocol)

vllm/entrypoints/mcp/:

• tool_server.py — MCP tool server that exposes vLLM-callable functions to MCP clients.
• tool.py — wrapper around individual tool definitions and authentication.

This is mainly relevant for agentic frameworks that want to expose vLLM models as MCP-callable tools.

SageMaker

vllm/entrypoints/sagemaker/ provides AWS SageMaker-specific request handlers (the /invocations and /ping endpoints SageMaker expects). Use the SageMaker integration when deploying vLLM as a SageMaker real-time inference endpoint — the routes are mounted alongside the OpenAI ones.

Pooling

vllm/entrypoints/pooling/ is not strictly an alternative frontend; it's the shared layer used by /v1/embeddings, /v1/score, /v1/rerank, and /v1/classify to convert HTTP payloads into pooling-mode engine inputs and back.

• factories.py::init_pooling_io_processors — builds the right IO processor chain based on configured task.
• scoring/ — IO processors for score / rerank.
• typing.py — Pydantic models for offline pooling I/O contexts.

See Pooling for the overall flow.

Shared infrastructure

All frontends share:

• vllm/entrypoints/launcher.py::serve_http — Uvicorn launcher
• vllm/entrypoints/utils.py — common request hooks, log helpers
• vllm/entrypoints/chat_utils.py — chat templating, multimodal item parsing
• vllm/entrypoints/openai/parser/ — tool-call parser plumbing
• vllm/reasoning/ — reasoning parsers
• vllm/v1/engine/async_llm.py::AsyncLLM — the actual engine adapter

This means that contributing a new endpoint mostly amounts to translating wire format ↔ SamplingParams/PoolingParams and forwarding to AsyncLLM.

Key source files

File	Purpose
vllm/entrypoints/anthropic/serving.py	Anthropic Messages handler (~36 KB)
vllm/entrypoints/anthropic/protocol.py	Anthropic request / response models
vllm/entrypoints/grpc_server.py	gRPC server
vllm/entrypoints/mcp/tool_server.py	MCP tool server
vllm/entrypoints/sagemaker/	SageMaker shims
vllm/entrypoints/pooling/factories.py	Pooling IO factories

Entry points for modification

• New API surface: write a router under vllm/entrypoints/<surface>/, mount it in api_server.py, reuse AsyncLLM and the existing parsers.
• Custom auth / middleware: add to vllm/entrypoints/openai/server_utils.py.
• New tool flavor: extend vllm/entrypoints/openai/parser/ and the tool parser in question.