OpenAI / Anthropic compat

Ollama exposes OpenAI-compatible (/v1/...) and Anthropic-compatible (/v1/messages) endpoints. Translation lives in middleware/ (request shape) and openai/ / anthropic/ (response shape). The handlers in server/routes.go are unchanged: they still consume /api/chat-style requests and emit /api/chat-style responses, but middleware wraps them on both sides.

Purpose

Make any OpenAI- or Anthropic-aware client point at http://localhost:11434/v1/... and just work. The cost is one translation step per request — well worth it because handlers stay simple and behavior stays consistent across native and compat surfaces.

Endpoints

Endpoint	Maps to	Middleware	Handler
POST /v1/chat/completions	/api/chat	middleware.ChatMiddleware	Server.ChatHandler
POST /v1/completions	/api/generate	middleware.CompletionsMiddleware	Server.GenerateHandler
POST /v1/embeddings	/api/embed	middleware.EmbeddingsMiddleware	Server.EmbedHandler
GET /v1/models	/api/tags	middleware.ListMiddleware	Server.ListHandler
GET /v1/models/:model	/api/show	middleware.RetrieveMiddleware	Server.ShowHandler
POST /v1/responses	/api/chat	middleware.ResponsesMiddleware	Server.ChatHandler
POST /v1/images/generations	/api/generate (imagegen)	middleware.ImageGenerationsMiddleware	Server.GenerateHandler
POST /v1/images/edits	/api/generate (imagegen)	middleware.ImageEditsMiddleware	Server.GenerateHandler
POST /v1/audio/transcriptions	/api/chat (transcription pipeline)	middleware.TranscriptionMiddleware	Server.ChatHandler
POST /v1/messages	/api/chat	middleware.AnthropicMessagesMiddleware	Server.ChatHandler

All but /v1/embeddings, /v1/models, and the audio/image endpoints also pass through cloudPassthroughMiddleware (see cloud proxy).

Key abstractions

Symbol	Location	Purpose
middleware.ChatMiddleware	middleware/openai.go	Translates messages, tools, tool_choice, response_format, stream, temperature, etc. to/from the canonical shape.
middleware.CompletionsMiddleware	middleware/openai.go	Translates legacy prompt-shaped completions.
middleware.EmbeddingsMiddleware	middleware/openai.go	Translates input (string or array) to Inputs []string.
middleware.ResponsesMiddleware	middleware/openai.go, openai/responses.go	Translates the OpenAI Responses API (reasoning, output_text, etc.).
middleware.ImageGenerationsMiddleware, ImageEditsMiddleware	middleware/openai.go	Image-generation request translation.
middleware.TranscriptionMiddleware	middleware/openai.go	Audio transcription request translation.
middleware.AnthropicMessagesMiddleware	middleware/anthropic.go	Anthropic Messages translation: content blocks, tool use, thinking.
openai.responses.go types	openai/responses.go	Strongly-typed Responses API shape.
anthropic.anthropic.go types	anthropic/anthropic.go	Strongly-typed Anthropic Messages shape.
anthropic/trace.go	anthropic/trace.go	Anthropic streaming event tracing.

How it works

graph TD
    Request[/v1/chat/completions/]
    Request --> CompatIn[middleware translate request]
    CompatIn --> Canonical[/api/chat shape/]
    Canonical --> Handler[ChatHandler]
    Handler --> ResponseStream[NDJSON]
    ResponseStream --> CompatOut[middleware translate response]
    CompatOut --> SSE[/SSE stream/]

Translation is bidirectional: incoming request body is rewritten to the canonical shape before the handler reads it, and outgoing tokens are rewritten back as the handler writes them. Because the translation is gin middleware, both directions plug in via c.Next() and a custom ResponseWriter.

Quirks worth knowing

• Reasoning effort → think: openai: map responses reasoning effort to think (#15789) made /v1/responses respect the reasoning field by setting the equivalent think flag. api: accept "max" as a think value (#15787) extended the value space.
• Cloud passthrough order: cloud middleware runs before the compat middleware. That means cloud-bound requests are forwarded with the original OpenAI/Anthropic body — the cloud handles its own translation.
• Encoding format: /v1/embeddings accepts encoding_format: float | base64 (golden-tested in openai/openai_encoding_format_test.go and middleware/openai_encoding_format_test.go).
• Tool calls: streamed tool call deltas are reconstructed across multiple chunks both ways. The biggest test files in the repo (middleware/anthropic_test.go at 3k lines) exist to cover these streaming corner cases.

Integration points

• All compat routes share the canonical handlers in server/routes.go.
• They feed the same scheduler, runner, prompt assembly, and tool-call parsing as native routes.
• Cloud proxy (cloud-proxy) sits between the route registration and the middleware.

Entry points for modification

• New OpenAI field → extend the request and response types in openai/ and the corresponding middleware translator. Add a golden test in openai_test.go or middleware/openai_test.go.
• New Anthropic content block → extend anthropic/anthropic.go and middleware/anthropic.go.

Key source files

File	Purpose
middleware/openai.go	OpenAI-compat middleware.
middleware/anthropic.go	Anthropic-compat middleware.
openai/openai.go	OpenAI types and helpers.
openai/responses.go	Responses API types.
anthropic/anthropic.go	Anthropic types.
anthropic/trace.go	Anthropic streaming tracing.