Server

The HTTP daemon. Built on gin-gonic/gin, it exposes the native Ollama API, OpenAI-compatible endpoints, Anthropic-compatible endpoints, and a few experimental routes.

Purpose

Be the single local endpoint that any client — the CLI, the Go API client, or third-party tooling — can call to pull, manage, and run models. Translation between API dialects (OpenAI, Anthropic, Ollama-native) happens here so individual handlers stay close to a canonical shape.

Directory layout

server/
├── routes.go                       # Server type + every route handler (~2,830 lines)
├── sched.go                        # Scheduler (separate page)
├── auth.go                         # Sign-in / sign-out handlers
├── cloud_proxy.go                  # Cloud passthrough (separate page)
├── create.go                       # Model creation pipeline
├── download.go                     # Pull from registry
├── upload.go                       # Push to registry
├── images.go                       # Local model store (manifests + blobs)
├── model.go                        # Model type + capability checks
├── model_recommendations.go        # Cached model recommendations
├── model_resolver.go               # Cloud-vs-local resolution
├── prompt.go                       # Prompt assembly for /api/generate
├── quantization.go                 # Quantization helpers
├── renderer_resolution.go          # Selects renderer/parser per model
├── inference_request_log.go        # Per-request structured logging
├── logprob.go                      # Log-probability helpers
├── fixblobs.go                     # On-startup blob repair
├── sparse_*.go                     # Sparse-file helpers per OS
├── gemma4_test.go, laguna_quantization_test.go, ...
└── internal/                       # client, registry, helpers

Key abstractions

Symbol	Location	Purpose
Server	server/routes.go	The composition root. Holds the scheduler, default num-ctx, request logger, and recommendation cache.
Server.GenerateRoutes	server/routes.go	Wires every gin route to a handler.
Server.scheduleRunner	server/routes.go	Validates the model, capabilities, and options, then asks the scheduler for a runner.
Server.GenerateHandler, ChatHandler, EmbedHandler, EmbeddingsHandler	server/routes.go	The inference handlers. They stream NDJSON for /api/generate and /api/chat, single JSON for embeddings.
Server.PullHandler, PushHandler, CreateHandler, CopyHandler, DeleteHandler, CreateBlobHandler, HeadBlobHandler	server/routes.go	Model lifecycle.
Server.ListHandler, ShowHandler, PsHandler, StatusHandler	server/routes.go	Read-side handlers.
Server.WhoamiHandler, SignoutHandler	server/auth.go and server/routes.go	Cloud account handlers.
Server.WebSearchExperimentalHandler, WebFetchExperimentalHandler, ModelRecommendationsExperimentalHandler	server/routes.go	Experimental endpoints, all forwarded through the cloud proxy.
withInferenceRequestLogging	server/inference_request_log.go	Wraps inference handlers so each request is captured for replay/debug.

Route table

The full route registration in Server.GenerateRoutes:

HEAD /                                        liveness
GET  /                                        liveness
HEAD /api/version                             version
GET  /api/version                             version
GET  /api/status                              StatusHandler
POST /api/pull                                PullHandler
POST /api/push                                PushHandler
HEAD /api/tags                                ListHandler
GET  /api/tags                                ListHandler
POST /api/show                                ShowHandler
DELETE /api/delete                            DeleteHandler
POST /api/me                                  WhoamiHandler
POST /api/signout                             SignoutHandler
DELETE /api/user/keys/:encodedKey             SignoutHandler
POST /api/create                              CreateHandler
POST /api/blobs/:digest                       CreateBlobHandler
HEAD /api/blobs/:digest                       HeadBlobHandler
POST /api/copy                                CopyHandler
POST /api/experimental/web_search             WebSearchExperimentalHandler
POST /api/experimental/web_fetch              WebFetchExperimentalHandler
GET  /api/experimental/model-recommendations  ModelRecommendationsExperimentalHandler
GET  /api/ps                                  PsHandler
POST /api/generate                            GenerateHandler        + inference logging
POST /api/chat                                ChatHandler            + inference logging
POST /api/embed                               EmbedHandler
POST /api/embeddings                          EmbeddingsHandler
POST /v1/chat/completions                     ChatHandler            + cloud passthrough + ChatMiddleware + logging
POST /v1/completions                          GenerateHandler        + cloud passthrough + CompletionsMiddleware + logging
POST /v1/embeddings                           EmbedHandler           + cloud passthrough + EmbeddingsMiddleware
GET  /v1/models                               ListHandler            + ListMiddleware
GET  /v1/models/:model                        ShowHandler            + cloud passthrough + RetrieveMiddleware
POST /v1/responses                            ChatHandler            + cloud passthrough + ResponsesMiddleware + logging
POST /v1/images/generations                   GenerateHandler        + cloud passthrough + ImageGenerationsMiddleware
POST /v1/images/edits                         GenerateHandler        + cloud passthrough + ImageEditsMiddleware
POST /v1/audio/transcriptions                 ChatHandler            + TranscriptionMiddleware
POST /v1/messages                             ChatHandler            + cloud passthrough + AnthropicMessagesMiddleware + logging

The + middleware columns refer to translators in middleware/ that map alternative API shapes onto the daemon's canonical handlers. Cloud passthrough is cloudPassthroughMiddleware from server/cloud_proxy.go.

How it works

sequenceDiagram
    participant Client
    participant Gin as gin Router
    participant MW as middleware (compat)
    participant Cloud as cloud_proxy
    participant Handler as ChatHandler / GenerateHandler
    participant Sched as Scheduler
    participant Runner as runner subprocess

    Client->>Gin: POST /v1/chat/completions
    Gin->>MW: cloudPassthroughMiddleware
    MW->>Cloud: is model cloud-only?
    Cloud-->>MW: no -> continue
    MW->>MW: ChatMiddleware (translate to /api/chat shape)
    MW->>Handler: ChatHandler
    Handler->>Sched: scheduleRunner(name, caps, opts)
    Sched-->>Handler: runnerRef
    Handler->>Runner: POST /completion
    Runner-->>Handler: token stream
    Handler-->>MW: NDJSON
    MW-->>Client: SSE / JSON (re-translated)

For native /api/chat, the middleware step is skipped — the handler reads the request directly.

Capabilities and options

Every model carries a list of declared capabilities (chat, tools, embedding, vision, thinking, completion, insert). Model.CheckCapabilities (server/model.go) verifies each request asks for a capability the model can provide. Server.modelOptions (server/routes.go) merges the model's default options with the per-request overrides, applying the daemon's defaultNumCtx when the model doesn't override it.

Inference request logging

Routes registered with s.withInferenceRequestLogging("path", handler)... go through server/inference_request_log.go. The logger captures each request/response into the configured log directory (controlled by OLLAMA_INFERENCE_LOG_DIR from envconfig/config.go) so production issues can be replayed offline.

Integration points

• Spawns the scheduler at startup.
• Calls into auth/ for SSH-key sign-in.
• Delegates to middleware/ for OpenAI/Anthropic translation.
• Calls into server/cloud_proxy.go for cloud-only models.
• Uses tools/, thinking/, harmony/, template/ when assembling and parsing prompts.
• Reads/writes the model store through server/images.go, server/download.go, server/upload.go.

Entry points for modification

• New native API endpoint → register in Server.GenerateRoutes and add a <Name>Handler method on Server.
• New OpenAI/Anthropic compat shape → add middleware in middleware/ that translates onto an existing handler. Avoid branching inside the canonical handler.
• New experimental route → place behind /api/experimental/... so it can move or be removed without breaking compatibility.

Key source files

File	Purpose
server/routes.go	Server type, route table, every handler.
server/sched.go	Scheduler — see scheduler page.
server/auth.go	Sign-in helpers.
server/inference_request_log.go	Per-request log capture.
server/cloud_proxy.go	Cloud passthrough.
server/model.go	Model type + capability checks.
server/prompt.go	Prompt assembly for /api/generate.
server/renderer_resolution.go	Selects renderer/parser per model family.