Architecture

Ollama ships as a single ollama binary that wears different hats depending on how it is invoked. The CLI process talks to a long-running daemon process, the daemon schedules per-model runner subprocesses, and the runner subprocesses host model inference. Everything between them goes over local HTTP.

High-level diagram

graph TD
    subgraph User["User process"]
        CLI[ollama CLI<br>cmd/cmd.go]
        TUI[ollama run<br>interactive TUI<br>cmd/interactive.go]
        Launch[ollama launch<br>cmd/launch/]
    end

    subgraph Daemon["ollama serve daemon"]
        Routes[Gin router<br>server/routes.go]
        Sched[Scheduler<br>server/sched.go]
        CloudProxy[Cloud proxy<br>server/cloud_proxy.go]
        Models[Model store<br>~/.ollama/models]
    end

    subgraph Runners["Per-model runner subprocesses"]
        LlamaRunner[llamarunner<br>llama.cpp via CGO]
        OllamaRunner[ollamarunner<br>pure-Go ml/ + model/]
        MLXRunner[mlxrunner<br>x/mlxrunner Apple MLX]
        ImageGen[imagegen<br>x/imagegen image models]
    end

    subgraph External["External integrations"]
        OllamaCloud[ollama.com cloud]
        ThirdParty[Claude Code, Codex,<br>Copilot CLI, Droid, OpenCode]
    end

    CLI -->|HTTP 11434| Routes
    TUI -->|HTTP 11434| Routes
    Launch -->|HTTP 11434| Routes
    Launch -.spawns.-> ThirdParty

    Routes -->|GetRunner| Sched
    Sched -->|spawns subprocess| LlamaRunner
    Sched -->|spawns subprocess| OllamaRunner
    Sched -->|spawns subprocess| MLXRunner
    Sched -->|spawns subprocess| ImageGen
    Routes -->|reads / writes| Models
    Routes -.web search, web fetch,<br>cloud-only models.-> CloudProxy
    CloudProxy -.HTTPS.-> OllamaCloud

Components

CLI (cmd/cmd.go)

main.go is two lines. All work happens in cmd.NewCLI(), which builds the cobra command tree:

• serve — start the daemon
• run, stop, ps, show, list, cp, rm — model lifecycle
• pull, push — registry I/O
• create — build a new model from a Modelfile or safetensors
• runner — internal hook that re-execs ollama as the inference runner subprocess (see runner subsystem)
• launch — orchestrate third-party integrations (see launch integrations)
• signin, signout, whoami — Ollama Cloud account state

The CLI is also the interactive shell: ollama run <model> enters the loop in cmd/interactive.go, which streams /api/chat responses and renders thinking blocks.

Daemon (server/)

ollama serve brings up a gin.Engine configured in Server.GenerateRoutes (in server/routes.go). Major route groups:

• Native API: /api/generate, /api/chat, /api/embed, /api/tags, /api/show, /api/pull, /api/push, /api/create, /api/copy, /api/delete, /api/ps, /api/blobs/:digest, /api/me, /api/signin, /api/signout.
• OpenAI compat: /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models, /v1/responses, /v1/images/generations, /v1/images/edits, /v1/audio/transcriptions — translated by middleware in middleware/ and handlers in openai/.
• Anthropic compat: /v1/messages — translated by middleware/anthropic.go and anthropic/anthropic.go.
• Experimental: /api/experimental/web_search, /api/experimental/web_fetch, /api/experimental/model-recommendations.

Routes that need a loaded model call Server.scheduleRunner, which delegates to the scheduler.

Scheduler (server/sched.go)

The scheduler tracks all loaded runnerRefs and serializes loads. Its workflow:

1. A handler sends an LlmRequest on pendingReqCh.
2. Scheduler.processPending checks if the model is already loaded; if so, it reuses the runner.
3. Otherwise it asks discover.GPUDevices for free VRAM, calls loadFn (default llm.NewLlamaServer), and waits for WaitUntilRunning.
4. processCompleted notices when keep-alive expires and triggers Close() on the runner subprocess.

Only one model can load at a time (activeLoading), but already-loaded models can serve concurrent requests.

Runner subprocesses (runner/, llm/, x/mlxrunner/, x/imagegen/)

When the scheduler loads a model, it forks ollama runner ... as a child process. The child re-enters the same binary at runner.Execute (runner/runner.go), which dispatches to one of:

• llamarunner.Execute — wraps llama.cpp through CGO (llama/llama.go). The default for most GGUF models.
• ollamarunner.Execute — selected with --ollama-engine. A pure-Go inference runtime that uses the model definitions in model/models/ and the tensor backends in ml/backend/.
• mlxrunner.Execute — selected with --mlx-engine. Runs models on Apple Silicon via the MLX framework (x/mlxrunner/).
• imagegen.Execute — selected with --imagegen-engine. For image-generation models (x/imagegen/).

The runner exposes a small HTTP API (/completion, /embedding, /tokenize, /detokenize, /health) on a port chosen by llm/server.go. The daemon talks to it through the LlamaServer interface.

Model store (server/images.go, server/download.go)

Models live as content-addressed blobs under $OLLAMA_MODELS (default ~/.ollama/models). A manifest references the blobs that make up a model: GGUF weights, projector for vision models, license, params, template. parser.Modelfile.CreateRequest (parser/parser.go) translates a Modelfile into a CreateRequest that walks every FROM/ADAPTER/MESSAGE/TEMPLATE directive and uploads the referenced blobs through /api/blobs/:digest.

Cloud proxy (server/cloud_proxy.go)

Some models are too large for any single user's hardware. Ollama Cloud provides remote inference for them. The cloud proxy forwards /api/chat (and the OpenAI/Anthropic compat endpoints) to ollama.com when the requested model is a cloud model, signing requests with the user's local SSH key. The same path serves /api/experimental/web_search and /api/experimental/web_fetch.

Launch integrations (cmd/launch/)

ollama launch <integration> sets up a third-party client to use Ollama as its model backend. Supported integrations include Claude Code, Codex, Copilot CLI, Droid, OpenCode, Hermes, Kimi, Pi, Poolside, Cline, VS Code, and OpenClaw — each implemented as a file under cmd/launch/ (e.g., claude.go, droid.go). The launcher writes integration-specific config files, ensures the daemon is running, picks a model, and execs the third-party tool.

Desktop app (app/)

The optional desktop app wraps the daemon with a tray icon and a webview. The Windows variant is built with wintray (app/wintray/), the macOS variant with native dialogs (app/dialog/). It includes an updater (app/updater/) and a webview (app/webview/) that opens the launch UI.

Request flow: ollama run gemma3 "hello"

sequenceDiagram
    participant User
    participant CLI as ollama (CLI)
    participant Daemon as ollama serve
    participant Sched as Scheduler
    participant Runner as llamarunner

    User->>CLI: ollama run gemma3 "hello"
    CLI->>Daemon: HEAD /api/version
    Daemon-->>CLI: 200 OK
    CLI->>Daemon: POST /api/show {model: gemma3}
    Daemon-->>CLI: capabilities, template, options
    CLI->>Daemon: POST /api/chat (stream=true)
    Daemon->>Sched: GetRunner(model, opts)
    Sched-->>Daemon: runnerRef (loads if needed)
    Daemon->>Runner: POST /completion
    Runner-->>Daemon: token stream
    Daemon-->>CLI: NDJSON stream
    CLI-->>User: rendered output

Build flow

The repository ships with both Go-native code and a vendored llama.cpp. CMake builds the C/C++/Metal/CUDA/HIP code under llama/llama.cpp/ into shared libraries, then go build links the Go code against them via CGO. See getting started for prerequisites per OS.

Where to learn more

• Server and HTTP routes
• Scheduler
• Runners
• LLM subprocess (llama.cpp wrapper)
• Modelfile and parser
• Launch integrations
• Cloud proxy