Background

Why some pieces of Ollama exist in their current shape. The structural details are on the per-system pages; this page collects design decisions and the more interesting pitfalls.

Why a single binary plays four roles

Distributing one executable simplifies installation: macOS users drop one .app in /Applications, Linux users curl | sh a script that fetches one binary. The same binary is the CLI, the daemon, the runner subprocess, and the launch host. Re-execing as ollama runner ... (see runner/runner.go) keeps the deployment story identical regardless of backend.

The cost is a cmd/cmd.go that's nearly 2,500 lines. The team accepts that — see CONTRIBUTING.md's preference for behavior tests over implementation tests; that big file is mostly handlers, not framework.

Why two runners (llamarunner and ollamarunner)

Adding ollamarunner was an alternate Go-native runtime, not a replacement for llama.cpp. Reasons inferable from the structure:

• Llama.cpp covers more architectures and quantization formats — keeping it as the default avoids regressions for users.
• A pure-Go runner is easier to extend with custom kernels (under ml/backend/) and to integrate with kvcache policies.
• Newer model formats (harmony, GPT-OSS, MLX-targeted models) experiment in ollamarunner without disturbing the default path.

The runner.Execute switch in runner/runner.go is the seam.

Why a separate cloud proxy

Some models are too large to ever run on consumer hardware, but Ollama wants the same "by name" UX. The cloud proxy in server/cloud_proxy.go routes those requests transparently through ollama.com so the user (and the third-party tool integrating with the daemon) doesn't have to know which models are local and which aren't. Authentication uses the local SSH key already used to sign registry pulls — no new credentials.

Why a launcher subsystem

ollama launch was added in late 2025 / 2026 as a way to make Ollama useful with the AI tools users already use (Claude Code, Codex, Droid, etc.). Each integration writes the third-party tool's config to point at the local daemon, picks an appropriate model, and execs the tool. The pattern fits behind a single subcommand because the integrations are otherwise unrelated (different config formats, different launch ergonomics) — putting them under cmd/launch/ keeps them out of the daemon and out of the rest of the CLI.

Why backwards compatibility is enforced

The README points users at the local daemon as a stable target for OpenAI/Anthropic clients. Breaking that contract would invalidate every tutorial, every blog post, and every third-party tool overnight. Hence CONTRIBUTING.md:

Changes that break backwards compatibility in Ollama's API (including the OpenAI-compatible API) ... may not be accepted.

parser.deprecatedParameters (parser/parser.go) is the smallest visible artifact of that policy: parameters that are no longer used are silently dropped instead of producing errors.

Pitfalls and danger zones

CGO drift

Updating llama.cpp can shift struct layouts that the Go side reads via CGO. The fix is documented in docs/development.md: go clean -cache and rebuild. The patch series in llama/patches/ plus Makefile.sync keep the divergence auditable.

Scheduler concurrency

The scheduler (server/sched.go) lets requests run in parallel on already-loaded runners but serializes loads. Subtle changes to the load path (e.g., touching activeLoading before all error paths reset it) can deadlock. The injection points used in server/sched_test.go (loadFn, newServerFn, getGpuFn) make these failure modes catchable in tests.

Tokenizer correctness

A tokenizer mismatch produces output that's "almost right" — usually undetectable until a downstream user notices. The fix tokenizer: fix multi-regex BPE offset handling (#15844) is a good example. The unit suite under tokenizer/ is the safety net.

Runner stderr capture

When a runner exits unexpectedly, the only useful diagnostic is its stderr. StatusWriter (llm/status.go) holds a tail; bugs there silently turn legible CUDA errors into "process has exited" with no detail.

Cloud disable

Cloud passthrough is gated by org-level / install-level config. Code paths that assume cloud is reachable should consult cloudStatusDisabled before failing — the launcher does this when warning about --model selecting a cloud model.

Modelfile FROM .

FROM . only works because of expandPath in parser/parser.go treating it as the Modelfile's directory and getModelfileName in cmd/cmd.go defaulting to Modelfile when no flag is provided. Changing either side independently breaks ollama create -f workflows.

Migration context

The repo contains a few visible migrations:

• llama3.2-vision projector format change — Server.scheduleRunner (server/routes.go) returns an error directing users to ollama pull llama3.2-vision.
• Removal of use_imagegen_runner from request options — Server.scheduleRunner deletes it on every request and routing now relies on model metadata.
• bubbletea selectors replacing raw terminal I/O in cmd.init().

Each migration kept the public API stable while changing the implementation underneath.