Ollama

Ollama is a tool for running and managing open large language models locally. It bundles a CLI, a local HTTP server, and the runners that execute model inference, packaged so a user can ollama run gemma3 and start chatting with a model without dealing with weights, GPU drivers, or compatibility flags.

The repository at github.com/ollama/ollama is a Go project (with CGO bindings into a vendored llama.cpp tree) that builds a single ollama binary. The same binary is the user-facing CLI, the background daemon (ollama serve), and — when invoked as a subprocess — the per-model inference runner.

What this repo contains

• A CLI built on cobra (main.go, cmd/cmd.go) with subcommands like run, pull, push, create, serve, ps, show, and launch.
• A local HTTP server (server/routes.go) that exposes the Ollama REST API on port 11434 plus OpenAI-compatible /v1/... and Anthropic-compatible /v1/messages endpoints.
• A scheduler (server/sched.go) that decides when to load and evict models, and how to fit them onto detected GPUs.
• An LLM runner subprocess (llm/server.go) that wraps llama.cpp via CGO and serves completions over a local HTTP port.
• A second-generation runner in pure Go (runner/ollamarunner/, runner/llamarunner/) plus an MLX runner under x/mlxrunner/.
• Model conversion (convert/) from safetensors / sentencepiece / pickle into GGUF.
• A Modelfile parser (parser/parser.go) that turns the user's FROM, TEMPLATE, SYSTEM, PARAMETER, etc. lines into a create request.
• A launch system (cmd/launch/) that integrates Ollama with third-party clients: Claude Code, Codex, Copilot CLI, Droid, OpenCode, Hermes, Kimi, Pi, Poolside, Cline, VS Code, and OpenClaw.
• An optional desktop app for macOS and Windows under app/ that wraps the daemon with a tray UI, dialogs, and an updater.
• A Go API client at api/client.go for talking to the server from external Go programs.

Who uses it

• End users running models locally on their own machine for chat, coding, embeddings, or image generation.
• Application developers who hit the local REST API or use the OpenAI/Anthropic compatibility layers from any language.
• Tooling vendors (editors, agents, RAG frameworks) that want a uniform on-device inference backend.

Tech stack

Area	Tools
Language	Go 1.24, plus C/C++/Objective-C/Metal in llama/llama.cpp/ and CMake glue
HTTP server	gin-gonic/gin, gin-contrib/cors
CLI	spf13/cobra
TUI	charmbracelet/bubbletea, lipgloss
Tensor / ML	Custom Go ML in ml/, model/, with backends bridging into llama.cpp and MLX
Build	Go modules, CMake (CMakeLists.txt, CMakePresets.json), Makefile.sync for vendoring llama.cpp

Quick links

• Architecture — how the CLI, server, scheduler, and runner fit together
• Getting started — build from source, run, and test
• Glossary — Ollama-specific vocabulary
• REST API — endpoint reference
• Apps — the ollama binary and the desktop wrapper
• Systems — server, scheduler, runners, model engine
• Features — launch integrations, cloud proxy, image generation, OpenAI/Anthropic compat