Systems

The internal building blocks of Ollama. Each subsystem here is shared across multiple commands and APIs and has a clear boundary in the source tree.

Pages

• Server — the gin HTTP server, routes, and request lifecycle.
• Scheduler — the model load/evict logic.
• Runners — the inference subprocess dispatcher and the four runner backends.
• LLM subprocess — the llm.LlamaServer interface and the llama.cpp wrapper.
• Model engine — the pure-Go inference stack (ml/, model/, kvcache/, sample/).
• Discover — GPU/CPU detection and capability negotiation.
• Storage and registry — manifests, blobs, pull, push.
• Conversion — safetensors / pickle → GGUF.
• Tokenizer — tokenizer implementations and BPE handling.
• Template — chat template engine.
• Tools — function-calling parser.
• Thinking — reasoning-token parser.
• Harmony — GPT-OSS structured prompt format.
• Auth — SSH-key based auth.
• Other subsystems — small but worth knowing about.