Image generation

Image-generation models run in their own runner backend under x/imagegen/ and are exposed through OpenAI-compatible /v1/images/generations and /v1/images/edits endpoints.

Purpose

Treat image-generation models as first-class Ollama models — pulled, stored, and run through the same scheduler — without forcing the LLM runner to grow into a multimodal generator. The imagegen runner is selected via runner dispatch in runner/runner.go (--imagegen-engine).

Directory layout

x/imagegen/
├── ...                # entry, model definitions
├── mlx/               # MLX-based image generation (the largest file is mlx.go at ~2,400 lines)
├── manifest/          # imagegen manifest types
└── ...

server/
└── routes.go          # /v1/images/generations and /v1/images/edits

middleware/
└── openai.go          # ImageGenerationsMiddleware, ImageEditsMiddleware

Endpoints

POST /v1/images/generations  -> ImageGenerationsMiddleware -> GenerateHandler -> imagegen runner
POST /v1/images/edits        -> ImageEditsMiddleware       -> GenerateHandler -> imagegen runner

The imagegen runner returns image data (typically PNG/WebP) instead of token streams. The middleware translates between OpenAI's image-API shape (prompt, size, n, response_format) and the GenerateRequest shape the handler expects.

Selection

Server.scheduleRunner (server/routes.go) explicitly removes the legacy use_imagegen_runner request option:

delete(requestOpts, "use_imagegen_runner")

Routing now relies on the model's metadata (and the dedicated middleware) to decide that an imagegen runner is needed. The scheduler then forks ollama runner --imagegen-engine ... (runner/runner.go) instead of the default llamarunner.

Backend

The MLX-based pipeline at x/imagegen/mlx/mlx.go is the primary backend. It hosts the diffusion / generation loop, image preprocessing, and the HTTP API the daemon talks to. Other directories under x/imagegen/ host model definitions, helpers, and platform shims.

x/imagegen/manifest/ carries imagegen-specific manifest extensions; the daemon's manifest/ package consumes them via the imports imagegenmanifest "github.com/ollama/ollama/x/imagegen/manifest" in server/routes.go.

Integration points

• Runners: runner/runner.go dispatches --imagegen-engine to imagegen.Execute.
• Cloud proxy: image-generation requests go through cloudPassthroughMiddleware (server/cloud_proxy.go) when the model is cloud-only.
• OpenAI compat: see features/openai-anthropic-compat.

Entry points for modification

• New image model → add it under x/imagegen/'s model definitions; matching entries in convert/ for the conversion path.
• Different generation API surface → adjust the imagegen middleware in middleware/openai.go to translate the new shape onto GenerateHandler.

Key source files

File	Purpose
x/imagegen/	Imagegen runner and model code.
x/imagegen/mlx/mlx.go	MLX-based generation pipeline.
runner/runner.go	Dispatcher entry.
middleware/openai.go	ImageGenerationsMiddleware, ImageEditsMiddleware.
server/routes.go	Route registration and handler.