Models

Active contributors: AUTOMATIC1111, w-e-w, Aarni Koskela, brkirch, light-and-ray

Purpose

Discovers checkpoint files on disk, loads their weights into a usable PyTorch model, switches between checkpoints on demand, and tracks the currently loaded model in shared.sd_model. Also handles VAE swaps, model hashing for infotext, and the in-memory cache that lets the user toggle between recently-used models without re-reading the file.

Directory layout

modules/
├── sd_models.py                 # checkpoint list, load/reload, ModelType enum
├── sd_models_config.py          # picks the right config YAML for a checkpoint
├── sd_models_xl.py              # SDXL-specific load helpers
├── sd_models_types.py           # WebuiSdModel base/protocol type
├── sd_vae.py                    # VAE list, load, switch
├── sd_vae_approx.py             # approximate VAE for live previews
├── sd_vae_taesd.py              # tiny AE distillation
├── sd_unet.py                   # custom UNet override (used by extensions)
├── sd_disable_initialization.py # skip default-weight init at load time
├── modelloader.py               # url-download / auto-download helpers
├── hashes.py                    # SHA-256 caching with disk cache
├── cache.py                     # general purpose disk cache
├── safe.py                      # `safe_open` and pickle restrictions
└── lowvram.py                   # weight-on-CPU mode
configs/                         # YAML model definitions (v1, v2, sdxl-inpaint, sd3, alt-diffusion)
models/
├── Stable-diffusion/            # checkpoints
├── VAE/                         # VAE files
├── ControlNet/                  # extension-managed
├── Lora/                        # extension-managed
└── ...

Key abstractions

Type	File	Description
CheckpointInfo	modules/sd_models.py	Per-file record: filename, name, hash (sha256), shorthash, type, ids, alias map.
ModelType	same	Enum: SD1, SD2, SDXL, SSD, SD3. Detected from state-dict shapes.
checkpoints_list	same	dict[str, CheckpointInfo] of all known checkpoints, keyed by display title.
checkpoint_aliases	same	Lookup table for legacy short names and hash-prefixed identifiers.
checkpoints_loaded	same	OrderedDict cache of state-dicts kept in CPU RAM (size limited).
WebuiSdModel	modules/sd_models_types.py	Protocol attached to every loaded model: is_sdxl, is_sd2, is_sd1, is_sd3, cond_stage_key, latent_channels, etc.
load_model_weights()	modules/sd_models.py	Reads safetensors/ckpt, applies VAE, hijacks the model, installs the result as shared.sd_model.
reload_model_weights()	same	Higher-level switch that reuses cached weights when possible.
find_checkpoint_config()	modules/sd_models_config.py	Picks a YAML from configs/ based on filename or detected model type.
get_sha256	modules/hashes.py	Returns the cached or freshly-computed sha256 of a file.

Discovery

On startup, list_models() walks models/Stable-diffusion/ and (optionally) --ckpt-dir. Each .ckpt / .safetensors becomes a CheckpointInfo. Names are made unique by suffixing the path; hashes are computed lazily when first needed. The result is exposed as a dropdown in shared.opts.sd_model_checkpoint and via /sdapi/v1/sd-models.

The first checkpoint loaded is determined by:

1. --ckpt if passed,
2. opts.sd_model_checkpoint from config.json,
3. The first checkpoint in alphabetical order.

If no checkpoint exists at all, modules/launch_utils.py downloads v1-5-pruned-emaonly.safetensors from Hugging Face. v1.10 added a hash check to detect corrupted downloads (#15602).

Loading

sequenceDiagram
    participant Caller
    participant SDM as sd_models
    participant Cache as checkpoints_loaded
    participant Disk
    participant Hijack as sd_hijack
    participant VAE as sd_vae

    Caller->>SDM: reload_model_weights(checkpoint_info)
    SDM->>Cache: state_dict cached?
    alt cached
        Cache-->>SDM: tensors
    else miss
        SDM->>Disk: load .safetensors with safe_open
        Disk-->>SDM: state_dict
        SDM->>Cache: store
    end
    SDM->>SDM: detect ModelType from keys
    SDM->>SDM: build LatentDiffusion via OmegaConf(config)
    SDM->>SDM: model.load_state_dict(weights)
    SDM->>VAE: load_vae(model, vae_file)
    SDM->>Hijack: model_hijack.hijack(model)
    SDM->>SDM: shared.sd_model = model
    SDM-->>Caller: model

The hijack step monkey-patches the loaded model so that prompt parsing, attention optimisation and embeddings work — see sd-hijack.md.

Configs

configs/ contains the YAML files that define each model architecture. The right one is picked by sd_models_config.find_checkpoint_config():

Config	Used for
v1-inference.yaml	Stable Diffusion 1.x
v1-inpainting-inference.yaml	SD 1.x inpainting variants
v2-inference.yaml, v2-inference-v.yaml	Stable Diffusion 2.x (in repositories/ upstream)
sd_xl_inpaint.yaml	SDXL inpainting
sd3-inference.yaml	Stable Diffusion 3
alt-diffusion-inference.yaml	Alt-Diffusion
instruct-pix2pix.yaml	InstructPix2Pix variant

For SDXL the config comes from repositories/generative-models. For SD 3, the entire model code lives under modules/models/sd3/ — added in v1.10 along with sd3-inference.yaml.

VAE

modules/sd_vae.py maintains a parallel list of .vae.pt/.safetensors files in models/VAE/. The active VAE is set by opts.sd_vae and applied at model load:

• Automatic — use the VAE bundled with the checkpoint.
• None — keep raw latents (useful for debugging).
• A specific filename — swap that VAE in.

Two approximate VAEs ship with the repo for live previews:

• Approx (modules/sd_vae_approx.py) — a tiny linear projection from latents to RGB, fast but blurry.
• TAESD (modules/sd_vae_taesd.py) — Ollin Boer Bohan's distilled tiny autoencoder; sharper, slightly slower.

The active preview decoder is selected by opts.live_preview_vae.

Hashing

Two hashes are computed per checkpoint:

• Old hash — first 9 chars of sha256(first 8 KB) from the legacy era. Fast but not secure.
• sha256 — full file hash, cached in cache.json.
• shorthash — first 10 chars of the sha256.

Hashes are written to infotext as Model hash: 1a2b3c4d and Model: my-model [1a2b3c4d]. The cache code is in modules/hashes.py; the disk cache lives in cache.json next to config.json.

Low-VRAM mode

modules/lowvram.py implements --medvram and --lowvram by registering _register_forward_hooks that move sub-modules between CPU and GPU around each forward pass. With --medvram, the UNet stays on GPU and other components shuttle in/out. With --lowvram everything shuttles, including UNet attention layers. SDXL adds significant memory pressure, so v1.7 introduced --medvram-sdxl as a more selective default for SDXL workflows.

Safe pickle loading

modules/safe.py wraps torch.load so that arbitrary __reduce__ payloads can't execute. By default only safe types are unpickled; --disable-safe-unpickle removes the restriction. Safetensors files are loaded directly, bypassing pickle entirely.

Integration points

• script_callbacks.on_model_loaded(callback) — extensions get notified after shared.sd_model is replaced.
• script_callbacks.list_unets_callback — extensions can register custom UNet implementations swapped in by modules/sd_unet.py.
• /sdapi/v1/sd-models, /sdapi/v1/refresh-checkpoints, /sdapi/v1/sd-vae, /sdapi/v1/refresh-vae — REST equivalents of the dropdowns.
• /sdapi/v1/unload-checkpoint, /sdapi/v1/reload-checkpoint — explicit memory management.
• extra_networks_lora (extra-networks.md) patches the loaded model's UNet/text-encoder linear layers in place.

Entry points for modification

• Add a new model architecture — add a YAML to configs/, extend ModelType and _detect_model_type_from_state_dict in modules/sd_models.py, and (if the architecture differs significantly) add a new directory under modules/models/ like the SD3 case.
• Add a new VAE preview — implement the same interface as sd_vae_taesd.py and register it in shared_options.
• Customise hash logic — change hashes.py. Be careful: hashes appear in saved infotext and changing them silently invalidates years of generation metadata.