Postprocessing

Active contributors: AUTOMATIC1111, Aarni Koskela, w-e-w, light-and-ray

Purpose

Owns image-level transformations that happen after sampling: face restoration (GFPGAN, CodeFormer), upscaling (ESRGAN, RealESRGAN, SwinIR, ScuNET, LDSR, DAT, HAT, latent), and the Extras tab pipeline. Distinct from "post-sample callbacks" which run per-step on latents.

Directory layout

modules/
├── postprocessing.py                  # the Extras tab handler (run() entry point)
├── extras.py                          # internal upscale helpers + Extras script wiring
├── scripts_postprocessing.py          # ScriptPostprocessing base class + runner
├── scripts_auto_postprocessing.py     # auto-applied postprocessors (chain after generation)
├── face_restoration.py                # FaceRestoration interface + registry
├── face_restoration_utils.py
├── codeformer_model.py                # CodeFormer impl
├── gfpgan_model.py                    # GFPGAN impl
├── upscaler.py                        # Upscaler / UpscalerData base classes
├── upscaler_utils.py                  # tile-based upscale helper
├── esrgan_model.py
├── realesrgan_model.py
├── dat_model.py
├── hat_model.py
└── modelloader.py                     # model auto-download
extensions-builtin/
├── LDSR/                              # latent diffusion super-resolution
├── ScuNET/                            # ScuNET upscaler
├── SwinIR/                            # SwinIR / Swin2SR
└── postprocessing-for-training/       # an alwayson postprocessor
scripts/
├── postprocessing_codeformer.py       # registers face restoration as an Extras step
├── postprocessing_gfpgan.py
└── postprocessing_upscale.py          # the main upscale step in Extras

Key abstractions

Type	File	Description
ScriptPostprocessing	modules/scripts_postprocessing.py	Base class for an Extras step. Implements name, order, ui(), process(pp, **args).
PostprocessedImage (pp)	same	The image being processed; pp.image is the current PIL image, pp.info accumulates infotext entries.
FaceRestoration	modules/face_restoration.py	Each restorer (GFPGAN, CodeFormer) registers an instance into shared.face_restorers.
Upscaler / UpscalerData	modules/upscaler.py	Base class for upscale impls. Each registers Upscaler.scalers: list[UpscalerData] to shared.sd_upscalers.
upscale_with_model(...)	modules/upscaler_utils.py	Tile-based upscale loop with progress reporting.
auto_postprocessing	modules/scripts_auto_postprocessing.py	Glue that runs a chain of ScriptPostprocessing instances after every generation if the user enables them in Settings.

The Extras tab pipeline

graph TD
    Image[Input image / batch] --> R[ScriptRunner.run_postprocess_image]
    R --> S1[ScriptPostprocessing 1<br/>(e.g. GFPGAN)]
    S1 --> S2[ScriptPostprocessing 2<br/>(e.g. Upscale)]
    S2 --> S3[ScriptPostprocessing N]
    S3 --> Save[images.save_image]

The Extras tab is built in modules/ui_postprocessing.py. When the user clicks Generate, control reaches postprocessing.run() in modules/postprocessing.py. This iterates the registered ScriptPostprocessing instances in order order and calls each one's process(pp, **args). Each step can mutate pp.image and append to pp.info.

The same pipeline can also run automatically after every txt2img/img2img generation (Settings → "Postprocessing" → "Enable postprocessing"); auto_postprocessing.py is the wiring.

Upscalers

Each upscaler registers UpscalerData records in shared.sd_upscalers. Records carry the model file path, scale, and a back-reference to the Upscaler that knows how to load the file. The "Upscaler" dropdown in Postprocessing → Upscale 1 reads this list directly.

Upscaler	Module	Notes
Lanczos / Nearest	modules/upscaler.py	Pure-PIL fallbacks; always available
ESRGAN	modules/esrgan_model.py	Loads .pth ESRGAN files via Spandrel
RealESRGAN	modules/realesrgan_model.py	Bundles a list of community RealESRGAN models
DAT	modules/dat_model.py	Dual aggregation transformer
HAT	modules/hat_model.py	Hybrid Attention Transformer
SwinIR / Swin2SR	extensions-builtin/SwinIR/
ScuNET	extensions-builtin/ScuNET/
LDSR	extensions-builtin/LDSR/	Latent Diffusion Super-Resolution
Latent	(txt2img only)	Bicubic on latents; only for hires fix

Almost all of these are wrappers around Spandrel, which provides a uniform load_model() for the various super-resolution architectures.

Face restorers

Two impls ship in-repo:

• GFPGAN (modules/gfpgan_model.py) — TencentARC's face restoration model.
• CodeFormer (modules/codeformer_model.py) — sczhou's CodeFormer with a fidelity slider.

Both register a FaceRestoration into shared.face_restorers. They are used:

1. As Extras tab postprocessing steps (each one is a ScriptPostprocessing in scripts/postprocessing_*.py).
2. As an inline step in process_images_inner when the user ticked "Restore faces" on the txt2img/img2img tab.

Tile-based upscaling

Most upscalers process tiles to keep VRAM bounded. upscaler_utils.upscale_with_model() handles tile splitting, padding, and reassembly with feathering. The tile size is per-upscaler (opts.ESRGAN_tile, opts.SCUNET_tile, etc.). Upscalers that don't need tiling (LDSR, Lanczos) bypass this helper.

Integration points

• Add a postprocessor to the Extras tab — drop a ScriptPostprocessing subclass under scripts/postprocessing_*.py (or in an extension's scripts/). It will be auto-registered.
• Add an upscaler — subclass Upscaler and put it in a module loaded at boot. It must populate Upscaler.scalers in __init__.
• Add a face restorer — implement FaceRestoration and append to shared.face_restorers from your on_app_started.
• Hook into image saving — use on_before_image_saved/on_image_saved (see script-callbacks.md) instead of writing a new postprocessor when the change is metadata-only.

Entry points for modification

• The Extras pipeline is one of the more contained subsystems — most user-facing changes happen in scripts/postprocessing_upscale.py (e.g., the v1.10 stop-mid-extras fix #16085).
• The CPU upscale path was rewritten in v1.10.1 (the only commit on master after v1.10.0) to fix a regression — see #16275.
• Extension authors typically don't need to touch postprocessing.py itself; registering a ScriptPostprocessing is enough.