Modular models

The newer mechanism for adding architectures while preserving the "One Model, One File" tenet. A contributor writes a small modular_<name>.py shard that inherits from another model; make fix-repo expands it into the full set of standalone files (modeling_<name>.py, configuration_<name>.py, tokenizer files, image-processor files) that end users read.

Why modular

Before this mechanism, adding a new architecture meant either:

1. Hand-writing a multi-thousand-line modeling_<name>.py from scratch, or
2. Copy-pasting from a similar model and decorating with # Copied from comments.

Both were tedious. Modular replaces them with a Pythonic diff: declare which classes you reuse, override what differs, and let the converter handle the rest.

What lives where

File	Purpose
src/transformers/models/<arch>/modular_<arch>.py	The shard authored by maintainers.
src/transformers/models/<arch>/modeling_<arch>.py	Generated; do not edit directly.
src/transformers/models/<arch>/configuration_<arch>.py	Generated.
utils/modular_model_converter.py (113K LOC)	The expansion engine.
utils/modular_model_detector.py (38K LOC)	Detects whether a model could be modular-ized.
utils/check_modular_conversion.py	CI guard: regenerates and diffs.
docs/source/en/modular_transformers.md	Authoritative reference.

Authoring a shard

A typical modular_<name>.py skeleton:

# Copyright …
"""<Arch> model implementation."""
from ..llama.modeling_llama import LlamaForCausalLM, LlamaModel, LlamaConfig

class <Arch>Config(LlamaConfig):
    model_type = "<arch>"
    # add or override attributes

class <Arch>Model(LlamaModel):
    pass

class <Arch>ForCausalLM(LlamaForCausalLM):
    # override forward, prepare_inputs_for_generation, etc. as needed
    pass

The converter:

1. Resolves every imported symbol back to its source file.
2. Inlines the source class bodies into the generated file.
3. Renames symbols (Llama → <Arch>).
4. Honours selective overrides (only methods you redefine differ from the source).
5. Emits standalone files (modeling_<arch>.py, configuration_<arch>.py, …) that look hand-written.

Run the converter

make fix-repo
# or directly
python utils/modular_model_converter.py --files-to-parse src/transformers/models/<arch>/modular_<arch>.py

CI runs python utils/check_modular_conversion.py to verify that every checked-in modeling_*.py matches what the converter would produce from its modular_*.py shard. Drift fails CI.

Inheritance and multi-parent reuse

Shards can compose pieces from multiple parents:

from ..llama.modeling_llama import LlamaForCausalLM
from ..mistral.modeling_mistral import MistralRMSNorm

class <Arch>RMSNorm(MistralRMSNorm):
    pass

class <Arch>ForCausalLM(LlamaForCausalLM):
    # uses <Arch>RMSNorm where Llama would have used LlamaRMSNorm
    pass

Override mechanics

Overrides happen by attribute name. To replace LlamaAttention.forward you write:

class <Arch>Attention(LlamaAttention):
    def forward(self, hidden_states, ...):
        # custom implementation
        ...

The converter inlines LlamaAttention.__init__ into <Arch>Attention and replaces forward with your version. Adding new methods is straightforward; deleting requires explicit del Cls.method.

What about # Copied from?

The two mechanisms coexist. # Copied from predates modular and remains useful for cross-model copies that aren't whole inheritance relationships (e.g., copying a single RMSNorm class without subclassing). make fix-repo runs both utils/check_copies.py and utils/check_modular_conversion.py so the codebase stays consistent.

230 of 462 model directories now ship a modular_*.py. The rest still rely on hand-written modeling_*.py (often older models that predate the converter).

Tests still target the generated files

You write tests against <Arch>ForCausalLM etc. just as before. The shrunk-down test fixtures generated by utils/create_dummy_models.py work identically.

Detecting candidates

utils/modular_model_detector.py heuristically identifies models that could be modular-ized. Run it locally to see suggestions; output is informational only.

Common pitfalls

• Editing the generated file. Your edit will disappear on the next make fix-repo. Always edit the shard.
• Forgetting to run make fix-repo. CI's check_modular_conversion will fail.
• Mismatched class names. The shard must use the canonical <Arch>Config, <Arch>Model, <Arch>ForCausalLM (or matching task) so the converter can rename consistently.
• Side effects in module top level. Modular shards should be importable without side effects. Move dynamic logic into class bodies or __init__.

Integration points

• Modeling — modular shards produce the same <Arch>PreTrainedModel/<Arch>Model surface.
• Auto classes — register the new architecture in auto_mappings.py after the converter generates files.
• Tooling — check_modular_conversion, check_copies, make fix-repo.

Entry points for modification

• Add a model → write a modular_<name>.py, register in auto_mappings.py, run make fix-repo.
• Improve the converter → utils/modular_model_converter.py. Tests are in tests/repo_utils/ and tests/test_modeling_common.py (round-trip parity).
• New override pattern → see existing examples like src/transformers/models/qwen2_moe/modular_qwen2_moe.py.