Systems

Internal subsystems that make up the library. Each is an architectural building block that does not map cleanly to a single deployable unit or a workspace package, so the "systems" lens applies.

Pages in this section

Page	Source roots	What it covers
Configuration	src/transformers/configuration_utils.py	PretrainedConfig, JSON I/O, deprecated kwargs
Modeling	src/transformers/modeling_utils.py, src/transformers/modeling_outputs.py, src/transformers/modeling_layers.py	PreTrainedModel, mixins, weight tying, forward dispatch
Tokenization	src/transformers/tokenization_*	Slow / fast / mistral-common backends, chat templates
Generation	src/transformers/generation/	generate(), decoding strategies, logits processors
Cache	src/transformers/cache_utils.py	KV cache hierarchy
Attention	src/transformers/modeling_flash_attention_utils.py, src/transformers/integrations/sdpa_attention.py, src/transformers/integrations/flex_attention.py, src/transformers/integrations/*_paged.py	Pluggable attention backends
Pipelines	src/transformers/pipelines/	25+ task pipelines
Trainer	src/transformers/trainer*.py, src/transformers/training_args*.py	Training loop, callbacks, distributed
Quantization	src/transformers/quantizers/, src/transformers/integrations/	bitsandbytes, GPTQ, AWQ, mxfp4, hqq, …
Integrations	src/transformers/integrations/	accelerate, deepspeed, peft, FSDP, TP, MoE, kernels
Processing	src/transformers/processing_utils.py, src/transformers/image_processing_*, src/transformers/feature_extraction_*, src/transformers/video_processing_utils.py	Multimodal preprocessors
Data	src/transformers/data/, src/transformers/hf_argparser.py	Data collators, GLUE/SQuAD processors, argparse helpers
CLI	src/transformers/cli/	transformers chat, serve, download, add-new-model-like, env, version

How they fit together

graph TD
    User[User code] --> Pipeline[Pipelines]
    User --> Trainer
    User --> Generate[generate]
    User --> CLI[transformers CLI]

    Pipeline --> Tokenization
    Pipeline --> Processing
    Pipeline --> Modeling
    Trainer --> Modeling
    Trainer --> Data
    Trainer --> Integrations
    Generate --> Modeling
    Generate --> Cache
    CLI --> Pipeline
    CLI --> Generate

    Modeling --> Configuration
    Modeling --> Cache
    Modeling --> Attention
    Modeling --> Quantization
    Modeling --> Integrations

See also

• Models for the per-architecture lens.
• Features for cross-cutting capabilities (auto classes, from_pretrained, modular models, continuous batching, …).