Getting started

Prerequisites

• Python: 3.10 to 3.14 (SUPPORTED_PYTHON_VERSIONS = (10, 14) in setup.py).
• PyTorch: 2.4+. Transformers v5 dropped TensorFlow and JAX backends; PyTorch is the only supported backend (see V5 migration).
• OS: Linux and macOS are first-class. Windows works for the library but several CI matrices skip it.

Install

For library users:

# Stable from PyPI
pip install "transformers[torch]"

# Or with uv
uv pip install "transformers[torch]"

For development:

git clone https://github.com/huggingface/transformers.git
cd transformers
pip install -e ".[dev]"   # all dev deps; heavy
# or, for a slim setup
pip install -e ".[torch,quality,testing]"

The setup.py file declares roughly 200 optional dependencies grouped into extras (torch, dev, quality, testing, audio, vision, integrations, etc.).

First inference

from transformers import pipeline

pipe = pipeline(task="text-generation", model="Qwen/Qwen2.5-1.5B")
print(pipe("Climate change is", max_new_tokens=20))

The pipeline factory (src/transformers/pipelines/__init__.py, 66K LOC) auto-detects the right preprocessor and model class from the checkpoint's config.json and instantiates the matching Pipeline subclass. See Pipelines.

First fine-tune

from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments
from datasets import load_dataset

ds = load_dataset("imdb", split="train[:2000]")
tok = AutoTokenizer.from_pretrained("distilbert/distilbert-base-uncased")
ds = ds.map(lambda x: tok(x["text"], truncation=True, padding="max_length"), batched=True)

model = AutoModelForSequenceClassification.from_pretrained("distilbert/distilbert-base-uncased", num_labels=2)

args = TrainingArguments(output_dir="out", num_train_epochs=1, per_device_train_batch_size=8)
Trainer(model=model, args=args, train_dataset=ds).train()

TrainingArguments (src/transformers/training_args.py, 2,868 LOC) defines every knob; Trainer (src/transformers/trainer.py, 4,418 LOC) drives the loop. See Trainer.

Common dev commands

The full Makefile is short; here are the targets you will use:

Command	What it does
make style	Runs ruff (utils/checkers.py ruff_check,ruff_format,init_isort,sort_auto_mappings --fix).
make typing	Runs the ty type checker and modeling-structure rules.
make check-repo	All of the above plus consistency checks (copies, modular conversion, dummies, doc TOC, docstrings, …). Read-only, never mutates.
make fix-repo	Applies all auto-fixes: copies, modular conversion, doc TOC, docstrings, plus make style. Run before opening any PR.
make test	Runs all unit tests with pytest -p random_order -n auto --dist=loadfile.
make benchmark	Generation throughput benchmark with benchmark/benchmark.py.

make fix-repo is mandatory because of the # Copied from ... and modular_*.py mechanisms; see Patterns and conventions.

Running tests

The full test suite is large (412K LOC under tests/). Most local development uses targeted runs:

# A single model
pytest tests/models/llama/

# Tests that match a pattern
pytest tests/models/llama/ -k "generate"

# Slow tests (skipped by default; require GPU and downloads)
RUN_SLOW=1 pytest tests/models/llama/

# Use the test fetcher to find tests covering your branch's changes
python utils/tests_fetcher.py

Many GPU-only tests are gated by @require_torch_gpu, @require_torch_multi_gpu, @require_flash_attn, etc. (see src/transformers/testing_utils.py, 159K LOC). Testing goes deeper.

CLI tools

pip install transformers[torch] registers the transformers CLI:

transformers env             # environment report
transformers chat <repo_id>  # interactive chat against a model
transformers serve <repo_id> # OpenAI-compatible server
transformers download <repo_id>
transformers add-new-model-like

See CLI.

Where things live

transformers/
├── src/transformers/
│   ├── models/<name>/      # 462 architecture directories
│   ├── pipelines/          # 25+ task pipelines
│   ├── generation/         # generate(), logits processors, caches, continuous batching
│   ├── quantizers/         # quantization adapters
│   ├── integrations/       # accelerate, deepspeed, peft, FSDP, TP, …
│   ├── data/               # data collators, GLUE/SQuAD processors
│   ├── cli/                # transformers CLI
│   └── utils/              # hub I/O, import dispatch, dummies, auto-docstrings
├── tests/                  # mirrors src/ structure
├── examples/pytorch/       # reference fine-tuning scripts
├── benchmark/, benchmark_v2/
├── docker/                 # CI Dockerfiles
├── utils/                  # repo consistency scripts (check_*, modular_model_converter.py)
└── docs/source/en/         # user docs (separate from this wiki)