CLI

Purpose

transformers ships a Typer-based CLI installed alongside the Python package. It exposes utilities for environment reporting, model downloads, an interactive chat REPL, an OpenAI-compatible HTTP server, and the model-scaffolding helper add-new-model-like.

The CLI was migrated from argparse to Typer on 2025-10-16 in PR #41487. The entry script (src/transformers/cli/transformers.py) is now 41 lines.

Sub-commands

Command	File	What it does
transformers env	src/transformers/cli/system.py	Print versions of transformers, torch, CUDA, etc. — paste into bug reports.
transformers version	same	Print the installed version.
transformers download	src/transformers/cli/download.py	Resolve and cache a model and its weights.
transformers chat	src/transformers/cli/chat.py (29K LOC)	Interactive chat REPL against a checkpoint or a transformers serve endpoint.
transformers serve	src/transformers/cli/serve.py + src/transformers/cli/serving/	OpenAI-compatible HTTP server.
transformers add-new-model-like	src/transformers/cli/add_new_model_like.py (33K LOC)	Scaffold a new model directory by copying and renaming an existing one.

The app Typer application is assembled in src/transformers/cli/transformers.py:

from huggingface_hub import check_cli_update, typer_factory

from transformers.cli.add_new_model_like import add_new_model_like
from transformers.cli.chat import Chat
from transformers.cli.download import download
from transformers.cli.serve import Serve
from transformers.cli.system import env, version

app = typer_factory(help="Transformers CLI")
app.command()(add_new_model_like)
app.command(name="chat")(Chat)
app.command()(download)
app.command()(env)
app.command(name="serve")(Serve)
app.command()(version)

transformers chat

Launches a REPL that runs model.generate (or pipes through transformers serve). Reads a YAML config for system prompts and tools, and supports readline navigation. Notable knobs:

• --system <prompt> — prepend a system prompt.
• --max-new-tokens N — generation length cap.
• --seed N — deterministic sampling.
• --config path.yaml — load defaults from disk.
• --remote-url <url> — talk to a transformers serve instance instead of running locally.

The implementation in src/transformers/cli/chat.py integrates httpx and huggingface_hub.AsyncInferenceClient for remote mode and transformers.GenerationConfig for local mode.

transformers serve

Production-grade OpenAI-compatible server. Highlights:

• --continuous-batching — turns on the paged-KV scheduler from src/transformers/generation/continuous_batching/.
• --cb-block-size, --cb-num-blocks, --cb-max-batch-tokens — KV pool tuning.
• --port, --host, --ssl-* — networking.
• --force-model <repo> — preload one model and refuse to switch.
• --enable-tools, --enable-vision — feature gates.

Exposes:

• POST /v1/chat/completions (OpenAI shape).
• POST /v1/completions (legacy completions).
• POST /v1/responses (Responses API).
• POST /v1/audio/transcriptions (Whisper-style).
• GET /v1/models (list loaded models).

Implementation lives under src/transformers/cli/serving/ (utilities and per-route handlers). The HTTP layer uses fastapi (declared in setup.py).

transformers download

Wraps huggingface_hub with transformers-aware defaults:

transformers download Qwen/Qwen2.5-1.5B

Resolves and downloads the config, weights, tokenizer, and any generation_config.json, populating the local ~/.cache/huggingface cache.

transformers add-new-model-like

Scaffolds a new model directory by copying an existing one and performing class/file renames. Used as the first step of "Add a new model" workflows. Output:

src/transformers/models/<new>/__init__.py
src/transformers/models/<new>/configuration_<new>.py
src/transformers/models/<new>/modeling_<new>.py
src/transformers/models/<new>/tokenization_<new>.py (if applicable)
src/transformers/models/<new>/<image|video|feature>_processing_<new>.py (if applicable)
tests/models/<new>/...
docs/source/en/model_doc/<new>.md

The implementation tracks every reference to the source model name and applies the rename consistently (config, modeling, tokenizer, processor, tests, docs, auto mappings). Run make fix-repo afterwards.

transformers env

Reports a sanitized version table that should be pasted into bug reports:

- `transformers` version: 5.8.0.dev0
- Platform: Linux-6.1.166-x86_64
- Python version: 3.10.14
- Huggingface_hub version: 1.5.0
- PyTorch version (GPU?): 2.4.0 (True)
- Tensorflow version: not installed
- Flax version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

Integration points

• Pipelines — transformers chat uses the text-generation pipeline (or serve).
• Generation — transformers serve --continuous-batching activates the paged scheduler.
• Tokenization — chat templates are applied before forwarding prompts.

Entry points for modification

• New sub-command → add a module under src/transformers/cli/, decorate with @app.command(), register it in transformers.py.
• Server route → add a route under src/transformers/cli/serving/ and wire it into the FastAPI app constructed by Serve.
• Tests → tests/cli/.