Debugging

The library ships a few in-tree tools for diagnosing modeling and tokenization bugs.

Logging

Logging is centralized at src/transformers/utils/logging.py. The package logger is transformers. Levels:

import transformers
transformers.logging.set_verbosity_info()    # default is WARNING
transformers.logging.set_verbosity_debug()
transformers.logging.disable_progress_bar()

The logger respects the TRANSFORMERS_VERBOSITY env var (debug, info, warning, error, critical).

Loading reports

When from_pretrained cannot match every parameter in the checkpoint to the model (or vice versa), it now emits a structured "loading report" instead of free-form warnings. Source: src/transformers/utils/loading_report.py. The report breaks down:

• Parameters that loaded cleanly.
• Missing keys (model wants but checkpoint lacks) — typically prediction heads.
• Unexpected keys (checkpoint has but model lacks) — typically dropped layers.
• Mismatched shapes — usually a config issue.

Model debugging utilities

src/transformers/model_debugging_utils.py adds opt-in instrumentation:

• model_addition_debugger_context — attaches forward-hook tracing to compare two implementations layer-by-layer.
• Tools for printing the shape and dtype of every intermediate tensor.

The matching docs file is docs/source/en/model_output_tracing.md.

Attention visualizer

src/transformers/utils/attention_visualizer.py renders attention masks (causal, sliding window, custom) as ASCII art in the terminal. Useful when debugging cache + attention interactions:

from transformers.utils.attention_visualizer import AttentionVisualizer
AttentionVisualizer(...).plot()

Trainer debugging knobs

TrainingArguments (src/transformers/training_args.py) has many debug-friendly flags:

• --debug overflow underflow runs DebugUnderflowOverflow (detects NaN/Inf during forward/backward).
• --debug tpu_metrics_debug for TPU.
• --include_inputs_for_metrics to surface inputs in compute_metrics.

The underflow/overflow detector is in src/transformers/debug_utils.py.

Generation debugging

• output_attentions=True, output_hidden_states=True, and return_dict_in_generate=True make generate return all intermediate tensors.
• streamer=TextStreamer(tokenizer) prints tokens as they are produced (src/transformers/generation/streamers.py).
• model.generate(..., logits_processor=...) lets you inject a custom processor that records the logits.

Common error patterns

Symptom	Likely cause	Where to look
RuntimeError: probability tensor contains either inf, nan or element < 0	NaN in logits during sampling	do_sample=False to confirm; check FP16 underflow; try bfloat16 or full precision
OSError: We couldn't connect to ...	Hub cache miss with no network	Set HF_HUB_OFFLINE=1 and ensure model is cached
IndexError: index out of range in self	Tokenizer / vocab mismatch	Confirm tokenizer and model come from the same checkpoint
KeyError after edits to auto_mappings.py	Forgot make fix-repo	Rerun fixers
AttributeError: ... has no attribute 'forward' after upgrade	Class renamed in a release	See MIGRATION_GUIDE_V5.md and CHANGELOG notes
Slow from_pretrained	First-time download or shard concatenation	Pre-cache checkpoints; consider low_cpu_mem_usage=True

Network logging

src/transformers/utils/network_logging.py records every HTTP call to the Hub. Useful for diagnosing why a download is slow or repeatedly retrying.

CI-specific debugging

For failures only seen in CI:

• Check compare_test_runs.py and the dashboard URLs surfaced in the Slack/Github Action notifications.
• Look at .github/workflows/TROUBLESHOOT.md for known infra issues.
• The script utils/check_bad_commit.py automates git bisect to find the commit that introduced a regression.

See also

• Patterns and conventions for what conventions cause subtle bugs (# Copied from, modular drift).
• Generation for the decoding pipeline.
• Cache for KV cache state.