Dynamo

Active contributors: jansel, anijain2305, mlazos, williamwen42

Purpose

torch._dynamo is the front-end of torch.compile. It captures Python bytecode into FX graphs at runtime by hooking into CPython's frame evaluation API (PEP 523). Where TorchScript scripting required Python source code that fit a typed subset, Dynamo accepts arbitrary Python and only specializes to graphs at the points where it can; everything else falls back to eager execution.

This is the hardest part of the compile stack to understand because it is a partial Python interpreter that runs inside CPython.

Directory layout

Path	Contents
torch/_dynamo/	The Dynamo package
torch/_dynamo/__init__.py	Public surface (optimize, disable, reset, config)
torch/_dynamo/convert_frame.py	Top-level frame compilation entry point
torch/_dynamo/symbolic_convert.py	The bytecode interpreter (the heart of Dynamo)
torch/_dynamo/variables/	VariableTracker subclasses, one per Python value kind
torch/_dynamo/output_graph.py	Builds the captured FX graph
torch/_dynamo/guards.py	Builds and checks recompilation guards
torch/_dynamo/eval_frame.py	The C-extension frame eval hook glue
torch/_dynamo/bytecode_transformation.py	Generates fallback bytecode
torch/_dynamo/backends/	Default compilation backends (inductor, eager, aot_eager, …)
torch/_dynamo/polyfills/	Re-implementations of stdlib behavior in tracing-friendly form
torch/_dynamo/repro/	Bug minifier + repro scripts
torch/csrc/dynamo/	C++ side: frame eval hook, guard evaluation

Key abstractions

Type	File	Purpose
InstructionTranslator	torch/_dynamo/symbolic_convert.py	The bytecode interpreter
VariableTracker	torch/_dynamo/variables/base.py	Symbolic value abstraction; one subclass per Python kind
OutputGraph	torch/_dynamo/output_graph.py	Accumulates the captured FX graph
GuardBuilder	torch/_dynamo/guards.py	Builds C++-callable guard expressions
Source	torch/_dynamo/source.py	How a value can be recovered from the original frame
BackendCompilerFunction	torch/_dynamo/backends/registry.py	Pluggable compile backend

The _dynamo/variables/ directory has one file per Python value kind — tensor.py, lists.py, dicts.py, functions.py, nn_module.py, higher_order_ops.py, ctx_manager.py, user_defined.py, etc. Each VariableTracker knows how to:

• emit FX nodes for ops it supports,
• enumerate guards that must hold for the compiled code to remain valid,
• and gracefully graph-break (fall back to eager) when it hits something it doesn't understand.

How it works

Frame evaluation hook

torch.compile(fn) returns a wrapper that installs a frame evaluation callback using _PyEval_SetFrameEvalFunc. Whenever Python is about to execute a frame whose code object is a compiled target, our hook is called instead of _PyEval_EvalFrameDefault.

The hook (defined in torch/csrc/dynamo/eval_frame.c) checks a per-code-object cache of compiled outputs. If a cache entry's guards all evaluate to true on the current frame, we run the cached compiled bytecode. Otherwise we call convert_frame to compile a new entry.

Bytecode interpretation

InstructionTranslator (symbolic_convert.py) walks the frame's bytecode op by op. For each LOAD_FAST/LOAD_GLOBAL/BINARY_OP/etc., it pushes/pops VariableTrackers on a symbolic stack rather than real values. Tensor-valued variables become FX Proxy nodes; integer/string/list variables become ConstantVariable/ListVariable/etc. that track Python-side reasoning.

When a tensor op is invoked, the corresponding TensorVariable calls output_graph.create_proxy("call_function", torch.add, ...), which appends a node to the FX graph being built.

Guards

For each fact the interpreter relied on, a guard is recorded. Examples: "this int argument equals 4", "this tensor has shape [8, dynamic] with the second dim a symbol", "this nn.Module has parameter weight with id X", "this global has not been monkey-patched". When the same code is called again, the guard list is checked first; if any fails, recompilation happens.

Guards are compiled to a C function (torch/csrc/dynamo/guards.cpp) for speed. They are also surfaced to users via torch._dynamo.config.guard_nn_modules, assume_static_by_default, etc.

Graph break

When Dynamo hits something it can't model — an unknown C extension, a side-effect on a tracked container, a print(), a try/except it doesn't support — it graph-breaks: it emits the partial graph compiled so far, generates resume bytecode that runs the rest of the original function in eager mode, then re-enters the frame eval hook for any remaining tracked region. The resume mechanism is in bytecode_transformation.py.

Symbolic shapes

Dynamo couples with torch.fx.experimental.symbolic_shapes to model dynamic shapes. Each tensor's shape can be static (specialized to a concrete int) or symbolic (carries a SymInt). The shape environment (ShapeEnv) accumulates equality and inequality constraints between symbols; these become guards.

The dynamic keyword on torch.compile controls whether sizes are auto-marked dynamic or specialized eagerly.

Backends

After the graph is captured, Dynamo hands it to a backend that compiles it. Default backends, in torch/_dynamo/backends/:

• inductor — the production backend; lowers to Triton/C++. See Inductor.
• eager — runs the FX graph in eager (debug-only).
• aot_eager — runs through AOT autograd then eager (debug-only).
• cudagraphs — wraps the eager run in CUDA graphs.
• tvm, onnxrt, openxla, tensorrt — third-party backends.

Each receives the FX graph + sample inputs and returns a callable.

graph LR
    Frame[Python frame] -->|_PyEval_SetFrameEvalFunc hook| Hook[Dynamo hook]
    Hook -->|cache miss| Convert[convert_frame]
    Convert --> Interp[InstructionTranslator<br/>walks bytecode]
    Interp -->|emit FX nodes| OG[OutputGraph]
    Interp -->|record guards| GB[GuardBuilder]
    OG -->|FX graph + sample inputs| Backend[Backend (default: inductor)]
    Backend -->|compiled callable| Cache[Per-code-object cache]
    Hook -->|cache hit + guards pass| Compiled[Run compiled bytecode]

Integration points

• Output to AOT Autograd / Inductor — Dynamo's FX graph is the input to AOT Autograd, which produces the joint forward+backward graph that Inductor compiles.
• Higher-order ops — torch._higher_order_ops (cond, while_loop, FlexAttention, etc.) are surfaced as HigherOrderOpVariable and traced through subgraphs.
• torch.export — uses Dynamo as its first stage; see Features / torch.compile.
• Logging — controlled by TORCH_LOGS=dynamo,recompiles,graph_breaks,guards,.... See torch/_logging/.

Entry points for modification

• New Python construct support → add or edit a VariableTracker subclass under torch/_dynamo/variables/.
• New default backend → register in torch/_dynamo/backends/registry.py.
• New guard kind → guards.py (Python build) + torch/csrc/dynamo/guards.cpp (C eval).
• Stdlib polyfills (e.g., functools.reduce) → torch/_dynamo/polyfills/.
• Bug repro / minifier → torch/_dynamo/repro/after_aot.py and after_dynamo.py.

Key source files

File	Purpose
torch/_dynamo/convert_frame.py	Top-level frame compilation
torch/_dynamo/symbolic_convert.py	Bytecode interpreter
torch/_dynamo/output_graph.py	FX graph builder
torch/_dynamo/guards.py	Guard construction
torch/_dynamo/variables/base.py	VariableTracker base
torch/_dynamo/variables/tensor.py	Tensor variable (the most important one)
torch/_dynamo/variables/higher_order_ops.py	HOP support
torch/_dynamo/eval_frame.py	Frame eval glue
torch/csrc/dynamo/eval_frame.c	Frame eval C hook
torch/csrc/dynamo/guards.cpp	Guard evaluation in C
torch/_dynamo/backends/inductor.py	Inductor backend wiring
torch/_dynamo/repro/after_aot.py	Post-AOT bug reproducer