torchgen

Active contributors: ezyang, bdhirsh, albanD

Purpose

torchgen is the in-tree code generator that produces most of the boilerplate around ATen ops: the C++ at::add(...) headers, the dispatch registration files, the autograd kernels, the Python argument parsers, and a small mountain of variable structures (_VF, _torch_docs, etc.). It is invoked at build time by CMake and lives at torchgen/.

If you have ever wondered why the Operators_*.cpp files are huge and look hand-written but always agree with native_functions.yaml, this is why.

Directory layout

Path	Contents
torchgen/	The codegen package
torchgen/gen.py	Top-level entry; wires everything together
torchgen/api/	Helpers for translating between native, dispatcher, structured, autograd, and Python signatures
torchgen/dest/	Per-target emitters (RegisterCPU, RegisterCUDA, register-meta, lazy, …)
torchgen/model.py	Parsed-YAML model (NativeFunction, FunctionSchema, OperatorName, …)
torchgen/native_function_generation.py	Out-of-line schema generation
torchgen/static_runtime/	Static runtime codegen for the JIT static runtime
torchgen/executorch/	Codegen for ExecuTorch
tools/autograd/	Autograd-specific codegen (gen_autograd.py, templates)
tools/autograd/derivatives.yaml	Derivatives that drive autograd codegen
aten/src/ATen/templates/	Mustache-style templates that get rendered

Key abstractions

Type	File	Purpose
NativeFunction	torchgen/model.py	Parsed entry from native_functions.yaml
FunctionSchema	torchgen/model.py	Parsed schema string
Argument / Return	torchgen/model.py	Schema components
BackendIndex	torchgen/model.py	Per-backend dispatch table
DispatchKey	torchgen/model.py	Mirrors c10's enum
*API modules	torchgen/api/	Per-flavor signature builders (native, dispatcher, structured, autograd, python)

How it works

Inputs

The single source of truth is aten/src/ATen/native/native_functions.yaml, supplemented by:

• tools/autograd/derivatives.yaml — differentiation rules.
• aten/src/ATen/native/tags.yaml — op tags (pointwise, inplace_view, data_dependent_output, …).
• Smaller per-feature YAMLs (functorch/test/, torch/_export/serde/schema.yaml).

Pipeline

graph TD
    YAML[native_functions.yaml<br/>derivatives.yaml<br/>tags.yaml] --> Model[Model parser<br/>torchgen/model.py]
    Model --> API[API translators<br/>torchgen/api/]
    API --> Dest[Per-target emitters<br/>torchgen/dest/]
    Dest --> Templates[Mustache templates<br/>aten/src/ATen/templates/<br/>tools/autograd/templates/]
    Templates --> Out[Generated files<br/>build/aten/src/ATen/<br/>torch/csrc/autograd/generated/<br/>...]

torchgen.gen.gen is the entry point. It parses YAML, runs each emitter, and writes outputs into the build directory. Outputs include:

• build/aten/src/ATen/Operators.h — declarations like at::add(Tensor, Tensor, Scalar).
• build/aten/src/ATen/RegisterCPU.cpp, RegisterCUDA.cpp, RegisterMPS.cpp, RegisterMeta.cpp, … — dispatcher registrations.
• build/aten/src/ATen/RegisterCompositeImplicitAutograd.cpp, RegisterCompositeExplicitAutograd.cpp — composite key registrations.
• torch/csrc/autograd/generated/Functions.h/.cpp — per-op *Backward0 Nodes.
• torch/csrc/autograd/generated/VariableType_*.cpp — autograd kernels.
• torch/csrc/autograd/generated/python_torch_functions.cpp, python_variable_methods.cpp, python_nn_functions.cpp, … — Python bindings.

API translators

The torchgen/api/ modules know how to translate a FunctionSchema into a C++ signature for each role:

• native.py — the signature used by at::native::* implementations.
• dispatcher.py — the boxed/unboxed dispatcher signature.
• structured.py — the meta + impl signature for structured kernels.
• autograd.py — the signature seen by autograd-generated VariableType wrappers.
• python.py — the args/kwargs handling for the Python binding.
• cpp.py — the user-facing at:: C++ API.

When you add an op, all of these "views" of it are produced from the one YAML entry.

Out-of-tree backends

External backends (XLA, MTIA, IREE, …) consume torchgen's library API through torchgen.gen_backend_stubs.py. They feed in their own dispatch tables and get back generated registration code that links against PyTorch's c10::Dispatcher.

ExecuTorch and selective build

torchgen/executorch/ produces the constrained op set used by ExecuTorch. The "selective build" feature lets users include only ops a model uses; torchgen reads a usage YAML and emits a stripped registration file.

Integration points

• CMake. Codegen runs as a CMake build step before the C++ libraries compile. See cmake/Codegen.cmake and tools/codegen/.
• ATen. Generated code lives next to ATen and is what users actually call. See ATen.
• Autograd. Per-op autograd kernels are generated. See Autograd.
• Python bindings. torch.add, Tensor.add_, Tensor.transpose_ etc. are all generated.
• Lazy / Functorch / Static Runtime. Each has its own emitter under torchgen/dest/ or its own subpackage.

Entry points for modification

• New op codegen target (e.g., a new accelerator) → add an emitter under torchgen/dest/, wire it into torchgen/gen.py.
• New autograd codegen behaviour → tools/autograd/gen_autograd.py and the templates in tools/autograd/templates/.
• New schema feature (e.g., a new argument flag) → extend torchgen/model.py to parse it, then propagate to API translators.

Key source files

File	Purpose
torchgen/gen.py	Entry point
torchgen/model.py	YAML model classes
torchgen/api/native.py	Native signature
torchgen/api/dispatcher.py	Dispatcher signature
torchgen/api/python.py	Python bindings
torchgen/dest/register_dispatch_key.py	Per-key Register*.cpp emitter
tools/autograd/gen_autograd.py	Autograd codegen
tools/autograd/templates/Functions.cpp	Backward Nodes template
tools/autograd/templates/VariableType.cpp	Autograd kernel template
aten/src/ATen/templates/TensorBody.h	C++ Tensor public class