Systems

PyTorch is a stack of cooperating subsystems. This section walks through the major ones — the things you have to understand to navigate the codebase. Each subsystem has its own page.

Page	What it is
c10	Header-only core: TensorImpl, Storage, dispatch keys, scalar types, allocators
ATen	Tensor library, op definitions, native kernels
Dispatcher	The op routing table that backends, autograd, autocast, vmap, etc. all plug into
Autograd	Reverse-mode autodiff engine
Dynamo	Bytecode-level graph capture (torch/_dynamo/)
AOT Autograd	Joint forward/backward tracing (torch/_functorch/_aot_autograd/)
Inductor	Compiler backend: Triton GPU codegen, C++ CPU codegen
FX	Symbolic tracer + IR + transformation toolkit
Functorch	vmap, grad, jvp, jacfwd, etc.
JIT / TorchScript	Legacy compilation stack
Distributed	c10d, FSDP, DTensor, RPC, pipeline parallelism
ONNX	ONNX export
Quantization (torch.ao)	Eager / FX / PT2E quantization
Profiler	Kineto-backed trace + memory/CPU profiler
Serialization	torch.save / torch.load, weights-only, package, export
CUDA backend	The CUDA-specific layers across c10/, aten/, torch/
MPS backend	Apple Silicon Metal backend
torchgen	Code generator for ops, autograd, Python bindings
Tensor subclasses	FakeTensor, FunctionalTensor, NestedTensor, MaskedTensor
Other subsystems	Smaller subsystems and pointers (lazy, nativert, mtia, sparse, package, …)

Some subsystems sit logically inside torch/ Python packages but are large enough to be treated as systems here (e.g., Dynamo, Inductor, the torch.distributed core). For top-level user-facing capabilities like "torch.compile" or "Distributed training" see Features.

Big picture

graph TB
    subgraph Eager["Eager runtime"]
        ATen2[ATen ops]
        Disp[Dispatcher]
        AG[Autograd engine]
    end
    subgraph Compile["Compile stack"]
        Dyn[Dynamo]
        AOT[AOT Autograd]
        FX2[FX graphs]
        Ind[Inductor]
    end
    subgraph DistSys["Distributed"]
        C10D[c10d backends]
        FSDP[FSDP / FSDP2]
        DT[DTensor]
        Pipe[Pipelining]
    end
    subgraph Export["Export and serving"]
        TE[torch.export]
        ONNX2[ONNX exporter]
        JIT2[JIT / TorchScript]
        Pkg[torch.package / AOTI]
    end
    Eager --> Compile
    Eager --> DistSys
    Eager --> Export
    Compile --> Export

Each subsystem page below documents its own purpose, key files, data flow, and the dispatcher keys / FX nodes / IR structures it owns or interacts with.