Lore

A timeline of major eras in the PyTorch repository, derived from git log and the dated comments in RELEASE.md/docs/source/notes/.

2012 – Torch 7 origins

The first commit in the repo is dated January 25, 2012:

053065ba23 initial revamp of torch7 tree
a8f1077567 initial revamp of torch7 tree
c7d7de315b initial revamp of torch7 tree

The initial code was a Lua-based scientific-computing toolkit (Torch 7) developed at NYU and IDIAP, with C kernels under TH/THC. The PyTorch repo started life as a fork of that history.

2016 – PyTorch announced

Soumith Chintala, Adam Paszke, Sam Gross, and the early team rewrote the front-end in Python on top of Torch's C/C++ kernels. The first public release was October 2016. The C kernels lived under torch/lib/TH, THC, THNN, THCUNN for years; their final removal didn't complete until PyTorch 1.x.

2017–2018 – ATen and the dispatcher

The C-style THTensor/THCTensor were gradually rewritten in C++ as at::Tensor (ATen). The "new dispatcher" (c10::Dispatcher) appeared in this era, replacing a hard-coded if-else chain with a key-set-driven lookup. Meta took over much of the development as Facebook AI Research scaled up.

2018–2019 – Caffe2 merger

In 2018, Caffe2's repo was merged into PyTorch's; this is why the codebase still has caffe2/ directories and a few caffe2:: namespaces in the C++ side. The Caffe2 proto/IR was the seed of what became TorchScript and torch.jit.

2019 – TorchScript and torch.jit.script

torch.jit shipped as the production-deployment story: a Python-typed-subset compiler with its own IR. For several years it was the recommended path for non-Python deployment.

2019–2020 – at::DispatchKey redesign

Multiple iterations on key-set semantics: separation of backend keys from autograd keys, the notion of "fallthrough" kernels, the introduction of Functionalize, FuncTorchBatched, and the Python key. The dispatcher and key-set machinery you see today crystallized in this period.

2021 – Functorch

The functorch library was developed externally as a JAX-style transform package (vmap, grad, vjp, jvp) that worked on PyTorch tensors. It was merged into the main repo in 2022 and rebranded as torch.func shortly after.

2022 – torch.compile foundations

In late 2022, three new subsystems landed simultaneously:

• TorchDynamo (torch/_dynamo/) — Python frame evaluation hook + symbolic bytecode interpreter.
• AOTAutograd (torch/_functorch/aot_autograd.py) — make-a-graph-functional pass plus joint forward+backward tracer.
• TorchInductor (torch/_inductor/) — Triton-based codegen backend.

These three together became what was announced as PyTorch 2.0 in December 2022 / March 2023. The torch.compile() user-facing entry point was added alongside.

2023 – PyTorch 2.0 and Triton

PyTorch 2.0 made torch.compile opt-in. Subsequent 2.x point releases promoted it from "beta" to "stable" for an ever-widening set of models. Triton became a hard dependency for CUDA inductor; CPU inductor uses C++ vector kernels generated with cpp_wrapper.

2023 – Float8, FlashAttention, scaled_dot_product_attention

Native fp8 dtypes (float8_e4m3fn, float8_e5m2) landed; torch.nn.functional.scaled_dot_product_attention was added as a unified entry to FlashAttention/MEM_EFF/Math kernels. The torch.ao quantization stack grew significantly.

2024 – DTensor, FSDP2, distributed.tensor

The distributed stack saw a major rewrite:

• torch.distributed.tensor — DTensor, a sharded tensor that composes naturally with torch.compile and autograd.
• FSDP2 (torch/distributed/_composable/fsdp/) — a re-implementation of FSDP on top of DTensor with fully_shard-style API.
• torch.distributed.pipelining — pipeline-parallel scheduling.
• tensor.parallel — tensor-parallel API.

Together these became the "composable distributed" stack that replaces (over time) the older DistributedDataParallel and original FSDP.

2024 – torch.export / AOTInductor

torch.export produces a strict ATen-IR ExportedProgram — the new portable model format. AOTInductor (torch/_inductor/aot_inductor.py) compiles an ExportedProgram to a self-contained .so runnable from C++ without Python. This is the modern non-Python-deployment story; it sits alongside (and is gradually superseding) TorchScript.

2024 – TorchScript deprecation

The jit.script path was officially declared "maintenance only" — bug fixes accepted, no new features. Deployment moves toward torch.export + AOTInductor.

2025 – Helion, autoheuristic, Triton 3.x integration

The compiler stack continues to evolve: Helion (helion/ as a sibling project, integrated via torch._inductor.kernel) is an AI-assisted Triton kernel generator; autoheuristic learning landed for tile-size / num-warp selection; CUTLASS-based GEMM templates were added to Inductor's choice set.

2026 – present

Recent activity (last 6 months: 8K+ commits, 860 contributors, ~315 commits per week). Headline themes:

• Stable C-API surface (torch/csrc/stable/) — out-of-tree extensions that survive PyTorch upgrades.
• GraphPickler — a pickler that's safe by default, paired with torch.load(..., weights_only=True) becoming the default.
• Increasing dynamic-shape coverage in torch.compile.
• MTIA backend: Meta's training accelerator, with its own dispatch keys and stream type.
• MPS / XPU parity — closing the gap with CUDA op coverage.

Where to read next

• By the numbers — the current snapshot.
• Fun facts — historical anecdotes.
• Systems / Inductor — the modern compiler.