Eager execution

What it is

The default execution mode: every torch.* call runs immediately, autograd is built dynamically, and Python control flow works because there is no Python control flow being captured. Eager is what the README is selling under "imperative experiences".

If you write torch.relu(x @ w + b) outside of torch.compile or torch.jit, you're using eager mode.

End-to-end path

sequenceDiagram
    participant User as Python
    participant PyBind as Generated Python binding
    participant Disp as Dispatcher
    participant AG as Autograd kernel
    participant Kernel as Backend kernel
    User->>PyBind: a + b
    PyBind->>Disp: at::add(a, b)
    Disp->>AG: AutogradCUDA registered for "add"
    AG->>AG: save inputs, build AddBackward0, set grad_fn
    AG->>Disp: redispatch with Autograd keys removed
    Disp->>Kernel: native::add_kernel_cuda
    Kernel-->>User: Tensor(grad_fn=AddBackward0)

The Python-side latency on a no-op call is dominated by PyArg_ParseTupleAndKeywords in the generated arg parser plus the dispatcher's keyset lookup. The C++ overhead is extremely tight; PyTorch's eager mode is one of the fastest in the industry on small-tensor workloads.

What "eager" means in practice

• Allocations happen on every call. The CUDA caching allocator hides most of the cost, but there is still a Python-side temporary tensor object.
• No fusion. Each op is a separate kernel launch.
• No graph specialization. Each call sees the actual shapes.
• Autograd graph is rebuilt each forward. Loops, conditionals, recursion all "just work".
• All Python features are available. Print, breakpoints, exceptions all work the way you'd expect.

Where it lives in the code

• The generated Python bindings (build/torch/csrc/autograd/generated/python_*.cpp, produced by torchgen) are the entry point.
• The Dispatcher routes the call.
• The Autograd kernels build the tape.
• ATen kernels run the math.
• c10 provides the underlying tensor objects and allocator.

Why you'd choose it (and why you wouldn't)

You'd choose eager mode for:

• Research. Experimenting with new architectures and loss functions where breakage is constant.
• Debugging. Stack traces point at your code; print(t) works.
• Small models. Where compile overhead would dominate.
• Workflows that don't fit graph capture. Heavy Python control flow, dynamic shapes that change every batch, custom autograd functions, extension modules.

You'd avoid eager (and use torch.compile instead) for:

• Production training of large models where kernel-launch overhead becomes a significant fraction of step time.
• Fused-attention / FlexAttention workloads that need compile-time fusion.
• Memory-tight training where AOT autograd's rematerialization saves real money.

Most PyTorch users start in eager and add @torch.compile selectively where it pays off.