JIT and tier 2

CPython has, since 3.13, an experimental tier-2 execution path: when a loop becomes hot, the adaptive interpreter records a sequence of micro-ops (uops), optimizes it, and then executes the result either through a small uop interpreter or — when --enable-experimental-jit is on — through machine code generated by a copy-and-patch JIT. The authoritative reference is InternalDocs/jit.md and the design notes in Python/tier2_engine.md.

This subsystem is the single most active area of CPython development today (see By the numbers).

Files

File	Role
Python/optimizer.c	Public entry points: _PyOptimizer_Optimize, _PyJit_TryInitializeTracing, executor lifecycle.
Python/optimizer_bytecodes.c	The DSL source for the optimizer — a parallel set of opcode definitions used by abstract interpretation.
Python/optimizer_cases.c.h	Generated dispatch tables for the optimizer (from optimizer_bytecodes.c).
Python/optimizer_analysis.c	Runs abstract interpretation over a recorded uop trace.
Python/optimizer_symbols.c	The abstract value lattice (constants, types, ranges) used during analysis.
Python/executor_cases.c.h	The tier-2 uop interpreter dispatch table (also generated from bytecodes.c).
Python/jit.c	The JIT engine: takes a uop trace and a stencil table, emits machine code via copy-and-patch.
Tools/jit/	Build-time tooling that compiles per-uop stencils with LLVM and emits jit_stencils.h.
Include/internal/pycore_optimizer.h	Data types: _PyExecutorObject, _PyExitData, etc.
Include/internal/pycore_jit.h	The jit_func signature and copy-and-patch helpers.
Include/internal/pycore_uop_ids.h	Generated uop ID enum.
Include/internal/pycore_uop_metadata.h	Generated per-uop metadata.
Include/internal/pycore_backoff.h	Counter helpers for "is this loop hot?" decisions.
Lib/test/test_capi/test_opt.py	The principal regression test for tier 2; one of the highest-churn files in the tree.

How a hot loop becomes a trace

sequenceDiagram
    participant Tier1 as Adaptive interpreter
    participant Counter as Backoff counter
    participant Recorder as Trace recorder
    participant Optimizer as Optimizer
    participant Executor as _PyExecutorObject
    participant JIT as JIT (jit.c)

    Tier1->>Counter: tick on JUMP_BACKWARD / RESUME
    Counter-->>Tier1: hot threshold reached
    Tier1->>Recorder: ENTER_TRACING
    Recorder->>Recorder: record uops, follow side exits
    Recorder-->>Optimizer: uop sequence
    Optimizer->>Optimizer: abstract interpretation, simplify
    Optimizer-->>Executor: build executor
    Executor->>JIT: compile (if enabled)
    JIT-->>Executor: jit_code function pointer
    Tier1->>Executor: ENTER_EXECUTOR
    Executor-->>Tier1: side exit / deopt

Entering tracing

The interpreter ticks a backoff counter on every JUMP_BACKWARD and RESUME. When it overflows, _PyJit_TryInitializeTracing (Python/optimizer.c) flips the thread into "tracing mode" via the ENTER_TRACING() macro. On platforms with computed-goto dispatch this swaps the dispatch table; on others it sets a flag.

Recording

Every dispatch in tracing mode goes through a synthetic TRACE_RECORD instruction that:

1. Captures the previous tier-1 instruction.
2. Translates it into a sequence of micro-ops via _PyJit_translate_single_bytecode_to_trace.
3. Resets adaptive counters so the trace recorder always sees up-to-date specialization.

Tracing stops based on heuristics (length, branch behaviour, repeated start). LEAVE_TRACING() restores normal dispatch.

Optimizing

_PyOptimizer_Optimize calls _Py_uop_analyze_and_optimize in Python/optimizer_analysis.c. This is an abstract interpreter over the symbolic lattice in Python/optimizer_symbols.c. It can:

• Eliminate redundant guards by tracking type constraints proven earlier in the trace.
• Constant-fold known values.
• Prove loads from frozen objects can be replaced with _LOAD_CONST_INLINE.
• Remove unnecessary refcount operations.

The optimizer's per-uop logic is itself a DSL — Python/optimizer_bytecodes.c — generated into Python/optimizer_cases.c.h by Tools/cases_generator/.

The output is wrapped in a _PyUOpExecutor_Type (_PyExecutorObject).

Executing

The trace's start instruction (the original JUMP_BACKWARD) is replaced with ENTER_EXECUTOR oparg=k, where k indexes the executor in code->co_executors. The ENTER_EXECUTOR implementation calls into the executor.

There are two execution modes:

• uop interpreter (tier2_dispatch: label in Python/ceval.c) — a small switch on uop IDs, used for debugging and on configurations without the JIT. Selected by --enable-experimental-jit=interpreter.
• JIT — jit_code points to a compiled C function. Selected by --enable-experimental-jit.

A _DEOPT uop returns to the tier-1 interpreter at the same Python instruction. An _EXIT_TRACE uop returns to the tier-1 interpreter and may trigger recording of a new trace from a side exit.

Copy-and-patch JIT

The JIT in Python/jit.c uses the technique from Haoran Xu's article. The build process:

1. At build time, Tools/jit/ takes the uop dispatch (Python/executor_cases.c.h) and generates a .c file per uop where CASE is replaced with the body. These are compiled by LLVM into object files.
2. The relocations and bytes from each object file are extracted into jit_stencils.h.
3. At runtime, _PyJIT_Compile (Python/jit.c) walks the optimized uop sequence, copies each stencil into a writable executable buffer, and patches the relocations with concrete pointers (the next stencil, the executor's data, refs to PyObject constants).

This means the JIT does no LLVM at runtime: all code generation cost is paid by the toolchain during the CPython build. The runtime cost is just memcpy + a handful of patches per uop.

The full pipeline is gated on --enable-experimental-jit; without it the JIT support code compiles to no-ops and the tier-2 path uses the uop interpreter.

Executor invalidation

Executors are stored both on the originating code object (co_executors) and in two contiguous arrays on the interpreter state: executor_blooms (a Bloom filter over assumed-immutable values) and executor_ptrs. When something the executor assumed changes — e.g. a class is mutated, a global rebinding fires LOAD_GLOBAL_VERSION invalidations — the relevant executors are dropped.

Configuration knobs

Build/runtime option	Effect
./configure --enable-experimental-jit	Build the full JIT path (requires LLVM).
./configure --enable-experimental-jit=interpreter	Build only the uop interpreter; useful for debugging.
./configure --disable-experimental-jit	Disable tier 2 entirely (the default for now).
_testinternalcapi.set_optimizer(None)	Disable tier-2 trace recording at runtime.
_testinternalcapi.get_optimizer_stats()	Return counters for trace recording, side exits, deopts, executor count.
--enable-pystats	Build with Python/pystats.c stat counters.

Entry points for modification

• New uop → Python/bytecodes.c (+ optimizer rules in Python/optimizer_bytecodes.c) → make regen-cases.
• Trace recording heuristics → Python/optimizer.c.
• Abstract value tracking → Python/optimizer_symbols.c and Python/optimizer_analysis.c.
• JIT stencil generation → Tools/jit/.
• Tests → Lib/test/test_capi/test_opt.py.