Threading and the GIL

CPython has supported OS threads since 1.5, but its concurrency story is dominated by the Global Interpreter Lock. Two related projects in flight reshape that story: per-interpreter GILs (PEP 684, 3.12) and the optional GIL-less build (PEP 703, 3.13+). This page covers all three configurations.

Files

File	Role
Python/ceval_gil.c	The GIL itself: acquire/release/drop/take. ~46k lines including the nogil branches.
Python/pystate.c	PyRuntimeState, PyInterpreterState, PyThreadState lifecycle.
Python/lock.c	_PyMutex / _PyOnceFlag / _PyEvent — the new per-object locks used by free-threaded code.
Python/parking_lot.c	Generic "park threads waiting on an address" used by _PyMutex. Modeled after WebKit/Rust.
Python/qsbr.c	Quiescent-state based reclamation, the safe-memory-reclamation primitive used by the nogil GC.
Python/thread.c	The portable thread API (uses thread_pthread.h or thread_nt.h).
Python/thread_pthread.h	POSIX implementation of the thread API.
Python/thread_nt.h	Windows implementation.
Modules/_threadmodule.c	The C _thread module backing threading.
Lib/threading.py	The high-level Thread, Lock, RLock, Event, Condition API.
Python/critical_section.c	Py_BEGIN_CRITICAL_SECTION macros for free-threaded mutation.
Python/brc.c	Biased reference counting helpers used in the nogil build.
Python/uniqueid.c	Per-object unique IDs for QSBR and biased refcounts.

Three lock topologies

graph TD
    subgraph "Classic (3.11 and earlier)"
        RT1[_PyRuntime: ONE GIL] --> I1A[interp 1]
        RT1 --> I1B[interp 2]
        I1A --> T1A[thread]
        I1A --> T1B[thread]
        I1B --> T1C[thread]
    end

graph TD
    subgraph "Per-interpreter GIL (3.12+)"
        RT2[_PyRuntime] --> I2A[interp 1: own GIL]
        RT2 --> I2B[interp 2: own GIL]
        I2A --> T2A[thread]
        I2A --> T2B[thread]
        I2B --> T2C[thread]
    end

graph TD
    subgraph "Free-threaded (--disable-gil)"
        RT3[_PyRuntime] --> I3A[interp 1: NO GIL]
        I3A --> T3A[thread]
        I3A --> T3B[thread]
        I3A --> T3C[thread]
    end

The default 3.13+ build is "per-interpreter GIL". --disable-gil opts into free threading; that build also has per-interpreter state, but no GIL at all and uses fine-grained locks instead.

How the GIL works

Conceptually the GIL is a single mutex that protects every interpreter data structure in a given PyInterpreterState. Implementation:

• A PyMutex-style flag (set via atomic ops) plus a wait list.
• A switch interval (sys.setswitchinterval, default 5 ms): every interval, the running thread sets a gil_drop_request flag.
• The interpreter checks eval_breaker between bytecodes. If gil_drop_request is set, it releases the GIL, briefly waits, and re-acquires. This gives other threads a chance.

This is in take_gil / drop_gil in Python/ceval_gil.c. The check is inserted into the dispatch macros (CHECK_EVAL_BREAKER()).

A C extension that does CPU-bound work without holding the GIL uses the macros Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS (Include/cpython/ceval.h). Inside this block the thread must not touch any PyObject*. Examples are everywhere in Modules/ — _zlib, _ssl, _bz2, socket, select, _io, etc.

Per-interpreter GIL (PEP 684)

In 3.12+, each PyInterpreterState has its own GIL by default. The big enabling change was making all mutable global state per-interpreter:

• Built-in module state (PyModuleDef.m_state populated by multi-phase init).
• Type objects with mutable internal state (e.g. global vars in C extensions are forbidden — Tools/c-analyzer/ enforces this).
• Small-int cache, intern dictionary, allocator pools.

Subinterpreters can run in parallel as long as they don't touch each other. Cross-interpreter communication is restricted to a small set of immutable / picklable types via Python/crossinterp.c. The user-facing API is in Modules/_interpretersmodule.c and Lib/test/test__interpreters.py.

Free threading (PEP 703)

--disable-gil removes the GIL entirely. Every refcount becomes atomic, and many internal data structures grow per-object locks:

• _PyMutex (Python/lock.c) — a tiny (1-byte) mutex backed by a parking lot. Used to protect mutable per-object state such as dict keys, list contents, and type slots.
• _PyOnceFlag — single-flight initialization, replaces lazy if (!initialized) patterns.
• Critical sections — Py_BEGIN_CRITICAL_SECTION(o) / Py_END_CRITICAL_SECTION() automatically lock o's _PyMutex. Nested critical sections handle deadlock by releasing locks in a deterministic order. Implementation in Python/critical_section.c.

Refcounting uses biased refcounts (Python/brc.c): each object has an "owner thread" whose Py_INCREF / Py_DECREF is non-atomic; cross-thread modifications go through a slower path with atomic CAS.

For memory reclamation, the nogil GC (Python/gc_free_threading.c) cooperates with QSBR (Python/qsbr.c, described in InternalDocs/qsbr.md). QSBR defers freeing until every thread has passed through a quiescent state, which avoids freeing memory another thread is still reading.

The threading module

Lib/threading.py is a thin wrapper over the C _thread module. It provides:

• Thread — start an OS thread that runs a callable.
• Lock, RLock — primitive mutex.
• Event, Condition, Semaphore, BoundedSemaphore.
• Barrier — synchronization barrier.
• local() — thread-local storage.

Lock is implemented in Modules/_threadmodule.c on top of _PyMutex. RLock is reentrant via a (thread_id, count) pair.

Async and threading

asyncio runs in a single thread by default; multi-thread coordination uses loop.run_in_executor(...) (a ThreadPoolExecutor from concurrent.futures) and asyncio.to_thread. The C-level integration with the eval loop is via loop._call_soon_threadsafe and the asyncio module's _asynciomodule.c implementation in Modules/_asynciomodule.c.

Pending calls and signals

_PyEval_AddPendingCall(...) schedules a function to run on the main thread between bytecodes. This is what signal.signal uses to deliver signals safely (signals can fire in any thread, but Python-level handlers always run on the main one).

The pending-call mechanism is also how the GIL drop request is implemented and how Py_AddPendingCall lets C code request a callback at a safe point.

Common pitfalls

• Holding refs without holding the GIL — illegal in classic and per-interp builds; legal in nogil but you still need a critical section to mutate the object.
• Releasing the GIL during a Py call — touching PyObject between Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS is undefined.
• Using static PyTypeObject in nogil / multi-interp — global state. Use heap types with PyType_FromModuleAndSpec.
• Calling Python code from a tp_dealloc — possible but dangerous; the object is being freed, so re-entrancy can cause use-after-free. The convention is "no Python calls from tp_dealloc".
• Shared mutable globals in C extensions — banned in PEP 684; Tools/c-analyzer/ flags them.

Entry points for modification

• The GIL's logic — Python/ceval_gil.c.
• New synchronization primitive — Python/lock.c + Python/parking_lot.c.
• Per-thread state field — Include/cpython/pystate.h.
• Per-interpreter state field — Include/internal/pycore_interp.h.
• New cross-interpreter type → Python/crossinterp.c and the lookup tables.
• Threading public API — Lib/threading.py and Modules/_threadmodule.c.