Open-Source Wikis

/

CPython

/

How to contribute

/

Tooling

python/cpython

Tooling

CPython has a long-running self-hosting bootstrap problem: most of the build is described in Python, but you can't run Python until you've built it. The tree resolves this by checking in every generated file and providing a layered set of regenerators that run after a working ./python exists.

Build system

File Role
configure.ac autoconf script (high-level checks for OS features, libs).
configure Generated from configure.ac. Don't edit by hand. Run autoreconf -ivf after editing the .ac file.
Makefile.pre.in The Makefile template. configure produces Makefile.
pyconfig.h.in Header template; configure produces pyconfig.h.
aclocal.m4 autoconf macros vendored from pkg.m4 and friends.
Modules/Setup The legacy "what to build statically" file. Mostly historical now.
Modules/Setup.bootstrap.in Modules linked into the bootstrap interpreter (_io, _freeze_module, …).
Modules/Setup.stdlib.in Modules built as .so after the interpreter is up.
PCbuild/ MSBuild project files for Windows.
Mac/BuildScript/ macOS .framework builder.

The bootstrap order is: (1) build a minimal interpreter linked statically against the modules in Setup.bootstrap.in; (2) that interpreter freezes a handful of bytecode modules (importlib, zipimport) into the binary via Programs/_freeze_module.c; (3) build the rest of the stdlib .sos.

Regenerators

Many tables and dispatchers are generated. The inputs and the corresponding make targets:

Generator Reads Writes Make target
Tools/peg_generator/ Grammar/python.gram, Grammar/Tokens Parser/parser.c regen-pegen
Tools/cases_generator/ Python/bytecodes.c Python/generated_cases.c.h, Python/executor_cases.c.h, Python/optimizer_cases.c.h, Python/opcode_targets.h, Lib/_opcode_metadata.py, several Include/internal/pycore_*.h regen-cases
Parser/asdl_c.py Parser/Python.asdl Python/Python-ast.c, Include/internal/pycore_ast.h regen-ast
Tools/clinic/ [clinic input] blocks in C files */clinic/*.c.h next to each source clinic
Tools/unicode/ UCD data files Modules/unicodedata_db.h, Modules/unicodename_db.h, Objects/unicodetype_db.h regen-unicodedata
Lib/keyword.py grammar itself regen-keyword
Tools/scripts/generate_global_objects.py pycore_* source Include/internal/pycore_global_strings.h, Include/internal/pycore_runtime_init_generated.h regen-global-objects
Tools/jit/ LLVM-compiled stencils jit_stencils.h regen-jit
Tools/scripts/generate_opcode_h.py Lib/opcode.py Include/opcode.h, Include/opcode_ids.h regen-opcode
Programs/_freeze_module Lib/...py Python/frozen_modules/*.h regen-frozen

The umbrella target is make regen-all, which runs every regenerator. CI explicitly checks that the regenerated outputs match what is checked in (see .github/workflows/lint.yml).

Argument Clinic

Tools/clinic/clinic.py reads [clinic input] blocks from C source and writes a parsing wrapper. Example block from a stdlib module:

/*[clinic input]
zlib.compress

    data: Py_buffer
        Binary data to be compressed.
    /
    level: int(c_default="Z_DEFAULT_COMPRESSION") = Z_DEFAULT_COMPRESSION
        Compression level, in 0-9 or -1.
    *
    wbits: int(c_default="MAX_WBITS") = MAX_WBITS

Returns a bytes object containing compressed data.
[clinic start generated code]*/

Clinic generates:

  • The PyMethodDef entry.
  • The argument-parsing function (zlib_compress).
  • A docstring PyDoc_VAR(zlib_compress__doc__).
  • The signature in inspect.signature form.

To regenerate after editing the input block:

make clinic                        # all files
./python Tools/clinic/clinic.py Modules/zlibmodule.c   # one file

The block above is valid only if you run with an in-tree Python (./python), since Clinic itself is Python code.

CI

Workflows under .github/workflows/:

Workflow What it builds / checks
build.yml The "main" matrix: Linux/macOS/Windows/WASI/Emscripten + several configurations.
jit.yml The JIT build (--enable-experimental-jit).
lint.yml ruff, mypy, regen-status, link-check.
mypy.yml Type-checks the in-tree Python tooling.
reusable-*.yml Shared per-OS jobs invoked by the others.
reusable-cifuzz.yml OSS-Fuzz integration entry point.
reusable-check-c-api-docs.yml Verifies every public C API is documented.
reusable-check-html-ids.yml Verifies stable Sphinx anchor IDs in docs.
regen-abidump.sh Compares ABI of the built libpython against the previous one.
require-pr-label.yml Refuses to merge without an awaiting merge label.

Linters

  • ruff — configured in .ruff.toml. Run ruff check Lib Tools locally.
  • mypy — config in Misc/mypy/. Targets internal Python tooling, not the stdlib (the stdlib is too large and dynamic for full mypy coverage).
  • pre-commit — full hook set in .pre-commit-config.yaml. Covers ruff, EOL/whitespace, dependabot config validation, blurb format.
  • patchcheck — see Tools/patchcheck/ and Development workflow.
  • Tools/check-c-api-docs/ — confirms every Include/cpython symbol shows up in the C-API reference.
  • Tools/c-analyzer/ — static-analysis tools used to enforce subinterpreter rules (no global mutable C state).

Code-formatting tools

CPython does not use a fully automated formatter on its C code — clang-format is invoked only on a few files (see .clang-format if present). Style is enforced by review and by PEP 7. For Python code, ruff format is the de facto formatter.

Tools you'll actually run

Beyond make, these are the most-used scripts:

  • ./python -m blurb add — create a NEWS entry. Documented under Tools/ and at https://blurb.readthedocs.io/.
  • ./python -m test — the test driver.
  • ./python Tools/clinic/clinic.py PATH — regenerate one Clinic block.
  • ./python Tools/cases_generator/generate_cases.py — regenerate dispatcher cases (the umbrella regen-cases target wraps this).
  • ./python Tools/peg_generator/pegen — regenerate the parser.
  • ./python Tools/build/check_warnings.py — find new warnings since the previous build.
  • ./python Tools/scripts/run_tests.py — the regrtest wrapper used in CI.

Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.

Tooling – CPython wiki | Factory