Open-Source Wikis

/

CPython

/

Systems

/

Import system

python/cpython

Import system

import x looks like a single keyword, but it triggers a multi-step protocol whose pure-Python implementation lives in Lib/importlib/ and whose C entry points live in Python/import.c. The system is bootstrapped in C, then "self-hosted" by frozen importlib bytecode.

Layered architecture

graph TD
    USER[Python code: import foo] --> BUILTINS[builtins.__import__]
    BUILTINS --> IMPORTLIB[importlib._bootstrap]
    IMPORTLIB --> METAPATH[Meta path finders]
    METAPATH --> SPEC[Find ModuleSpec]
    SPEC --> LOADER[Loader.create_module / exec_module]
    LOADER -->|.py| SOURCE[SourceFileLoader → compile + exec]
    LOADER -->|.pyc| BYTECODE[SourcelessFileLoader]
    LOADER -->|.so / .pyd| EXT[ExtensionFileLoader]
    LOADER -->|builtin| BUILTIN[BuiltinImporter]
    LOADER -->|frozen| FROZEN[FrozenImporter]
    EXT --> IMPORTDL[Python/importdl.c]
    BUILTIN --> RUNTIME[builtin module table]
    FROZEN --> FROZEN_TABLE[Python/frozen_modules/]

Files

File / dir Role
Python/import.c The C entry points: _PyImport_BootstrapImp, PyImport_ImportModule, builtin and frozen module tables.
Python/importdl.c The OS-portable wrapper that loads a .so/.pyd extension module.
Python/dynload_*.c Per-platform dlopen/LoadLibrary wrappers (dynload_shlib.c, dynload_win.c, …).
Python/frozen.c Looks up a module name in the frozen-modules table.
Python/frozen_modules/ Generated .h files with bytecode for _bootstrap, _bootstrap_external, zipimport, etc.
Lib/importlib/ The Python implementation of import machinery.
Lib/importlib/_bootstrap.py The protocol: _find_and_load, _call_with_frames_removed, etc. Frozen at build time.
Lib/importlib/_bootstrap_external.py File-system-aware finders/loaders. Frozen at build time.
Lib/importlib/metadata/ The importlib.metadata API for installed-package discovery.
Lib/importlib/resources/ The resource loader API (open_binary, read_text, …).
Lib/importlib/util.py Conveniences (spec_from_file_location, find_spec).
Lib/zipimport.py Frozen ZIP-archive importer.
Programs/_freeze_module.c The "freezer" tool: turns a .py into a Python/frozen_modules/<name>.h.

Bootstrapping

CPython can't import importlib until it has an importer for importlib. The chicken-and-egg is broken by freezing:

  1. At build time, Programs/_freeze_module compiles Lib/importlib/_bootstrap.py and _bootstrap_external.py to bytecode and writes the bytes into Python/frozen_modules/<name>.h.
  2. At interpreter init (Python/pylifecycle.c), _PyImport_BootstrapImp looks up the frozen _bootstrap module, executes it, and installs its __import__ as builtins.__import__.
  3. From there on out, import runs the Python code in _bootstrap.py like a normal stdlib module.

Hence the in-tree Lib/importlib/_bootstrap.py is the source; the .h files under Python/frozen_modules/ are generated. Editing one without regenerating the other will produce silent skew. make regen-frozen regenerates them.

You can disable frozen modules at startup with python -X frozen_modules=off, useful when debugging the import bootstrap.

Finders and loaders

Once _bootstrap is loaded, Python's import goes through:

  1. sys.meta_path — list of meta-path finders. Defaults: BuiltinImporter, FrozenImporter, then the path-based one. Each finder's find_spec(fullname, path, target) returns a ModuleSpec or None.
  2. PathFinder — the path-based finder; consults sys.path_hooks and sys.path_importer_cache to map each sys.path entry to a finder.
  3. Loader — the chosen ModuleSpec carries a loader. create_module(spec) is optional; exec_module(module) is required and runs the module body in module.__dict__.

The pure-Python implementations of all this are in Lib/importlib/_bootstrap.py and Lib/importlib/_bootstrap_external.py.

Built-in and frozen modules

  • Built-in modules — modules linked statically into the interpreter (sys, _thread, _io, gc, _imp, …). Listed in _PyImport_Inittab (Python/import.c) and instantiated by BuiltinImporter.
  • Frozen modules — Python source compiled at build time into .h files and embedded in the interpreter. The set is configured by Tools/freeze/ and the table is emitted in Python/frozen.c.

Both are checked before sys.path is consulted, which is why import sys cannot be shadowed by a sys.py in the cwd.

Extension modules (.so / .pyd)

Loading a compiled extension goes through ExtensionFileLoader.exec_module_imp.create_dynamicPython/importdl.c → the platform dynload_*.c shim. The shim:

  1. dlopen / LoadLibrary the file.
  2. Look up PyInit_<modname> (or, in the modern multi-phase init protocol, the PyModuleDef it returns).
  3. Call the init slot; the module returns a PyObject*.

CPython now strongly prefers multi-phase initialization (PEP 489) — see the example in Modules/_testmultiphase.c. Multi-phase modules can be safely loaded into multiple subinterpreters and free-threaded builds.

sys.path and sys.path_hooks

sys.path is a list of directories or other strings. Each entry is mapped to a path entry finder through sys.path_hooks (with caching in sys.path_importer_cache). The default hooks are:

  • zipimport.zipimporter — for .zip paths.
  • FileFinder — for actual directories. Uses path-based path importers for .py, .pyc, and extensions.

You can install your own hook by appending to sys.path_hooks and clearing sys.path_importer_cache. The setup.py develop / editable install workflows used to do this; modern editable installs use a .pth file that imports a hook indirectly.

.pyc files

Bytecode caches live under __pycache__/<name>.<tag>.pyc. The tag includes the Python version and optimization level (e.g. cpython-315). The header has:

  • A magic number (changes when the bytecode format changes — see MAGIC_NUMBER in Lib/importlib/_bootstrap_external.py).
  • A timestamp or hash of the source (the "hash-based pyc" introduced by PEP 552).
  • A marshalled code object.

If the source .py is newer than the .pyc (or the hash mismatches), CPython recompiles. The marshalling format is implemented in Python/marshal.c.

Lazy modules

PEP 690-style lazy imports are not yet on by default in CPython, but Objects/lazyimportobject.c implements the building block (a LazyImport object whose attribute access triggers the import). Some stdlib modules use it internally to defer expensive imports; importlib.util.LazyLoader is the user-facing helper.

Subinterpreter notes

Each subinterpreter has its own sys.modules. Built-in and frozen modules can be shared (their m_state is cached per-interpreter via multi-phase init); .so extensions that don't support multi-phase init refuse to load into a subinterpreter.

The cross-interpreter import test surface lives in Modules/_testsinglephase.c and Modules/_testmultiphase.c.

Entry points for modification

  • New finder/loader → ship a _bootstrap_external.py patch (rare) or a third-party meta-path finder.
  • New frozen module → add to Tools/freeze/ and Python/frozen.c, run make regen-frozen.
  • New built-in module → add to _PyImport_Inittab in Python/import.c and add a build rule.
  • Bytecode magic bump → MAGIC_NUMBER in Lib/importlib/_bootstrap_external.py; requires a NEWS entry.
  • Platform dlopen quirks → Python/dynload_*.c for the relevant platform.

Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.

Import system – CPython wiki | Factory