Serialization and module loading

Active contributors: jrose-apple, hamishknight, slavapestov

Purpose

lib/Serialization/ writes and reads the binary .swiftmodule format. A .swiftmodule captures everything the compiler needs to use a Swift module from a client: declarations, types, generic signatures, conformances, and (optionally) the SIL bodies that allow @inlinable cross-module inlining. The textual .swiftinterface complements it for ABI-stable distribution.

Directory layout

lib/Serialization/                  (25 files)
├── Serialization.cpp                # write a Swift module to disk
├── Deserialization.cpp              # 9,661 lines — read decls/types/conformances
├── ModuleFile.cpp                   # the in-memory representation of a loaded .swiftmodule
├── ModuleFileSharedCore.cpp
├── SerializedModuleLoader.cpp       # the FileUnit subclass for loaded modules
├── DeclTypeRecordNodes.def          # the BCRecord-bytecodes used in .swiftmodule
├── SerializeSIL.cpp                 # SIL bodies inside .swiftmodule
├── DeserializeSIL.cpp
├── ScanningLoaders.cpp              # used by lib/DependencyScan
└── ...

include/swift/Serialization/
├── Validation.h
└── SerializedModuleLoader.h

Key abstractions

Type	File	Description
swift::ModuleFile	include/swift/Serialization/ModuleFile.h	A loaded .swiftmodule.
swift::SerializedModuleLoader	include/swift/Serialization/SerializedModuleLoader.h	Finds and loads .swiftmodule files from import search paths.
swift::serialization::Serializer	lib/Serialization/Serialization.cpp	Writes a .swiftmodule for a SourceFile.
swift::serialization::ModuleFileSharedCore	lib/Serialization/ModuleFileSharedCore.h	Backing data shared between many ModuleFile instances.

How it works

graph LR
    SourceFile -->|"emit-module"| Serializer
    Serializer --> SwiftModule[".swiftmodule"]
    Serializer --> SwiftDoc[".swiftdoc"]
    Serializer --> SwiftSourceInfo[".swiftsourceinfo"]
    SwiftModule -->|"-import"| Loader[SerializedModuleLoader]
    Loader --> ModuleFile
    ModuleFile -->|lazy lookup| Decls[Swift Decls in ASTContext]

Wire format

.swiftmodule is an LLVM Bitstream container with custom record types defined in DeclTypeRecordNodes.def. Each kind of AST node (struct, class, function, type alias, protocol, conformance, ...) has a record type. Serialization is lazy on read -- only the records actually accessed by the client are deserialized.

The associated files written together:

• .swiftmodule -- decls + types + (optional) SIL.
• .swiftdoc -- documentation comments.
• .swiftsourceinfo -- source location data for IDE jump-to-definition.
• .abi.json (when -abi-descriptor is on) -- ABI surface for diffing.

.swiftinterface

For ABI-stable libraries (and any library compiled with -enable-library-evolution), the compiler also emits a textual .swiftinterface. This is essentially Swift source containing only the public API. The advantage: it survives compiler upgrades because it can be re-typechecked by a newer compiler. See lib/Frontend/ModuleInterfaceLoader.cpp for the loader and lib/Frontend/PrintingDiagnosticConsumer.cpp for printing.

When a client imports such a module:

1. Look for a .swiftmodule next to the interface that was built by this compiler.
2. If absent, type-check the .swiftinterface into a fresh .swiftmodule (in the cache) and proceed.
3. The ModuleInterfaceLoader handles all this automatically.

Cross-module inlining

If a .swiftmodule was built with -emit-module-source-info and contains the SIL bodies of @inlinable / @_alwaysEmitIntoClient functions, the SIL deserializer (DeserializeSIL.cpp) can pull them in for cross-module inlining and specialization at the call site.

Dependency scanning

lib/DependencyScan/ builds a graph of module imports without fully type-checking. It uses ScanningLoaders.cpp to inspect .swiftmodule and .swiftinterface files and produce a JSON dependency graph for build systems (swift -scan-dependencies). Drives features like prebuilt module caches and explicit module builds.

Integration points

• Frontend (lib/Frontend/) wires the loaders into the ASTContext import-search machinery.
• Sema consumes deserialized decls as if they were locally type-checked.
• SILGen / IRGen sometimes deserialize SIL bodies for cross-module inlining.
• The build system treats .swiftmodule as an output artifact alongside .o.
• lib/APIDigester/ consumes serialized modules to compare ABI/API surfaces between toolchains (swift-api-digester).

Entry points for modification

• Adding a new Decl kind: register a new record in DeclTypeRecordNodes.def, write a serializer in Serialization.cpp, write a deserializer in Deserialization.cpp. Bump the format version constant in include/swift/Serialization/ModuleFormat.h.
• Investigating a deserialization crash: run with -Xfrontend -dump-deserialized-modules. Many bugs are due to stale .swiftmodules; clean the module cache first.
• Adding a .swiftinterface flag: consider whether new APIs need @_alwaysEmitIntoClient to ship on older runtimes.

Related pages

• AST -- the data this module reads/writes.
• Frontend and driver -- where module loading is configured.
• ClangImporter -- another ModuleLoader peer.