pulumi/pulumi

Language hosts

Active contributors: Ian Wahbe, Thomas Gummerer, Julien, Fraser Ashyggton

Purpose

A language host is a Go process the engine spawns to run a user program in a specific language. It implements proto/pulumi/language.proto (the LanguageRuntime gRPC service) and brokers between the engine and the user's runtime. Each supported language has one host binary.

The hosts in this repo

Binary	Source	Notes
`pulumi-language-nodejs`	`sdk/nodejs/cmd/pulumi-language-nodejs/`	Boots Node.js, runs `index.js` from the project root
`pulumi-language-python`	`sdk/python/cmd/pulumi-language-python/`	Boots a Python interpreter (selected by `Pulumi.yaml`'s `runtime.options.toolchain` — pip/poetry/uv)
`pulumi-language-go`	`sdk/go/pulumi-language-go/`	Compiles and runs Go programs
`pulumi-language-pcl`	`sdk/pcl/cmd/pulumi-language-pcl/`	Runs the PCL intermediate language

pulumi-language-dotnet, pulumi-language-java, and pulumi-language-yaml live in their own repos.

What a host does

sequenceDiagram
    participant E as Engine
    participant H as Language host
    participant R as Language runtime (e.g. Node)
    participant P as User program

    E->>H: spawn process, --tracing, monitor address
    E->>H: GetRequiredPackages
    H-->>E: list of plugins / providers required
    E->>H: Run(monitor address, project, stack, config, ...)
    H->>R: spawn runtime
    R->>P: execute program
    P->>E: ResourceMonitor.RegisterResource(...) [via SDK]
    E-->>P: outputs
    P->>E: SignalAndWaitForShutdown
    R-->>H: exit
    H-->>E: Run returns

The host's responsibilities, broadly:

Bootstrap the language runtime with the right environment (PATH, NODE_OPTIONS, virtualenv, etc.).
Translate the engine's requests into idiomatic invocations (run-program, install-deps, list-required-plugins, generate-program, ...).
Stream stdout/stderr back to the engine as engine logs.
Surface errors with language-specific stack traces.

Modern hosts also implement codegen RPCs — GenerateProgram, GeneratePackage, Pack — which let the same host drive pulumi convert and pulumi package gen-sdk.

Key abstractions

Symbol	Lives in	Role
`LanguageRuntime` service	`proto/pulumi/language.proto`	Contract every host implements
`nodeLanguageHost`	`sdk/nodejs/cmd/pulumi-language-nodejs/main.go`	Node host implementation
`pythonLanguageHost`	`sdk/python/cmd/pulumi-language-python/main.go`	Python host implementation
`goLanguageHost`	`sdk/go/pulumi-language-go/main.go`	Go host implementation
`pclLanguageHost`	`sdk/pcl/cmd/pulumi-language-pcl/main.go`	PCL host implementation
`LanguagePlugin`	`pkg/codegen/sdks.md`, `pkg/codegen/programs.md`	Documentation of the codegen-side plugin contract

How the engine picks a host

The engine reads runtime from Pulumi.yaml:

runtime: nodejs
# or:
runtime:
  name: python
  options:
    toolchain: poetry

pkg/engine/plugins.go resolves runtime.name to the corresponding language host binary in the plugin cache. If not present, it installs it from the same release as the engine. The host process is spawned with environment variables containing the gRPC address of the engine's resource monitor.

SDK / host pairing

Each host expects to find a matching language SDK on disk:

Host	Expected SDK	Resolution
`pulumi-language-nodejs`	`@pulumi/pulumi` from npm or `node_modules`	Standard Node.js resolution from project dir
`pulumi-language-python`	`pulumi` from PyPI or virtualenv	Toolchain-driven (pip/poetry/uv)
`pulumi-language-go`	`github.com/pulumi/pulumi/sdk/v3/go/pulumi`	`go.mod`
`pulumi-language-pcl`	n/a — PCL has no user SDK	(built-in)

A version mismatch between host and SDK is the most common deployment-time failure. The host reports it with a "language host expects SDK ≥ X" error.

Entry points for modification

A new RPC method on LanguageRuntime — define it in proto/pulumi/language.proto, regenerate, implement in every host (Go, Node, Python, PCL here; .NET/Java/YAML in their repos), and call it from the engine.
Per-language project layout changes — touch the host (e.g. sdk/python/cmd/pulumi-language-python/main.go) for boot logic, and the SDK runtime entry point (sdk/python/lib/pulumi/runtime/) for in-program behavior.
Toolchain support (Python only at present) — sdk/python/toolchain/ defines the toolchain abstraction; add a new implementer there.

Tests

Per-host Go tests: *_test.go next to host source.
Conformance tests: scripts/run-conformance.sh cross-tests every language.
Integration tests: tests/integration/<feature>/<lang>/.