Architecture

Pulumi is a multi-process, gRPC-driven system. A single pulumi up invocation spawns the CLI, an engine, a language host, and one or more provider plugins, and they communicate exclusively through Protobuf-defined RPCs. This page traces those processes and explains where each one lives in the repo.

The four roles

graph TD
    User[Developer] -->|invokes| CLI[pulumi CLI<br/>pkg/cmd/pulumi]
    CLI -->|creates| Engine[Deployment engine<br/>pkg/engine, pkg/resource/deploy]
    Engine -->|spawns + RPCs| LangHost[Language host<br/>sdk/&lt;lang&gt;/cmd/pulumi-language-&lt;lang&gt;]
    Engine -->|spawns + RPCs| Provider[Provider plugins<br/>e.g. pulumi-resource-aws]
    LangHost -->|runs| UserProg[User program<br/>links against sdk/&lt;lang&gt;]
    UserProg -->|RegisterResource etc.<br/>via ResourceMonitor| Engine
    Engine -->|persists snapshot| Backend[Storage backend<br/>pkg/backend/diy or httpstate]

Every Pulumi operation conforms to this shape. The arrows are gRPC connections defined in proto/pulumi/*.proto.

Components in this repo

pulumi CLI — pkg/cmd/pulumi/

The CLI is the only binary a user runs directly. Its entry point is pkg/cmd/pulumi/main.go, which calls into pkg/cmd/pulumi/pulumi.go (the giant NewPulumiCmd() function — ~33 KB) to register every subcommand. Subcommand groups live in their own subpackages: pkg/cmd/pulumi/stack, pkg/cmd/pulumi/config, pkg/cmd/pulumi/policy, pkg/cmd/pulumi/state, pkg/cmd/pulumi/plugin, etc. The CLI uses Cobra for command parsing.

For a deep-dive see apps/cli.

Deployment engine — pkg/engine/ and pkg/resource/deploy/

The engine is what the CLI spins up to actually do anything. It plans, executes, and records the side-effects of every operation. Two complementary subsystems:

• pkg/engine/ — operation entry points (update.go, destroy.go, refresh.go, import.go), event emission (events.go), plugin and policy management, and the journaled-snapshot infrastructure (journal_snapshot.go).
• pkg/resource/deploy/ — the actual deployment: source iterators (source_eval.go is the largest file in the codebase at ~123 KB), the step generator (step_generator.go, ~137 KB), the step executor (step_executor.go), and the resource graph (snapshot.go).

The lifecycle test infrastructure under pkg/engine/lifecycletest/ is a fuzzing harness that exercises the engine through synthetic programs.

For details see systems/engine and primitives/resources-and-urns.

Language hosts — sdk/<lang>/cmd/pulumi-language-<lang>/

Each language Pulumi supports ships with a host — a Go process that wraps the language runtime in a gRPC interface defined in proto/pulumi/language.proto. The engine spawns a host, asks it to run the user's program, and the host streams ResourceMonitor calls back to the engine.

The hosts in this repo:

• sdk/nodejs/cmd/pulumi-language-nodejs/ — boots Node.js and runs index.js
• sdk/python/cmd/pulumi-language-python/ — boots a Python interpreter
• sdk/go/pulumi-language-go/ — compiles and runs Go programs
• sdk/pcl/cmd/pulumi-language-pcl/ — runs PCL (the intermediate language)

.NET, Java, and YAML hosts live in their own repositories.

Providers — external repositories

Provider plugins (pulumi-resource-aws, pulumi-resource-kubernetes, etc.) implement proto/pulumi/provider.proto. They live outside this repo. The engine talks to them over gRPC just like it talks to language hosts. Two notable exceptions are present here:

• Default providers — synthetic providers materialized by the engine when the user doesn't construct one explicitly (pkg/resource/deploy/providers/).
• Test providers — built-in fake providers used by lifecycle tests (tests/testprovider/).

Storage backends — pkg/backend/

Backends persist the state snapshot that the engine produces. Two implementations:

• pkg/backend/diy/ — do-it-yourself backend (filesystem, S3, GCS, Azure Blob via gocloud.dev/blob).
• pkg/backend/httpstate/ — Pulumi Cloud (the hosted service at app.pulumi.com).

Both implement the Backend interface from pkg/backend/backend.go. See systems/backend.

gRPC protocol — proto/pulumi/

Everything between processes is defined in Protobuf. The major service surfaces:

File	Purpose
proto/pulumi/resource.proto	ResourceMonitor — language host → engine (Register, Read, Invoke, Call)
proto/pulumi/engine.proto	Engine — language host → engine (logging, root URN, etc.)
proto/pulumi/provider.proto	ResourceProvider — engine → provider plugin (Check, Diff, Create, Update, Delete)
proto/pulumi/language.proto	LanguageRuntime — engine → language host (GetRequiredPackages, Run, Pack)
proto/pulumi/analyzer.proto	Analyzer — engine → policy pack
proto/pulumi/converter.proto	Converter — pulumi convert plumbing
proto/pulumi/plugin.proto	Generic plugin metadata
proto/pulumi/callback.proto	Callbacks (transforms, hooks)

Generated code is committed under sdk/proto/go/, sdk/nodejs/proto/, and sdk/python/lib/pulumi/runtime/proto/. Regenerate via make build_proto. CI enforces freshness via make check_proto (Makefile).

Code generation — pkg/codegen/

The codegen subsystem has two distinct jobs:

• SDK generation — produce a typed library for a given Pulumi schema (pkg/codegen/{go,nodejs,python}).
• Program generation — convert PCL programs into target-language programs, and (via converter plugins) convert from Terraform/CloudFormation/Kubernetes YAML into PCL. See pkg/codegen/hcl2/ for PCL-side handling and pkg/codegen/convert/ for the conversion pipeline.

See systems/codegen.

End-to-end: what happens during pulumi up

sequenceDiagram
    participant User
    participant CLI as pulumi CLI
    participant Engine as Deployment engine
    participant Host as Language host
    participant Prog as User program
    participant Prov as Provider plugin
    participant BE as Backend

    User->>CLI: pulumi up
    CLI->>BE: load latest snapshot
    BE-->>CLI: previous state
    CLI->>Engine: Update(target=stack, source=program)
    Engine->>Host: spawn pulumi-language-<lang>
    Engine->>Host: Run(program path, monitor address)
    Host->>Prog: execute
    Prog->>Engine: RegisterResource(type, name, inputs) [via ResourceMonitor]
    Engine->>Prov: spawn + Configure
    Engine->>Prov: Check(inputs), Diff(old,new), Create/Update/Delete
    Prov-->>Engine: physical id + outputs
    Engine-->>Prog: resource outputs
    Prog->>Engine: SignalAndWaitForShutdown
    Engine->>BE: persist new snapshot
    Engine-->>CLI: exit code + summary

The engine's job is to interleave the language-host events ("here is the desired state") with provider RPCs ("here is what changed in the cloud") and persist the result.

Where to start reading

If you have one hour: read docs/architecture/README.md, then pkg/cmd/pulumi/pulumi.go, then pkg/engine/update.go, then pkg/resource/deploy/deployment.go. After that the rest of the engine starts to make sense.

If you have one day, follow the lifecycle of a single resource: sdk/go/pulumi/resource.go (NewResource) → sdk/go/pulumi/context.go (registerResource) → engine pkg/resource/deploy/source_eval.go (monitor) → pkg/resource/deploy/step_generator.go → pkg/resource/deploy/step.go → provider RPC.

See systems/engine for that walkthrough in more depth.