gRPC protocol

Active contributors: Fraser Waters, Ian Wahbe, Justin Van Patten

Purpose

Every inter-process call in Pulumi goes over gRPC. The schema lives in proto/pulumi/*.proto and is the source of truth. This page enumerates the services, what they do, and how to evolve them.

Services at a glance

File	Service(s)	Direction	Used by
proto/pulumi/resource.proto	ResourceMonitor	language host → engine	RegisterResource, Read, Invoke, Call, hooks
proto/pulumi/resource.proto	ResourceStatus	engine → engine	Resource state during a step
proto/pulumi/engine.proto	Engine	language host → engine	Log, GetRootResource
proto/pulumi/provider.proto	ResourceProvider	engine → provider	Configure, Check, Diff, Create, Update, Delete, Read, Invoke, Construct, Call
proto/pulumi/language.proto	LanguageRuntime	engine → language host	GetRequiredPackages, Run, InstallDependencies, GenerateProgram, GeneratePackage, Pack
proto/pulumi/analyzer.proto	Analyzer	engine → policy pack	Analyze, AnalyzeStack, Remediate, GetAnalyzerInfo
proto/pulumi/converter.proto	Converter	engine → converter plugin	ConvertState, ConvertProgram
proto/pulumi/plugin.proto	(no service)	n/a	Common plugin metadata
proto/pulumi/callback.proto	Callbacks	engine → language host	Per-resource callbacks (transforms, hooks)

The big two are ResourceMonitor and ResourceProvider. They embody the entire deployment loop.

ResourceMonitor (resource.proto)

The language host's call surface. The user program — through its SDK — invokes these:

RPC	Effect
RegisterResource	"I want a resource of type T with name N and inputs ..."
RegisterResourceOutputs	Component resources call this to publish outputs
ReadResource	Read-only / external resource
Invoke	Provider data-source call (e.g. aws.getAmi(...))
Call	Provider method call on a resource (e.g. bucket.getPolicy())
SupportsFeature	Probe for engine capabilities
RegisterStackTransform	Stack-wide transform on every resource
RegisterStackInvokeTransform	Stack-wide transform on every invoke
RegisterResourceHook	Per-step lifecycle hook
RegisterErrorHook	Retryable-error hook
RegisterPackage	Package registration (multi-package programs)
SignalAndWaitForShutdown	"I'm done generating events; wait for steps to finish."
GetDeploymentInfo	Modern feature-probe consolidator

SignalAndWaitForShutdown is what allows hooks and dynamic providers to keep running after the user's main program returns. Until it's called, the engine assumes the program may register more resources.

ResourceProvider (provider.proto)

The contract every cloud provider implements. Largest proto file in the repo (~70 KB).

RPC	Effect
Handshake	Initial protocol negotiation
Parameterize	Parameterized providers (turn one provider into another via params)
GetSchema	Return the provider's resource/function schema
CheckConfig / Check	Validate provider config / resource inputs
DiffConfig / Diff	Compute difference between old and new
Configure	Apply provider config
Invoke	Call a data source
Call	Call a method on a resource
Create	Provision a new resource
Read	Refresh / import / read-only inspection
Update	Update an existing resource in-place
Delete	Tear down a resource
Construct	Build a remote component (multi-language component)
Cancel	Cancel in-flight ops on shutdown
GetPluginInfo	Plugin version
Attach	Attach to a debugger

Construct and Call enable remote components: components written in any language, surfaced behind the provider interface. They're what makes "publish your component as a Pulumi package" possible.

LanguageRuntime (language.proto)

What language hosts implement. Engine-side calls during an op:

RPC	Effect
GetRequiredPackages	Inspect program, list provider plugins required
Run	Execute the program against a ResourceMonitor address
InstallDependencies	pulumi install plumbing
GeneratePackage	SDKgen — emit a typed library
GenerateProgram	Programgen — emit user-language source
Pack	Package the language program for distribution
Cancel	Cancellation
RuntimeOptionsPrompts / About / GetProgramDependencies	Misc helpers

Analyzer (analyzer.proto)

The policy-pack interface. Analyze runs per-resource; AnalyzeStack runs after every resource has been processed; Remediate proposes auto-fixes.

Converter (converter.proto)

ConvertProgram takes input source (e.g. a Terraform module) and emits PCL. ConvertState translates state files. The driver in pkg/codegen/convert/ calls these.

Asset / Archive — proto/pulumi/asset.proto (in resource.proto)

Assets and archives are first-class wire types. Encoded with the magic constant c44067f5952c0a294b673a41bacd8c17 (the asset marker) and 0def7320c3a5731c473e5ecbe6d01bc7 (archive marker) so they can't be confused with ordinary string properties.

Generated code

Run make build_proto to regenerate. CI enforces freshness via make check_proto (Makefile):

make build_proto    # regenerate
make check_proto    # CI gate — fails if generated code differs from committed

Generated paths committed to the repo:

• Go: sdk/proto/go/ (package pulumirpc)
• Node: sdk/nodejs/proto/
• Python: sdk/python/lib/pulumi/runtime/proto/

The pinned grpc-tools and protoc-gen-go versions are in .mise.toml and proto/grpc_version.txt.

Adding a new RPC

1. Edit the relevant proto/pulumi/*.proto.
2. make build_proto to regenerate.
3. Implement on every server (engine + every host + every provider, depending on the service).
4. Implement on every client.
5. Add a test (pkg/... for engine; per-language for SDKs).
6. Add a changelog entry.

Be aware that adding an RPC obligates every implementer, including out-of-tree providers. Default to optional features that older clients/servers can ignore.

Adding a feature flag

Use SupportsFeature for capability probes:

// in resource.proto
message SupportsFeatureRequest { string id = 1; }
message SupportsFeatureResponse { bool hasSupport = 1; }

Engine clients call SupportsFeature with a string id (e.g. "resourceHooks") and only use the new behavior if the engine reports support. Newer clients should prefer GetDeploymentInfo (which returns a list of supported features in one round-trip).

Wire-level conventions

• Resource property values use google.protobuf.Struct so they can carry arbitrary nested JSON-like data. Special markers (asset/archive/secret/computed/output/resource-reference) are detected by reserved keys with magic UUID-like values.
• URNs are strings on the wire.
• IDs are strings.
• Type tokens are strings.
• Versions are strings (semver).

Entry points for modification

• proto/pulumi/<file>.proto — edit the schema.
• proto/generate.sh — code generator config.
• Engine-side handlers — pkg/resource/deploy/source_eval.go (for ResourceMonitor), pkg/engine/eventsink.go (for Engine logging), pkg/engine/plugins.go (for plugin lifecycle).
• Per-language plumbing — every SDK has a runtime/ directory with the generated stubs and the SDK-side wrappers.

See also

• Apps: language hosts
• Apps: CLI
• Systems: plugins
• Systems: deployment engine