Resource graph and step model

Active contributors: Ian Wahbe, Thomas Gummerer, Fraser Ashyggton, Julien

Purpose

pkg/resource/deploy/ is where Pulumi's state machine for resources lives. Given a previous snapshot and a stream of "I want this resource" events from a user program, it computes the minimal sequence of provider operations to reconcile the two, then executes them in dependency order. This page documents the data model and the step state machine.

The data model

URN — sdk/go/common/resource/urn/

A URN uniquely identifies a resource within a stack:

urn:pulumi:<stack>::<project>::<parent-type>$<type>::<name>

• <stack> is the stack name (e.g. dev).
• <project> is the project name from Pulumi.yaml.
• <parent-type>$<type> is the type chain (component parents prepended with $).
• <name> is the user-supplied resource name.

URNs are stable across runs as long as the user doesn't rename a resource. Renames require explicit aliases (proto/pulumi/alias.proto).

State — sdk/go/common/resource/resource_state.go

State is the per-resource record persisted in the snapshot:

Field	Meaning
URN	Identity
Custom	True for provider-backed resources
Delete	Pending-delete marker (replace pattern)
ID	Provider-side identifier (e.g. AWS instance id)
Type	Type token
Inputs	What the user asked for
Outputs	What the provider returned
Parent	Parent URN
Protect	Protection flag
External	Read-only resource (Read step)
Dependencies	Other URNs this resource depends on
InitErrors	Errors from a partial Create
Provider	URN of the provider that manages this resource
PropertyDependencies	Per-property dependency map
PendingReplacement	Replace step in progress
AdditionalSecretOutputs	Outputs to mark secret post-hoc
Aliases	Old URNs this resource has occupied
CustomTimeouts	Per-op timeout overrides
ImportID	Set if the resource was imported
RetainOnDelete	Skip delete in destroy
DeletedWith	Co-deletion link
Created / Modified	Timestamps
SourcePosition	Where in user code this was registered

A snapshot is []State plus metadata (manifest, secrets manager, etc.).

Snapshot — sdk/go/common/resource/snapshot.go

type Snapshot struct {
    Manifest          Manifest
    SecretsManager    secrets.Manager
    Resources         []*State
    PendingOperations []Operation
    Metadata          SnapshotMetadata
}

Snapshots are append-only during a run — every mutation goes through state_builder.go to produce a new state, never edit-in-place.

The step state machine — pkg/resource/deploy/step.go

step.go is ~88 KB because every step type has its own bookkeeping. The variants:

Step	Trigger	Provider call
SameStep	New inputs match old, no diff	none
CreateStep	URN didn't exist before	Create
UpdateStep	Diff says in-place update is fine	Update
DeleteStep	URN gone from desired state	Delete
ReplaceStep	Diff says replace required (Create-then-Delete or Delete-then-Create)	Create + Delete
RefreshStep	pulumi refresh	Read
ReadStep	Resource.Read(...) (read-only resource)	Read
ImportStep	pulumi import or Resource(.., {import: ...})	Read then Create (no-op)
RemovePendingReplaceStep	Cleanup from a partially-replaced Replace	none

Steps emit events: OnResourceStepPre, OnResourceStepPost, OnResourceOutputs. Failures throw structured errors (*ResourceFailedError) that the engine surfaces to the user.

Step generation

step_generator.go (~137 KB) is the single-threaded heart of the engine.

graph TD
    Event[SourceEvent from language host] --> SG[stepGenerator]
    SG -->|provider Check| Inputs[Validated inputs]
    SG -->|provider Diff| Diff[Diff result]
    Diff -->|NoDiff| Same[SameStep]
    Diff -->|Update| UpdateS[UpdateStep]
    Diff -->|Replace| Replace[Create-replacement<br/>+ Delete-old]
    Diff -->|DeleteBeforeReplace| DBR[Delete-old<br/>+ Create-replacement]
    SG -->|missing in desired| Del[DeleteStep]
    SG -->|unchanged| ImportCheck[Existing import?]

Notable subtleties:

• Delete-before-replace vs Create-before-replace is per-resource (the provider may force one). Default is Create-before-replace.
• Targeted updates (pulumi up --target) prune the dependency graph; non-targeted resources get SameSteps.
• Continue-on-error mode collects step failures instead of aborting.
• Aliases let the engine recognize a renamed resource and avoid Replace.

Step execution

step_executor.go (~28 KB) runs steps in dependency order. Up to N steps execute in parallel; the bound is set on the deployment options. Cross-resource dependencies (parent, explicit dependsOn, or property dependency) serialize execution.

Goroutine panics are caught by goroutine_panic_recovery.go and surfaced as engine errors.

Sources — source.go, source_eval.go, source_null.go

A source feeds events to the step generator:

• EvalSource (source_eval.go, ~123 KB — the largest file in the repo) drives a real language host. It owns the ResourceMonitor gRPC server, translates registrations into SourceEvents, and handles invokes/calls. Most engine complexity around concurrency lives here.
• NullSource (source_null.go) emits no events. Used for pulumi destroy (no desired state, only old state).

Default providers

When the user constructs a resource without explicitly supplying a provider, the engine creates a default provider on demand. pkg/resource/deploy/providers/ synthesizes these. They look like ordinary pulumi:providers:<pkg> resources in the snapshot.

Plans — plan.go

Plan is a frozen sequence of steps. pulumi preview --save-plan=plan.json produces one; pulumi up --plan=plan.json executes only it. Useful for gated promotion: review the plan, get approval, run with the plan locked in.

Resource hooks — resource_hooks.go

Hooks fire before/after Create, Update, Delete (and an error hook for retryable failures). User programs register them via RegisterResourceHook (proto/pulumi/resource.proto). They run in the language host but are dispatched by the engine.

Targets — target.go

pulumi up --target <urn> and --target-dependents are implemented here. Targets prune the desired set; non-targeted resources keep their previous state via SameStep.

Built-in invokes — builtins.go

A handful of "invokes" don't need a provider at all (e.g. pulumi:pulumi:getStack, pulumi:pulumi:readStackReference). builtins.go short-circuits them in the engine.

Entry points for modification

• A new step type — define it in step.go (interface + implementation), teach step_generator.go when to emit it, and step_executor.go how to run it. Lifecycle test coverage required.
• A new diff outcome — extend the diff result types and update step_generator.go.
• A new resource option — add to resource_options.go, plumb through RegisterResource in proto/pulumi/resource.proto, regenerate, and propagate to user-program SDKs.
• A snapshot integrity rule — add to analyze_snapshot.go. Will run on every snapshot save via validating_persister.go.

See also

• Deployment engine — drives this layer.
• Backends — what stores the snapshots.
• Primitives: resources and URNs — the data model in more depth.
• The internal write-up at docs/architecture/deployment-execution/.