pulumi/pulumi

Automation API

Active contributors: Ian Wahbe, Justin Van Patten, Thomas Gummerer

Purpose

The Automation API lets you drive Pulumi from inside your own program, without invoking the pulumi CLI yourself. You write Go/Node/Python code that creates a stack, runs up, reads outputs, etc. Internally the API still spawns the CLI as a subprocess (or runs in-process for some operations) — but the API surface you write against is a typed library.

Use cases:

Building higher-level CLIs / platforms on top of Pulumi (custom dashboards, multi-stack orchestrators).
Test harnesses for Pulumi programs (the conformance tests use it).
CI/CD glue that needs typed outputs.
"Inline programs" — declaring infrastructure as a function rather than a separate index.js.

Where it lives

Language	Path
Go	`sdk/go/auto/`
Node	`sdk/nodejs/automation/`
Python	`sdk/python/lib/pulumi/automation/`
.NET	external repo
Java	external repo

The Go implementation is the reference. Each language's API surface is generated from tools/automation/specification.json (which is itself produced by make generate-cli-spec).

How an inline program works

sequenceDiagram
    participant Host as Host program (your code)
    participant Auto as Automation API
    participant CLI as pulumi CLI subprocess
    participant Eng as Engine
    participant Inline as User program (your function)

    Host->>Auto: NewStackInlineSource(... fn ...)
    Auto->>CLI: spawn pulumi up
    CLI->>Eng: Update(stack, source=inline)
    Eng->>Auto: spawn language host (special)
    Auto->>Inline: invoke fn(ctx)
    Inline->>Eng: RegisterResource(...) [via SDK as usual]
    Eng-->>Inline: outputs
    Inline-->>Auto: return
    Auto-->>CLI: done
    CLI-->>Auto: stdout (events)
    Auto-->>Host: typed Result with outputs

The "inline" trick is that the Automation API is the language host for the duration of the operation: it implements proto/pulumi/language.proto itself instead of spawning pulumi-language-<lang>.

Local vs remote workspaces

sdk/go/auto/ exposes both:

LocalWorkspace — files on disk: a working dir, Pulumi.yaml, Pulumi.<stack>.yaml. Mirrors a normal CLI invocation but driven by your code.
RemoteWorkspace — Pulumi Deployments. The work runs on Pulumi's infrastructure; your host program kicks it off and polls.

The RemoteWorkspace and the optremote* packages (optremoteup, optremotedestroy, etc.) parametrize cloud-side execution.

Key abstractions (Go)

Symbol	File	Role
`Stack`	`sdk/go/auto/stack.go` (~62 KB)	The handle — Up/Preview/Refresh/Destroy/Outputs
`Workspace`	`sdk/go/auto/workspace.go`	Project-level operations (create stack, list stacks, ...)
`LocalWorkspace`	`sdk/go/auto/local_workspace.go` (~60 KB)	Disk-backed implementation
`RemoteWorkspace`	`sdk/go/auto/remote_workspace.go`	Pulumi Deployments
`optup`, `optdestroy`, `optpreview`, `optrefresh`, `optimport`, `opthistory`, ...	`sdk/go/auto/opt*/`	Per-operation option packs
`Cmd`	`sdk/go/auto/cmd.go`	CLI process abstraction

Equivalent surfaces exist in Node and Python with idiomatic naming.

CLI specification flow

The CLI knows how to dump its own command tree to JSON:

make generate-cli-spec      # produces tools/automation/specification.json

Boilerplate templates under sdk/{nodejs,python}/tools/automation/boilerplate/ are merged with the spec to produce automation/interface/ files. The result is type-safe wrappers for every CLI flag — and they always track the CLI surface because regenerating is a one-liner.

Inline programs in detail

Inline mode is enabled when you create a stack with a PulumiFn:

fn := func(ctx *pulumi.Context) error {
    bucket, err := s3.NewBucket(ctx, "my-bucket", &s3.BucketArgs{ ... })
    ...
}
stack, _ := auto.UpsertStackInlineSource(ctx, "dev", "my-project", fn)
res, _ := stack.Up(ctx)

UpsertStackInlineSource creates a LocalWorkspace whose program field is the PulumiFn. When the engine asks the workspace to run the program, the workspace dials back into the host and invokes the function in-process. The user-program SDK (the same pulumi.Context everyone uses) bridges to the resource monitor.

This is why the Automation API can produce typed Output values: it never leaves the host process for the user-program path.

Tests

Per-SDK unit tests are extensive. The Go file sdk/go/auto/example_test.go (~~42 KB) and local_workspace_test.go (~~124 KB) define dozens of end-to-end scenarios.
Conformance tests use the Go Automation API as the driver: scripts/run-conformance.sh.

Entry points for modification

A new operation flag — extend opt<op>/ package with the new option, propagate through Stack.<Op> and the CLI command construction.
A new operation (rare) — define it in the spec, generate, then implement in Stack/Workspace.
Cloud-deployments-only feature — RemoteWorkspace plus the optremote*/ packages.