pulumi/pulumi

Codegen

Active contributors: Fraser Waters, Justin Van Patten, Ian Wahbe, Tom Harding

Purpose

pkg/codegen/ is two things in one package:

SDKgen — given a provider's schema (a JSON document describing resources, functions, and types), emit a typed library for a target language. Powers pulumi package gen-sdk and the build-time generation in every provider repo.
Programgen — given a Pulumi program written in PCL (the intermediate language), emit an idiomatic program in a target language. Powers pulumi convert.

The two share infrastructure: type system, name munging, schema validation, and the per-language emitters.

Directory layout

pkg/codegen/
├── docs.go                    # Doc generator (Sphinx fragments)
├── docs_integration_test.go
├── utilities.go               # Common helpers
├── utilities_types.go         # Shared type utilities
├── go/                        # Go SDKgen + Programgen
├── nodejs/                    # Node SDKgen + Programgen
├── python/                    # Python SDKgen + Programgen
├── pcl/                       # PCL parser/checker (used by Programgen)
├── hcl2/                      # Lower-level HCL2 parsing (PCL is built on HCL2)
├── schema/                    # Schema bind / validate / link
│   └── pulumi.json            # The metaschema
├── convert/                   # Driver for `pulumi convert`
├── cgstrings/                 # Naming helpers (camelCase / snake_case / PascalCase)
├── gen_program_test/          # Programgen test fixtures
└── testing/                   # Shared codegen test infrastructure

Schemas

A schema describes a provider:

name: aws
version: 6.0.0
resources:
  aws:s3/bucket:Bucket:
    properties:
      bucket: { type: string }
      arn: { type: string }
    requiredOutputs: [bucket, arn]
functions:
  aws:s3/getBucketObject:getBucketObject:
    inputs: { ... }
    outputs: { ... }
types:
  aws:s3/BucketLifecycleRule:BucketLifecycleRule:
    properties: { ... }

The metaschema — pkg/codegen/schema/pulumi.json — is what the schema package validates against. It is itself a valid JSON Schema document and lints with biome (make lint_pulumi_json).

pkg/codegen/schema/ handles binding (resolving cross-package references, normalizing types) and validation. Once bound, every emitter gets the same in-memory model.

SDKgen pipeline

graph LR
    Schema[provider schema.json] --> Bind[schema.Bind]
    Bind --> Model[bound *Package]
    Model --> Go[codegen/go.GeneratePackage]
    Model --> Node[codegen/nodejs.GeneratePackage]
    Model --> Python[codegen/python.GeneratePackage]
    Model --> Java[external: pulumi-java]
    Model --> DotNet[external: pulumi-dotnet]
    Go --> GoSDK[generated *.go]
    Node --> NodeSDK[generated *.ts]
    Python --> PythonSDK[generated *.py]

Each emitter is independent. Adding a new language means adding a new directory under pkg/codegen/<lang>/ (or, in practice, adding a new repo with a corresponding language host that implements GeneratePackage over gRPC).

Emitters read configuration from the schema's language block:

language:
  go:
    importBasePath: github.com/pulumi/pulumi-aws/sdk/v6/go
  nodejs:
    packageName: '@pulumi/aws'
  python:
    packageName: pulumi_aws

Programgen pipeline (`pulumi convert`)

graph LR
    Source[input program<br/>e.g. *.tf] -->|converter plugin| PCL
    PCL[PCL program] -->|hcl2.Parse + Bind| Bound[bound *Program]
    Bound --> Go[codegen/go.GenerateProgram]
    Bound --> Node[codegen/nodejs.GenerateProgram]
    Bound --> Python[codegen/python.GenerateProgram]

pulumi convert takes:

A converter plugin (e.g. pulumi-converter-terraform) that turns the source format into PCL. Lives outside this repo.
PCL → in-memory bound program (hcl2.BindProgram, pcl/).
Per-language emitter that produces idiomatic source code.

The driver is pkg/codegen/convert/. Converter plugins implement proto/pulumi/converter.proto.

PCL — `pkg/codegen/pcl/`, `pkg/codegen/hcl2/`, `sdk/pcl/`

PCL is HCL with a Pulumi-specific type system layered on top. It supports:

Resources: resource "name" "type" { ... }
Variables: <expr> (referenced by name)
Outputs: output "name" { value = ... }
Configuration: config "name" "type" { default = ... }
Components: component "name" "path/to/component" { ... }
Function calls: fn(args)

PCL's role is to be a minimal IR — small enough that every language can emit it cleanly, expressive enough that conversion from Terraform retains intent.

pkg/codegen/hcl2/ is the lower-level HCL2 wrapper; pkg/codegen/pcl/ adds Pulumi-specific types (resources, outputs, the Output<T> flow). sdk/pcl/ ships PCL as a runnable language with its own host.

Test infrastructure

Codegen tests are golden-file tests:

pkg/codegen/testdata/
  schemas/<schema-name>/             # Input schemas
    <schema>.json
  output/
    <emitter>/<schema>/              # Expected output
      <files>

PULUMI_ACCEPT=1 regenerates the goldens — diff and review before committing.

pkg/codegen/gen_program_test/ is the equivalent for programgen.

Documentation generator

pkg/codegen/docs.go produces Sphinx-flavored documentation fragments from schemas. Used by pulumi.com/registry/ and the Sphinx docs build.

Entry points for modification

Fixing a bug in Node/Python/Go SDKgen — pkg/codegen/<lang>/gen.go (and friends). Tests in pkg/codegen/<lang>/gen_test.go + golden files.
Adding a new schema feature — extend pkg/codegen/schema/, update pulumi.json metaschema, propagate to every emitter, regenerate goldens.
PCL changes — pkg/codegen/pcl/ for binding, pkg/codegen/<lang>/gen_program.go for emit.
A new converter (e.g. another IaC format) — implement proto/pulumi/converter.proto in a new repo. The driver in pkg/codegen/convert/ plugs it in automatically.