07 - Infrastructure as Code

📋 Jump to Takeaways

🎁 What if you could describe your entire production environment in a Go file, and a single command would make reality match it exactly?

Imports in code snippets are trimmed for brevity. See the full example linked at the end for complete, compilable source.

Declarative vs Imperative Automation

Infrastructure automation falls into two paradigms. Imperative code describes how to reach a state, with step-by-step commands. Declarative code describes what the desired state is and lets the engine figure out the steps.

// Imperative: explicit steps
func provisionImperative(ctx context.Context, client *ec2.Client) error {
    vpc, err := client.CreateVpc(ctx, &ec2.CreateVpcInput{CidrBlock: aws.String("10.0.0.0/16")})
    if err != nil {
        return err
    }
    _, err = client.CreateSubnet(ctx, &ec2.CreateSubnetInput{
        VpcId:     vpc.Vpc.VpcId,
        CidrBlock: aws.String("10.0.1.0/24"),
    })
    return err
}

Compare that with the declarative approach, where you describe the end state and let the engine handle execution order:

// Declarative: describe desired state, engine handles execution
func infrastructure(ctx *pulumi.Context) error {
    vpc, err := ec2.NewVpc(ctx, "main", &ec2.VpcArgs{
        CidrBlock: pulumi.String("10.0.0.0/16"),
    })
    if err != nil {
        return err
    }
    _, err = ec2.NewSubnet(ctx, "web", &ec2.SubnetArgs{
        VpcId:     vpc.ID(),
        CidrBlock: pulumi.String("10.0.1.0/24"),
    })
    return err
}

Declarative approaches excel for infrastructure because they handle idempotency, dependency ordering, and partial failure automatically.

State Management & Drift Detection

Every IaC system maintains a state file, a snapshot of what it believes exists. Drift occurs when real infrastructure diverges from state (manual changes, external processes).

// Drift detection pattern: compare desired state to actual state
type Resource struct {
    ID       string
    Type     string
    Desired  map[string]string
    Actual   map[string]string
}

func detectDrift(resources []Resource) []Resource {
    var drifted []Resource
    for _, r := range resources {
        for key, desired := range r.Desired {
            if actual, ok := r.Actual[key]; ok && actual != desired {
                drifted = append(drifted, r)
                break
            }
        }
    }
    return drifted
}

State backends (S3, GCS, Pulumi Cloud) provide locking so concurrent operations don't corrupt state. Always use remote state in production. Local files cause collaboration nightmares.

Diff/Apply Pattern

The core IaC loop is: read desired state → read actual state → compute diff → apply changes.

import "reflect"

type Action string

const (
    Create Action = "create"
    Update Action = "update"
    Delete Action = "delete"
    NoOp   Action = "noop"
)

type Change struct {
    Resource string
    Action   Action
    Before   map[string]string
    After    map[string]string
}

func computeDiff(desired, actual map[string]map[string]string) []Change {
    var changes []Change
    for name, desiredProps := range desired {
        if actualProps, exists := actual[name]; !exists {
            changes = append(changes, Change{Resource: name, Action: Create, After: desiredProps})
        } else if !reflect.DeepEqual(desiredProps, actualProps) {
            changes = append(changes, Change{Resource: name, Action: Update, Before: actualProps, After: desiredProps})
        }
    }
    for name := range actual {
        if _, exists := desired[name]; !exists {
            changes = append(changes, Change{Resource: name, Action: Delete, Before: actual[name]})
        }
    }
    return changes
}

This pattern separates planning from execution. You can preview changes before applying them, exactly like terraform plan or pulumi preview.

Pulumi Automation API

Terraform uses HCL; Pulumi uses real programming languages including Go. Since this is a Go course, Pulumi's Automation API lets us define and deploy infrastructure in the same language as our tools. The patterns (state, diff, apply) are identical, only the interface differs.

The Pulumi Automation API lets you drive Pulumi programmatically from Go, no CLI subprocess needed. You create stacks, set config, and run operations as library calls.

import (
    "context"
    "fmt"

    "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/s3"
    "github.com/pulumi/pulumi/sdk/v3/go/auto"
    "github.com/pulumi/pulumi/sdk/v3/go/auto/optup"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

// deployFunc defines the infrastructure
func deployFunc(ctx *pulumi.Context) error {
    bucket, err := s3.NewBucket(ctx, "my-bucket", &s3.BucketArgs{
        Website: &s3.BucketWebsiteArgs{
            IndexDocument: pulumi.String("index.html"),
        },
    })
    if err != nil {
        return err
    }
    ctx.Export("bucketName", bucket.Bucket)
    return nil
}

func main() {
    ctx := context.Background()

    // Create or select a stack
    stackName := "dev"
    projectName := "infra-service"

    stack, err := auto.UpsertStackInlineSource(ctx, stackName, projectName, deployFunc)
    if err != nil {
        panic(err)
    }

    // Set AWS region config
    stack.SetConfig(ctx, "aws:region", auto.ConfigValue{Value: "us-west-2"})

    // Preview changes
    preview, err := stack.Preview(ctx)
    if err != nil {
        panic(err)
    }
    fmt.Printf("Preview: %d changes\n", len(preview.Steps))
    // Output: Preview: 3 changes

    // Apply changes with streaming output
    result, err := stack.Up(ctx, optup.ProgressStreams(os.Stdout))
    if err != nil {
        panic(err)
    }
    fmt.Printf("Bucket: %s\n", result.Outputs["bucketName"].Value)
    // Output: Bucket: my-bucket-a1b2c3d
}

Stack Configuration

// Set secrets (encrypted in state)
stack.SetConfig(ctx, "dbPassword", auto.ConfigValue{
    Value:  "supersecret",
    Secret: true,
})

// Set multiple config values
stack.SetAllConfig(ctx, auto.ConfigMap{
    "aws:region":   {Value: "us-west-2"},
    "app:replicas": {Value: "3"},
    "app:apiKey":   {Value: "key-123", Secret: true},
})

Resource Dependencies & Ordering

Pulumi infers dependencies from output-to-input references. When you pass one resource's output as another's input, the engine builds a DAG and parallelizes where possible.

func infrastructure(ctx *pulumi.Context) error {
    // VPC is created first (no dependencies)
    vpc, _ := ec2.NewVpc(ctx, "vpc", &ec2.VpcArgs{
        CidrBlock: pulumi.String("10.0.0.0/16"),
    })

    // Subnet depends on VPC (uses vpc.ID())
    subnet, _ := ec2.NewSubnet(ctx, "subnet", &ec2.SubnetArgs{
        VpcId:     vpc.ID(),
        CidrBlock: pulumi.String("10.0.1.0/24"),
    })

    // Instance depends on subnet
    _, _ = ec2.NewInstance(ctx, "web", &ec2.InstanceArgs{
        SubnetId:     subnet.ID(),
        InstanceType: pulumi.String("t3.micro"),
        Ami:          pulumi.String("ami-0c55b159cbfafe1f0"),
    }, pulumi.DependsOn([]pulumi.Resource{subnet}))

    return nil
}

Use pulumi.DependsOn for implicit ordering relationships the engine can't infer from data flow.

Error Recovery & Rollback

Pulumi tracks resource state during operations. If a deployment fails mid-way, state records which resources were created.

import "time"

func deployWithRecovery(ctx context.Context, stack auto.Stack) error {
    result, err := stack.Up(ctx, optup.ProgressStreams(os.Stdout))
    if err != nil {
        fmt.Printf("Deploy failed: %v\n", err)
        // Output: Deploy failed: updating urn:pulumi:dev::infra::aws:s3:Bucket::my-bucket: access denied

        // Option 1: Destroy partially-created resources
        _, destroyErr := stack.Destroy(ctx)
        if destroyErr != nil {
            return fmt.Errorf("deploy failed and cleanup failed: %w (original: %v)", destroyErr, err)
        }
        return fmt.Errorf("deploy failed, resources cleaned up: %w", err)
    }

    fmt.Printf("Deploy succeeded: %s\n", result.Summary.Message)
    // Output: Deploy succeeded: 3 resources created
    return nil
}

// Retry with exponential backoff for transient failures
func deployWithRetry(ctx context.Context, stack auto.Stack, maxRetries int) error {
    for attempt := 0; attempt <= maxRetries; attempt++ {
        _, err := stack.Up(ctx)
        if err == nil {
            return nil
        }

        if attempt == maxRetries {
            return fmt.Errorf("deploy failed after %d attempts: %w", maxRetries, err)
        }

        // Refresh state to reconcile with reality before retrying
        _, _ = stack.Refresh(ctx)

        backoff := time.Duration(1<<uint(attempt)) * time.Second
        time.Sleep(backoff)
    }
    return nil
}

Always call stack.Refresh(ctx) after failures. It syncs state with actual cloud resources before the next attempt.

Pulumi Stack Deployer

A self-contained Go program that uses the Pulumi Automation API to create a stack, set config, preview changes, and deploy infrastructure, all as library calls with no CLI subprocess.

Input: A stack name, project name, and AWS region. Output: Provisioned S3 bucket with website hosting and streaming deploy output.

Full source: examples/pulumi-stack-deployer

Key Takeaways

Declarative IaC describes desired state; the engine computes and applies the diff.
State files track what exists. Remote backends with locking prevent corruption.
The diff/apply pattern separates planning from execution for safe deployments.
Pulumi Automation API embeds full IaC lifecycle into Go programs without CLI dependencies.
Resource dependencies form a DAG. Pulumi infers most from data flow; use DependsOn for implicit edges.
Always refresh state after failures and implement retry logic for transient cloud errors.

🎁 What if your Go program could clone repos, create branches, commit changes, and push, all without shelling out to the git CLI?