03 - Docker SDK

🎁 The docker CLI is for humans. What do you use when a program needs to manage containers?

devctl ssh can run commands on the fleet. Now you need devctl container deploy <image> <name> — a subcommand that replaces the bash deploy script your team has been maintaining.

Your team has a deploy script. It looks something like this:

IMAGE="myregistry/api:$VERSION"
docker pull $IMAGE
docker stop api-prod || true
docker rm api-prod || true
docker run -d --name api-prod -p 8080:8080 --env-file .env $IMAGE
sleep 3
docker inspect api-prod --format '{{.State.Status}}'

It works until it doesn't. The docker stop fails silently because the container doesn't exist yet. The sleep 3 is a guess. The inspect output is a string you're grepping. There's no retry, no structured error handling, no way to tell a "container not found" error from a network blip to the daemon. When something goes wrong at 2am, you get a non-zero exit code and a wall of text.

This is the problem the Docker SDK solves. It replaces fragile shell orchestration with typed Go code that talks directly to the Docker daemon's API.

Imports in code snippets are trimmed for brevity. See the full example linked at the end for complete, compilable source.

Why Not Shell Out to `docker`?

exec.Command("docker", "run", ...) gets you one thing: a subprocess. You get a combined stdout/stderr blob, a raw exit code, and nothing else.

The Docker SDK gives you:

Typed errors. client.ContainerStart returns an error you can inspect. errdefs.IsNotFound(err) tells you the container doesn't exist. You can't do that with an exit code.
Streaming. Log tailing, build output, stats — all streamed as structured data, not text you parse.
Cancellation. Every SDK call takes a context.Context. Pass a deadline and the operation cancels cleanly.
Portability. Your tool works against a local socket, a remote daemon over TCP, or a Docker-in-Docker container without changing a line.

Every docker CLI command is itself a call to this same API over a Unix socket. The SDK is how the CLI is built.

go get github.com/docker/docker@latest
go get github.com/docker/go-connections@latest

Connecting to the Daemon

The central type is client.Client. You create one and reuse it. It manages connection pooling and API version negotiation.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/docker/docker/client"
)

func main() {
    // Auto-detects DOCKER_HOST, falls back to unix:///var/run/docker.sock
    cli, err := client.NewClientWithOpts(
        client.FromEnv,
        client.WithAPIVersionNegotiation(),
    )
    if err != nil {
        log.Fatal(err)
    }
    defer cli.Close()

    info, err := cli.Info(context.Background())
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Docker %s — %d containers, %d images\n",
        info.ServerVersion, info.Containers, info.Images)
    // Output: Docker 24.0.7 — 12 containers, 45 images
}

WithAPIVersionNegotiation() is important. Docker daemons and clients can be on different versions. Without it, your tool can break if it's compiled against a newer API than the daemon it's talking to. With it, the client automatically downgrades.

For remote daemons over TCP with mutual TLS (common in CI environments):

cli, err := client.NewClientWithOpts(
    client.WithHost("tcp://10.0.1.50:2376"),
    client.WithTLSClientConfig("/certs/ca.pem", "/certs/cert.pem", "/certs/key.pem"),
    client.WithAPIVersionNegotiation(),
)

Pulling Images

Pulling feels like it should be simple, but there's a trap.

package main

import (
    "context"
    "fmt"
    "io"
    "log"

    "github.com/docker/docker/api/types/image"
    "github.com/docker/docker/client"
)

func pullImage(ctx context.Context, cli *client.Client, ref string) error {
    reader, err := cli.ImagePull(ctx, ref, image.PullOptions{})
    if err != nil {
        return fmt.Errorf("pull %s: %w", ref, err)
    }
    defer reader.Close()

    // You MUST consume this reader. If you don't, the pull stalls.
    _, err = io.Copy(io.Discard, reader)
    return err
}

func main() {
    cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
    if err != nil {
        log.Fatal(err)
    }
    defer cli.Close()

    if err := pullImage(context.Background(), cli, "nginx:latest"); err != nil {
        log.Fatal(err)
    }
    fmt.Println("pull complete")
}

Stopping and Removing

Before running a new container, you need to stop and remove any existing one with the same name. This is where typed errors pay off immediately.

import "github.com/docker/docker/errdefs"

func stopAndRemove(ctx context.Context, cli *client.Client, name string) error {
    timeout := 10
    if err := cli.ContainerStop(ctx, name, container.StopOptions{Timeout: &timeout}); err != nil {
        return err
    }
    return cli.ContainerRemove(ctx, name, container.RemoveOptions{
        Force:         true,
        RemoveVolumes: true,
    })
}

ContainerStop sends SIGTERM and waits up to timeout seconds for a graceful shutdown, then sends SIGKILL. Always set a timeout — the default is to wait forever.

The errdefs package is the typed error layer over the Docker daemon's REST responses. errdefs.IsNotFound(err) returns true when the container doesn't exist — which is exactly the error the bash script was hiding with || true. The difference: you know why it failed, not just that it did.

Running a Container

Now that stopAndRemove is defined, the deploy function can safely clean up before creating the new container:

import (
    "github.com/docker/docker/api/types/container"
    "github.com/docker/go-connections/nat"
)

func deployContainer(ctx context.Context, cli *client.Client, image, name string) (string, error) {
    // Stop and remove any existing container with this name.
    // Not an error if it didn't exist yet.
    if err := stopAndRemove(ctx, cli, name); err != nil {
        if !errdefs.IsNotFound(err) {
            return "", fmt.Errorf("cleanup: %w", err)
        }
    }

    resp, err := cli.ContainerCreate(ctx,
        &container.Config{
            Image: image,
            Env:   []string{"APP_ENV=production", "LOG_LEVEL=info"},
            ExposedPorts: nat.PortSet{
                "8080/tcp": struct{}{},
            },
        },
        &container.HostConfig{
            PortBindings: nat.PortMap{
                "8080/tcp": []nat.PortBinding{{HostIP: "0.0.0.0", HostPort: "8080"}},
            },
            RestartPolicy: container.RestartPolicy{Name: "unless-stopped"},
        },
        nil, nil, name,
    )
    if err != nil {
        return "", fmt.Errorf("create: %w", err)
    }

    if err := cli.ContainerStart(ctx, resp.ID, container.StartOptions{}); err != nil {
        return "", fmt.Errorf("start: %w", err)
    }

    return resp.ID, nil
}

No sleep. No string parsing. errdefs.IsNotFound replaces docker stop || true — but unlike || true, it only swallows the "not found" error, not every possible failure.

func main() {
    cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
    if err != nil {
        log.Fatal(err)
    }
    defer cli.Close()

    id, err := deployContainer(context.Background(), cli, "nginx:latest", "my-nginx")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("container started: %s\n", id[:12])
}

Waiting for Health, Not Sleeping

The bash script sleeps 3 seconds and hopes. The SDK lets you wait for a specific condition.

Poll ContainerInspect until the container reports healthy or a deadline is reached:

func waitUntilHealthy(ctx context.Context, cli *client.Client, containerID string) error {
    deadline := time.Now().Add(30 * time.Second)
    for time.Now().Before(deadline) {
        info, err := cli.ContainerInspect(ctx, containerID)
        if err != nil {
            return err
        }

        // State.Health is nil when the image has no HEALTHCHECK defined.
        // Fall back to checking State.Status instead.
        if info.State.Health == nil {
            if info.State.Running {
                return nil
            }
            return fmt.Errorf("container exited with status %d", info.State.ExitCode)
        }

        switch info.State.Health.Status {
        case "healthy":
            return nil
        case "unhealthy":
            return fmt.Errorf("container reported unhealthy")
        }
        // "starting" — keep waiting
        time.Sleep(500 * time.Millisecond)
    }
    return fmt.Errorf("container not healthy after 30s")
}

State.Health is nil when the image has no HEALTHCHECK in its Dockerfile — nginx, postgres, and most public images don't. Without the nil check, accessing .Status panics. When there's no healthcheck, State.Running is the right signal.

Streaming Logs

docker logs -f is a familiar command. The SDK equivalent streams in real time:

import "github.com/docker/docker/pkg/stdcopy"

func streamLogs(ctx context.Context, cli *client.Client, containerID string) error {
    opts := container.LogsOptions{
        ShowStdout: true,
        ShowStderr: true,
        Follow:     true,
        Timestamps: true,
    }

    reader, err := cli.ContainerLogs(ctx, containerID, opts)
    if err != nil {
        return err
    }
    defer reader.Close()

    _, err = stdcopy.StdCopy(os.Stdout, os.Stderr, reader)
    return err
}

The non-obvious part: Docker multiplexes stdout and stderr into a single stream with an 8-byte header on each frame (1 byte for stream type, 3 bytes padding, 4 bytes for payload size). If you use plain io.Copy instead of stdcopy.StdCopy, those binary headers appear in your output and corrupt everything downstream. This is one of the most common mistakes people make with this API.

stdcopy.StdCopy knows the frame format and demultiplexes correctly. Always use it when reading container logs.

Networks and Volumes

The sections above cover the single-container deploy script. For automation that spins up multi-container environments — integration tests, local dev stacks — you also need to manage networks and volumes programmatically:

import (
    "github.com/docker/docker/api/types/network"
    "github.com/docker/docker/api/types/volume"
    "github.com/docker/docker/api/types/mount"
)

func createIsolatedEnvironment(ctx context.Context, cli *client.Client, name string) (networkID, volumeName string, err error) {
    net, err := cli.NetworkCreate(ctx, name+"-net", network.CreateOptions{
        Driver: "bridge",
        Labels: map[string]string{"managed-by": "devctl"},
    })
    if err != nil {
        return "", "", fmt.Errorf("network: %w", err)
    }

    _, err = cli.VolumeCreate(ctx, volume.CreateOptions{
        Name:   name + "-data",
        Driver: "local",
        Labels: map[string]string{"managed-by": "devctl"},
    })
    if err != nil {
        return "", "", fmt.Errorf("volume: %w", err)
    }

    return net.ID, name + "-data", nil
}

Attach the network and volume when creating a container via HostConfig:

&container.HostConfig{
    NetworkMode: container.NetworkMode(networkID),
    Mounts: []mount.Mount{
        {
            Type:   mount.TypeVolume,
            Source: volumeName,
            Target: "/data",
        },
    },
}

Label everything with managed-by: devctl. When you clean up, list resources by that label and delete only what your tool created, without touching anything else on the host.

Here's createIsolatedEnvironment in use — spin up a named environment, start a container into it, then run a command inside:

func main() {
    cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
    if err != nil {
        log.Fatal(err)
    }
    defer cli.Close()

    ctx := context.Background()

    networkID, volumeName, err := createIsolatedEnvironment(ctx, cli, "mytest")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("network: %s  volume: %s\n", networkID[:12], volumeName)

    resp, err := cli.ContainerCreate(ctx,
        &container.Config{Image: "nginx:latest"},
        &container.HostConfig{
            NetworkMode: container.NetworkMode(networkID),
            Mounts: []mount.Mount{
                {Type: mount.TypeVolume, Source: volumeName, Target: "/data"},
            },
        },
        nil, nil, "mytest-nginx",
    )
    if err != nil {
        log.Fatal(err)
    }
    if err := cli.ContainerStart(ctx, resp.ID, container.StartOptions{}); err != nil {
        log.Fatal(err)
    }
    fmt.Printf("container %s running on isolated network\n", resp.ID[:12])
}

Putting It Together: `devctl container deploy`

Here's what the deploy script from the intro looks like when rewritten with the SDK and wired into devctl. The deploy function pulls the image, replaces the running container, and waits for a confirmed healthy state. The Cobra command wires it into the devctl container subcommand tree:

var containerCmd = &cobra.Command{
    Use:   "container",
    Short: "Manage containers via the Docker SDK",
}

var containerDeployCmd = &cobra.Command{
    Use:   "deploy <image> <name>",
    Short: "Pull an image and replace the named container",
    Args:  cobra.ExactArgs(2),
    RunE: func(cmd *cobra.Command, args []string) error {
        cli, err := client.NewClientWithOpts(
            client.FromEnv,
            client.WithAPIVersionNegotiation(),
        )
        if err != nil {
            return err
        }
        defer cli.Close()
        return deploy(context.Background(), cli, args[0], args[1])
    },
}

func init() {
    containerCmd.AddCommand(containerDeployCmd)
    rootCmd.AddCommand(containerCmd)
}

func deploy(ctx context.Context, cli *client.Client, image, name string) error {
    fmt.Printf("pulling %s...\n", image)
    if err := pullImage(ctx, cli, image); err != nil {
        return fmt.Errorf("pull: %w", err)
    }

    fmt.Printf("replacing container %s...\n", name)
    id, err := deployContainer(ctx, cli, image, name)
    if err != nil {
        return fmt.Errorf("deploy: %w", err)
    }

    fmt.Printf("waiting for healthy state...\n")
    if err := waitUntilHealthy(ctx, cli, id); err != nil {
        // Container started but isn't healthy. Stream the last 5s of logs before bailing.
        ctx2, cancel := context.WithTimeout(context.Background(), 5*time.Second)
        defer cancel()
        streamLogs(ctx2, cli, id)
        return fmt.Errorf("health check: %w", err)
    }

    fmt.Printf("deployed %s successfully\n", image)
    return nil
}

Every failure has a specific type, every operation has a deadline, and if the health check fails you get the last 5 seconds of logs automatically. That's the whole bash script, done properly.

Container Lifecycle Manager

A self-contained program that demonstrates the full Docker container lifecycle using the Go SDK: pulling an image, creating and starting a container, streaming logs for 5 seconds with proper multiplexed output, then gracefully stopping and removing the container.

Input: A running Docker daemon (connects via DOCKER_HOST or the default socket).

Output: Progress messages for each lifecycle stage, 5 seconds of container logs, and cleanup confirmation.

Full source: examples/container-lifecycle

Key Takeaways

The SDK replaces fragile shell orchestration with typed errors, streaming, and context-aware cancellation
Always use client.WithAPIVersionNegotiation() so your tool works across daemon versions
ImagePull returns a stream you must fully consume or the pull stalls
Docker multiplexes stdout and stderr in log streams. Use stdcopy.StdCopy, not io.Copy
errdefs.IsNotFound distinguishes "container doesn't exist" from other errors — the typed replacement for || true
Poll ContainerInspect with a deadline instead of sleeping arbitrary seconds
Always set a timeout on ContainerStop. The default is infinite

🎁 devctl container deploy can run images that already exist. What if devctl image build could compile your binary, build the OCI image, and push it to a registry — no Docker socket required?