Hedronite · Dev Lesson · Polyglot-Dev / Go · Sat 2026-09-05

AWS SDK for Go v2 backend inventory — ListObjectsV2 on the state bucket

LoadDefaultConfig. List .tfstate keys. Paginate. Do not parse state.

Lesson Class: Dev (Go / DevOps-Go, non-terratest)
API: config.LoadDefaultConfig + s3.ListObjectsV2
Grounding: Brikman Shared Storage · Lab 12/04 bucket keys · SDK tome gap logged
Chain
Same credentials path as the AWS CLI.
List
Paginate. Filter .tfstate. Print keys.
Refuse
Not terratest. Not state parsing.
The backend bucket holds state objects. List them with the default config chain.

<!-- hal:authoritative:yaml -->

The backend bucket holds state objects. An ops tool should list them with the same credential chain the AWS CLI already uses. That tool is Go. It is not terratest.

§I — Frame

Recent Go lessons on this track lived inside terratest: copy folders, poll HTTP, call cloud SDKs to assert Terraform outputs. Terratest remains the TF-day Dev triad when the counter says so. Today the counter says Python for Dev. The quatro Go lane is separate: Go for ops tooling.

Ops today names S3 keys network/prod/terraform.tfstate and app/prod/terraform.tfstate. Someone still has to answer "what state objects exist in the bucket right now?" without opening the Terraform CLI. That someone is a small Go program using AWS SDK for Go v2.

Brikman's Shared Storage for State Files section builds the bucket, enables versioning, blocks public access, and points Terraform at object keys. Lab 12 and Lab 04 treat that bucket as the shared store. Today's Go program lists the store.

There is no Go AWS SDK tome on the shelf this cycle. Lattice returned empty for the SDK phrase. Cite Brikman and Bootcamp for the objects under management. Treat SDK API shapes as primary tooling knowledge and log the tome gap.

§II — Language Idiom: config.LoadDefaultConfig and the credentials chain

AWS SDK for Go v2 centralizes configuration in config.LoadDefaultConfig. One call loads region and credentials from the same chain operators already know from the AWS CLI: environment variables, shared config and credentials files, IAM role for task/instance/irsa, and related providers.

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
	ctx := context.Background()
	bucket := os.Getenv("TF_STATE_BUCKET")
	prefix := os.Getenv("TF_STATE_PREFIX") // e.g. "network/" or ""
	if bucket == "" {
		log.Fatal("TF_STATE_BUCKET is required")
	}

	cfg, err := config.LoadDefaultConfig(ctx)
	if err != nil {
		log.Fatalf("load aws config: %v", err)
	}

	client := s3.NewFromConfig(cfg)
	var token *string
	for {
		out, err := client.ListObjectsV2(ctx, &s3.ListObjectsV2Input{
			Bucket:            aws.String(bucket),
			Prefix:            aws.String(prefix),
			ContinuationToken: token,
		})
		if err != nil {
			log.Fatalf("list objects: %v", err)
		}
		for _, obj := range out.Contents {
			fmt.Printf("%s\t%d\t%s\n", aws.ToString(obj.Key), aws.ToInt64(obj.Size), obj.LastModified.UTC().Format("2006-01-02T15:04Z"))
		}
		if aws.ToBool(out.IsTruncated) {
			token = out.NextContinuationToken
			continue
		}
		break
	}
}

Four Go facts matter for ops use.

Fact one. LoadDefaultConfig fails closed when nothing resolves. Do not invent a static access key in source control to "make the demo work." Fix the environment. The chain is the product.

Fact two. Region must resolve. If AWS_REGION / AWS_DEFAULT_REGION / shared config lack a region, LoadDefaultConfig errors or you must pass config.WithRegion. Backend buckets live in one region. Match it.

Fact three. ListObjectsV2 is paginated. Ignoring IsTruncated silently drops keys after the first page. Backend buckets with years of state objects will truncate. Loop with ContinuationToken.

Fact four. Context carries deadlines. For cron inventory, wrap context.WithTimeout. A hung list should die, not occupy a runner forever.

§III — Code Worked Example: inventory against today's keys

Operators want a filtered view:

func listTFState(ctx context.Context, client *s3.Client, bucket, prefix string) ([]string, error) {
	var keys []string
	var token *string
	for {
		out, err := client.ListObjectsV2(ctx, &s3.ListObjectsV2Input{
			Bucket:            aws.String(bucket),
			Prefix:            aws.String(prefix),
			ContinuationToken: token,
		})
		if err != nil {
			return nil, err
		}
		for _, obj := range out.Contents {
			key := aws.ToString(obj.Key)
			if len(key) >= 11 && key[len(key)-11:] == ".tfstate" {
				keys = append(keys, key)
			}
		}
		if !aws.ToBool(out.IsTruncated) {
			return keys, nil
		}
		token = out.NextContinuationToken
	}
}

Run it with TF_STATE_BUCKET=acme-tf-state and prefixes network/ then app/. Compare the printed keys to the remote_state key arguments in Ops. A missing network/prod/terraform.tfstate means the producer never wrote, the prefix is wrong, or IAM cannot see the object.

Do not download and parse every state file in this tool. Parsing state is a different program with a different blast radius. Listing keys is enough for today's inventory.

Versioned buckets (Brikman enables versioning) mean delete markers and old versions exist. ListObjectsV2 shows current object versions' keys. Use ListObjectVersions only when you are debugging a restore. Default inventory stays on current keys.

§III.B — Credentials chain details operators hit

Order operators should reason about:

  1. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_PROFILE, region vars).
  2. Shared files (~/.aws/config, ~/.aws/credentials) selected by profile.
  3. Container/task/instance role credentials when running on AWS compute.
  4. SSO / identity center flows when configured in the shared config.

LoadDefaultConfig walks this for you. Explicit static credentials in code skip the chain and create a secret-sprawl incident. Prefer profiles locally and roles in CI.

When CI assumes a role with sts:AssumeRole before the job, the environment variables the assume-role action exports become the chain inputs. Your Go program should not re-implement assume-role unless it is an identity broker. One chain. One place that assumes.

IAM for the inventory role needs s3:ListBucket on the bucket ARN with prefix conditions if you scope it, and does not need s3:GetObject if you only list. Least privilege: listing keys is not reading state contents. If you later add a get for metadata-only heuristics, tighten the review.

§III.C — Failure modes

Empty list with a 200. Wrong prefix. Wrong bucket. Wrong account. Print bucket, prefix, and caller identity (sts.GetCallerIdentity) in debug mode.

AccessDenied. Role lacks ListBucket. Do not broaden to s3:* on *. Fix the resource policy and role policy.

Wrong region. PermanentRedirect or empty weirdness. Align region with the bucket.

Terratest confusion. This program is not a test. It has no t *testing.T. It does not call terraform.InitAndApply. Keep the binary in cmd/tfstate-ls. Keep terratest under test/.

§II.B — Why Go for this inventory

Python could list the bucket with boto3. Today's Dev slot already spends Python on plan JSON. The quatro Go lane exists to keep Go ops muscle warm without collapsing into terratest. S3 list plus default config is a clean first ops binary: small API surface, obvious IAM, obvious pagination, obvious exit codes.

Compile to a static binary. Ship in the same container image as your Terraform runners or as a separate tiny image. Either way, operators get one command: tfstate-ls.

§III.E — Comparing CLI and SDK

aws s3api list-objects-v2 --bucket ... --prefix ... already answers the question. Why write Go?

If the team is shell-only, keep the AWS CLI. Do not invent a Go binary for fashion. This lesson assumes you want the embeddable form.

Define a small interface for testability:

type ObjectLister interface {
	ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error)
}

Pass *s3.Client in production. Pass a fake in tests that returns two pages then stops. Assert your loop requests the continuation token. That test is the whole point of writing Go instead of a one-liner shell alias.

§III.F — Security notes next to Brikman

Brikman blocks public access on the state bucket. Your lister must not weaken that. No public ACL changes. No bucket policy edits. Read-only list.

State objects can contain secrets. Listing keys leaks environment topology (which stacks exist) but not secret values. Still treat the key list as internal. Do not publish it to a world-readable dashboard.

CloudTrail will record ListBucket. That is desirable. Inventory jobs should use a named role so the trail is attributable.

§IV.B — Wiring into the day's pipeline

Suggested job order:

  1. Network apply (Ops producer)
  2. tfstate-ls with prefix network/ (Go)
  3. App plan (Ops consumer)
  4. plan_census.py (Dev)
  5. Approval
  6. App apply

If step 2 does not show the expected key, fail before plan. That is a stronger signal than a confusing remote_state refresh error later.

§IV — Connection to Today's Ops, Dev, and Cert

Ops declares the keys. Go lists them. If Ops says the consumer reads network/prod/terraform.tfstate and Go cannot see that key, fix produce/apply or IAM before you debug HCL.

Dev censuses plan actions. Go censuses bucket keys. One is desired-change inventory. One is stored-state inventory.

Cert workspaces create additional state objects under the same bucket. A workspace-heavy backend will show more keys. That visibility supports Brikman's critique: many named states behind one IAM door.

§V — Prior-Lesson Reach

08-30 / 08-21 terratest lessons taught cloud SDK calls inside tests. Today's SDK call is outside tests. Same vendor libraries, different job.

08-18 Ops taught the lock table next to the bucket. Listing S3 does not list DynamoDB lock items. If you need lock inventory, that is a DynamoDB scan/query tool. Do not pretend ListObjects shows locks.

§V.B — Extended example: JSON output mode

type item struct {
	Key          string `json:"key"`
	Size         int64  `json:"size"`
	LastModified string `json:"last_modified"`
}

func emitJSON(keys []item) error {
	enc := json.NewEncoder(os.Stdout)
	enc.SetIndent("", "  ")
	return enc.Encode(keys)
}

CI can pipe JSON to jq to assert any(.key == "network/prod/terraform.tfstate"). That assertion is an ops gate without parsing state contents.

Keep TSV as default for humans. JSON is opt-in via -json. Flag parsing with flag standard library is enough. Do not pull a heavy CLI framework for two flags.

§V.C — Region and endpoint overrides

Some stacks use S3-compatible endpoints. SDK v2 allows custom endpoints through config. Only enable that behind an explicit env var such as TF_STATE_S3_ENDPOINT. Default behavior should talk to real AWS S3. Document the override for LocalStack labs if you use them. Bootcamp labs today are plan-mode and may not need LocalStack; do not force it.

§V.D — Relationship to DynamoDB lock table

08-18 taught the lock table. A complete backend health tool would list S3 keys and summarize open locks. That is a second client (dynamodb.NewFromConfig) and a second IAM grant. Out of scope for today's binary. Mention it in the README as future work so nobody assumes ListObjects shows lock health.

§VI.E — Word to the quartet

Four programs now touch one platform story:

Each refuses to become the others. That refusal is the design.

§VI — Closing

LoadDefaultConfig. ListObjectsV2 with pagination. Filter .tfstate. Print keys. Stop. Leave state parsing and terratest for their own programs. The inventory binary earns its keep by staying small enough to read in one sitting.

§VI.B — Classroom drill

  1. Point TF_STATE_BUCKET at a non-prod backend bucket you own.
  2. Run the lister with prefix network/.
  3. Confirm keys match Ops remote_state config.
  4. Revoke ListBucket temporarily in a sandbox role and confirm AccessDenied handling.
  5. Restore the grant.
  6. Write three bullets in your notes: chain source used, prefixes checked, keys expected vs seen.

No Bootcamp Go lab exists for this. Bootcamp supplies the Terraform backend referent. That dual-corpus split is intentional and logged.

§VI.C — Module layout

cmd/tfstate-ls/main.go
internal/statebucket/list.go
go.mod // module github.com/example/tfstate-ls

Keep internal/statebucket free of log.Fatal. Return errors. main decides process exit. That boundary makes the lister callable from a larger ops binary later without rewriting.

Add a -json flag when you need machine output. Default stay TSV for humans. Do not default to colored tables in CI logs.

Pin SDK module versions in go.mod. Upgrade deliberately. AWS SDK v2 modules move quickly enough that floating latest in CI will surprise you on Monday.

§VI.D — What this lesson refuses

It refuses terratest wrappers. It refuses embedding access keys. It refuses parsing full state JSON on a list path. It refuses claiming a vault tome for the SDK that does not exist. It refuses to unload anything on launchd. It ships as a Go ops tool beside today's Terraform shirts.

When the SDK tome eventually lands on the shelf, revisit this lesson's tome_refs and add the chapter that covers config.LoadDefaultConfig. Until then, Bootcamp plus Brikman remain the honest cites for the managed object.

§III.G — Minimal go.mod and build

// go.mod
module github.com/example/tfstate-ls

go 1.22

require (
        github.com/aws/aws-sdk-go-v2 v1.30.0
        github.com/aws/aws-sdk-go-v2/config v1.27.0
        github.com/aws/aws-sdk-go-v2/service/s3 v1.58.0
)

Versions above are illustrative pins. Resolve real current minors at ship time with go get. The point is pinning, not chasing floating latest.

Build:

CGO_ENABLED=0 go build -o tfstate-ls ./cmd/tfstate-ls

Run on a runner with an instance role or exported profile. Confirm with a sandbox bucket before aiming at prod state.

§VI.F — Closing expansion

If LoadDefaultConfig succeeds and ListObjects returns empty for a prefix you know should exist, believe the API. Check the account, the bucket name spelling, and whether the producer apply actually ran. Go is not wrong because HCL exists. Both can be right about different accounts.

Ship the binary. Document the env vars. Link Ops for key naming. Link Dev for the plan census. Link Cert for workspace-prefixed keys. Then leave the code alone until a real pagination bug shows up in production logs.

One more operational habit: store the last successful inventory as a small artifact next to the plan census markdown. When a remote_state refresh fails tomorrow, compare yesterday's key list to today's. Missing keys are producer problems. Extra keys are workspace spikes or abandoned stacks. Both are actionable without opening Terraform state files.

Do not alert on key count alone without a baseline. A growing bucket can be healthy versioning. Alert on absence of expected prod keys. That alert belongs next to the apply pipeline, not in a generic empty-bucket check.

Expected key list can live as a YAML file next to the binary:

required_keys:
  - network/prod/terraform.tfstate
  - app/prod/terraform.tfstate

Load it, list the bucket, diff. Exit 1 only in the gate job that owns that contract. The plain lister still exits 0 after printing. Same split Dev taught for census versus policy: report first, gate second, different processes. Keep the YAML allowlist in the platform repo so app teams do not each invent required keys.

Related