Terraform partial backend config — S3 keys at init time

The backend is configured before the graph exists. Put environment identity in the init command, not in an expression.

The backend is configured before the graph exists. Put environment identity in the init command, not in an expression.

§I — Frame

Tuesday this arc put lifecycle preconditions on AWS resources. Saturday taught a consumer stack how to read network outputs through terraform_remote_state. August spent the DynamoDB LockID claim and the force-unlock hazard. Keep those spends.

Today the track is Terraform on AWS. The concrete referent is an S3 backend whose bucket, key, and region stay empty in committed HCL and arrive at terraform init -backend-config=backend.hcl. Lab 31 names that pattern. Brikman names the rule behind it: Limitations with Terraform's Backends. The backend block is not a resource. It cannot read var, local, or a data source. Init runs before the graph.

File layout still isolates environments: network/dev.tfstate versus network/prod.tfstate as separate keys. Partial config is how CI and laptops inject those keys without forking the root module.

§II — Foundations: three rules

Fact one. Backend configuration is init-time.

terraform {
  backend "s3" {
    bucket = ""
    key    = ""
    region = ""
  }
}

Lab 31 leaves those strings empty on purpose. The working fill lives beside the root:

bucket         = "acme-tfstate-prod"
key            = "network/prod.tfstate"
region         = "us-east-1"
dynamodb_table = "acme-tf-locks"
encrypt        = true

Run terraform init -backend-config=backend.hcl. Terraform merges the partial block with the file. The merged result is what .terraform/ remembers until you reconfigure.

Fact two. Expressions inside `backend` fail closed.

locals {
  network_state_key = format("network/%s.tfstate", var.environment)
}

terraform {
  backend "s3" {
    bucket = "acme-tfstate"
    key    = local.network_state_key
    region = "us-east-1"
  }
}

Lab 04 breaks on that shape. Brikman's counter-example puts var.bucket in the backend and labels it as code that will not work. The graph that evaluates local and var does not exist at init. Treat the backend as a static declaration plus optional init overlays.

Fact three. Isolation is the key path, not a clever expression.

One bucket can hold many states. One DynamoDB table can lock many keys. Production and staging still need different object paths. Brikman State File Isolation prefers separate root modules and separate keys over one file wearing many names. Partial config is how each root receives its own key at init without baking prod paths into the shared module source.

§III — Worked path: network stack across three environments

Commit the partial backend once. Ship three backend-*.hcl files (or generate them in CI):

# backend-dev.hcl
bucket         = "acme-tfstate"
key            = "network/dev.tfstate"
region         = "us-east-1"
dynamodb_table = "acme-tf-locks"
encrypt        = true

# backend-prod.hcl
bucket         = "acme-tfstate"
key            = "network/prod.tfstate"
region         = "us-east-1"
dynamodb_table = "acme-tf-locks"
encrypt        = true

Dev laptop:

terraform init -backend-config=backend-dev.hcl
terraform plan

Prod pipeline:

terraform init -backend-config=backend-prod.hcl -reconfigure
terraform plan -input=false

-reconfigure is required when the same working directory already pointed at another key. Without it, Terraform may refuse to switch backends silently or ask interactively. CI must be non-interactive.

Lab 31's expected_backend_key local is a documentation hint for humans and tests. It is not a substitute for the real key in backend.hcl. The hint can use var.environment. The backend key cannot.

§IV — Failure modes

Hardcoding prod keys in the shared root. Every clone plans against production state. Partial config exists so the root stays environment-blind.

Using workspaces as the only bulkhead. Named workspaces under one configuration share the same code and tempt a single apply path. Prefer file layout plus distinct keys (prior GCS workspace spend stays prior). Partial config serves the file-layout model.

Forgetting `-reconfigure` in CI. The runner reuses a workspace directory from a previous job. Init keeps the old backend. Plans look green against the wrong state. Always pass -reconfigure (or -migrate-state when you intend a move) in automation.

Confusing partial backend with `terraform_remote_state`. Remote state is a data source that reads another stack's outputs after init. Partial backend configures *this* stack's own memory. Saturday's consumer lesson stays the read path. Today is the write-path address.

Putting secrets only in `backend.hcl` and committing it. Prefer CI-injected files or -backend-config=key=value flags from a secret store. The pattern is init-time injection, not a new place to commit credentials.

§V — Connection to prior lessons

09-08 taught assumptions next to resources (check blocks). 09-05 taught reading another stack's outputs. 08-18 taught acquiring and releasing the DynamoDB lock. Today's lesson sits under those: before any plan can lock or read, init must know which object path is authoritative. Lab 31 drills the empty bucket/key/region shape. Lab 04 drills the expression ban. Brikman Limitations is the page that fails both labs when ignored.

§VI — Closing

Leave bucket, key, and region partial in the committed root when environments share code. Supply them at init. Keep DynamoDB locking as already taught. Keep terraform_remote_state for cross-stack reads. Examine your roots: every backend "s3" that interpolates var or local is a latent init failure. Replace the expression with a backend.hcl and an init flag.

Related

• Paired Dev (HCL):
• Paired Cert:
• Paired Go:
• Prior Ops (remote_state):
• Prior Ops (S3 lock):
• Lab 31:
• Language hub: