Hedronite · Dev Lesson · Polyglot-Dev / Python · Sat 2026-09-05

Python terraform show JSON census — apply-risk resource_changes report

Count creates, updates, deletes, and replaces. Print markdown. Exit zero.

Lesson Class: Dev (Python-around-TF)
Idiom: subprocess.run + terraform show -json + resource_changes
Paired Ops: remote_state consumer plan is the artifact
Grounding: Python for DevOps subprocess · Brikman plan files referenced
Census
Report counts. Do not gate.
Replace
Set equality on delete+create actions.
Exit
Zero on success. Policy lives elsewhere.
A saved plan is a JSON document. Count the actions. List the replaces.

<!-- hal:authoritative:yaml -->

A saved plan is a JSON document. Count the actions. List the replaces. Post the census. Do not pretend counting is the same as denying.

§I — Frame

08-09 taught a policy checker: read resource_changes, fail the build when a rule fires. That tool exits non-zero on purpose. It is a gate.

Today's Ops lesson repairs a consumer stack that reads network outputs through terraform_remote_state. After that repair, every merge still needs a human-readable answer to a simpler question: what will this apply do? How many creates, updates, deletes, and replaces? Which addresses are replaces?

That answer is a census, not a policy. The census always exits zero. It prints markdown a pull request can quote. It refuses to become Rego.

08-18 already wrapped Terraform in subprocess to chase lock errors. Do not redo lock parsing. Today's subprocess call is terraform show -json against a plan file the pipeline already saved. Python for DevOps (Gift et al.) spends subprocess.run as the standard-library way to call CLI tools and capture stdout. That is the idiom.

§II — Language Idiom: plan file, show -json, resource_changes

Terraform can write a binary plan with terraform plan -out=tfplan. Brikman's Plan files section warns that plan files can hold secrets. Treat the artifact as sensitive. Do not upload it to a public cache.

terraform show -json tfplan prints a JSON document on stdout. The interesting array is resource_changes. Each element carries address, mode, type, name, and change.actions. Actions are a list drawn from no-op, create, read, update, delete. A replace appears as ["delete", "create"] (or the reverse order depending on create_before_destroy). That list is the whole census.

import json
import subprocess
import sys
from collections import Counter


def load_plan(path: str) -> dict:
    proc = subprocess.run(
        ["terraform", "show", "-json", path],
        check=True,
        capture_output=True,
        text=True,
    )
    return json.loads(proc.stdout)


def census(plan: dict) -> tuple[Counter, list[str]]:
    counts: Counter = Counter()
    replaces: list[str] = []
    for rc in plan.get("resource_changes") or []:
        actions = tuple(rc.get("change", {}).get("actions") or [])
        if actions == ("no-op",) or actions == ("read",):
            continue
        if set(actions) == {"delete", "create"}:
            counts["replace"] += 1
            replaces.append(rc["address"])
        elif actions == ("create",):
            counts["create"] += 1
        elif actions == ("update",):
            counts["update"] += 1
        elif actions == ("delete",):
            counts["delete"] += 1
        else:
            counts["other"] += 1
    return counts, replaces

Three Python facts keep the tool honest.

**Fact one. subprocess.run with check=True and capture_output=True.** Gift et al. show command-line tools built this way. If terraform show fails, you want the exception, not a partial JSON parse of stderr mixed into stdout. Do not shell=True. Pass the argument list.

**Fact two. Treat missing resource_changes as empty, not fatal.** A plan against an empty configuration still returns JSON. Defensive .get keeps the census useful when Terraform adds fields.

Fact three. Classify replace by set equality, not by stringifying the list. ["delete","create"] and ["create","delete"] are both replaces. Set equality catches both. Order-sensitive tuple checks will lie under create_before_destroy.

§III — Code Worked Example: markdown report for the consumer plan

def render(counts: Counter, replaces: list[str]) -> str:
    lines = [
        "## Terraform apply-risk census",
        "",
        f"- create: {counts.get('create', 0)}",
        f"- update: {counts.get('update', 0)}",
        f"- delete: {counts.get('delete', 0)}",
        f"- replace: {counts.get('replace', 0)}",
    ]
    if replaces:
        lines.append("")
        lines.append("### Replace addresses")
        for addr in replaces:
            lines.append(f"- `{addr}`")
    else:
        lines.append("")
        lines.append("_No replaces in this plan._")
    return "\n".join(lines) + "\n"


def main(argv: list[str]) -> int:
    if len(argv) != 2:
        print("usage: plan_census.py <tfplan>", file=sys.stderr)
        return 2
    plan = load_plan(argv[1])
    counts, replaces = census(plan)
    sys.stdout.write(render(counts, replaces))
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

Pipeline shape against today's Ops consumer:

  1. terraform plan -out=tfplan in app/
  2. python plan_census.py tfplan >> $GITHUB_STEP_SUMMARY
  3. Continue to a human approval job if replace count is greater than zero
  4. Never let this script's exit code fail the build. Approval is a different step.

That last line is the split from 08-09. The policy checker fails closed. The census informs. If you need both, run both. Do not overload one script with two jobs.

Optional enrichment for remote_state days: scan plan["configuration"]["root_module"].get("resources", []) for type == "terraform_remote_state" and print the backend keys named in config. That is documentation of which producer keys this consumer plan refreshed. It is not a substitute for Ops §III refresh discipline.

§III.B — Schema details the census must respect

The plan JSON is versioned. Top-level format_version tells you which schema you parsed. Pin your expectations. When HashiCorp ships a new format version, re-read the changelog before you assume change.actions still means what it meant last quarter.

Each resource_changes entry may include change.before and change.after. The census above ignores those objects on purpose. Diffing before/after for a human report is a second tool. It balloons output, risks printing secret values that landed in the plan, and duplicates terraform show without -json. If you add a before/after mode later, redact attribute names matching password, secret, token, and private_key by default.

mode is managed or data. Data sources often show read actions. Skipping read and no-op keeps the census focused on mutating managed resources. A remote_state data source refresh that only reads should not inflate create counts.

Modules nest under change addresses with module paths: module.network.aws_subnet.private[0]. Keep the full address. Truncating to the resource type hides which module instance will replace.

Provider aliases appear in configuration, not always in the address string. If your census ever groups by provider, walk configuration, not address prefixes.

§III.C — Testing the census without a cloud account

You do not need AWS to unit-test the classifier. Save a fixture JSON that looks like a trimmed plan:

FIXTURE = {
    "format_version": "1.2",
    "resource_changes": [
        {
            "address": "aws_instance.app",
            "change": {"actions": ["delete", "create"]},
        },
        {
            "address": "aws_lb_listener_rule.api",
            "change": {"actions": ["update"]},
        },
        {
            "address": "data.terraform_remote_state.network",
            "change": {"actions": ["read"]},
        },
    ],
}

def test_census_counts_replace_and_skips_read():
    counts, replaces = census(FIXTURE)
    assert counts["replace"] == 1
    assert counts["update"] == 1
    assert counts.get("create", 0) == 0
    assert replaces == ["aws_instance.app"]

Run that under pytest in CI next to the script. Brikman's Plan testing section aims at Terraform-native tests. Your Python fixture test is the bridge while HCL tests catch up. Both can coexist.

When you do have a plan file from Lab 12's repaired consumer, run the census once and paste the markdown into the lab notes. That is the practice loop: Ops repairs HCL, Dev summarizes the plan, humans decide.

§III.D — Failure modes

terraform not on PATH. subprocess.run raises FileNotFoundError. Catch it and print a clear message. Do not catch Exception.

Wrong working directory. terraform show -json tfplan needs the same directory context the plan was created in, or an absolute path to a plan that still matches provider plugins. Prefer absolute paths in CI.

JSON decode errors. Usually mean stdout mixed with progress noise. Ensure TF_IN_AUTOMATION=1 and no wrapper printing banners into stdout. Capture stderr separately; log it on failure.

Huge plans. A plan with tens of thousands of resource_changes will stress memory if you also keep before/after. The census that only keeps actions and addresses stays small.

Sensitive values in addresses. Addresses rarely hold secrets, but before/after do. Stick to addresses in the default report.

§III.E — Why exit zero is a feature

Teams ask for "fail the build if replace > 0." That policy belongs in the 08-09 checker or in an explicit approval gate. If the census exits non-zero, humans stop reading the markdown and start arguing about thresholds in the wrong file. Keep the census boring. Let an approval job read the markdown artifact and branch.

Document the contract in the script's module docstring: "Reporting only. Exit codes: 0 success, 2 usage, 1 terraform/show failure." That is enough.

§IV — Connection to Today's Ops Lesson

Ops demands producer apply, then consumer refresh, then consumer apply. The census sits on the consumer plan. If remote_state refreshed into a new subnet ID and the instance resource must replace, the census lists that address under replaces. Reviewers see the risk without opening the raw JSON.

If the census shows zero changes after a network apply that published new outputs, either the consumer does not read those outputs or refresh did not run. That is an Ops finding surfaced by a Dev tool.

§II.B — Wrapping the CLI without reinventing Terraform

A common temptation is to parse the human terraform plan text with regular expressions. That path breaks when columns shift, when colors wrap, when a provider prints a note. The JSON document exists so you stop scraping prose.

Another temptation is to call the cloud APIs directly and invent your own drift detector. That path duplicates the provider. The plan already reconciled desired state with refreshed state. Read the plan.

The third temptation is to import HashiCorp's Go libraries from Python. You do not need them for a census. subprocess plus json is enough. Gift et al. place command-line tools in the standard library on purpose: ops glue should stay thin.

Put the script in tools/plan_census.py next to the app stack, or in a shared platform repo with a pinned version. Pinning matters once two teams disagree about whether other actions should be listed. Version the script. Tag releases. Do not copy-paste the function into five repositories.

Environment variables worth supporting:

Default format stays markdown. JSON format should emit {"create":N,"update":N,"delete":N,"replace":N,"replaces":["addr",...]} so Go or Ruby jobs can read the same census without scraping.

§IV.B — Pairing with remote_state days

On a day like today, run the census twice in the notes. Once against the network producer plan. Once against the app consumer plan. Compare replace lists. A replace on the producer that does not appear on the consumer may still matter if outputs changed. The census will not see output-only changes unless a managed resource in the consumer interpolates them. That is expected. Output-only producer changes still require the Ops refresh habit even when the consumer census is all zeros.

If you want a cheap signal for output changes, add a second small script later that diffs terraform output -json before and after. That script is not today's lesson. Today's lesson stays on resource_changes.

§V — Prior-Lesson Reach

08-09: policy checker over the same schema. Keep it. Do not merge it into the census.

08-18: subprocess around lock errors. Different stderr grammar. Different purpose.

08-27: boto3 STS around assume-role. Today's tool does not call AWS APIs. It calls Terraform. Go will list the S3 bucket that stores the state objects; Python here lists the planned actions against one of those objects' configurations.

Brikman Plan testing (Ch.9) shows automated assertions against plans. The census is the lightweight sibling you can ship before a full terratest suite exists. It does not replace terratest. It feeds humans.

Also print the plan's terraform_version field at the top of the markdown. Reviewers should know whether the plan was produced by the same major version the module authors tested. Drift between 1.5 and 1.9 is rare for action lists, but it is not imaginary.

When the consumer plan is empty, the census still prints zeros. Empty is a finding on days Ops expects a subnet change to propagate. Empty is a comfort on days nobody touched the producer. The markdown does not know which day it is. The reviewer does.

§V.B — What this lesson refuses

It refuses to become an OPA competitor. It refuses to call terraform apply. It refuses to rewrite plan files. It refuses to pretty-print every attribute. It refuses banned filler about "end-to-end visibility platforms." The job is a census.

It also refuses the 08-18 lock story. If terraform show fails because the working directory has a stale lock from a crashed apply, that is an Ops incident. The census should surface the subprocess error and stop. Do not add lock-force logic to a reporting tool.

§V.C — Classroom drill (twenty minutes)

  1. From a repaired Lab 12 consumer directory, produce tfplan.
  2. Run python plan_census.py tfplan.
  3. Force a replace by changing an instance ami (or whatever Lab resource is cheap).
  4. Re-plan and re-run the census.
  5. Confirm the replace address appears.
  6. Confirm exit code stays 0.
  7. Paste both markdown reports into your notes.

That drill ties Bootcamp Lab 12 to Python for DevOps subprocess pages without inventing a Python-specific Bootcamp lab. Dual-corpus stays honest: Bootcamp supplies the plan; the tome supplies the CLI wrapper idiom.

When you later teach Go ListObjects against the backend bucket, keep this census on the plan artifact. Two different views of the same platform: object inventory in S3, action inventory in the plan. Neither replaces the other.

Ship the script with a one-line README that states the contract, the exit codes, and the non-goals. Link the Ops lesson for refresh order and the 08-09 lesson for gating. Future you will thank present you when someone asks why the census did not fail the build on a replace.

If markdown tables render better in your forge than bullet lists, emit a table. Keep the columns boring: action, count. Replaces stay as a list under the table. Do not invent sparkline charts for three integers.

§VI — Closing

Save the plan. Show it as JSON. Count actions. List replaces. Print markdown. Exit zero. Leave gating to a policy tool that already knows how to fail.

Remember the split one last time. 08-09 denies. Today reports. Ops refreshes remote_state so the plan you census is the plan that will apply. Go inventories the bucket that stores the state behind that plan. Four views, one platform day.

If a reviewer only reads one artifact from the consumer pipeline, make it this census. It is short enough to finish before coffee cools and specific enough to name the instance that will be replaced. That is the bar. Meet it, then stop adding features. Extra charts can wait for a week when the census has already earned trust. Three integers and a replace list beat a dashboard nobody opens. Ship that. Then rest.

Related