Rust TF integration tests validate -json, plan -json diagnostics, and the variable that must fail

A negative test passes when Terraform refuses. Check the refusal, not the summary.

A negative test passes when Terraform refuses. Check the refusal, not the summary.

§I. Frame

On 09-29 the Rust test crate ran init, apply, and output -json and checked that the round trip came back whole. That is a positive test. Today's paired Ops lesson puts a policy in a variable validation: rotation_period must exceed 86400s and stay at or under 90 days. A policy needs negative tests. Feed it 3600s and prove the plan fails, for the right reason, every time.

The problem for today: write Rust integration tests that drive a real terraform binary, parse both of its JSON output shapes with serde, and assert on errors as data.

§II. Two shapes of JSON

Terraform speaks JSON two ways, and a test crate needs a type for each.

1. **terraform validate -json** prints one document: valid, error_count, warning_count, and a diagnostics array. Each diagnostic carries severity, summary, detail, and a source range.
2. **terraform plan -json** prints a stream. Every line is its own JSON object with @level, @message, and a type such as version, planned_change, change_summary, outputs, or diagnostic.

The crate gives the stream one struct and lets serde skip what it does not read:

#[derive(Debug, Deserialize)]
pub struct UiLine {
    #[serde(rename = "type")]
    pub kind: String,
    #[serde(rename = "@level")]
    pub level: String,
    pub diagnostic: Option<Diagnostic>,
    pub changes: Option<Changes>,
}

type is a Rust keyword and @level is not an identifier, so both need #[serde(rename)]. Fields that are absent on a given line land as None. The fold walks lines with a tuple match:

match (ui.kind.as_str(), ui.diagnostic, ui.changes) {
    ("diagnostic", Some(d), _) if d.severity == "error" => errors.push(d),
    ("change_summary", _, Some(c)) => summary = Some(c),
    _ => {}
}

§III. The zero before the error

Here is the run that shaped the crate. Terraform 1.16.5 on the box, fixture good/, -var rotation_period=3600s. The stream, trimmed:

{"@level":"info","@message":"Terraform 1.16.5","type":"version",...}
{"@level":"info","@message":"Plan: 0 to add, 0 to change, 0 to destroy.","changes":{"add":0,...},"type":"change_summary"}
{"@level":"error","@message":"Error: Invalid value for variable","diagnostic":{"severity":"error",...},"type":"diagnostic"}
exit 1

The Zero Before the Error (named technique). A failed plan still emits a change_summary, and it reads zero. A test that asserts "no changes" by reading changes.add == 0 passes against a plan that never ran. So PlanReport::succeeded reads in a fixed order: exit code, then error diagnostics, then the summary.

pub fn succeeded(&self) -> bool {
    self.exit_code == 0 && self.errors.is_empty()
}

The negative test then pins the reason. One error, summary Invalid value for variable, exit 1. A different failure, such as a typo in the fixture, also exits 1, and the summary check catches it.

§IV. Validate is not where variable rules fire

Brikman's Table 9-1 lists terraform validate as built-in static analysis with "syntax checks only" (PDF p.542). The box confirmed the edge. Terraform 1.16.5 accepts -var on validate, and its help says it checks the module "regardless of any provided variables." With -var rotation_period=3600s, validate -json returned "valid": true and exit 0. The same value fails at plan.

So the suite splits along that line:

§V. The test rig

Workdir::copy_of copies a fixture's .tf files into a fresh temp directory, so tests that run in parallel never share .terraform/. Drop removes it, pass or panic. tf() wraps std::process::Command::output with TF_IN_AUTOMATION=1 and TF_INPUT=0 and returns (exit code, stdout, stderr).

The fixtures use only terraform_data, which is built into Terraform. init -backend=false downloads nothing, so the suite runs offline in under a second. Three unit tests parse lines captured from the real run, so the parser is tested even where no terraform binary exists.

Run on the box:

running 3 tests (unit)       ... 3 passed
running 5 tests (terraform)  ... 5 passed; finished in 0.07s

rustc 1.99.0, edition 2024, terraform 1.16.5. The lab Mac has no terraform on PATH, so the integration tests ran on the box.

§VI. Close

The Zero Before the Error: read exit code, then diagnostics, then the summary. Pin each negative test to its diagnostic summary so the wrong failure cannot pass. Add a sixth test of your own: rotation_period=7776001s must fail with the same summary, and write the expected exit code before you run it.

Paired Ops wrote the validation this suite proves. Paired Cert covers the lock file that init writes next to these fixtures when a real provider is involved.

Related

• Tome: TRPL ch11 Writing Automated Tests (grounded-in)
• Tome: Brikman, Terraform: Up and Running 3e, ch9 Table 9-1, PDF p.542 (grounded-in)
• Prior Dev: TF apply round trip 09-29 · plan JSON drift 10-05 · replace in two actions 09-26
• Web: machine-readable UI · terraform validate · std::process::Command