Hedronite · Curriculum · Data · Rust · Tue 2026-11-10

A ledger is a claim until you replay it A ledger auditor in Rust

A schema check says a ledger is well formed. Only a replay says it is true.

Lesson Class: Curriculum · B-Data · rung A
Tested with: Rust 1.98 · serde_json 1
Lab: lab/ · sh check.sh (7 result lines) · CPU only, no network
Source: Designing Data-Intensive Applications ch. 11 · CloudTrail log file integrity validation · CloudWatch Logs filter patterns

Lab download. Starter is the tree you edit. Worked answer is a separate file; open it after the starter passes. Rust 1.85 or newer, tar, and grep. One online cargo vendor vendor. The zip has no vendored crates. All lab requirements.

Claims
Every ledger line is a claim. A claim the plan does not support was edited in.
Omissions
Every outcome the plan implies needs a line. A missing line was deleted.
Honest Limit
Edit the plan and the ledger together and this audit passes. Signed, hash-chained logs close that gap.
Well formed is not the same as true.

B-Data · A · arc: self-healing-edge · langs: rust, go · DOP-4.2, DOP-1.4

By Evan Eugene Pincham.

Concept

A release dashboard says v2 is confirmed, so v2 is the version the system will fall back to the next time a deploy goes wrong. Before anyone trusts that, one question needs an answer: is the record behind the dashboard true? The record is a file, and a file can be edited, truncated, or have a line removed. A check that only confirms every line is well formed will pass an edited file, because an edited line is still well formed. This lesson builds a program that checks whether the ledger's claims are true, by working out from the original inputs what the ledger must say and comparing the two in both directions.

A ledger is an append-only record of what a program did, one entry per action. This article audits the ledger of a deployer. A deployer that writes a ledger works like this: it promotes a candidate release (a new version put into service), watches it for a soak window of 30 000 ms, and restarts it when it exits. The restart budget is 2, so the third exit inside the window triggers a rollback to the last confirmed release. If the window closes first, the release is confirmed. Every action lands in a ledger as JSONL, which means one JSON object per line, so a program can read the file one line at a time.

The deployer's input is a plan: a JSONL file of observed events with a millisecond timestamp ts, an event and a version. The events are serving (the release already running), promoted (a candidate takes over and its window starts), exited (a release process died) and tick (time passes; the line carries nothing but a timestamp). The ledger is the deployer's output. Its lines carry ts, an event (restarting, rolled_back, confirmed or restarted_post_soak), a version, and per-event fields.

The ledger is a claim. It says "v2 was confirmed at 31 000", and nothing in the file proves it. Anyone who can write the file can change the claim. A schema check, which verifies that ts is a number and event is a string, tells you the line is well formed. It does not tell you the line is true. A ledger where confirmed was moved from 31 000 to 12 000 passes every schema check, and it describes a release that was declared safe before its window had closed.

Pitfall: a validator that reports "all lines valid" gives false comfort. Shape and meaning are separate properties, and only meaning matters when someone decides whether to trust a release.

To check meaning, derive what the ledger should say. Kleppmann, writing about event sourcing, says the transformation from event log to state "should be deterministic, so that you can run it again and derive the same application state from the event log". Deterministic means the same input always gives the same output, with no hidden inputs such as the wall clock. The auditor applies this idea one step further along. It replays the plan through the deployer's rules and works out which outcome each promotion must have produced. The only time it reads is the ts values written in the plan, so two runs over the same files always agree.

Pitfall: if the derivation reads the current time, a process list or the network, the same files can produce different verdicts on different days, and the audit stops being evidence.

The comparison then runs in both directions. First, every ledger line is a claim, and a claim the plan does not support was edited in or altered. Second, every outcome the plan implies must have a ledger line, and an implied outcome with no line was deleted. A violation is one named finding, printed as VIOLATION line=<n|none> rule=<rule> version=<v> <detail>. An omission has no line to point at, so its line field is none and the version names what is missing.

Pitfall: checking each line alone can never find a deletion. A tool that looks only at lines that exist is blind to the line that does not. The second direction is the only thing that sees it.

The security reason is plain. An edited or deleted ledger line is a forged history. Kleppmann describes the accounting practice for immutable ledgers: a mistake is corrected by adding a compensating entry, and the incorrect entry stays "because it might be important for auditing reasons". An honest correction therefore appends. A line that changed in place, or vanished, has no honest explanation.

The honest limit matters as much as the checks. This is not tamper-proof storage. Someone who edits the plan and the ledger consistently, so that the forged ledger is what a replay of the forged plan produces, is not caught. The auditor finds inconsistency between two files, not tampering with both. Hash-chained or signed logs are the stronger tool, covered under Other paths.

Reading logs for signs of trouble is also a named skill in the AWS DevOps Engineer Professional exam guide (Task Statement 4.2, "Audit, monitor, and analyze logs and metrics to detect issues"), and the ledger is the evidence of a deployment strategy decision (Task Statement 1.4).

Worked example

The auditor is a Rust program named audit. It takes two files, audit PLAN LEDGER, and applies ten rules. It prints each violation and a summary line.

Here is the whole program. The pieces after it are read one at a time.

use serde_json::Value;
use std::{env, fs, process::exit};

const SOAK_MS: u64 = 30_000;
const MAX_RESTARTS: usize = 2;
const LEDGER_EVENTS: [&str; 4] = ["restarting", "rolled_back", "confirmed", "restarted_post_soak"];

struct Ev {
    ts: u64,
    event: String,
    version: String,
}

#[derive(Clone, PartialEq)]
enum Outcome {
    Confirmed(u64),
    RolledBack { ts: u64, to: String },
    Open,
}

struct Promo {
    version: String,
    start: u64,
    restarts: usize,
    outcome: Outcome,
}

struct Claim {
    line: usize,
    ts: u64,
    event: String,
    version: String,
    to: Option<String>,
}

fn read_plan(path: &str) -> Result<Vec<Ev>, String> {
    let text = fs::read_to_string(path).map_err(|e| format!("{path}: {e}"))?;
    let mut evs = Vec::new();
    for (i, line) in text.lines().enumerate() {
        if line.trim().is_empty() {
            continue;
        }
        let v: Value = serde_json::from_str(line).map_err(|e| format!("{path}:{}: {e}", i + 1))?;
        let ts = v["ts"].as_u64().ok_or(format!("{path}:{}: ts", i + 1))?;
        let event = v["event"].as_str().ok_or(format!("{path}:{}: event", i + 1))?.to_string();
        let version = v["version"].as_str().unwrap_or("").to_string();
        evs.push(Ev { ts, event, version });
    }
    Ok(evs)
}

// Derive, from the plan alone, what each promotion must have produced.
fn derive(plan: &[Ev]) -> Vec<Promo> {
    let mut last_good = plan.iter().find(|e| e.event == "serving").map(|e| e.version.clone()).unwrap_or_default();
    let mut promos: Vec<Promo> = Vec::new();
    for (i, p) in plan.iter().enumerate().filter(|(_, e)| e.event == "promoted") {
        let mut exits = 0usize;
        let mut outcome = Outcome::Open;
        for e in &plan[i + 1..] {
            if e.ts >= p.ts + SOAK_MS {
                outcome = Outcome::Confirmed(e.ts);
                break;
            }
            if e.event == "promoted" {
                break;
            }
            if e.event == "exited" && e.version == p.version {
                exits += 1;
                if exits > MAX_RESTARTS {
                    outcome = Outcome::RolledBack { ts: e.ts, to: last_good.clone() };
                    break;
                }
            }
        }
        if let Outcome::Confirmed(_) = outcome {
            last_good = p.version.clone();
        }
        promos.push(Promo { version: p.version.clone(), start: p.ts, restarts: exits.min(MAX_RESTARTS), outcome });
    }
    promos
}

fn main() {
    let args: Vec<String> = env::args().collect();
    if args.len() != 3 {
        eprintln!("usage: audit PLAN LEDGER");
        exit(2);
    }
    let plan = match read_plan(&args[1]) {
        Ok(p) => p,
        Err(m) => {
            eprintln!("audit: {m}");
            exit(2);
        }
    };
    let bytes = match fs::read(&args[2]) {
        Ok(b) => b,
        Err(e) => {
            eprintln!("audit: {}: {e}", args[2]);
            exit(2);
        }
    };
    let promos = derive(&plan);
    let mut raw: Vec<&[u8]> = bytes.split(|b| *b == b'\n').collect();
    if raw.last().is_some_and(|l| l.is_empty()) {
        raw.pop();
    }
    let lines = raw.len();
    let mut violations: Vec<String> = Vec::new();
    let mut claims: Vec<Claim> = Vec::new();
    let mut prev_ts = 0u64;
    for (i, b) in raw.iter().enumerate() {
        let n = i + 1;
        let Ok(text) = std::str::from_utf8(b) else {
            violations.push(format!("VIOLATION line={n} rule=schema version=- not utf-8"));
            continue;
        };
        let v: Value = match serde_json::from_str(text) {
            Ok(v) => v,
            Err(e) => {
                violations.push(format!("VIOLATION line={n} rule=schema version=- {e}"));
                continue;
            }
        };
        let (Some(ts), Some(event), Some(version)) = (v["ts"].as_u64(), v["event"].as_str(), v["version"].as_str()) else {
            violations.push(format!("VIOLATION line={n} rule=schema version=- ts, event or version missing or of the wrong type"));
            continue;
        };
        if !LEDGER_EVENTS.contains(&event) {
            violations.push(format!("VIOLATION line={n} rule=unknown_event version={version} event {event}"));
            continue;
        }
        if ts < prev_ts {
            violations.push(format!("VIOLATION line={n} rule=ts_order version={version} ts {ts} is earlier than {prev_ts}"));
        }
        prev_ts = ts;
        claims.push(Claim {
            line: n,
            ts,
            event: event.to_string(),
            version: version.to_string(),
            to: v["to"].as_str().map(String::from),
        });
    }
    // Direction 1: every claim must be supported by the plan.
    for c in &claims {
        let promo = promos.iter().rev().find(|p| p.version == c.version);
        match c.event.as_str() {
            "confirmed" => match promo {
                None => violations.push(format!("VIOLATION line={} rule=confirmed_unsupported version={} never promoted in the plan", c.line, c.version)),
                Some(p) if c.ts < p.start + SOAK_MS => violations.push(format!(
                    "VIOLATION line={} rule=confirmed_before_window version={} confirmed at {} but the window closes at {}",
                    c.line, c.version, c.ts, p.start + SOAK_MS
                )),
                Some(p) if p.outcome != Outcome::Confirmed(c.ts) => violations.push(format!(
                    "VIOLATION line={} rule=confirmed_unsupported version={} the plan does not confirm this release at {}",
                    c.line, c.version, c.ts
                )),
                Some(_) => {}
            },
            "rolled_back" => match promo.map(|p| &p.outcome) {
                Some(Outcome::RolledBack { ts, to }) if *ts == c.ts => {
                    if c.to.as_deref() != Some(to.as_str()) {
                        violations.push(format!(
                            "VIOLATION line={} rule=rollback_target version={} to {} but the last confirmed release is {}",
                            c.line,
                            c.version,
                            c.to.as_deref().unwrap_or("-"),
                            to
                        ));
                    }
                }
                _ => violations.push(format!("VIOLATION line={} rule=rollback_unsupported version={} the plan has no rollback at {}", c.line, c.version, c.ts)),
            },
            _ => {}
        }
    }
    // Direction 2: every outcome the plan implies must have a claim.
    for p in &promos {
        let has = |ev: &str| claims.iter().any(|c| c.event == ev && c.version == p.version);
        match &p.outcome {
            Outcome::Confirmed(t) if !has("confirmed") => violations.push(format!("VIOLATION line=none rule=missing_confirmed version={} the plan confirms it at {t}", p.version)),
            Outcome::RolledBack { ts, .. } if !has("rolled_back") => violations.push(format!("VIOLATION line=none rule=missing_rollback version={} the plan rolls it back at {ts}", p.version)),
            _ => {}
        }
        let n = claims.iter().filter(|c| c.event == "restarting" && c.version == p.version).count();
        if n != p.restarts {
            violations.push(format!("VIOLATION line=none rule=restart_count version={} ledger has {n} restarting lines, plan implies {}", p.version, p.restarts));
        }
    }
    for v in &violations {
        println!("{v}");
    }
    println!("ledger: lines={lines} violations={}", violations.len());
    exit(if violations.is_empty() { 0 } else { 1 });
}

The fixtures

The program is exercised on three plans, each with an honest ledger. The honest ledgers are exactly what the deployer wrote for those plans, byte for byte. The stable pair shows the shape. The plan has a baseline v1, a promotion of v2, one exit of v2, and a tick event at 31 000:

{"ts":0,"event":"serving","version":"v1"}
{"ts":1000,"event":"promoted","version":"v2","previous":"v1"}
{"ts":3000,"event":"exited","version":"v2"}
{"ts":31000,"event":"tick"}

The ledger the deployer wrote for it records one restart and a confirmation once the window closed (promotion at 1 000 plus 30 000 is 31 000):

{"attempt":1,"event":"restarting","max":2,"ts":3000,"version":"v2"}
{"event":"confirmed","restarts":1,"ts":31000,"version":"v2"}

The flap plan has v2 exit at 3, 6 and 9 seconds, so its ledger has two restarting lines and a rolled_back to v1. The chain plan rolls back v2, then promotes v3 with a stale previous field of v2, and rolls v3 back as well. The auditor never reads previous, and the reason becomes clear in the target check below.

derive

derive turns the plan into a list of promotions, each with the outcome the rules force. This is the loop at its centre:

        for e in &plan[i + 1..] {
            if e.ts >= p.ts + SOAK_MS {
                outcome = Outcome::Confirmed(e.ts);
                break;
            }
            if e.event == "promoted" {
                break;
            }
            if e.event == "exited" && e.version == p.version {
                exits += 1;
                if exits > MAX_RESTARTS {
                    outcome = Outcome::RolledBack { ts: e.ts, to: last_good.clone() };
                    break;
                }
            }
        }

For each promoted event, the loop walks the events after it. The first event at or past promotion plus 30 000 closes the window, so the outcome is Confirmed at that event's ts. Any event type counts, which is why a bare tick is enough to close the window. A later promoted event ends the scan with the outcome left Open, because the release was replaced before anything was decided. Each exited event for the same version bumps exits, and when it passes the budget of 2 the outcome is RolledBack at that exit's ts. The window test comes first, so an exit at or after the window's end confirms rather than rolls back.

The target comes from last_good, a variable that starts as the serving baseline and moves to a version only after its derived outcome is Confirmed. In chain, v2 is rolled back, so it never becomes last_good, and v3 rolls back to v1. The plan's previous field says v2, which is stale. An auditor that trusted the field would accept a ledger that rolls back to v2.

Pitfall: derive leaves a promotion Open when the plan ends before the window closes. An open promotion implies no ledger line at all, so a ledger that confirms it is reported by the claim loop, and a ledger that stays silent is fine.

The claim loop

The first direction goes through every parsed ledger line. Before that, the parsing loop turns each line into a Claim, or into a violation. A cut-off line fails serde_json::from_str, gets a schema violation with its line number, and the loop moves on with continue. Value::as_u64 and Value::as_str return None for a value of another type, which also becomes a schema violation. The check on prev_ts only advances over valid lines, so a damaged line does not hide an ordering problem after it.

With the claims collected, each one is compared with the promotion it names:

    for c in &claims {
        let promo = promos.iter().rev().find(|p| p.version == c.version);
        match c.event.as_str() {
            "confirmed" => match promo {
                None => violations.push(format!("VIOLATION line={} rule=confirmed_unsupported version={} never promoted in the plan", c.line, c.version)),
                Some(p) if c.ts < p.start + SOAK_MS => violations.push(format!(
                    "VIOLATION line={} rule=confirmed_before_window version={} confirmed at {} but the window closes at {}",
                    c.line, c.version, c.ts, p.start + SOAK_MS
                )),
                Some(p) if p.outcome != Outcome::Confirmed(c.ts) => violations.push(format!(
                    "VIOLATION line={} rule=confirmed_unsupported version={} the plan does not confirm this release at {}",
                    c.line, c.version, c.ts
                )),
                Some(_) => {}
            },

A confirmed claim has four possible fates, tried in order. If the release was never promoted, it is unsupported. If the claimed ts is earlier than the promotion plus the window, it is confirmed_before_window. If it is late enough but the plan's derived outcome is not Confirmed at exactly that ts, it is unsupported again, which catches a confirmation of a release the plan rolled back. Otherwise the claim stands. The order means one bad line produces one named violation, with the most specific rule first.

The rolled_back arm requires a derived RolledBack at the same ts, then compares the to field with the target derive computed. Anything else is rollback_unsupported. The other two ledger events, restarting and restarted_post_soak, fall into _ => {}. The first is checked by count in the second direction, and the second is checked only for shape.

Pitfall: promos.iter().rev().find(...) pairs a claim with the latest promotion of that version. A plan that promotes the same version twice would be judged against the second promotion for every claim, including claims about the first.

The omission loop

The second direction starts from the plan and asks whether the ledger said what it should have:

    for p in &promos {
        let has = |ev: &str| claims.iter().any(|c| c.event == ev && c.version == p.version);
        match &p.outcome {
            Outcome::Confirmed(t) if !has("confirmed") => violations.push(format!("VIOLATION line=none rule=missing_confirmed version={} the plan confirms it at {t}", p.version)),
            Outcome::RolledBack { ts, .. } if !has("rolled_back") => violations.push(format!("VIOLATION line=none rule=missing_rollback version={} the plan rolls it back at {ts}", p.version)),
            _ => {}
        }

The closure has asks whether any valid claim has a given event and version. If the plan implies Confirmed and no confirmed line exists for that version, the output is missing_confirmed, with line=none because there is no line to point at. The same goes for a rollback. A second check in the loop counts the restarting lines for the version and compares the count with restarts, which derive capped at the budget of 2. A mismatch is restart_count.

Exit codes and the four forgeries

The program ends with the summary line and an exit code:

    println!("ledger: lines={lines} violations={}", violations.len());
    exit(if violations.is_empty() { 0 } else { 1 });

Exit 0 means clean, exit 1 means at least one violation, and exit 2 means a file could not be opened (or the plan itself is malformed, or the argument count is wrong). A caller can tell "the ledger is suspect" from "the audit could not run".

Four forgeries were built from the honest ledgers for the checker, and a fifth was run by hand. Each output below is copied from a real run, and each ends with the shell exit status.

Early confirmation. The stable ledger with "ts":31000 changed to "ts":12000. The line is well formed, so a schema check passes it. The window for v2 closes at 31 000, and the claim is 19 000 ms early:

VIOLATION line=2 rule=confirmed_before_window version=v2 confirmed at 12000 but the window closes at 31000
ledger: lines=2 violations=1
exit=1

Deleted rollback. The flap ledger with its rolled_back line removed. Every remaining line is true, and the file is still valid. The claim loop finds nothing wrong, and the omission loop does:

VIOLATION line=none rule=missing_rollback version=v2 the plan rolls it back at 9000
ledger: lines=2 violations=1
exit=1

Wrong target. The chain ledger with line 6 changed so that v3 rolls back "to":"v2". The plan's stale previous field also says v2, which makes the lie believable. The derivation knows v2 was never confirmed:

VIOLATION line=6 rule=rollback_target version=v3 to v2 but the last confirmed release is v1
ledger: lines=6 violations=1
exit=1

Damaged line. The stable ledger's first line, then a confirmed line cut off after "ts":31, then a restarting line with ts 2000, earlier than the 3000 before it. Four violations come out, and the audit does not stop at the first:

VIOLATION line=2 rule=schema version=- EOF while parsing an object at line 1 column 41
VIOLATION line=3 rule=ts_order version=v2 ts 2000 is earlier than 3000
VIOLATION line=none rule=missing_confirmed version=v2 the plan confirms it at 31000
VIOLATION line=none rule=restart_count version=v2 ledger has 2 restarting lines, plan implies 1
ledger: lines=3 violations=4
exit=1

The damaged line is counted in lines=3 and named by number. Because it never became a claim, the omission loop then reports the confirmation it should have carried. The two restarting lines (line 1 and line 3) make a count of 2 where the plan implies 1, so restart_count fires too. One damaged line produced consequences in both directions, each named.

A rollback rewritten as a confirmation. The checker has no fixture for this fifth case, so it was run separately on the reference build (the output is saved in the lab as revived.run.log). Take the flap ledger, delete the rolled_back line, and append a confirmed line for v2 at 31 000, the timestamp a healthy release would have had. Every line is well formed and the timestamps rise, so nothing about the file looks wrong by itself. The plan says v2 exited three times inside its window, and the audit reports both halves of the rewrite, the false claim and the missing true one:

VIOLATION line=3 rule=confirmed_unsupported version=v2 the plan does not confirm this release at 31000
VIOLATION line=none rule=missing_rollback version=v2 the plan rolls it back at 9000
ledger: lines=3 violations=2
exit=1

This is the forgery with the most at stake, because it turns a failed release into the next rollback target. Both directions are needed to name it fully: the claim loop alone would report the false confirmed and miss that a rollback was erased, and the omission loop alone would report the erased rollback and miss the false claim.

The three honest ledgers, run against their plans, print violations=0 and exit 0:

flap: ledger: lines=3 violations=0 exit=0
stable: ledger: lines=2 violations=0 exit=0
chain: ledger: lines=6 violations=0 exit=0

That is the other half of the test. An auditor that flags forgeries but also flags the deployer's real output is useless, so the honest ledgers must match what the deployer wrote exactly.

Lab

The lab has two trees and one checker:

lab/
  check.sh
  starter/     Cargo.toml  Cargo.lock  src/main.rs  vendor.tgz
  reference/   Cargo.toml  Cargo.lock  src/main.rs  vendor.tgz

Each tree is a Rust crate (edition 2024) that builds a binary named audit with serde_json 1. The vendor.tgz file holds serde and serde_json as vendored sources, so the build needs no network. check.sh is POSIX shell and needs cargo, tar and grep. It writes the three plans, the three honest ledgers and the four forged ledgers itself, builds the crate in the current directory, and prints one result line per check, each ending in PASS or FAIL. Copy it into the tree you want to test and run it from there:

cp check.sh starter/
cd starter
sh check.sh

The seven checks are build, honest_ledgers_pass, early_confirm_caught, deleted_rollback_caught, wrong_rollback_target_caught, damaged_line_named and missing_file_exit_2.

The starter validates shape only. It reads each ledger line, checks that ts is an unsigned integer and event and version are strings, and exits 0 if everything parses. It never looks at the plan beyond confirming the file opens. That passes the honest ledgers and the missing-file check, because they need nothing more, and it fails everything that needs meaning. These are the failing lines from a run of the starter:

ledger: lines=2 violations=0
exit=0
RESULT early_confirm_caught FAIL
ledger: lines=2 violations=0
exit=0
RESULT deleted_rollback_caught FAIL
ledger: lines=6 violations=0
exit=0
RESULT wrong_rollback_target_caught FAIL
VIOLATION line=2 rule=schema version=- malformed line
ledger: lines=3 violations=1
exit=1
RESULT damaged_line_named FAIL

The early, deleted and wrong-target ledgers all come back with violations=0. That is the failure this lab exists to show: a forged ledger that passes validation.

Task. Edit starter/src/main.rs so the three forgery checks pass without breaking the four that already do. Read the plan, derive what each promotion must have produced, then compare in both directions.

Suggested order.

  1. early_confirm_caught. Parse the plan, find each promoted event, and flag any confirmed line earlier than promotion plus 30 000. This needs only a start time and a constant.
  2. deleted_rollback_caught. Add the third-exit rule (the third exited event for the candidate inside the window is a rollback), then report a derived rollback that has no ledger line, with line=none.
  3. wrong_rollback_target_caught. Track the last release whose derived outcome was confirmed, and compare the to field of each rolled_back line with it.
  4. damaged_line_named. The starter already names the damaged line with rule=schema, but the check also wants rule=ts_order on line 3 and lines=3 in the summary. Track the previous valid ts, and keep going after a bad line.

When it is done, a run prints the seven passing lines:

RESULT build PASS
RESULT honest_ledgers_pass PASS
RESULT early_confirm_caught PASS
RESULT deleted_rollback_caught PASS
RESULT wrong_rollback_target_caught PASS
RESULT damaged_line_named PASS
RESULT missing_file_exit_2 PASS

Extension. Add a rule that a restarted_post_soak line is valid only after a confirmed line for the same version. Before running it, predict which fixtures change. None of the three honest ledgers and none of the four forgeries contains a restarted_post_soak line, so every existing fixture should print exactly what it printed before. To see the rule fire you need a new ledger, for example the stable ledger with a restarted_post_soak line for v2 placed before the confirmed line.

Other paths

Go sketch (hypothetical, not compiled and not run). The same two-way compare works in Go. Build a map[string]outcome from the plan, where the key is the version and outcome holds the kind, the ts and the target. Read the ledger with bufio.Scanner, check each confirmed or rolled_back line against the map and record the versions seen, then report every map entry never seen.

// Sketch only: not compiled, not run.
type outcome struct {
	kind string // "confirmed" or "rolled_back"
	ts   uint64
	to   string
}

func audit(expected map[string]outcome, ledger io.Reader) int {
	violations := 0
	seen := map[string]bool{}
	sc := bufio.NewScanner(ledger)
	line := 0
	for sc.Scan() {
		line++
		var rec struct {
			Ts      uint64 `json:"ts"`
			Event   string `json:"event"`
			Version string `json:"version"`
			To      string `json:"to"`
		}
		if err := json.Unmarshal(sc.Bytes(), &rec); err != nil {
			fmt.Printf("VIOLATION line=%d rule=schema version=- %v\n", line, err)
			violations++
			continue
		}
		if rec.Event != "confirmed" && rec.Event != "rolled_back" {
			continue
		}
		want, ok := expected[rec.Version]
		if !ok || want.kind != rec.Event || want.ts != rec.Ts ||
			(rec.Event == "rolled_back" && want.to != rec.To) {
			fmt.Printf("VIOLATION line=%d rule=unsupported version=%s\n", line, rec.Version)
			violations++
			continue
		}
		seen[rec.Version] = true
	}
	if err := sc.Err(); err != nil { // bufio.ErrTooLong ends the scan early
		fmt.Printf("VIOLATION line=%d rule=schema version=- %v\n", line+1, err)
		violations++
	}
	// Sort the keys before this loop in a real version: map order is random.
	for v, want := range expected {
		if !seen[v] {
			fmt.Printf("VIOLATION line=none rule=missing_%s version=%s\n", want.kind, v)
			violations++
		}
	}
	return violations
}

Three things change from the Rust. First, Scanner has a default token limit of 64 KiB per line. A longer line stops the scan with ErrTooLong, and the loop simply ends as if the file were finished, so the code must check sc.Err() after the loop. A truncated audit that reports "no violations" is the worst result this tool can give. Second, Go's json.Unmarshal leaves a missing field at its zero value, where Rust's as_u64 returns None. A real version would use pointer fields to tell "absent" from 0. Third, Go randomises map iteration order, so the omission loop would print in a different order on each run unless the keys are sorted first, which would break the determinism the whole audit depends on.

CloudTrail log file integrity validation (prose only, not run). The audit here detects a ledger that disagrees with its plan. It cannot detect tampering done to both files. The stronger tool is a hash chain with signatures. The AWS CloudTrail User Guide says: "To determine whether a log file was modified, deleted, or unchanged after CloudTrail delivered it, you can use CloudTrail log file integrity validation". It uses SHA-256 for hashing and SHA-256 with RSA for digital signing. CloudTrail writes a digest file every hour, each one signed and containing the signature of the previous digest file. Because each digest covers the hashes of the log files and links back to the one before, a modified, deleted or forged log file breaks the chain, and someone who can edit a plan and a ledger together still cannot produce a valid signature without the key.

A log filter does not close the gap either. The Amazon CloudWatch Logs User Guide says: "Filter patterns only return the log events that contain the terms you define". A filter sees only events that exist, so it cannot see a deleted line. The deleted-rollback case above is the same blind spot, and only a comparison against an independent source of truth finds it.

Citations