Hedronite · Curriculum · Data · Frontend · Tue 2026-11-03

A log health page that shows what it could not read A static export in Bun and TypeScript

Twelve lines in, six readable, six listed by line number, and a page whose totals add up or the build fails.

Lesson Class: Curriculum · F-Data · rung A
Tested with: Bun 1.3 (lab image, v1.3.14)
Lab: lab/ · sh check.sh (9 checks) · no network
Source: JSON Lines · Designing Data-Intensive Applications ch. 4 · Bun File I/O · MDN TextDecoder and JSON.parse · Lustre element
Read Bytes, Not Text
Split the file on the newline byte and decode each line strictly, so a bad byte is named instead of repaired.
Every Line in One Bucket
Readable or unreadable with a kind and a line number, and readable plus unreadable equals lines.
Numbers That Add Up
A page whose totals disagree is a build error, and a damaged log gets a warning block.
A health page that hides the lines it cannot read is healthy only in the places it looked.

F-Data · A · arc: self-healing-edge · langs: ts, gleam · DOP-4.2

# A deploy log health page that shows the lines it could not read

By Evan Eugene Pincham.

Concept

A small deployer program starts a new release beside the old one and keeps the old one unless the new one proves ready in time. Each step it takes is appended to a log file as one line of JSON. That format is called JSON Lines: a text file in which every line is one complete JSON value, with \n (the newline byte, 0x0A) as the line terminator, encoded as UTF-8. UTF-8 is the standard way of turning text into bytes. Not every sequence of bytes is valid UTF-8, so a file can hold bytes that are not text at all. The JSON Lines rules say that "a blank line is not" a valid JSON value.

The deployer's lines follow a fixed schema, which is an agreement about which keys a record has and what type each value is:

keytype
tsunsigned integer
eventstring, one of candidate_started, promoted, rolled_back
versionstring
detailstring

The operator question this lesson answers is "how many releases rolled back, and why?" You answer it by reading the log and building a page. That is a small case of what the AWS Certified DevOps Engineer - Professional exam guide (version 1.6) lists as Task Statement 4.2, "Audit, monitor, and analyze logs and metrics to detect issues": the page is an audit view of a log. The service in the examples is a demo called edge-svc, and the page is named after it. The page is a static export: HTML files written ahead of time by a build step, so a plain file server can hand them out. No code runs per visitor. This content needs no accounts and no data store, so there is no reason to run a server.

A report is a claim about the whole file

The natural first version of a reader reads the file as text, parses each line, and skips any line that fails. It looks tidy, and it makes the page say less than the file holds while still looking complete. The starter for this lab behaves this way. Given a twelve-line file it prints 6 lines, and nothing on the page hints that six lines went missing.

This matters for security as well as for tidiness. A truncated or tampered audit log is evidence, and a reader that silently drops lines it cannot parse hides the gaps. A log is only as trustworthy as its gaps are visible. The fix has three parts:

  1. Put every physical line into exactly one bucket: readable, or unreadable with a named kind.
  2. Print the bucket sizes.
  3. State the invariant, a fact that must always be true, and check it in code: readable + unreadable = lines.

If the invariant fails, the build fails. A page whose numbers do not add up is a build error.

Read bytes, then decode each line

Bun's file API can hand you the same file as a string or as bytes. The Bun documentation lists text(), arrayBuffer() and bytes() as BunFile methods. This lesson uses arrayBuffer() and splits on the 0x0A byte itself, and the reason is visible in a short probe (a scratch script, not part of the lab) that reads the damaged fixture both ways:

$ bun probe-text.ts
text() gives line 8 starting with: U+FFFD U+FFFD U+0020
JSON.parse on it throws: SyntaxError
fatal decoder on the same two bytes throws: TypeError

The output shows text() decoding leniently, which matches what MDN documents as the default for a TextDecoder: fatal is false, "which means that the decoder will substitute malformed data with a replacement character." So the two bad bytes 0xFF 0xFE at the start of line 8 became two U+FFFD replacement characters, and the line then fails JSON parsing like any other garbage. The damage turns into a SyntaxError, and the reader would file the line under the wrong kind (or, in the starter, drop it). A TextDecoder built with { fatal: true } throws a TypeError on the same bytes, and that throw is how the reference recognises the not utf-8 kind.

Forward compatibility

Kleppmann's Designing Data-Intensive Applications, chapter 4, separates two directions. Backward compatibility means newer code can read data written by older code. Forward compatibility means older code can read data written by newer code. He notes that "Forward compatibility can be trickier, because it requires older code to ignore additions made by a newer version of the code."

A newer deployer might add a key such as host, or a new event name such as paused. A reader written today must treat both as readable and counted, not as corruption. Extra keys are ignored. An event name the reader does not know goes into an unknown events row. Compare that with a missing detail key or a ts that holds the string "yesterday". Those lines break the schema the reader depends on, so they are unreadable.

Why the page re-parses the log

The page builder reads the log itself in TypeScript. It does not consume the text printed by a separate command-line report, although such a report exists and reads the same log in Rust. The reasons:

  1. The log is the single source of truth. A report's human-readable text is a presentation format, and its line order and wording can change. Scraping it would add a second place to be wrong.
  2. The builder runs anywhere Bun runs. It needs no Rust toolchain.
  3. Two independent readers are a cross-check. The same fixtures must give 12 lines, 6 readable and 6 unreadable from both.

The cost is that two readers can drift apart. The lab's fixtures pin the numbers on both sides. The error messages differ on purpose. The Rust report quotes its JSON library's messages. The TypeScript reader writes its own, because the documented behaviour of JSON.parse is only that it throws a SyntaxError for invalid JSON. The message text belongs to the JavaScript engine. In the Bun used for this lab the message for a cut-off line is JSON Parse error: Unexpected EOF, while the examples on the MDN page read like Unexpected token ] in JSON at position 13. A page that published either wording would change when the engine changes. The kind names (truncated, not json, blank, schema, not utf-8) match on both sides.

Exit codes: when should the build fail?

The command-line report exits 1 when any line is unreadable, because a command's exit code is its result. The static build is different. It exits 0 when it renders a damaged log, because the page is the result and the page says "incomplete". It fails only when the numbers cannot be reconciled. CI should not go red because the data is damaged, since that is the very thing the page exists to show. CI should go red when the page would lie.

Untrusted input

Every string that comes from the log and lands in the page passes through esc, which HTML-escapes it. A log line is untrusted input and the page is HTML. A rollback reason like <script>… must show up as text, not run. The lab's checker does not test escaping, so this protection depends entirely on you.

Worked example

The reader splits the bytes into lines, classifies each line, and checks the totals. report.ts:

// Reads a JSON Lines deploy log (ts, event, version, detail) and puts EVERY physical line in exactly one bucket.
export type Kind = "truncated" | "not json" | "blank" | "schema" | "not utf-8";
export type Bad = { line: number; kind: Kind; message: string };
export type Report = {
  lines: number;
  readable: number;
  unreadable: Bad[];
  events: Record<string, number>; // known events only
  unknownEvents: number; // readable lines whose event name this reader does not know
  reasons: Map<string, number>; // rollback reason -> count
};

const KNOWN = ["candidate_started", "promoted", "rolled_back"];
const utf8 = new TextDecoder("utf-8", { fatal: true }); // throws on bytes that are not UTF-8

// Split on the 0x0A byte ourselves: decoding the whole file first would hide a bad line.
export function splitLines(bytes: Uint8Array): Uint8Array[] {
  const out: Uint8Array[] = [];
  let start = 0;
  for (let i = 0; i < bytes.length; i++) {
    if (bytes[i] === 10) {
      out.push(bytes.subarray(start, i));
      start = i + 1;
    }
  }
  if (start < bytes.length) out.push(bytes.subarray(start)); // last line without a final newline
  return out;
}

export function assertTotals(lines: number, readable: number, unreadable: number): void {
  if (readable + unreadable !== lines) {
    throw new Error(`totals do not add up: ${lines} lines but ${readable} readable + ${unreadable} unreadable`);
  }
}

export function readLog(bytes: Uint8Array): Report {
  const parts = splitLines(bytes);
  const unreadable: Bad[] = [];
  const events: Record<string, number> = Object.fromEntries(KNOWN.map((k) => [k, 0]));
  const reasons = new Map<string, number>();
  let readable = 0;
  let unknownEvents = 0;

  parts.forEach((raw, i) => {
    const line = i + 1; // 1-based, like an editor
    const bad = (kind: Kind, message: string) => unreadable.push({ line, kind, message });
    let text: string;
    try {
      text = utf8.decode(raw);
    } catch {
      return bad("not utf-8", "bytes are not valid UTF-8");
    }
    const t = text.trim();
    if (t === "") return bad("blank", "empty line");
    let v: unknown;
    try {
      v = JSON.parse(t);
    } catch {
      return t.startsWith("{") && !t.endsWith("}")
        ? bad("truncated", `object never closed; the line ends after ${t.length} characters`)
        : bad("not json", "not a JSON value");
    }
    if (typeof v !== "object" || v === null || Array.isArray(v)) return bad("schema", "not a JSON object");
    const o = v as Record<string, unknown>;
    for (const key of ["ts", "event", "version", "detail"]) {
      if (!(key in o)) return bad("schema", `missing key \`${key}\``);
    }
    if (typeof o.ts !== "number" || !Number.isInteger(o.ts) || o.ts < 0) {
      return bad("schema", `\`ts\` must be an unsigned integer, got ${typeof o.ts} ${JSON.stringify(o.ts)}`);
    }
    for (const key of ["event", "version", "detail"]) {
      if (typeof o[key] !== "string") return bad("schema", `\`${key}\` must be a string, got ${typeof o[key]}`);
    }
    // Readable. Extra keys are ignored (forward compatibility); an event name we do not know is counted, not rejected.
    readable++;
    const ev = o.event as string;
    if (KNOWN.includes(ev)) events[ev]++;
    else unknownEvents++;
    if (ev === "rolled_back") {
      const why = (o.detail as string).split(";")[0].trim();
      reasons.set(why, (reasons.get(why) ?? 0) + 1);
    }
  });

  assertTotals(parts.length, readable, unreadable.length);
  return { lines: parts.length, readable, unreadable, events, unknownEvents, reasons };
}

Each design decision is one visible line or block of this file:

The page builder, build.ts, uses Bun's Bun.file(...).arrayBuffer() and Bun.write. For listing the directory it uses node:fs, which Bun's file I/O documentation recommends for operations such as readdir that the Bun.file and Bun.write APIs do not cover.

// Static export: one "deploy log health" page per fixture log, written to dist/. No server, no client JS.
import { readdirSync } from "node:fs";
import { assertTotals, readLog } from "./report.ts";

const esc = (s: string) =>
  s.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;").replace(/"/g, "&quot;");
const plural = (n: number, word: string) => `${n} ${word}${n === 1 ? "" : "s"}`;

for (const file of readdirSync("fixtures").filter((f) => f.endsWith(".log")).sort()) {
  const r = readLog(new Uint8Array(await Bun.file(`fixtures/${file}`).arrayBuffer()));
  const u = r.unreadable.length;
  assertTotals(r.lines, r.readable, u); // a page whose numbers do not add up is a build error

  const warning = u > 0
    ? `<section class="warning" data-warning role="alert">
<p><b>${u} of ${plural(r.lines, "line")} could not be read.</b> The counts below describe only the ${r.readable} readable lines.</p>
</section>
` : "";
  const events = Object.entries(r.events)
    .map(([k, n]) => `<tr data-event="${k}"><td>${k}</td><td>${n}</td></tr>`).join("\n") +
    `\n<tr data-event="unknown"><td>unknown events</td><td>${r.unknownEvents}</td></tr>`;
  const reasons = [...r.reasons].sort((a, b) => a[0].localeCompare(b[0]))
    .map(([why, n]) => `<li data-reason-count="${n}">${n} &times; ${esc(why)}</li>`).join("\n");
  const bad = r.unreadable
    .map((b) => `<li data-line="${b.line}" data-kind="${esc(b.kind)}">Line ${b.line}: <span class="kind">${esc(b.kind)}</span> <span class="msg">${esc(b.message)}</span></li>`)
    .join("\n");
  const html = `<!doctype html>
<html lang="en"><head><meta charset="utf-8"><title>edge-svc deploy log health</title>
<link rel="stylesheet" href="health.css"></head>
<body><main data-state="${u > 0 ? "incomplete" : "complete"}">
<h1>edge-svc deploy log health</h1>
<p class="totals" data-lines="${r.lines}" data-readable="${r.readable}" data-unreadable="${u}"><span>${plural(r.lines, "line")}</span><span>${r.readable} readable</span><span>${u} unreadable</span></p>
${warning}<h2>Events</h2>
<table>
${events}
</table>
<h2>Rollback reasons</h2>
<ul>
${reasons || "<li>none</li>"}
</ul>
${u > 0 ? `<h2>Unreadable lines</h2>\n<ol class="unreadable">\n${bad}\n</ol>\n` : ""}</main></body></html>
`;
  await Bun.write(`dist/${file.replace(/\.log$/, ".html")}`, html);
  console.log(`${file} -> dist/${file.replace(/\.log$/, ".html")} lines=${r.lines} readable=${r.readable} unreadable=${u}`);
}
await Bun.write("dist/health.css", await Bun.file("health.css").text());

Things to notice in build.ts:

Supporting files:

{
  "name": "edge-svc-log-health",
  "private": true,
  "type": "module",
  "scripts": { "build": "bun build.ts" }
}
body { font: 16px/1.5 system-ui, sans-serif; margin: 2rem; }
.totals span { margin-right: 1rem; font-weight: 600; }
table { border-collapse: collapse; }
td, th { padding: .15rem .75rem .15rem 0; text-align: left; }
.warning { border-left: 4px solid #b4462f; background: #fbeeea; padding: .5rem 1rem; margin: 1rem 0; }
.unreadable li { margin: .25rem 0; }
.kind { font-weight: 600; }
.msg { color: #555; }

The two fixtures

A fixture is a small input file made for a test. The checker builds two of them and writes them into fixtures/.

clean.log has six lines from a real deployer run. Version v2 was promoted. Version v3 was rolled back with not ready after 1500 ms. Version v4 was rolled back with candidate exited with exit status: 78 before ready.

damaged.log has twelve physical lines, six readable and six damaged in six different ways. Among the readable ones is a line with an extra key host, as a newer deployer might write, and a line with a new event name paused. The six damaged lines are on lines 3 to 8:

linewhat is wrongkind
3a line cut off mid-writetruncated
4a stray panic messagenot json
5an empty lineblank
6the detail key is missingschema
7ts is the string "yesterday"schema
8bytes that are not UTF-8not utf-8

Line 8 begins with the two bytes 0xFF 0xFE. Those bytes cannot be shown as text, so this lesson does not print the fixture file.

The built page

This is the page the reference builds from damaged.log:

$ cat dist/damaged.html
<!doctype html>
<html lang="en"><head><meta charset="utf-8"><title>edge-svc deploy log health</title>
<link rel="stylesheet" href="health.css"></head>
<body><main data-state="incomplete">
<h1>edge-svc deploy log health</h1>
<p class="totals" data-lines="12" data-readable="6" data-unreadable="6"><span>12 lines</span><span>6 readable</span><span>6 unreadable</span></p>
<section class="warning" data-warning role="alert">
<p><b>6 of 12 lines could not be read.</b> The counts below describe only the 6 readable lines.</p>
</section>
<h2>Events</h2>
<table>
<tr data-event="candidate_started"><td>candidate_started</td><td>2</td></tr>
<tr data-event="promoted"><td>promoted</td><td>1</td></tr>
<tr data-event="rolled_back"><td>rolled_back</td><td>2</td></tr>
<tr data-event="unknown"><td>unknown events</td><td>1</td></tr>
</table>
<h2>Rollback reasons</h2>
<ul>
<li data-reason-count="1">1 &times; candidate exited with exit status: 78 before ready</li>
<li data-reason-count="1">1 &times; not ready after 1500 ms</li>
</ul>
<h2>Unreadable lines</h2>
<ol class="unreadable">
<li data-line="3" data-kind="truncated">Line 3: <span class="kind">truncated</span> <span class="msg">object never closed; the line ends after 66 characters</span></li>
<li data-line="4" data-kind="not json">Line 4: <span class="kind">not json</span> <span class="msg">not a JSON value</span></li>
<li data-line="5" data-kind="blank">Line 5: <span class="kind">blank</span> <span class="msg">empty line</span></li>
<li data-line="6" data-kind="schema">Line 6: <span class="kind">schema</span> <span class="msg">missing key `detail`</span></li>
<li data-line="7" data-kind="schema">Line 7: <span class="kind">schema</span> <span class="msg">`ts` must be an unsigned integer, got string &quot;yesterday&quot;</span></li>
<li data-line="8" data-kind="not utf-8">Line 8: <span class="kind">not utf-8</span> <span class="msg">bytes are not valid UTF-8</span></li>
</ol>
</main></body></html>

Check it against the invariant: 12 lines, 6 readable and 6 unreadable. The unreadable list names each of the six lines with its number. The two rollback reasons from the readable lines both appear. unknown events shows 1, which is the paused line. The host line is counted like any other, and nothing marks it as odd. Line 7's message contains the quotation marks from the log value as &quot;, which is esc at work.

Limits

Lab

Copy the starter files report.ts, build.ts, health.css and package.json into an empty folder and put check.sh beside them. Bun must be installed. The checker needs nothing else and uses no network: it writes the two fixtures itself, runs bun run build, and reads the pages it wrote into dist/.

Task: make the starter's page account for every line. In order:

  1. Read the file as bytes and split it on 0x0A.
  2. Decode each line with a fatal UTF-8 decoder.
  3. Classify each line into one bucket.
  4. Add assertTotals and call it.
  5. Add the warning block and the unreadable list to the page.

Run the checker with:

$ sh check.sh

It has nine checks and prints one RESULT line for each:

checkwhat it looks at
static_buildthe build runs and writes a page for each log
clean_page_countsthe clean page shows the right event counts
clean_no_warningthe clean page has no warning block
damaged_totalsthe damaged page shows 12, 6 and 6
totals_add_upreadable plus unreadable equals lines, read back from the built HTML
unreadable_lines_listedeach damaged line appears with its number
warning_when_damagedthe damaged page has a warning
newer_writer_stays_readablethe host and paused lines stay readable and counted
invariant_fails_buildassertTotals throws on 12 lines, 6 readable and 5 unreadable

The checker reads the built HTML and compares figures from the data-* attributes and the listed rows. It proves that assertTotals throws on a wrong total, and that the build prints the three totals. It does not tamper with the reader in the middle of a build. The diagnostics below show what that looks like, from a one-off scratch edit that is not part of the checker.

The output below was produced in a clean container from the oven/bun:1.3-debian image with networking disabled (Bun v1.3.14, Linux arm64). The reference passes all nine:

$ sh check.sh
clean.log -> dist/clean.html lines=6 readable=6 unreadable=0
damaged.log -> dist/damaged.html lines=12 readable=6 unreadable=6
built 2 pages
RESULT static_build PASS
clean page: totals=6 6 0 promoted=1 rolled_back=2
RESULT clean_page_counts PASS
RESULT clean_no_warning PASS
damaged page: totals=12 6 6
RESULT damaged_totals PASS
RESULT totals_add_up PASS
RESULT unreadable_lines_listed PASS
RESULT warning_when_damaged PASS
damaged page: rolled_back=2 unknown=1
RESULT newer_writer_stays_readable PASS
invariant threw: totals do not add up: 12 lines but 6 readable + 5 unreadable
RESULT invariant_fails_build PASS

The starter passes static_build and clean_page_counts and fails the other seven:

$ sh check.sh   # starter
clean.log -> dist/clean.html lines=6
damaged.log -> dist/damaged.html lines=6
built 2 pages
RESULT static_build PASS
clean page: totals= promoted=1 rolled_back=2
RESULT clean_page_counts PASS
RESULT clean_no_warning FAIL
damaged page: totals=
RESULT damaged_totals FAIL
RESULT totals_add_up FAIL
RESULT unreadable_lines_listed FAIL
RESULT warning_when_damaged FAIL
damaged page: rolled_back=2 unknown=
RESULT newer_writer_stays_readable FAIL
RESULT invariant_fails_build FAIL
SyntaxError: Export named 'assertTotals' not found in module '/lab/report.ts'.
Bun v1.3.14 (Linux arm64)

Diagnostics

Symptom: the page says 6 lines for a twelve-line file.

Symptom: a line is listed under the wrong kind, or is missing from the unreadable list.

Symptom: the build stops with totals do not add up.

$ bun run build
$ bun build.ts
clean.log -> dist/clean.html lines=6 readable=6 unreadable=0
error: totals do not add up: 12 lines but 5 readable + 6 unreadable
error: script "build" exited with code 1
exit=1

The clean page was written first, then the damaged log failed the build, so no damaged.html exists. Twelve lines, five readable and six unreadable leaves one line in no bucket: the paused line. (The full run also prints a source excerpt and a stack trace, omitted here.)

Symptom: the checker prints SyntaxError: Export named 'assertTotals' not found.

Stretch exercises

These are for you to try. The lesson does not claim results for them.

  1. Add a hostile.log fixture. The reference page never prints version, so putting <script> there tests nothing. Put it in the detail of a rolled_back line, where it reaches the page as a rollback reason, or in a string ts value. Check that the built page contains &lt;script&gt;.
  2. Add a third fixture whose last line has no final newline. Predict the three totals first, using the start < bytes.length line in splitLines, then check them.

Other paths

Gleam with Lustre (hypothetical sketch, not compiled, not run). In TypeScript the guarantee that every line lands in one bucket rests on discipline: each branch has to return, and assertTotals catches mistakes afterwards. Gleam has custom types, so the same idea can be moved into the type of the parser:

// Sketch only. Not compiled, not run.
import gleam/list

pub type Kind {
  Truncated
  NotJson
  Blank
  Schema
  NotUtf8
}

pub type Line {
  Readable(event: String, detail: String)
  Unreadable(kind: Kind, message: String)
}

// Must return one of the two variants for every input.
pub fn parse_line(raw: BitArray) -> Line {
  todo
}

pub fn counts(raw_lines: List(BitArray)) -> #(Int, Int, Int) {
  let parsed = list.map(raw_lines, parse_line)
  let readable =
    list.filter(parsed, fn(l) {
      case l {
        Readable(..) -> True
        Unreadable(..) -> False
      }
    })
    |> list.length
  let total = list.length(parsed)
  #(total, readable, total - readable)
}

The design claim, which is a claim and not a result of running anything: because parse_line has to return a Line for every input, a line that belongs to no bucket cannot be written. The counts are lengths over one list, so readable + unreadable = lines holds by construction, though total - readable makes that nearly tautological. What the types add is that the parser, not the counter, decides each line's bucket. The Lustre documentation says Lustre can be used "as a HTML templating library, without using its runtime features, by passing elements to functions like to_string_tree or to_document_string", and that to_document_string "prepends a doctype declaration". That is the static export path. The cited page does not say whether text escapes <, and it says unsafe_raw_html is not escaped. Escaping would have to be confirmed before relying on it, and unsafe_raw_html should never be given a string from a log.

Citations