English · Español

Lab 01 — Render the dashboard as a self-contained HTML file¶

Goal: turn streaming stats into a single static HTML file with seven panels, no external assets.

Estimated time: 90-120 minutes.

Prereq: Lab 00 (Inspector) committed.

What you produce¶

A new file:

src/minitrain/dashboard.py — the renderer.

A first generated dashboard:

experiments/19-healthy/dashboard.html — Phase-18 healthy training, this time with the Inspector enabled.

Plus a sample diagram:

docs/phase-19-training-dynamics/diagrams/dashboard-layout.png — annotated screenshot of what the panels should look like (for the docs site).

TODOs¶

Block A — define the dashboard data structure¶

src/minitrain/dashboard.py consumes a "log file" written by the Inspector — a JSONL where each line is:

{
  "step": 0,
  "train_loss": 6.234,
  "val_loss": null,
  "loss_regular": 6.21,
  "loss_irregular": 6.34,
  "lr": 0.0,
  "grad_norm_pre": 5.21,
  "grad_norm_post": 1.0,
  "activations": {"layer_0": {"l2": 0.95, "mean": 0.01, ...}, ...},
  "spectral": {"layer_0.W_q": 1.23, ...},
  "dead_neurons": {"layer_0": 3, "layer_1": 8, ...},
  "dead_heads": {"layer_0": 0, "layer_1": 0, ...}
}

This log file is written by src/minitrain/loop.py with the Inspector active. The loss_regular / loss_irregular fields come from partition_batch_loss in src/minitrain/per_class_loss.py (Lab 00, Block D). If a batch has zero examples of one class, the corresponding field is null and the dashboard's Panel 7 interpolates over it.

Block B — render seven matplotlib panels¶

In src/minitrain/dashboard.py:

def render(log_path: Path, out_html: Path, config: dict, manifest: dict) -> None:
    log = read_jsonl(log_path)
    panels = []
    panels.append(render_panel_loss(log, config))           # Panel 1
    panels.append(render_panel_lr(log, config))             # Panel 2
    panels.append(render_panel_grad_norm(log, config))      # Panel 3
    panels.append(render_panel_activations(log))            # Panel 4
    panels.append(render_panel_spectral(log))               # Panel 5
    panels.append(render_panel_dead(log))                   # Panel 6
    panels.append(render_panel_reg_vs_irr(log))             # Panel 7

    # Each render_panel_* returns a base64-encoded PNG bytes blob.
    html = build_html(panels, config, manifest)
    out_html.write_text(html, encoding='utf-8')

Panel 7 plots loss_regular and loss_irregular as two lines on the same axes (log y-scale), with the gap τ = loss_irregular − loss_regular shaded between them. Annotate the final-step gap in the legend.

Each render_panel_* calls matplotlib, saves to an in-memory PNG via io.BytesIO, returns the base64-encoded bytes:

def fig_to_b64(fig) -> str:
    buf = io.BytesIO()
    fig.savefig(buf, format='png', bbox_inches='tight', dpi=100)
    plt.close(fig)
    return base64.b64encode(buf.getvalue()).decode('ascii')

Block C — assemble the HTML¶

The HTML template should be a single file with:

<style> block defining a responsive grid (e.g., 2 columns × 4 rows, with the last cell empty or holding the run summary). CSS Grid, no external framework.
Seven <img src="data:image/png;base64,..."> tags, one per panel.
A header summary box showing run name, config hash, manifest summary (seed, hardware, versions), final val perplexity.
A footer with a link back to the manifest.json (relative path).
No CDN. No external CSS. No JavaScript. Pure HTML + inline CSS + embedded PNGs.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Run: {{ run_name }}</title>
  <style>
    body { font-family: -apple-system, sans-serif; max-width: 1400px; margin: 1rem auto; }
    .panels { display: grid; grid-template-columns: repeat(2, 1fr); gap: 1rem; }
    .panel img { width: 100%; }
    /* etc. */
  </style>
</head>
<body>
  <header>...</header>
  <div class="panels">
    <div class="panel"><h3>1 Loss</h3><img src="data:image/png;base64,{{ panel_1 }}"></div>
    <!-- ... -->
    <div class="panel"><h3>7 Regular vs Irregular</h3><img src="data:image/png;base64,{{ panel_7 }}"></div>
  </div>
  <footer>...</footer>
</body>
</html>

Use plain Python string formatting or string.Template — no Jinja2 (extra dep).

Block D — generate the healthy dashboard¶

In experiments/19-healthy/:

Re-run Phase-18 training with --inspector enabled.
The training loop writes inspector.log.jsonl.
After training, call dashboard.render('inspector.log.jsonl', 'dashboard.html', config, manifest).
Open the resulting dashboard.html in Firefox AND Chromium. Verify all seven panels are visible and correctly labeled. Panel 7 should show the regular-line clearly below the irregular-line by ~step 500, with the gap narrowing by the end of training (the expected §A13 pattern).

Block E — annotate the diagram¶

Take a screenshot of dashboard.html and save to docs/phase-19-training-dynamics/diagrams/dashboard-layout.png. Add arrow annotations identifying the six panels (use GIMP, Inkscape, or just matplotlib's annotate to draw on top).

Constraints¶

Self-contained HTML. Open from a USB stick with no network. Tests this by literally firefox file:///path/to/dashboard.html.
No external CSS/JS. No Bootstrap, no Plotly, no Tailwind, no jQuery, nothing CDN-hosted.
Renders in both Firefox and Chromium. Browser-specific CSS is forbidden.

Stop conditions¶

Done when:

experiments/19-healthy/dashboard.html opens in both Firefox and Chromium with all seven panels visible.
Disconnecting the network does not change rendering.
The header shows the run name + config hash + final val perplexity.

Pitfalls¶

base64 makes the HTML file large. A typical dashboard is 200-500 KB. Acceptable. If it's > 2 MB, your panel PNGs are too high-resolution; drop dpi to 80.
Matplotlib font issues if a non-default font is used. Stick to the default (DejaVu Sans on Linux).
CSS Grid in old browsers. Fine for modern Firefox/Chromium; if you need to support older browsers, use flexbox. Not a Phase-19 concern.
JSONL parsing speed. If logs get huge (e.g., for the overfit run with 8000 steps and many logging fields), naïve for line in file: json.loads(line) is fine but verify it parses in <2 s.

When to consult `solutions/`¶

After the healthy dashboard renders correctly in both browsers. The solution at solutions/01-build-dashboard-ref.md (written at phase open) discusses the choice of matplotlib vs Plotly vs Bokeh and why we land on matplotlib.

Next lab: lab/02-break-it.md.