Skip to content

English · Español

Lab 03 — Knowledge graph: visualising the 40 phases

🇪🇸 Un grafo donde los nodos son conceptos clave y las aristas son dependencias entre fases. Un futuro aprendiz puede leer el grafo y entender qué encaja con qué. Es la cartografía del proyecto.

Objective

Produce a single SVG file: docs/phase-40-hardening-postmortem/diagrams/knowledge-graph.svg. Nodes are concepts (one per phase, roughly); edges are dependencies ("Phase N's understanding requires Phase M"). Annotated so a stranger can read it in 5 minutes.

Setup

  • graphviz (dot) for rendering, or mermaid if you prefer (uv add or system install).
  • The 40 PHASE_NN_README.md files for the headline concept of each phase.
  • The "What this phase does NOT cover" sections, which often name dependencies on later phases.

Tasks

  1. Pick the node set. Aim for 30-50 nodes. Each node is a concept, not necessarily a phase. Many phases contribute multiple concepts; some phases contribute one. Examples:

  2. Phase 02 → "fp32/fp16/bf16 numerical representation" (one node).

  3. Phase 17 → "Pre-LN transformer block" + "tied embeddings" + "Mini-GPT assembly" (three nodes).
  4. Phase 21 → "Sampling: greedy/temperature/top-k/top-p" (one node).
  5. Phase 22 → "KV cache" (one node).

  6. Phase 33 → "FastAPI serving" + "continuous batching" + "Little's law" (three nodes).

  7. Pick the edge set. Edges are prerequisite relationships. If understanding concept B requires concept A, draw A → B. Examples:

  8. "fp32/fp16/bf16" → "quantization" (Phase 02 → Phase 26)

  9. "log-sum-exp" → "softmax sampling at temperature" (Phase 05 → Phase 21)
  10. "Pre-LN transformer block" → "KV cache" (Phase 17 → Phase 22)
  11. "KV cache" → "continuous batching" (Phase 22 → Phase 33)

Aim for ≤ 3 incoming edges per node. If a node has more, it's too central — split it into two concepts or push some prereqs to be "indirect" (i.e., go through an intermediate node).

  1. Render with dot:
digraph LynxCortex {
    rankdir=LR;
    node [shape=box, fontname="Helvetica", fontsize=10];

    // Foundations
    "Reproducibility (P00)" -> "Numerical Representation (P02)";
    "Numerical Representation (P02)" -> "Quantization (P26)";

    "Log-sum-exp (P05)" -> "Softmax sampling at τ (P21)";
    "Cross-entropy (P05)" -> "Training loop (P18)";

    "Tied embeddings (P17)" -> "LM head (P17)";
    "Pre-LN block (P17)" -> "Mini-GPT assembly (P17)";
    "Mini-GPT assembly (P17)" -> "Training loop (P18)";
    "Training loop (P18)" -> "Training dynamics (P19)";
    "Training loop (P18)" -> "Mini-GPT (P17)";

    "KV cache (P22)" -> "Continuous batching (P33)";
    "FastAPI serving (P33)" -> "Continuous batching (P33)";
    "Little's law (P33)" -> "Capacity sizing (P34)";

    // ... continue for ~30-50 nodes total
}

Render: 

dot -Tsvg knowledge-graph.dot -o knowledge-graph.svg

  1. Cluster the graph. Use subgraph cluster_X { ... } to group concepts by theme:

  2. Foundations (Phases 00-09): reproducibility, math, autograd.

  3. Tokenization & data (Phases 11-13): BPE, corpus design, embeddings.
  4. Sequence modeling (Phases 14-17): attention, positional, Mini-GPT.
  5. Training & evaluation (Phases 18-20): loops, dynamics, harness.
  6. Inference (Phases 21-22, 27): sampling, KV cache, modern attention.
  7. Optimization (Phases 23-28): GPU, CUDA, quantization, LoRA.
  8. Application (Phases 29-32): RAG, structured gen, tools, agents.

  9. Operations (Phases 33-39): serving, observability, distributed, security, MLOps.

  10. Render with annotations. Use label= on key edges to denote why the dependency exists:

"KV cache (P22)" -> "Continuous batching (P33)" [label="per-request cache"];
  1. Verify legibility. Open the SVG in a browser. Walk through it as if you'd never seen it before:
  2. Can you identify the entry points (foundational concepts, no incoming edges)?
  3. Can you identify the terminal nodes (capstone concepts, no outgoing edges)?
  4. Do clusters visibly group related concepts?

  5. Is any node isolated (no edges)? Either delete or fix.

  6. Link from the phase README. Add to docs/phase-40-hardening-postmortem/README.md:

![Knowledge graph](diagrams/knowledge-graph.svg)

Deliverable

docs/phase-40-hardening-postmortem/diagrams/knowledge-graph.svg — the rendered graph.

Plus the source: docs/phase-40-hardening-postmortem/diagrams/knowledge-graph.dot (or .mmd for mermaid).

Acceptance

  • 30-50 nodes; 50-100 edges.
  • Clustered into 5-8 logical groups.
  • Renders cleanly to SVG.
  • No isolated nodes.
  • A stranger could trace a path from "Reproducibility" to "Continuous batching" by following the arrows, with each step justified.

Pitfalls

  • Too many nodes. Past ~60, the graph becomes unreadable. Aggregate. ("Numerical hygiene" can be one node, not three.)
  • Too few edges. Concepts in an AI system are dense. If you have 50 nodes and 10 edges, you're under-connecting. Aim for ~2-3 edges per node on average.
  • Cycles. The graph should be acyclic (it's a curriculum dependency graph). If you find a cycle, it's a bug in your conceptualisation — break the cycle by being more precise about which sub-concept depends on which.
  • Rendering hell. Graphviz default layouts can be ugly. Try rankdir=LR (left-to-right) for curriculum-style graphs; use splines=ortho for clean orthogonal edges.
  • Conflating "phase" with "concept." A phase is a chapter; a concept is a notion. Some phases produce many concepts; some produce one.

Stretch

  • Add color to encode something useful: e.g., red for foundations, blue for inference, green for ops. Don't over-color — 5 categories max.
  • Add edge weights by, say, the strength of the dependency. Visually fades the weak ones.
  • Render twice: once with all nodes (the master), once collapsed by cluster (the executive summary). Link both from the README.

Next: 04-final-reflection.md