English · Español

02 — Memory: scratchpad vs long-term

🇪🇸 La palabra "memoria" en literatura de agentes confunde dos cosas muy distintas: lo que el agente está pensando ahora (scratchpad, efímero) y lo que el agente quiere recordar entre conversaciones (long-term, persistente). Mantenerlas separadas evita errores estructurales.

Two kinds, kept apart

Kind	Lifetime	Format	What it holds	Read by
Scratchpad	One correction	Append-only list of (thought, action, observation) triples	The current trace	The planner, every step
Long-term	Across corrections	Structured store (JSON file)	Aggregate facts: per-learner mistake counters, per-verb regularity overrides	Hand-fetched when relevant

The scratchpad is the agent's working memory. The long-term store is a tool the agent can read. Mixing them — e.g., dumping long-term content into the scratchpad prompt blindly — bloats the context and confuses the planner.

The scratchpad

Concrete shape (src/miniagent/memory.py):

@dataclass
class ScratchpadMemory:
    steps: list[Step] = field(default_factory=list)

    def append(self, thought: str, action: ToolCall, observation: ToolResult) -> None: ...

    def format_for_prompt(self) -> str:
        """Render the trace as the 'TRACE:' block in the planner prompt."""

    def clear(self) -> None:
        """Reset to empty. Called at the start of every correct() call."""

Lifecycle:

NEW → ACCUMULATING → FROZEN_FOR_ANSWER → DISCARDED

• NEW: fresh ScratchpadMemory(), zero steps.
• ACCUMULATING: planner appends after each tool call.
• FROZEN_FOR_ANSWER: planner is about to emit FinalAnswer; no more steps.
• DISCARDED: after the correction is returned, the scratchpad is dropped.

Critical: the scratchpad never survives a correction. If you accidentally make it a class-level attribute that persists across correct() calls, you'll see corrections "remember" the previous sentence's verb — confusing and wrong.

Why append-only

Two reasons:

1. Replay. With an append-only trace, you can replay the correction step by step for debugging or audit.
2. The planner sees consistent history. Mutating earlier steps mid-loop confuses the planner — it would re-condition on history that didn't actually happen.

Why structured, not free text

A naive "scratchpad" is the prose the model wrote between actions ("Let me think... I should call lookup_irregular_verb..."). We could use this, but it's lossy and easy to manipulate via prompt injection.

We use a structured trace: (thought_summary, action: ToolCall, observation: ToolResult). The thought_summary is a 1-2 sentence summary extracted from the planner's JSON if present, or empty. The action and observation are typed and validated. This makes the trace machine-readable — we can serialise it to JSON and reload it for replay or analysis.

Long-term memory

LongTermMemory is a small persistent store. Default backend: a JSON file at experiments/32-tutor-demo/longterm.json. (SQLite would be overkill at this scale.)

Three classes of fact get persisted:

@dataclass
class LongTermMemory:
    # Per-learner mistake counters.
    # Key: (learner_id, verb, error_type) → count
    mistake_counts: dict[tuple[str, str, str], int]

    # Per-verb regularity overrides (e.g., "spelled" vs "spelt" — both acceptable).
    # Key: verb → list of acceptable past forms
    verb_overrides: dict[str, list[str]]

    # Per-learner per-tense accuracy.
    # Key: (learner_id, tense) → {correct, total}
    accuracy_stats: dict[tuple[str, str], dict[str, int]]

These are small (Phase 32's scope is 20 verbs × 1 learner × 5 tenses), so the entire store fits in RAM and serialises to a few KB.

What does not go in long-term

• Raw sentences. Privacy + noise. Save aggregate stats, not user input.
• Free-text feedback. Hard to query, easy to drift. Force the structure.
• Embedding indexes / vector DBs. Phase 29's RAG is a separate retrieval tool the agent can call; it's not the agent's own memory. (See theory/00-motivation.md for why.)

Reading from long-term into the prompt

When the agent is corrects a sentence with verb V, it can choose to inject:

LEARNER HAS PREVIOUSLY ERRED ON THIS VERB 3 TIMES.
Most common error: incorrect past tense (used "goed" instead of "went").

This becomes part of the planner's system prompt for this correction. The injection is explicit and bounded — we never dump the entire long-term store. The bound is, say, the top-3 most relevant facts (by simple recency or count).

When to inject:

• The current correction involves a verb the learner has erred on before → inject the count.
• The current correction is in a tense the learner has low accuracy on → inject the stat.
• Otherwise → inject nothing.

This kind of selective injection is the real purpose of long-term memory in a tutoring agent: not "remember everything," but "personalise the explanation."

Persistence guarantees

The long-term store is append-or-update only. Mistakes can be incremented but not deleted (we want the history). Overrides can be added but not removed (deletion requires manual intervention).

On agent start: load longterm.json if it exists; otherwise empty. On correction end: update counters, save to disk.

For Phase 32, this is sufficient. A real production system would add: versioning, schema migrations, lock files for concurrent writes, backups. None of that is in scope.

A common pitfall: memory leak across corrections

It is very easy to write:

class GrammarTutorAgent:
    def __init__(self):
        self.scratchpad = ScratchpadMemory()  # <-- BUG: shared across corrections

    def correct(self, sentence):
        ...

This makes the scratchpad a class-level attribute, which means the second correction inherits the trace from the first. Result: the agent's planner sees stale context and behaves erratically.

The fix:

class GrammarTutorAgent:
    def __init__(self): ...

    def correct(self, sentence):
        scratchpad = ScratchpadMemory()  # fresh per call
        ...

tests/test_memory.py should explicitly test that two consecutive correct() calls produce independent traces. The test is dead simple but catches a critical bug.

Memory ≠ context window

A common confusion: "the model's context window is its memory." This is technically true at the LM-call level — but the agent loop turns the model into a stateful system by:

• Constructing the prompt from the scratchpad (which is real persistent storage between LM calls).
• Persisting long-term facts to disk between corrections.

The model is stateless; the agent is stateful. Memory lives in the loop, not in the model.

What this file does NOT cover

• Vector-DB retrieval as memory. That's Phase 29's RAG, available to the agent as a tool, not its core memory.
• Episodic vs semantic memory. Cognitive-science distinction. Out of scope.
• Memory consolidation, sleep cycles, etc. Out of scope (and somewhat speculative for LLM agents anyway).

Next: 03-sandboxing.md