Correcting Long-Term Memory in AI Agents: Making Wrong Notes Refutable

The Problem in Abstraction: Wrong Premises Corrupt New Observations

Consider an abstracted case. An agent exploring a game-like environment has a navigation tool and a route-memory catalog. It needs to pass through a regional landmark to reach the next target city. The resupply point is actually located near the western entrance of the landmark, but the agent’s notes place it on the east side. Following that note, exiting the resupply point appears equivalent to having passed the landmark, so the agent declares “the landmark is behind us” and updates the note to “landmark completed.”

Later, it crosses a one-way staircase and falls back onto the path it came from. Since the staircase cannot be climbed back directly, the “already completed” judgment is clearly suspect. But the navigation tool does not show this elevation change, so the agent interprets returning to the old path as “having overshot to the west.” In its explanatory framework, the earlier position was correct — it simply walked too far afterward. The wrong landmark note determined the route, the wrong route’s result was explained away as an overshoot, and the landmark note survived. New observations came in, but the premises used to interpret them never changed.

This case exposes not a simple “retrieval failure” but three deeper engineering problems:

1. Lost provenance: notes keep only conclusions, not how those conclusions were reached;
2. Compressed conditions: temporary workarounds and one-off observations get compressed into general rules;
3. Explanatory absorption of evidence: new evidence fails to trigger contradiction handling and is instead reinterpreted through the old premise.

Long-term memory therefore cannot be just “summarize and archive.” Every stored experience should be managed as a verifiable, revisable hypothesis.

Raw Records Must Remain Queryable — Summaries Can’t Do It All

Many systems store only a summary when writing memory, such as “landmark X completed” or “tests must set the variable.” Summaries are readable, but they lose provenance, conditions, and context. When a contradiction later appears, the system cannot answer: Which tool call produced this experience? What user constraints applied at the time? Did the agent’s own interpretation contribute to the conclusion?

A more robust approach is to store raw records in layers, distinguishing at least four types of information:

• User constraints: explicit restrictions from the user, e.g., “no code changes allowed this session”;
• Tool inputs: the queries, parameters, and actions the agent actually passed to tools;
• Tool results: the raw results, error codes, and observations returned by tools;
• Agent interpretation: the agent’s inferences and attributions — always marked as interpretation, not fact.

An illustrative data structure:

memory_record = {
  claim: "The resupply point is east of the landmark",
  source: [
    "tool_result: navigation did not show the one-way elevation change",
    "observation: leaving the resupply point led onto the east path"
  ],
  conditions: ["one-way staircase near the resupply point", "navigation tool does not show elevation"],
  agent_interpretation: "leaving the resupply point means the landmark has been crossed",
  confidence: 0.7,
  status: "active"
}

Summaries can be generated separately, but they must not replace raw records. Summaries serve fast retrieval; raw records serve auditing and error correction. Without raw records, downstream processes see only conclusions and cannot verify how an experience was derived.

Experiences Must Carry Applicability Conditions and Provenance

Temporary workarounds are the most likely to harden into wrong rules. For example, in one task the user says “no code changes this time,” so the agent works around a problem by setting a variable. If that experience gets compressed into “tests must set the variable,” a one-time workaround becomes a general rule; in the next task, without that constraint, the agent may still route around the issue with the variable instead of fixing the root cause.

Memory entries should therefore include at least:

• Claim: what is being asserted;
• Provenance: whether it came from a user instruction, tool result, documentation, or the agent’s own inference;
• Scope of validity: which environments, tasks, and constraints it holds under;
• Confidence: how certain the conclusion is;
• Invalidation conditions: what evidence should trigger re-evaluation.

An update strategy in pseudocode:

function reconcile(note, evidence):
    if evidence.type == "absent":
        return note  # not mentioned in a single session — keep for now
    if evidence.type == "contradiction":
        if evidence.strength >= threshold:
            note.status = "contradicted"
            note.confidence = max(0, note.confidence - 0.5)
        else:
            note.status = "needs_review"
    if evidence.source_quality == "low":
        note.confidence = note.confidence * 0.8
    return note

The point of this pseudocode is not the specific algorithm but the handling principles: silence, contradiction, and vague sourcing must each be treated differently. If a single session doesn’t mention something, keep the note; if clear contradicting evidence appears, correct it or flag the conflict; if the source is uncertain, lower the note’s certainty. Treating all incoming information as “supporting the original conclusion” is the most dangerous failure mode a memory system can degrade into.

Passing Tests Isn’t Enough — Check Which Premises Were Inherited

In software tasks, passing tests is often treated as proof of correctness. But both old and new notes can make tests go green. If the agent inherited a wrong premise, the tests only verified that things run under that premise — not that the premise itself is correct.

During verification, check three things:

1. What it looked up: which notes were retrieved, and whether stale entries were hit;
2. How it judged: which observations its conclusions rested on, and whether interpretations were treated as facts;
3. What it executed: whether actions only hold under old conditions, and whether alternative paths were ignored.

A stricter practice is to add counterfactual checks: if a particular old note is removed, does the agent still reach the same conclusion? If it is replaced with the opposite note, does the system detect a conflict? If only the correct note is placed in context, does the task succeed? These questions separate “happened to pass” from “passed based on correct premises.”

Distinguish Retrieval Failures from Usage Failures

When a memory system misbehaves, two common failure modes are easily conflated:

• Retrieval failure: the correct note exists but was not recalled;
• Usage failure: the note was recalled, but the agent misread its conditions or chose the wrong action.

To diagnose, place the correct note directly into context and rerun the same task. If the task now succeeds, the problem likely lies in the retrieval pipeline; if it still fails, the problem likely lies in condition understanding, conflict handling, or action selection. This controlled experiment is cheap but significantly narrows the search space.

Building a Correctable Memory Loop

The engineering loop for long-term memory can be summarized as: write, retrieve, execute, verify, revise. Preserve provenance and conditions at write time; return confidence and invalidation conditions at retrieval time; record actual actions and observations after execution; check the premises behind conclusions during verification; and update by evidence-based grading at revision time rather than simple appending.

The ultimate goal is not to make the agent “remember more,” but to give it the capacity to abandon a prior judgment when facing new evidence. Only a note that carries provenance, conditions, confidence, and a contradiction-handling mechanism can ever be overturned. Otherwise, the more the agent remembers, the more often wrong premises get re-cited — and the harder it becomes for the agent to find its way out of old paths.