Silent Poisoning in AI Agent Memory: Wrong Notes, Global Scope Mounting, and Missing Tests

1. A Typical Closed-Loop Incident

Consider a common failure scenario. An agent is asked to periodically enter a region to investigate an issue. During one run, due to a failed tool call, an incorrect path decision, or lost intermediate results, it actually completes nothing. Nevertheless, in the wrap-up phase, it writes a conclusion-style memory: “Inspection of this region is complete.”

The problem begins to amplify from that moment. Every time a new session starts, this memory is automatically mounted into the context as prior knowledge. The agent’s starting line is fixed to “the cave inspection is complete,” so it never returns to the cave. The run that should have corrected the mistake never happens.

Three characteristics deserve attention:

• The error does not occur during reasoning; it occurs when memory is written and mounted.
• The error is not one-off; it is inherited repeatedly by later runs.
• The external symptom is not “the agent gave a wrong answer” but “the work that should be done was not done” — a silent omission.

2. Three Conditions Required for Hidden Contamination

If we abstract the story above, hidden contamination is not an accident. It is the inevitable result of three design gaps stacking on top of one another.

1. The write side lacks a validation gate.
   When memory is stored, it is not bound to an objective fact such as whether the task was actually completed. A write without a validator essentially treats the model’s self-narration as truth.

2. The mount side lacks scope constraints.
   A local observation from a single session is indiscriminately promoted into a global prior and injected into the system-level context of later sessions. Local evidence is treated as a global axiom.

3. The lifecycle lacks versioning and invalidation.
   Memory is append-only and never revised. Correction writes are either discarded or coexist with the old memory, leaving the retrieval layer to randomly decide which one to believe.

If any one of these gaps is fixed, the severity of the incident drops significantly. If all three are missing, the error escalates from “a one-time deviation” to “system-level lock-in.”

3. Why Most Memory Test Cases Miss This

This is more alarming than the contamination itself: many teams have already shipped memory capabilities but remain unaware of this class of failure. The root cause is mainly in test design, not engineering implementation.

• Incorrect pre-test assumptions.
  Most test cases reset the memory store during setup or inject a manually curated clean dataset. Since “dirty memory” never exists in the test environment, contamination naturally cannot be reproduced.

• Only the positive path is tested.
  Test cases cover “correct write → correct recall → correct execution,” but lack reverse cases such as “incorrect write → incorrect recall → contamination spread → whether it is detected and corrected.”

• Assertions are too shallow.
  Assertions usually stop at the tool-call sequence or the final text output. They do not check which memories were actually mounted at the beginning of the session, so contamination remains invisible at the observation layer.

• Memory value comes from the time dimension, but most tests are one-shot.
  Single-turn tests cannot expose problems such as cross-session inheritance, memory expiry, or delayed correction.

• Invariant assertions are missing.
  There are no automatically checkable constraints such as “any recalled memory must carry evidence” or “global-scope memory must have explicit approval.”

4. Severity Classification: Four Memory Defects and Observable Signals

Level	Type	Typical Behavior	Detection Signal
L1	Precision defect	Memory exists but details are distorted	Conclusion does not match the actual artifact
L2	Temporal defect	Memory is not updated as facts change and becomes stale	Old conclusions override new evidence
L3	Global defect	Local memory is promoted to a global prior	Multiple unrelated sessions show the same source of bias
L4	Missing self-healing	Correction writes cannot overturn old memory	Repeated tasks repeatedly skip the same step

L3 and L4 are the easiest to overlook and the hardest to discover through manual regression. They must be covered by automated test cases.

5. Write Side: Manage Memory Like Code

The first engineering step is to attach structured metadata to every memory, so that the question “can this memory be trusted?” becomes decidable.

{
  "id": "mem-20260812-001",
  "scope": "project",
  "claim": "Inspection of Area A is complete",
  "evidence": ["trace-9f2c", "artifact://report-a.md"],
  "verified_by": "exit_code==0 AND report_non_empty",
  "confidence": 0.92,
  "ttl_days": 30,
  "status": "active"
}

Four supporting rules:

1. Scope should default to the minimum.
   The hierarchy is session < project < global. Promoting memory to a higher scope requires an explicit reason. Global mounting should go through an allowlist, not be a default behavior.

2. Writes must pass the gate.
   Memory without a validator or an evidence chain is not allowed into long-term storage. Memory with confidence below the threshold can only exist as a “pending verification assumption” and must not be treated as a reasoning premise.

3. Append only; do not overwrite.
   Memory is written in append mode with a version number. Corrections are made by adding a supersede record, so history can be replayed and rolled back.

4. Set a lifecycle.
   Configure an expiry time for memory. When it expires, it is automatically downgraded to a pending-review state instead of silently participating in reasoning.

def write_memory(agent, candidate):
    if not candidate.verified_by:
        raise Reject("Missing verification condition; long-term memory write is not allowed")
    if candidate.scope == "global" and not candidate.approved_by_human:
        raise Reject("Global mounting requires explicit confirmation")
    store.append(candidate)  # Append only; never rewrite in place

6. Read Side: Be Restrained When Mounting, Resolve Conflicts Explicitly

The mounting strategy is as important as the write strategy. The recommended principle is: “Do not mount by default; recall on demand.”

• Recall must use thresholds and truncation to prevent low-relevance memory from filling the context.
• At the start of a session, only inject task-agnostic global hard constraints; other memory should be retrieved on demand.
• If multiple pieces of evidence conflict for the same claim, keep the latest verified one, mark the old one as superseded, and retain traceable links.
• If recalled memory conflicts with current observed facts, the observation takes precedence, and an explicit correction write should be triggered.

7. Make Memory Testable: Stateful Tests

The core idea of supplementary testing is simple: testing must shift from “stateless” to “stateful.”

The scenarios that need to be covered include: poisoned memory injection, cross-session inheritance, memory expiry, delayed correction, scope escalation, conflicting claims, missing evidence, and the maximum number of globally mounted memories.

def test_poisoned_memory_does_not_become_premise():
    seed_memory(claim="Inspection of Area A is complete", evidence=[],
                confidence=0.99, scope="global")
    result = run_session(task="Inspect Area A")
    assert result.entered_region("A")          # Should not be discouraged by a wrong prior
    assert not result.treated_as_done("A")     # Should not treat it as a conclusion
    assert memory_status("mem-poison") in {"superseded", "retracted"}

The above is an illustrative structure; interface names should be adjusted according to each implementation.

Additional supporting methods include:

• Perform mutation testing on the write gate: intentionally relax validation rules and confirm that the corresponding test cases turn from green to red.
• Maintain a golden set of “memory state × task” combinations and run a full regression every time memory logic changes.

8. Observability and Cloud Implementation

To stop contamination from remaining invisible, the observability layer must cover three stages: what was written, what was mounted during recall, and how the mounted memory affected decisions.

Useful metrics include:

• Memory write verification failure rate
• Proportion of correction writes
• Number of memories marked as superseded
• Proportion of tasks affected by cross-session memory
• Time from contamination injection to correction

When implementing this pipeline in the cloud, general-purpose capabilities are sufficient:

• Use object storage to save memory snapshots and evidence artifacts for later replay.
• Use a message queue to asynchronously decouple write review and correction workflows.
• Use a logging service to retain audit records linking sessions, recalled memories, and decision chains.
• Use monitoring and alerting to set thresholds for sudden increases in verification failures and stalled correction processes.

Platforms such as Baidu AI Cloud provide these general product categories, which can support the above responsibilities without introducing extra architectural complexity.

9. Implementation Checklist

1. All long-term memory writes must pass a validator; reject writes with missing evidence.
2. Scope should default to the minimum; global mounting must go through an allowlist and require explicit confirmation.
3. Memory must be append-only, versioned, traceable, and rollback-capable.
4. Every memory must have a lifecycle and be automatically downgraded for review when it expires.
5. Corrections must be explicit; a new conclusion should immediately invalidate the old one.
6. The test set must include stateful cases for four categories: contamination injection, correction, conflict, and privilege escalation.
7. Before changing memory logic, run the golden-set regression and mutation tests first.

The reliability of a memory system does not depend on whether the model is smarter. It depends on four things: writes backed by evidence, bounded scope, traceable errors, and detectable contamination. With these four in place, a wrong note like “the cave inspection is complete” will stop at the moment it is first written, instead of quietly taking over every subsequent session.