Hidden Memory Pollution in AI Agents: Auto-Mounted Wrong Notes and Test Blind Spots

As large language model agents gain more autonomy, memory systems grow more complex—and a silent failure mode appears: incorrect memories that are automatically mounted into every new session. A recent research-lab write-up on agent memory components described this “memory pollution” and noted that most existing memory test cases miss it. This article examines the technical nature of the problem and how to build tests that intercept the spread of bad memories.

Four-layer memory architecture and the pollution source

Modern agent memory is often described in four layers:

1. Session logs: full context of a single interaction; the basis of short-term memory.
2. Memory notes: key items distilled from session logs for cross-session experience.
3. Structured representations: wiki-like long-term facts, stored as graphs or documents.
4. Injected context: environment or experience automatically loaded into the prompt at the start of every new session.

The most dangerous layer is injected context. It is loaded unconditionally as a default. Once it contains errors, the effect is global. The more automated the system, the faster errors spread.

Case study: lifetime of a wrong memory

In a classic experiment, an agent played an exploration RPG and had to remember a cave location. While writing notes it hallucinated: “the cave is east of the building” became “east of the cave.” After leaving the building it inferred “the cave is behind me,” then wrote “cave already cleared.” It then looped on a one-way stair, jumped three times, gave up, and recorded “I decided to stop rather than keep looping.”

People also err; that is not the core issue. The crisis is that “cave already cleared” was written into injected context. On every restart the agent loaded that instruction and skipped the cave. It never reached the next city.

Researchers later added an offline “dreaming” pass: idle rereading of logs and comparison of notes against original context. Dreaming found the contradiction and restored the correct location. With the fix, the agent entered the cave and progressed. Without it, the run stayed in a dead loop.

Test paradigm shift: from accuracy to survival and spread

Traditional memory tests ask whether the agent remembered a fact correctly. Hallucinations cannot be fully eliminated. The useful questions become: How long can a wrong fact survive? How far can it spread?

Typical cases only check that writes to the note store match expectations. They ignore whether inconsistent data is intercepted, isolated, or corrected. That is why most memory suites miss the failure.

Practice: pollution-resistant memory test gates

Accuracy checks must be extended into a full anti-pollution suite. Four dimensions and an engineering gate:

Dimension 1: Injected-context volume and duplication

Volume sets the scale of pollution. Automate checks on character or token count of injected context at session start. Abnormal growth or highly duplicated historical notes suggests failed merge and stacking of redundant or wrong information. Log injected token counts; alert if they exceed a threshold (for example, 20% of the total context window).

Dimension 2: Error lifetime and isolation

Inject “poison-pill” data (wrong clear status, nonexistent file paths) and start multiple new sessions.

• Goal: See whether the pill is loaded unconditionally and after how many rounds self-correction finds and cleans it.
• Gate: If the pill is not intercepted for three consecutive new sessions and reaches the execution layer (for example, causing abnormal exit), treat it as a severe defect.

Dimension 3: Effectiveness of offline reconciliation (“dreaming”)

Reconciliation is the main correction line. Simulate idle review:

1. Construct contradictory session logs and memory notes.
2. Trigger the offline pass.
3. Verify that contradictions are detected and notes are rolled back or corrected using logs as authority.

Cover time-series state jumps so reconciliation does not introduce new conflicts.

Dimension 4: Boundary tests for layered writes

Not every fact belongs in globally loaded injected context. Cover routing with this table:

Information type	Expected layer	Forbidden layer	How to verify
Temporary exploration path	Session logs	Injected context	Global load list must not contain temp variables
Fixed steps of a completed task	Memory notes	Session logs	Reusable across sessions without polluting current logs
Objective facts (e.g. API docs)	Structured representations	Memory notes	Notes must not rote-memorize API parameters
Environment init config	Injected context	Session logs	Every start correctly identifies environment state

Cloud-native external defenses

Enterprise agents often face huge codebases or complex workflows. Agent-internal tests are not enough. Use infrastructure as an outer line: object storage and version control for note snapshots so every update is traced; message queues to push write/update events to an independent audit service. If a rule engine finds a severe conflict between a note and the business knowledge graph (structured representations), block promotion of that note into injected context via API and cut the spread path at the architecture level.

Conclusion

Agent memory is not only a store; it is the engine of autonomous action. Globally mounted wrong notes are impurities in that engine; damage grows exponentially with sessions. Move beyond “memory accuracy” to a system of volume gates, lifetime checks, offline reconciliation tests, and layer-routing boundary tests. Treating “how long an error lives and how far it travels” as a first-class metric is what makes agents industrially robust.