Agent Long-Term Memory: Background Replay and Experience Correction

Cross-session long-term memory is what lets an agent move from a one-shot Q&A tool toward a system that can collaborate over time. Real long-term memory is not an ever-growing paste of past dialogue into the context window. It is the use of idle compute, between interactions, to replay, reorganize, correct, and consolidate history. This article describes a background “dreaming” memory architecture: its layers, boundaries, critical-event marking, learning loop, and engineering requirements.

1. The Core Problem: Memory Is Experience Governance, Not Context Stacking

Everyday agent work produces a large interaction log: user preferences, task constraints, verified facts, failed attempts, tool results, and temporary hypotheses. If the agent relies only on the current session, that information disappears when the session ends. If history is simply compressed into a summary, wrong assumptions, stale facts, and important details tend to mix together and pollute memory.

A cross-session memory system therefore has to govern three things:

1. What to keep: which observations deserve to persist across sessions, and which are only temporary context.
2. How to correct: when later evidence conflicts with an older note, how to mark, replay, and update it.
3. How to inject: how to recall what the next task needs, instead of stuffing every memory back into the context.

The architecture in the source material splits memory into raw records, memory notes, memory representations, and injected context, with an offline “dreaming” process running across those layers. The essential idea is to separate writing memory from organizing it: the live session captures; the offline cycle reflects, checks, and reorganizes.

2. Layered Architecture: Four Memory Layers and Continuous Offline Replay

A practical agent memory system can be abstracted into four layers.

2.1 Raw interaction logs: an immutable record

Raw logs store task inputs, tool returns, user feedback, and answers the system publishes. The model should not rewrite them freely. They are the factual baseline for audit and replay. When a memory note is disputed, the system can return to the raw log and check it. In engineering terms, logs should keep timestamps, session identifiers, sources, sensitivity levels, and traceable references.

2.2 Memory notes: readable, editable experience

Memory notes are the agent’s distillation of past experience—for example, “this kind of query should check field meanings first,” “this user prefers tabular output,” or “this rule does not hold under certain conditions.” Notes should separate confirmed details from unconfirmed ones and attach evidence references. Compared with raw logs, notes are shorter and more reusable, but they are also easier to pollute with bad generalizations, so they need versioning and conflict markers.

2.3 Memory representations: index and compression

A representation is not a plain summary. It is an index built for recall: topic tags, entities, task types, confidence, freshness, and links to evidence. It helps the system decide quickly which notes matter for a later task. The layer may use vector indexes, keyword indexes, or structured metadata, but it must not replace the original evidence.

2.4 Injected context: recall on demand

The injection layer selects a small set of highly relevant, high-confidence memory fragments for the current task and turns them into plain-text context for the model. The policy has to limit noise and conflict: prefer verified notes, keep counterexamples, and avoid treating expired experience as a hard rule. Injected content should state its source and confidence so the model can tell facts from experience during reasoning.

“Dreaming” runs through all of these layers. During idle time, the system replays recent logs, checks whether notes match the evidence, merges duplicate experience, marks conflicts, writes new notes or revises old ones, and updates the representation index. It does not change model weights. It governs experience in an external memory space.

3. Memory Boundaries: Exclude Weight Updates, In-Session Learning, and Summary Replacement

Here, agent memory means experience gained from tasks and user interaction, retained across sessions, and later read back as plain-text context. It strictly excludes three mechanisms:

• Model weight changes: this is not fine-tuning or a parameter update.
• In-context learning within a single conversation: temporary adaptation that vanishes when the session ends is not long-term memory.
• Replacing old dialogue with a compressed summary: a summary can help, but it cannot replace traceable memory governance.

A clear boundary makes the system easier to build. Memory should exist as an external, auditable component, not as a vague claim that the model itself “has” long-term memory.

4. A Cue from Human Sleep: Replay, Marking, and Consolidation

In the moment, the hippocampus quickly captures experience and holds it as temporary memory. During sleep and rest, it replays recent experience, extracts what matters, and integrates it into the cortex as longer-term knowledge. Emotional or chemical signals can mark a memory at a critical moment so it gets more replay.

Mapped onto an agent, the live session is capture and temporary storage; offline dreaming is replay and consolidation. During a session, the agent can tag critical events or write them into an external memory catalog so the offline cycle treats them first. The hard part is knowing, in the moment, which event will matter later. Too many marks make replay expensive; too few can drop important experience. A workable approach combines several signals: explicit user feedback, task success or failure, contradictory tool results, constraints that keep recurring, and hypotheses corrected by an answer the system later publishes. A mark is not a final conclusion. It only raises offline replay priority.

5. A Note Is a Prediction: Closing the Learning Loop

Writing a note is itself a prediction about the future. A rule predicts that the pattern will be useful again. A verified fact predicts that a later call will not need to recheck it. Memory should therefore form a predict–verify–revise loop:

1. During the live task, record observations and hypotheses.
2. During offline replay, test those hypotheses against later evidence.
3. Promote verified content to high-confidence notes.
4. Keep unverified content as hypotheses still to be checked.
5. Mark conflicting content as a counterexample or as expired.
6. Inject the result into later sessions and keep collecting feedback.

In a long-horizon memory evaluation based on database lookup question answering, the agent must look up tables to answer a series of questions, and the system publishes the correct answer after each response. The agent can write experience into a memory catalog for later sessions. Different setups learn in different ways. Early setups may produce many unchecked guesses, with fragmented theories scattered across folders. More advanced setups tend to consolidate notes into a single reference document and separate confirmed details from unconfirmed ones. The value of the memory system is not only how much it stores, but how clearly it organizes and verifies what it stores.

6. Pseudocode: The Offline Dream Loop and Memory Injection

# Generic pseudocode that describes the flow only; it does not depend on a specific interface
def online_session(task, memory_store):
    context = inject_memory(task, memory_store)
    result = agent_run(task, context)
    log = record_interaction(task, result)
    memory_store.raw_logs.append(log)
    if is_critical(log):
        memory_store.notes.append(draft_note(log, status="unverified"))
    return result
def offline_dream(memory_store, budget):
    logs = select_recent_logs(memory_store.raw_logs, budget)
    for batch in replay_batches(logs):
        claims = extract_claims(batch)
        for claim in claims:
            evidence = find_evidence(claim, memory_store.raw_logs)
            if evidence.confirms(claim):
                memory_store.upsert_note(claim, status="verified", refs=evidence.refs)
            elif evidence.contradicts(claim):
                memory_store.mark_conflict(claim, evidence)
            else:
                memory_store.keep_as_hypothesis(claim)
    memory_store.notes = merge_duplicates(memory_store.notes)
    memory_store.index = rebuild_index(memory_store.notes)

In practice, the system also needs permissions, privacy redaction, expiration policies, conflict resolution, and audit logs. Memory injection should cap how much is added and label confidence, so old experience does not override current facts.

7. Conclusion

The point of long-term agent memory is not to make the model remember every conversation. It is to build an external experience layer that can be replayed, verified, and corrected. With layered memory and background “dreaming,” an agent can fix mistakes between interactions, consolidate rules, and gradually close a learning loop across sessions. For developers, the first priorities are a traceable evidence chain, explicit memory boundaries, and a controlled injection policy. Replay scheduling and note organization can then be improved on top of that foundation.