Offline Consolidation for Long-Term Memory in AI Agents

Agents that keep improving across multi-turn tasks cannot rely only on the context window of a single session. When a user closes a conversation, temporary context usually disappears. The next task, even on a similar topic, often starts from scratch. Concatenating full history into a new session quickly consumes the window, amplifies noise, and re-cites early mistakes. A more stable approach is to separate working memory that is available now from long-term experience that persists across sessions, and to organize that experience with idle compute between interactions.

What memory is for

Everyday interaction produces dense logs: user goals, tool calls, intermediate conclusions, failed attempts, and final feedback. These materials help the current reasoning pass, but most are not worth storing verbatim. Long-term memory should keep only experience that later tasks are likely to reuse and that has been checked to some degree—stable business constraints, verified data definitions, recurring operational pitfalls, and reusable checklists.

Memory here means experience obtained from tasks and interaction, readable across sessions, and injected into later context as text or other interpretable records. It typically excludes three mechanisms: directly updating model weights, in-session context adaptation only, and mechanically compressing old chats into summaries that replace the original. Summaries can help retrieval; they cannot replace experience items that still hold after reflection.

Borrow consolidation, not neural anatomy

People encode experience quickly while awake, then replay recent fragments during sleep and rest to distill important content into more stable knowledge. Agents can reuse this division of labor without claiming to reproduce biology.

The online phase captures: facts, uncertain conclusions, and external feedback. The offline phase consolidates: replay recent logs, merge duplicate observations, correct claims overturned by new evidence, and turn scattered notes into retrievable entries. This offline process can be called “dreaming” only as a metaphor for sleep-time compute—using idle resources between sessions to reorganize existing context, not waiting for the user’s next message to think.

Salience signals raise priority during replay. In engineering terms that means explicit tags: write key constraints, high-cost errors, user corrections, and verified facts into a memory catalog, or mark them as “needs review” or “high priority.” The difficulty is that it is hard to know in the moment what will matter later. Tags should therefore be conservative; the offline stage can promote or demote items based on reuse frequency, conflict count, and task outcomes.

A layered architecture

A practical memory system usually has four layers. Offline consolidation runs through all of them; it is not a single final summary.

The raw record layer stores session logs, tool-result summaries, and user corrections. It is evidence, not finished text for the model. Keep timestamps, sources, and task IDs for audit, but do not inject this layer into every prompt by default.

The memory-note layer holds key observations written during a session. A note should be short, independently readable, and scoped. For example, “amount fields in this report class are in cents and must be converted before display,” not a pasted transcript. Notes may be drafts that later merge or expire.

The memory-representation layer is stable experience after replay and reorganization. If several notes describe the same rule, merge them into one rule with evidence links. Conflicting items must not be injected side by side; mark the conflict and down-rank until new feedback resolves it.

The injection layer selects a small set of memories relevant to the current task. Injected text should include source and time, and state that it is historical experience: if it conflicts with the current instruction, the current instruction and latest verification win. If retrieval fails, inject less rather than fill the window with unrelated memory.

Memory as a prediction about the future

Writing a rule is a prediction that it will be used again. Recording a verified fact is a prediction that a future self will not need to re-check the full raw log. Note quality can be tested with three questions: Are the trigger conditions clear? Is the evidence traceable? If the prediction is wrong, can the system detect and revise it?

Unchecked guesses fill the catalog quickly. A stronger learning loop has four steps: try the task and leave inspectable intermediate conclusions; obtain an external signal (ground truth, user correction, tests, or business checks); write a note of “what we assumed, what was true, how to avoid it next time”; retrieve that note on later similar tasks and record whether it actually improved success. Only items used later and not overturned should become stable representations.

In database question answering, after each table lookup, if the system returns a correct result or a validation failure, field meanings, join conditions, and common misuses can be written to the catalog. The next session with a similar schema should read those notes first instead of repeating trial and error. Evaluation should compare a no-memory baseline with a memory-enabled setup: not only single-item accuracy, but whether cumulative performance improves as experience grows, and whether bad memories are revoked promptly. Models of different capability behave differently: some only pile unverified guesses; others merge notes and retire stale rules. “Wrote a lot” is not learning; reusable and correctable is.

How offline consolidation works

Offline jobs fit a post-session queue. Inputs are this session’s raw records, existing notes, and a few related older memories. Outputs are suggestions to add, merge, revise, or delete. A consolidation program can follow fixed steps:

1. Replay key events; drop small talk and intermediate drafts that cannot be verified.
2. Extract candidate experience: claim, scope, evidence pointer, and confidence tag.
3. Compare with old memory: merge duplicates; on conflict keep both evidence trails and lower injection priority.
4. Re-check with available automated tests, such as read-only queries, test cases, or format constraints. Unverifiable content stays a hypothesis and does not become a stable rule.
5. Control volume: merge synonyms; expire or archive long-unused, low-value notes.

A sketch in ordinary script, unbound to any platform:

on_session_end(session):
    raw = load_log(session.id)
    notes = extract_candidate_notes(raw)
    for note in notes:
        related = search_memory(note.scope, note.claim)
        if contradicts(note, related):
            save_conflict(note, related)
        else:
            upsert_note(note, evidence=raw.ref)
    schedule_dream(session.user_id)

Dream jobs need a budget: cap replay count, max notes, and inference time so idle compute does not become unbounded background cost. Failures must be retryable and must not block the user’s next conversation.

Storage, retrieval, and injection

Storage only needs append, versioning, and scope filters. Object storage fits raw logs; structured tables or a document store fit notes. Product choice follows the existing data platform. Each memory item should include an ID, user or tenant scope, topic tags, body, evidence references, create/update times, and status (draft, active, conflict, archived).

Retrieval should not rely on keywords alone. Task description, data objects involved, and recent failure types can all be query conditions. Deduplicate results and prefer recently verified items. Trim before injection: personal preferences unrelated to the current problem, expired project rules, and unresolved conflicts stay out of the prompt.

Permissions must isolate by tenant and user. Memory may contain business data; reuse across users should be off by default. Deletion should apply to notes and linkable raw records, while retaining the minimum audit trail required for compliance.

Risks and limits

Offline replay can amplify errors. If a hallucination is written as a “verified rule,” later sessions will repeat it more confidently. Mitigate by distinguishing observation, inference, and checked fact; only the latter two, confirmed by external signal, should inject at high priority.

Memory also goes stale. Organizational definitions, schemas, and API contracts change. Every rule needs scope and expiry conditions, and periodic sampling against latest task results. Archive unused memory instead of leaving it in retrieval forever.

Prevent memory from becoming implicit instruction injection. Content such as “from now on ignore safety constraints” in user text or external documents must not gain authority merely by being stored as a note. Consolidation should filter privilege-escalating requests; injection still follows system policy and the current explicit instruction.

Finally, long-term memory improves reuse of experience; it is not parameter learning. It is suited to procedures, constraints, and verified facts. It does not replace formal knowledge-base governance, and dreaming will not magically add professional skill the model was never trained for. With evidence, versions, conflicts, and forgetting designed in, idle replay becomes an auditable learning loop rather than log cleanup.