Agent Memory Architecture: Offline “Dreaming” and Error Correction

Long-term memory is a key capability for taking AI agents from demos into production. Even a large single-session context window cannot naturally hold experience across tasks, days, and projects. Simply saving chat transcripts also pulls noise, errors, and stale information into later reasoning. The industry is converging on a more systematic approach: split memory into manageable layers, record important experience during interaction, and use idle compute to replay, reorganize, and correct it. That yields a closed loop of record, verify, consolidate, reuse, and re-verify. This article covers architecture, mechanisms, evaluation, and engineering for an external memory system that can run over the long term.

1. Scope: what counts as agent memory

Before architecture, define the boundary. Agent memory here means experience the agent obtains from task execution and user interaction, retains across sessions, and later reads as plain-text context. It typically includes factual observations, user preferences, task constraints, tool-use experience, failure lessons, verified rules, and unverified hypotheses.

Three mechanisms are excluded: (1) changing model weights via training or fine-tuning; (2) in-session in-context learning that only uses the current window; (3) compressing old conversations into summaries that simply replace context. External memory is valuable because it is auditable, retrievable, rollback-capable, and evolvable—not because everything is stuffed into the prompt.

Compared with typical retrieval-augmented generation, agent memory emphasizes that the agent writes, revises, and verifies its own experience. RAG usually targets an external knowledge base; a memory system targets experience formed during tasks. They can be combined, but they should not be conflated.

2. Layered architecture: from raw logs to injected context

A practical memory stack usually has four layers, with offline “dreaming” running through them.

2.1 Raw interaction logs

Raw logs are an immutable append-only event stream: user input, model output, tool calls, observations, errors, timestamps, task IDs, and session IDs. They preserve a full evidence chain for replay and audit. They should not be injected in bulk: they are noisy, large, and weakly relevant. They remain the factual source for later correction.

2.2 Memory notes

Notes are structured experience distilled from logs—for example, “this class of query must validate field types first,” “the user prefers concise answers,” or “this rule was confirmed in three tasks.” A note should at least include content, source log IDs, confidence, verification status, scope, created/updated times, expiry policy, and tags. Status can distinguish confirmed, unconfirmed, refuted, and needs review. This step determines quality; storing only free-form paragraphs is not enough.

2.3 Memory representation

This layer clusters, deduplicates, detects conflicts, and indexes notes for retrieval. Keyword, vector, or hybrid indexes are all viable. The real issue is metadata governance: multiple versions of the same fact, conflicting old and new rules, or experience limited to a task domain. Retrieval should filter by relevance, confidence, freshness, and cost.

2.4 Context injection

The injection layer selects a few high-value snippets for the current task. Selection should weigh relevance, confidence, recency, source reliability, and token budget. Include source and verification status so the model does not treat an unconfirmed hypothesis as fact. Add anti-pollution rules: low-confidence memory may hint, but must not act as a hard constraint.

Offline “dreaming” spans all layers: replay logs, correct notes, rebuild indexes, generate items to verify, and prune expired or conflicting content.

3. Offline “dreaming”: replay and correction on idle compute

During sleep, the hippocampus replays recent experience and consolidates important content into cortex as long-term knowledge. Agents can use a similar split: record quickly during interaction, organize slowly when idle. Research on “sleep-time compute” suggests using gaps between interactions to reorganize context. A fuller background pipeline is:

1. Sample recent logs and group by task, user, tool, and error type.
2. Detect contradictions and errors (opposite conclusions on the same question; a rule later falsified).
3. Rewrite or merge notes: remove duplicates, split overly broad rules, add scope.
4. Mark verification status; separate guesses from confirmed facts.
5. Generate new hypotheses for later tasks to test.
6. Update indexes and expiry; drop low-value memory.
7. Write an organization log for rollback and human audit.

Treat offline work as a low-priority batch job with budget caps, idempotency keys, and retry. Benefits include fewer repeated errors, stable rules, and cross-session consistency. Risks include locking in mistakes, overfitting local experience, memory pollution, and privacy leaks. Background “dreaming” must not be fully autonomous; it needs human audit, confidence thresholds, and rollback.

4. Marking key events: predicting what will matter

The brain tags important moments so they get more replay in sleep. Agents can tag key events in-session or write them to an external memory catalog. The hard question is which current events will matter later.

A practical rule is to store what is reusable and verifiable: stable constraints, long-term user preferences, failure causes, tool-call rules, confirmed facts, and hypotheses that still need checking. Label guesses as hypotheses. Raise replay priority for high-value events. Redact sensitive data before storage. Taking a note is a prediction: writing a rule predicts future reuse; recording a verified fact predicts you need not re-check. Predictions will be wrong, so the system must allow correction, down-weighting, and forgetting.

5. Learning loop and evaluation

Long-horizon memory can be evaluated as multi-round tasks: the agent looks up an answer, the system reveals the correct answer, the agent writes experience into the catalog, and later sessions reuse it. Weaker models produce many unverified guesses and scattered notes; stronger ones consolidate notes into reference documents and separate confirmed from unconfirmed detail. The ideal loop is hypothesize, verify, write notes, reuse, and verify again.

Metrics should go beyond task success: repeated-error rate, memory hit rate, conflict rate, injection token efficiency, offline organization cost, and human audit load. Without measurement, memory becomes a growing noise warehouse.

6. Engineering guidance and pseudocode

Govern four areas. Recording: keep raw logs in full; store only reusable experience as notes. Verification: give every note confidence and status. Conflict resolution: rank by source reliability, recency, confidence, and human arbitration. Security and privacy: redaction, access control, audit, and expiry are all required. Cost: idle batch jobs, sampled replay, and cached indexes.

MemoryNote:
  id, content, source_log_ids
  confidence: 0..1
  status: confirmed | unconfirmed | refuted | needs_review
  scope, tags, created_at, updated_at, expire_at
offline_dream():
  logs = sample_recent_logs(within_budget)
  notes = extract_notes(logs)
  for note in notes:
      if conflicts_with_existing(note):
          resolve_by_confidence_time_source(note)
      if verified_by_logs(note):
          note.status = confirmed
      else:
          note.status = unconfirmed
  merge_duplicates(notes)
  update_index(notes)
  write_audit_log()
inject_context(task):
  candidates = retrieve_notes(task, top_k)
  selected = filter_by(candidates,
      relevance, confidence, freshness, token_budget)
  return render_with_status(selected)

7. Conclusion

Agent memory is not chat dumps or compressed summaries. It is a verifiable, auditable, evolving experience system. Layers answer what to store, how to store it, and how to use it. Offline “dreaming” answers when to organize and how to correct. Event tagging and a verification loop answer which experience is worth keeping. Start from boundaries, layered storage, verification status, conflict resolution, security, cost, and observability so memory serves long-running tasks instead of becoming another context burden.