Dual-Mode Memory Architecture in Practice

Why Dual-Mode Memory Is Needed

Conversational agents must stay coherent across turns and sessions. The core tension is a limited context window versus facts, preferences, and task state that keep accumulating. The usual approach compresses history into one “master summary” and updates it as the dialogue continues. That path is simple, but it hits three bottlenecks.

The first is capacity. Even a tight summary is still a single text; when there is too much information, the system must drop or over-generalize, and detail is lost irreversibly. The second is noise. A full-history summary mixes unrelated topics, so the model scans irrelevant content every turn, wasting tokens and increasing crosstalk. The third is weak governance. A summary is a black-box compression: users cannot locate a specific memory, cannot correct errors precisely, and auditing and compliance become expensive.

Dual-mode memory targets those problems: keep classic memory for a fast global profile, and add file memory so conversations settle into retrievable, editable structured documents. The system no longer depends on one all-purpose summary; it loads what the current task needs.

How Classic Memory and File Memory Split the Work

Classic memory is a compressed view of the user and the session: identity, long-term preferences, current goals, unfinished work. It belongs in the system prompt or a fixed prefix so every turn has a stable persona and constraints. Updates are usually incremental merges: write new facts, resolve conflicts by recency or confidence, and decay stale items.

File memory splits information into independent documents. Typical splits include technical conventions and coding standards, project structure notes, travel and schedule preferences, account and environment configuration (without raw secrets), and decision logs. Each document has a title, tags, timestamp, source session id, and version. Load technical docs when talking about code, travel preferences when talking about trips, and avoid dumping everything.

The two modes complement each other. Classic memory answers who this person is and what they are doing now; file memory answers what the details of a topic are. If retrieval fails, fall back to classic memory, then run a background organization job to fill in files.

File-System Style Memory Management

Persisting memory as files means a controlled knowledge base, not a raw archive of chat logs. A minimal metadata model includes document id, namespace (user / project / session), topic tags, a short snippet, body, update time, and a checksum. Storage can be object storage or a document store. Indexing can combine inverted search and vectors: exact fields via keywords, semantic neighbors via vector recall.

A three-step write pipeline works well. First, extract: identify facts worth persisting and filter one-off chat and sensitive data. Second, merge: decide whether to append to an existing document or create a new one, to avoid fragmentation. Third, validate: check conflicts with classic memory; if they conflict, mark for confirmation instead of silently overwriting.

Reads use intent routing plus a token budget. Classify the current question, retrieve top-K documents, then truncate to budget. Reserve budget for system instructions, classic memory, and the current turn; file memory only uses the elastic remainder. For coding tasks, prefer API conventions and directory notes; for planning, prefer constraints and past decisions.

In theory, document count can grow with the user, so the bottleneck shifts from “single summary length” to retrieval quality and governance cost. In practice, still set limits: max documents per user, max length per document, and hot/cold tiers. Archive cold files; keep hot files in the live index.

Retrieve On Demand, Not the Full Memory

Loading everything looks more “user-aware” but lowers signal-to-noise. On-demand retrieval depends on how the query is built. Do not use only the last user sentence. Concatenate the current question, a short recap of recent turns, and relevant slots from classic memory. Queries that are too short miss recall; queries that are too long drift topics.

Reranking is not optional. Vector recall finds semantic neighbors, but technical docs often share names. Apply rule features for a second pass: tag match, time decay, user-pinned items. For high-risk writes (config changes, long-term preference changes), require a higher match threshold and confirm with the user before writing.

Caching matters. Topics are often stable within a session, so cache session-level retrieval briefly to avoid repeat recall. Invalidate on topic switch. That controls latency and keeps old-topic files from polluting a new topic.

Editability: Treat Memory as a Governable Asset

File memory is valuable not only for capacity but for governance. Users should browse, edit, and delete memory files like knowledge entries. At minimum, provide list and search, full-text view, conflict comparison, and one-click undo of the latest write.

Permissions and audit must be designed together: who can read, who can write, whether a change needs a second confirmation, and a record of each change. Deleting a wrong memory should also drop it from the index and write back to classic memory so a bad conclusion does not linger in the summary. Keep a limited version history so bad merges can be rolled back.

On the model side, state clearly that file content outranks parametric priors, but the user’s explicit instruction this turn outranks old files. That handles preference changes: if the user says they want a different default style from now on, update the file and refresh the matching slot in classic memory.

Implementation Notes and Risk Control

Extraction quality sets the ceiling. Over-recall creates junk documents; under-recall looks forgetful. Combine rules and models: rules catch explicit statements (“my name is… / I prefer…”); the model summarizes long dialogues. Mark uncertain facts with low confidence and keep them out of high-privilege documents.

Privacy and minimization come first. Do not store passwords, keys, or ID numbers. Refer to necessary environment data instead of pasting secrets. Encrypt in transit and at rest for cross-device sync. Offer one-click wipe and export for compliance.

Evaluation should not only count “how much was remembered” but “whether it was used correctly.” Useful metrics include off-topic document rate on topic switches, retention of key facts, recurrence of a fact after the user corrected it, and end-to-end latency. Use fixed dialogue scripts offline; monitor implicit feedback online (whether the user corrects the same fact again).

A Practical Rollout Path

Evolve in three stages. Stage one: classic memory plus a few hand-written template files, to verify that on-demand loading reduces crosstalk. Stage two: automatic extract and merge, plus retrieval and editing UI. Stage three: hot/cold tiers, conflict arbitration, and integration with a general document store / object storage / search service—without binding to a specific vendor API.

For developer assistants, prioritize project conventions, build methods, and interface contracts. For personal assistants, prioritize stable preferences and long-term plans. In both cases, memory is a revisable structured asset, not an unexplainable ultra-long summary.

Dual-mode memory is not mysterious: it combines a compressed representation with a retrievable knowledge base so engineering can address capacity, noise, and governance together. With solid extraction, retrieval budgets, conflict handling, and user editability, an agent can remember over the long term and still use memory accurately.