Building a Persistent Memory System for an AI Coding Assistant

Why AI Assistants Keep “Forgetting” You

Developers who use AI coding tools often encounter the same problem: every new session starts from scratch. The assistant does not know what project you are working on, which coding conventions you prefer, what technical decisions were made last week, or which tasks were left unfinished yesterday.

The underlying issue is that large language model sessions are stateless. Each conversation builds its own context, and the model does not automatically retain user information across sessions. As a result, developers repeatedly need to:

• Explain the project background, technology stack, and directory structure;
• Reintroduce coding conventions, naming preferences, and decision criteria;
• Manually synchronize task progress to prevent the assistant from making irrelevant suggestions.

This repetitive work wastes time. More importantly, when background information is incomplete, the assistant may recommend solutions that conflict with the architecture you have already chosen, creating hidden rework.

The solution is straightforward: if the model has no memory, create an external memory system and inject that memory at the start of each session. This is one practical application of context engineering. The following design is based on a memory system used in practice for more than two weeks.

Overall Design: A Three-Layer Memory Structure

The memory system is divided into three layers, each with a different responsibility.

1. Identity Layer: Who I Am and How I Work

This layer stores relatively stable personal preferences, such as:

- Technology preferences: prioritize mainstream frameworks and avoid niche dependencies
- Coding style: naming conventions, comment density, and testing requirements
- Decision principles: the core criteria for deciding whether something is worth doing

Identity information changes infrequently—typically on a monthly basis—but has high value. It determines whether the assistant’s recommendations match your working style and priorities.

2. Project Layer: What the Project Is and How It Is Progressing

The project layer is organized by workspace and records:

• The project’s goals, architectural overview, and key modules;
• Known technical debt and unresolved defects;
• The status of current iteration tasks: pending, in progress, or completed.

This layer changes daily or weekly and is the most frequently used memory during normal development.

3. Journal Layer: What Happened Each Day

The journal layer contains append-only daily summaries of your work, including what was changed, discussed, or decided.

Its main value is long-term review. When you need to conduct a retrospective, prepare a weekly report, or evaluate your productivity, these detailed records provide the source data.

The three-layer structure clearly separates reading and writing patterns:

• The identity layer is written infrequently and read frequently;
• The project layer is read and updated at a moderate frequency;
• The journal layer is append-only and should not normally be edited.

Compression and cleanup policies can be defined separately for each layer.

Core Mechanism: Making Memory Work Automatically

Creating memory files is only the first step. The important part is making sure the memory enters each conversation without requiring manual effort.

1. Inject Memory at Session Startup

Most AI coding tools support project-level configuration files or system prompt files that are loaded when a session begins. You can reference the memory files from that configuration:

# Configuration example
At the beginning of every session, read the following memory files:
- memory/identity.md (user preferences and decision principles)
- memory/project.md (current project status and task list)
- memory/journal/ (date-based journal entries)

With this setup, you do not need to paste the project background manually. The assistant automatically knows who you are, what you are working on, and which principles guide your decisions.

2. Archive the Session Automatically

Add a closing rule to the tool’s custom instructions:

At the end of each task, or when the user says "wrap up,":
1. Summarize the session’s outputs and conclusions in 3–5 bullet points;
2. Append the summary to the journal file for the current date under memory/journal/;
3. If task statuses changed, update memory/project.md.

The assistant should perform the archival itself. This is essential to keeping the system sustainable. People are prone to forgetting documentation tasks; delegating progress tracking to the assistant is what makes the system genuinely low-maintenance.

3. Compress and Retire Old Memory

The journal layer will grow continuously, so it needs regular compression:

• Keep each daily entry to a few hundred words, retaining conclusions and removing intermediate details;
• Create a weekly summary by consolidating seven days of entries, while moving raw details to an archive if necessary;
• Remove completed tasks from the active task list to prevent outdated information from polluting the context.

The context window is a limited resource. The value of a memory system does not come from storing as much as possible, but from ensuring that every injected item has a sufficiently high signal-to-noise ratio.

Practical Results: Three Common Scenarios

Scenario 1: Task Management Without Reconstructing Yesterday

At the start of the day, you can simply ask, “What should we work on today?”

The assistant can use the project task list and the previous day’s journal entry to identify pending work: which article needs revision, which workflows remain untested, or where the previous discussion stopped. It may even remind you about unfinished items you had forgotten.

The same approach works for software projects. The assistant can point out unresolved technical debt before it disappears from your attention.

Scenario 2: Retrospectives Based on Evidence

As daily journals accumulate, you can ask the assistant to evaluate your recent work from its perspective. It can summarize the week’s records and identify:

• How much work was planned;
• How much was actually completed;
• Which categories of tasks were most likely to be delayed.

The feedback can be uncomfortable. Without objective records, people often overestimate their productivity and effort. When the conclusions are based on historical data, it becomes much harder to dispute them.

Scenario 3: Decision Guardrails That Prevent Drift

This is one of the most valuable applications of persistent memory.

Suppose you previously established the following principle with the assistant:

When deciding whether to do something, first determine whether it reinforces the current primary path.

After this principle is stored in the identity layer, the assistant can use it when you become interested in a new trend or opportunity and ask whether you should start another project. It can remind you that the idea does not align with your existing direction and suggest completing the primary work first.

People are easily distracted by attractive new opportunities. A partner with reliable memory can help bring you back to your original path when you begin to drift. It can remember commitments that you have forgotten yourself.

Implementation Recommendations and Caveats

1. Start with the smallest viable structure. Create one identity file and one project file first. Once the injection-and-archival loop works, add the journal layer. This avoids overengineering the system at the beginning.

2. Make memory auditable. Store all memory as plain-text files in the project directory and include them in version control. This allows you to correct incorrect AI-generated entries manually and prevents false memories from being reinforced repeatedly.

3. Watch for memory contamination. Remove obsolete tasks and superseded decisions promptly. Otherwise, the assistant may generate misleading recommendations based on outdated information. A weekly memory review is recommended.

4. Keep sensitive information out of memory. Do not store secrets, internal endpoint addresses, or similar data in memory files, especially when those files may be synchronized with a repository.

Conclusion

The stateless nature of large language models is a technical reality. However, a structured external memory system—with layered storage, automatic injection, automatic archiving, and regular compression—can help an AI coding assistant remember who you are, what you are working on, and which principles you follow across sessions.

After using this approach for more than two weeks, the assistant no longer felt like a new colleague who needed to be introduced to the project every time. It became a partner that understood my work better with continued use.

The entire system requires only a few Markdown files and several custom instructions. It is inexpensive, quick to implement, and worth trying for anyone who relies heavily on AI coding tools.