Why AI Coding Agents Forget Across Sessions: Memory Mechanisms and Practical Implementation

1. Memory Is Not the Same as Persistent Storage

Imagine this scenario: last week, you spent 30 minutes explaining three project conventions to a coding assistant:

• Do not refactor the authentication module because legacy clients still depend on it.
• Start the local cache before running tests.
• Run the formatter or linter before committing code.

The assistant acknowledged all three rules and generated code that followed them. This week, however, you start a new session, and its first suggestion is to refactor the authentication module.

It is easy to blame the context window. The context window does determine how much information can fit into a single session, but it is more like a computer’s memory: once the session ends, its contents are cleared. Cross-session continuity requires something closer to persistent disk storage. Even a context window with millions of tokens does not automatically preserve information after the session is closed.

Another point is often overlooked: everything you tell an agent is not necessarily stored as a memory. Whether information is written, what gets written, and when it is written depend on the product’s design. Some products make these decisions automatically, some require manual configuration, and some do not provide automatic memory at all.

2. Four Layers of AI Memory

AI memory systems are generally discussed in four layers:

1. Manual rules files: Project rules written by developers, such as AGENTS.md or team-specific configuration files.
2. Single-session conversation compression: Dynamic summaries of older content in a long conversation to preserve continuity within the current session. This does not persist after the session ends.
3. Agent-managed notebooks: Local, cross-session notes maintained by the model during operation.
4. External memory databases: Large collections of historical interactions stored in vector databases or dedicated memory services.

The boundaries between these layers are clear, but real products often combine several of them. Before choosing an approach, clarify who maintains each layer, where it is stored, and when it takes effect.

3. Five Ways Memory Gets Written

The main difference between memory implementations is who decides what should be written to the “notebook.” Common approaches can be grouped into five categories:

Approach	When it is written	Who decides	Typical storage location	How it is deleted	Main risk
Periodic background extraction	After a session has been idle for some time	A background system process	The tool’s memory directory	Usually manual cleanup or a system policy	Extraction is delayed, so recent information may not be immediately available
Mandatory manual rules	When a developer edits a file	Developer	Documentation files in the repository	Manual editing	High maintenance cost and a risk of becoming outdated
Direct configuration changes	During an interaction	Model	The tool’s configuration files	Manual configuration edits	An incorrect change may affect all future sessions
Tool-based CRUD operations	When the model decides a write is needed	Model	A backend selected by the developer	A delete tool call	The model may mistake a temporary instruction for a long-term rule
Repository write-back	After a task is completed	Workflow or developer	The code repository	Revert through version control	Memory becomes part of the version history and must be reviewed

This table summarizes implementation patterns at a general level. Whether a specific product uses any of them should be verified against its current official documentation and release notes. Before selecting an approach, test it in a dedicated repository.

4. The Challenges of Storing, Retrieving, and Deleting Memory

A mature memory system is not defined merely by whether it can store information. The difficult part is coordinating storage, retrieval, and deletion. One practical engineering principle is that incorrect memory can be more dangerous than no memory at all.

Storage: distinguish temporary instructions from long-term rules. During a debugging session, for example, a model might be told to temporarily disable a security check. If it stores that instruction as a permanent practice, future code may repeatedly include the same security weakness. The system must distinguish between “do this for the current task” and “do this from now on.”

Retrieval: load only information relevant to the current task. Irrelevant historical memories consume tokens and distract the model, reducing generation quality. Retrieval should be filtered by task scope. For example, load only the rules related to the module currently being modified.

Deletion: verify before removing a memory. As a project evolves, many rules become obsolete. A retired service or migrated test framework should no longer influence decisions. Keeping such rules creates technical debt and unnecessary interference. Deletion itself also carries risk: removing a rule that is still valid may cause the agent to repeat old mistakes without making the regression obvious.

Before deleting a memory, check:

• Timestamp: Has the relevant module undergone a major change since the memory was created?
• Code status: Do the files, services, or dependencies mentioned by the rule still exist?
• User instruction: Has anyone explicitly stated that the rule is no longer valid?

Keep a deletion log that records what was removed, when it was removed, and why. This makes it possible to investigate problems and restore a memory if necessary.

5. Three Lightweight Ways to Implement Agent Memory

Most teams do not need to deploy a vector database on day one. The following three approaches are inexpensive and can address many common cross-session memory problems. Their coverage will vary by task type, so each team should validate the results in its own workflows rather than relying on a universal percentage.

Approach 1: Maintain a Structured Project Rules File

Place a rules file in the project root, such as AGENT_RULES.md. Some tools conventionally read AGENTS.md; in that case, write the rules directly there or reference the separate rules file from it.

There is no universal length requirement. Keep the file small enough for the team to review and maintain each rule individually. When it becomes too large, split it by module and place a separate file in each relevant subdirectory.

A rules file could use the following structure:

# Project Development Conventions
## Prohibited Changes
- [Prohibited] Do not refactor the authentication logic under `src/auth`. Legacy clients still depend on this interface.
## Environment Requirements
- [Environment] Ensure the local Redis service is running on port 6379 before executing tests.
## Commit Requirements
- [Standard] Run `make lint` before committing code to check for syntax and style errors.
## Architecture Notes
- Services communicate through gRPC.
- Apply database schema changes through the framework's built-in migration tool. Do not modify the schema manually.

Each rule should include a brief reason when possible. The reason helps the model understand the rule’s boundaries. For example, “legacy clients still depend on this interface” explains why the authentication code cannot be changed and helps the model generalize to similar situations.

When maintaining the file, check the following:

• Does every rule have a clearly assigned owner?
• Does each rule include a last-confirmed date, with an agreed review interval?
• Do any rules conflict, and if so, which one takes precedence?

Approach 2: Write a Handoff Note at the End of Each Session

At the end of a long task, or before closing a session, ask the agent to create a handoff note using a fixed template and append it to the work log.

The template should include:

• Task objective
• Files changed
• Open issues
• Validation commands and results
• The entry point for the next session, such as the file or command from which work should continue

A handoff note helps the next session get up to speed quickly and allows other team members to understand the agent’s progress. Do not include secrets, tokens, or internal addresses. If the work log is stored in the repository, protect it with the same access controls used for regular project documentation.

Approach 3: Expose Memory as Tools When Building a Custom Agent

If a team is building its own coding agent on top of a model inference service that supports tool calling, it can implement tool-based memory. Provide three interfaces:

• add_memory
• search_memory
• delete_memory

The system prompt should define when these tools may be used. For example: “Write a memory only when the user explicitly establishes a long-term convention.”

This design allows the model to make the decision while hiding the storage backend behind an interface. If the team later migrates from local files to a hosted retrieval service, only the tool implementation needs to change; the model’s calling pattern can remain the same.

The deletion interface should require a memory ID and a deletion reason, and it should write every deletion to a log. If the model deletes something incorrectly, the team can trace the decision and restore the memory.

6. Common Mistakes

• Treating a larger context window as the solution. The window only affects a single session and does not preserve content after the session ends.
• Allowing memory to grow without limits. Without cleanup, outdated rules accumulate and interfere with new decisions.
• Writing rules that are too vague. Statements such as “pay attention to code quality” do not give the model an actionable instruction.
• Allowing automatic memories and manually maintained rules to conflict without defining which source takes precedence.
• Deleting memories without an audit trail. When a problem occurs, it becomes impossible to determine which memory was removed.

7. Frequently Asked Questions

How should a team choose a cross-session memory approach?

Start by evaluating how stable the team’s rules are. If the rules are stable and rarely change, a manually maintained rules file may be sufficient. If tasks frequently span multiple sessions and the agent needs to accumulate experience independently, consider tool-based memory or an external memory system.

Where should a team start with agent memory management?

Start with one project rules file and one handoff-note template. This has the lowest cost and can solve many common continuity problems before the team invests in a more complex memory architecture.