Local-First Agent Memory in 5 Steps: From Write to Recall

    Copying an entire previous conversation into a new session may look like giving an AI agent "memory," but it breaks down in real coding and multi-tool workflows—diluting context, making information hard to retrieve, and losing everything when the session ends. This guide explains what a local-first memory runtime is and how it turns cross-session, cross-tool experience into searchable, editable, versionable local assets. You'll follow the five-step workflow: install the runtime and run the no-key demo to see the memory lifecycle, generate a configuration and start the service, write memories with `add`, `flush` them to disk and build an index, and `search` to recall relevant entries into the prompt. It also covers the key model constraints (same-provider models and matching embedding dimensions), a troubleshooting checklist for missing memories, and advanced practices for privacy, versioning, observability, and quality.

    Introduction: Agent memory isn’t just replaying the conversation

    Many people assume that copying an entire previous conversation into a new session gives an Agent “memory.” This seems to work in a single demo, but problems surface quickly once you enter real coding collaboration and multi-tool workflows.

    The “memory” flow most people imagine looks like this:

    1. full previous conversation → copy and paste → inject into the new session's system prompt

    Another common approach keeps a complete session log locally and concatenates the most recent entries when a question comes in. Both approaches are essentially the same “wholesale transport,” and they share four main problems:

    1. Context dilution. The longer the conversation, the more content gets concatenated into the prompt, and token consumption and response latency rise together. Irrelevant information also crowds out useful context and interferes with the model’s judgment.
    2. Hard to retrieve. Concatenated text can only be passed as one block; it can’t be matched precisely by topic, time, or keyword. Whether the model actually uses a relevant memory comes down to luck.
    3. Can’t edit or audit. Memories are scattered across chat logs with no unified format and no source information. When something is recorded incorrectly, it’s very hard to trace which turn or which file introduced it.
    4. Can’t reuse across sessions or tools. Switch to a new session or a new tool, and the experience from the previous turn resets to zero.

    By contrast, a local-first memory runtime uses this flow:

    1. conversation snippets → accumulate in a buffer → LLM extracts them into structured memory entries → write to readable files
    2. → build an index asynchronously in the background → retrieve relevant entries by query → inject into the prompt

    The key difference between the two flows lies in the “extraction” step in the middle and in the form the memory entries take on disk: human-readable and editable Markdown files, rather than data blocks locked inside a closed dashboard.

    So the problem these solutions address isn’t “how to stuff more text into the context,” but “how to turn cross-session, cross-tool experience into searchable, editable, versionable local assets.” We’ll start with the memory lifecycle, then run through the five steps to get the service working.

    1. Clarifying the concept: What is a “local-first memory runtime”

    1.1 Core definition

    A Memory Runtime is a middle layer between the Agent and persistent storage. It handles the collection, processing, storage, and recall of memory entries. It exposes a unified read/write interface upward for the Agent and manages local data files and indexes downward.

    “Local-first” emphasizes two points:

    • Memory data is stored in a local directory by default;
    • The service runs without depending on a separate database, search engine, or cache middleware.

    This does not mean fully offline. The large model responsible for “extraction” can be deployed in the cloud, but “ownership” of the memory entries always stays local. Data is stored as plain text, and users can view, edit, migrate, or delete it at any time.

    1.2 The essential difference between the two paths

    Dimension Conversation dumping Memory runtime
    Processing target Raw conversation text Structured facts extracted from the conversation
    Storage form Chat logs, inconsistent format Readable files such as Markdown, with source metadata
    Retrieval Whole block injected into the prompt Retrieved by query, only relevant entries
    Lifecycle Expires when the session ends Persists across sessions, editable and archivable
    Reuse scope Single tool, single session Shared across multiple Agents and tools

    1.3 Data unit: the memory entry

    The basic unit is the memory entry (think of it as a “memory cell,” or MemCell). Each entry typically contains the following elements:

    • Body: a standalone, complete fact or conclusion, for example “the project reads its database connection uniformly from environment variables”;
    • Source: the session identifier or file path that produced the entry, for traceability;
    • Time: creation time and last-updated time;
    • Tags: topic, project name, type, and so on, used for filtering and grouping;
    • Metadata: importance, confidence, expiry, and so on, used for ranking and filtering during retrieval.

    Entry design follows a “one entry, one fact” principle. The clearer the granularity, the higher the retrieval hit rate. Cramming multiple unrelated facts into one entry makes recall results fuzzy.

    2. Step 1: Install and demo—understand the memory lifecycle in five minutes

    2.1 Installation

    Before configuring keys and starting the service, it’s best to install the runtime through a package manager. Prefer an officially maintained distribution channel, and run the installation in an isolated virtual environment so that dependencies don’t interfere across projects. After installation, run a version query command to confirm it’s ready.

    2.2 A no-key demo

    The package usually ships with a teaching demo that runs without any keys:

    • Interactive mode: in a full-screen terminal interface, enter one “memory” and one “recall question,” and the screen walks through the five stages the memory passes through, ending with a celebration animation:
      • ingest
      • extraction
      • indexing
      • recall
      • source reveal
    • Replay and exit: press r to replay, q to exit.
    • Plain-text mode: in non-interactive environments such as CI pipelines or SSH, run it with plain-text parameters.

    Note that the extraction and recall results in the demo are preset content; the point is to make the abstract flow “visual,” not to validate real behavior. But that’s exactly the value of the demo: it builds the right mental model before you move on to real configuration.

    2.3 The core insight to take away

    The single most important thing to remember from the demo: writing is not the same as being readable. After data is written in, it still has to pass through the “extraction” and “indexing” steps before it can be retrieved. Mapped to the real service:

    • Extraction is triggered by a flush operation;
    • Indexing is completed asynchronously by a background process.

    This gives a practical debugging intuition: when freshly written content can’t be found, don’t suspect the retrieval logic first—in most cases, the index simply hasn’t caught up yet.

    3. Step 2: Generate the config and start the service

    3.1 Generating the config file

    After the demo, you move on to production-oriented configuration. The initialization command usually generates a config file in the current directory or the user’s home directory. Key parameters to focus on:

    • Model access: provider type, endpoint, API key, extraction model name, embedding model name;
    • Storage path: the root directory for memory files and the index (e.g., ~/.agent-memory/);
    • Extraction parameters: max entries extracted per run, timeout, temperature, and so on;
    • Retrieval parameters: number of results returned, similarity threshold, weight configuration, and so on.

    3.2 Choosing a model service

    Both extraction and embedding require calls to a large model, so choose a service that fits your network environment and compliance needs:

    • Regional or cloud providers: some teams prefer a regional cloud platform that bundles model selection, key management, and usage monitoring in one place. Baidu AI Cloud’s Qianfan platform is one example of this kind of service.
    • OpenAI-compatible aggregators: others use aggregation/routing services that support OpenAI-style Chat Completions and Embeddings APIs, which makes it easy to switch between model vendors.

    One point deserves special attention: the extraction model and the embedding model must come from the same provider, and the embedding vector dimensions must match. If the embedding API returns 1536-dimensional vectors while the index is built at 1024 dimensions, queries will fail with a dimension-mismatch error, or the recall results will be completely distorted.

    3.3 Model capability constraints

    Not every model is suited to memory extraction:

    • Structured output: extraction results need to parse reliably into a “body + tags + source” structure, so prefer models that support JSON structured output;
    • Instruction following: extraction prompts usually carry many constraints, and the model needs to follow them accurately;
    • Cost and rate: extraction is called frequently, so choose models with good cost-performance and ample concurrency quota.

    For data-sensitive scenarios, you can switch the model service to a private model deployed on your internal network, keeping memory data inside that network.

    3.4 API key security

    Keys should be injected only through environment variables, for example:

    1. # Inject the key via an environment variable to avoid writing it into a config file
    2. export MEMORY_API_KEY="your-api-key"

    At the same time, set minimum permissions on the config file (e.g., chmod 600) and confirm the config directory is excluded from version control. Never commit any key to a shared code repository.

    3.5 Starting the service and the index process

    The start command usually listens on a local loopback address (127.0.0.1). After starting, confirm:

    • The service port is reachable;
    • The background index process (cascade) is running, responsible for writing pending entries into the index file.

    If you want it to start automatically on boot, register it as a system service. Then watch the log files to observe indexing progress and any errors.

    4. Step 3: Write memory (add)

    4.1 The target: the buffer

    The add command appends conversation snippets or manual notes to a buffer, rather than writing directly to the formal memory files. The buffer solves two problems:

    1. Decoupling write from processing: you can record continuously and cheaply during a conversation, without calling the model every turn;
    2. Aggregation before extraction: related snippets gather together first, so the extraction stage can identify duplicates and produce more concise entries.

    4.2 Basic usage

    1. # Append a conversation snippet
    2. memory add --text "API response fields consistently use snake_case naming"
    3. # Tag the source for later traceability
    4. memory add --source ./notes/api-spec.md --text "The order query API must support cursor-based pagination"
    5. # Streaming append (must be paired with a flush to read)
    6. memory add --stream

    4.3 Writing tips

    • Clear source: add a source path for manual notes; for automatic collection, the runtime records the session identifier;
    • One thing at a time: avoid piling multiple topics into a single record;
    • Flush promptly: after a session ends or a milestone task is done, run a flush as soon as possible to prevent the buffer from growing without bound.

    5. Step 4: Flush to disk and build the index (flush)

    flush is the most critical step in the whole chain. It does three things in sequence:

    1. Read the buffer contents;
    2. Call the extraction model to generate structured memory entries;
    3. Write the entries into local Markdown files and notify the background process to build the index.

    5.1 Basic usage

    1. # Extract and persist the buffer; by default processes the most recent session
    2. memory flush
    3. # Process a specific source
    4. memory flush --source ./notes/api-spec.md
    5. # Preview the extraction results without persisting
    6. memory flush --dry-run

    When you first start out, it’s recommended to use --dry-run (if your version supports it) to preview the extraction results, then persist for real once you’ve confirmed the format and quality are correct.

    5.2 On-disk structure

    After persisting, the local directory usually looks like this:

    1. ~/.agent-memory/
    2. ├── config.toml # config file
    3. ├── buffer/ # pending buffer
    4. ├── cells/ # memory entries (Markdown)
    5. │ └── 2026-06/
    6. │ └── api-20260622.md
    7. ├── index/ # index data
    8. └── logs/ # runtime logs

    Example of a single memory entry:

    1. ---
    2. id: cell_01J8X2K4M9
    3. source: ./notes/api-spec.md
    4. created: 2026-06-22T10:31:00+08:00
    5. tags: [api, naming-convention]
    6. ---
    7. # API Field Naming Convention
    8. Response fields for order-related APIs consistently use snake_case.

    Such a file can be opened directly in an editor, searched full-text with grep, and brought into version control for diff comparison.

    5.3 Example extraction prompt

    Extraction quality depends heavily on prompt design. A referenceable template:

    1. You are a memory extractor. Extract the facts from the conversation snippet below
    2. that are worth keeping across sessions. Output each fact as a separate entry, and
    3. return a JSON array.
    4. Rules:
    5. 1. Keep only stable facts, preferences, conclusions, and decisions; ignore
    6. pleasantries and one-off actions;
    7. 2. Each entry contains three fields: content, tags, confidence;
    8. 3. Do not fabricate missing tags; return an empty array [] when there is no
    9. useful information;
    10. 4. Do not write secrets, tokens, or other sensitive information into content.
    11. Conversation snippet:
    12. {{buffer_content}}

    5.4 Idempotency and deduplication

    The same conversation may be flushed multiple times. To avoid duplicate entries, it’s recommended to:

    • Clear or mark the processed buffer after flushing;
    • Hash the entry content and check for duplicates before persisting;
    • Merge semantically similar entries and update their “last-updated” time.

    6. Step 5: Search and recall (search)

    6.1 The search flow

    Once the index is built, the search command works through these steps:

    1. Parse the query: extract keywords and, when necessary, generate a query vector;
    2. Hybrid recall: run keyword matching and vector-similarity retrieval at the same time;
    3. Fusion ranking: combine lexical and semantic scores, with tag filtering and time-decay ranking;
    4. Return results: each result includes the body, the score, and the source path.

    Example:

    1. # Basic search
    2. memory search "API field naming"
    3. # Filter by tag and return the top 5
    4. memory search "pagination" --tag api --top-k 5
    5. # Return JSON for easy integration with scripts or Agents
    6. memory search "order query" --json

    Return example:

    1. [
    2. {
    3. "id": "cell_01J8X2K4M9",
    4. "content": "Response fields for order-related APIs consistently use snake_case.",
    5. "score": 0.87,
    6. "source": "~/.agent-memory/cells/2026-06/api-20260622.md",
    7. "tags": ["api", "naming-convention"]
    8. }
    9. ]
    • Extraction quality: entries must be clearly expressed and single-purpose for recall to be precise. Long or mixed entries drag down similarity scores.
    • Index granularity and weights: entry-level indexes offer high precision, while session-level indexes offer broader recall; combine them per scenario. The weight of keywords versus semantics should be tuned to your data—raise the keyword weight for terminology-dense domains, and raise the semantic weight when there’s more natural-language description.
    • Filtering and reranking: first narrow the candidate set by tag and time range, then use time decay to lower the priority of stale memories, and finally rerank the candidates with a smaller reranking model.

    6.3 Injecting the search results

    Recalled memories should be injected into the prompt in a compact form, for example:

    1. Below are historical memories relevant to this task; please refer to them. If they
    2. conflict with the current instruction, the current instruction takes precedence:
    3. 1. [api/naming-convention] Response fields for order-related APIs consistently use snake_case
    4. 2. [preference/comm-style] Keep code-change notes concise and list only key changes

    When injecting, you should:

    • Limit the count: avoid having too many search results crowd out the context;
    • Keep the source tags: so the model can distinguish facts from instructions;
    • Make priority explicit: to prevent old memories from overriding the user’s new requirements.

    7. Troubleshooting: What to do when you can’t find a memory

    “Can’t find it” is the most common problem in the early stages. Work through it in this order:

    1. Is indexing complete? After a flush, indexing is built asynchronously by the background process. For large files or a first full index, wait a few seconds to tens of seconds and retry.
    2. Did extraction produce entries? Check whether new files appeared in the cells/ directory. If the buffer was cleared but no new entries were produced, the extraction model returned an empty result or the call failed—check the logs.
    3. Is the query hitting? Try different keywords, or filter by tag first to narrow the range, to determine whether the problem is “not extracted” or “not searched.”
    4. Do the model and dimensions match? Confirm the extraction model and embedding model are configured consistently, and that the index dimension matches the embedding output dimension.
    5. Is the similarity threshold too high? Lower the threshold appropriately and check whether the recall results look reasonable.
    6. Is the config taking effect? Distinguish global config from project-level config, and confirm the config that was actually read at startup is the one you expect.

    8. Advanced practice: Keeping local memory useful over the long term

    8.1 Layered management

    It’s recommended to divide memory into three layers to avoid cross-contamination:

    • Personal preference layer: long-term and stable, such as reply style and coding-standard preferences;
    • Project knowledge layer: architecture decisions, API contracts, environment configuration, and so on, bound to a specific codebase;
    • Session temporary layer: short-term task context that can be archived or expired once the task is done.

    8.2 Redaction and privacy

    A memory store easily becomes a collection point for sensitive information. It’s recommended to filter out API keys, tokens, ID numbers, phone numbers, and similar data before extraction, and to periodically review the memory files and clean up content that’s no longer needed. For memories that involve personal information, provide a clear way to view and delete them.

    8.3 Version control and backup

    Memory files are plain text, so they can be brought into version control, giving you a record of every change and the ability to roll back. It’s also recommended to back up the memory directory to object storage regularly, or to deploy the service containerized on a container platform, to achieve cross-device synchronization.

    8.4 Observability

    Set up basic monitoring for the memory system:

    • Collect metrics such as flush duration, index latency, and recall hit rate;
    • Configure a logging service and monitoring alerts;
    • Trigger warnings promptly when index latency keeps rising or the extraction failure rate climbs.

    8.5 Quality evaluation

    “Searchable” doesn’t mean “correct.” You can build a small evaluation set that covers common questions, manually annotate the memories that should be hit, and periodically measure the recall rate and false-recall rate to keep tuning the extraction prompt and retrieval parameters.

    9. Summary

    Moving from “wholesale transport” to a “local-first memory runtime,” the core changes come down to three points:

    • Memory is extracted from conversation text into structured entries;
    • Memory changes from temporary context into editable, versionable local assets;
    • Memory changes from “stuffing everything into the prompt” to precise recall by query.

    The five-step flow is itself a complete chain of understanding:

    • Install and demo: understand that “writing is not the same as being readable”;
    • Config generation and service startup: choose a model service, inject keys, and bring up the index process;
    • Write memory (add): continuously accumulate experience into the buffer;
    • Flush to disk (flush): complete extraction, persist to disk, and trigger indexing;
    • Search and recall (search): retrieve relevant memory by query and inject it into the prompt.

    Once you’ve mastered this chain, an Agent’s memory is no longer temporary context that vanishes when the session ends, but a local knowledge asset that keeps accumulating across sessions and tools.