Long-Term Agent Memory: Consolidation and Context Injection Budget Practices

Long-term agents generate vast amounts of intermediate information during continuous tasks: user preferences, task constraints, tool call results, failure reasons, and interim conclusions. Without a memory mechanism, agents tend to repeat questions and trial-and-error cycles. Conversely, naively stuffing all historical information into the context introduces issues with cost, latency, and attention dilution. Recent industry discussions highlighting the capability of “automated memory injection budgets at the level of hundreds of lines or tens of kilobytes” indicate that the real focus should not be on the slogan of “infinite memory.” Instead, it should be on dividing memory into three stages: in-session notes, offline consolidation, and context injection budgets, acknowledging the hard constraints on the injection side. This approach aligns closer to engineering reality: memory is not about having more, but about placing information of the correct granularity and credibility into a limited context at the right time.

1. The Memory Trilemma: Wrong, Fragmented, and Exploded

Long-term agents primarily fear three types of issues. The first is “recording wrong”: writing model guesses, temporary conclusions, or outdated information as long-term facts, which continuously pollutes subsequent tasks. The second is “recording fragmented”: similar information scattered across multiple notes without merging or thematic structure, leading to retrieval hits of numerous low-value fragments. The third is “recording exploded”: memory entries growing infinitely, exceeding the context budget during injection, or, even if not exceeding the limit, displacing attention with irrelevant information and causing key constraints to be ignored. The root cause of all three is the lack of layering in the memory lifecycle: writing, consolidation, reading, and eviction are tangled together, making the system unstable.

2. Three-Layer Architecture: Session Notes, Offline Consolidation, Injection Budget

A more rational approach is to split the memory system into three layers. The in-session notes layer is responsible for “timely recording”: during task execution, it logs candidate facts, todos, constraints, tool result summaries, and failure experiences, but only as drafts, not directly equivalent to long-term memory. The offline consolidation layer handles “dream-like consolidation”: at the end of a session or during off-peak hours, it performs deduplication, merging, conflict resolution, summary compression, hot/cold tiering, and expiration marking on candidate notes. The injection budget layer is responsible for “restrained reading”: it retrieves relevant memories based on the current task, assembles context fragments by priority and budget, and sets hard limits on line count, byte count, and entry count. Once split into three layers, writing can be loose, consolidation must be strict, and injection must be restrained.

3. Why Injection Budgets Must Be Hard Limits

No matter how large the context window is, there are costs. Full injection brings four problems: cost grows with entries, latency grows with context, attention is diluted, and the spread of erroneous memories expands. The value of a hard budget lies in forcing the system to make choices: which memories are task-essential, which only need summaries, and which should remain in external storage waiting for retrieval. Engineering-wise, multi-level budgets can be set: total byte budget, single-item byte budget, single-topic entry budget, and low-confidence entry budget. The budget is not a simple truncation; it involves ranking by “task relevance, confidence, freshness, usage frequency, conflict status” and then assembling. For over-budget content, prioritize generating summaries or retaining reference identifiers rather than pouring in the original text.

4. Designing the Offline “Dreaming” Consolidation Mechanism

Offline consolidation can be designed as a pipeline: candidate extraction, merge and dedupe, conflict resolution, summary compression, hot/cold tiering, and version rollback. Candidate extraction identifies stable facts, user preferences, task constraints, and reusable experiences from session notes, annotating source and confidence. Merge and dedupe combines similar entries by topic, entity, and time window to avoid multiple copies of the same fact. Conflict resolution should not simply overwrite; it should retain old and new versions, sources, and times, marking them as “pending confirmation” or “subject to latest user confirmation.” Summary compression turns long notes into short facts and searchable keywords, leaving the original text in external storage. Hot/cold tiering places high-frequency, highly relevant memories in the hot layer, low-frequency ones in the cold layer, and expired information into archives. Version rollback ensures recovery if consolidation goes wrong, preventing a single erroneous merge from polluting long-term memory.

5. Injection Strategies: Retrieval-Based, Tiered, and Budget-Based

For injection, it is recommended to use “retrieval-based injection” rather than “full injection.” First, generate a retrieval intent based on the current task, recall relevant topic memories, and then assemble them according to the budget. The assembly order can be: hard constraints and safety rules first, explicit user preferences second, current task status and incomplete todos third, and historical experience and background summaries last. Each memory entry should carry confidence, update time, source type, and conflict markers. Low-confidence content should only serve as a hint, not a factual assertion. When over budget, prioritize retaining summaries and references, not full text. For conflicting entries, a prompt like “two versions exist, old and new” can be injected to let the model explicitly handle uncertainty.

6. Engineering Implementation Example

Below is an illustrative pseudocode independent of specific product APIs:

def build_memory_context(task, memory_store, budget_bytes):
    candidates = retrieve(task, memory_store)  # Retrieve relevant memories
    ranked = rank(candidates, keys=["relevance", "confidence", "freshness", "usage"])
    selected, used = [], 0
    for item in ranked:
        piece = compress_if_needed(item, remaining=budget_bytes - used)
        if used + len(piece) <= budget_bytes:
            selected.append(piece)
            used += len(piece)
    return assemble(selected, include_conflicts=True)

Offline consolidation example:

def consolidate(notes):
    grouped = group_by_topic_and_entity(notes)
    merged = [merge_and_dedupe(g) for g in grouped]
    resolved = [resolve_conflicts(m) for m in merged]
    summarized = [summarize(m) for m in resolved]
    return assign_hot_cold_and_ttl(summarized)

7. Observability and Evaluation Metrics

The memory system needs observability: record the number of injected entries, byte count, retrieval hit rate, summary compression ratio, number of conflicting entries, number of expired entries, and rollback count. Evaluation metrics can include task success rate, repeated question rate, constraint forgetting rate, error memory propagation rate, context cost, and time to first token. Offline consolidation should also conduct sample audits to check whether merging lost key information or whether conflict resolution favored incorrect sources. Only by linking memory quality with task effectiveness can we avoid “consolidating vigorously without improving task performance.”

8. Implementation Recommendations

First, treat automated memory as a managed agent or research-oriented capability, rather than an instant toggle in a chat interface; production systems should clarify the boundaries between writing, consolidation, and injection. Second, the injection side must set hard budgets and prioritize retaining high-confidence, strongly relevant, and relatively fresh memories. Third, offline consolidation needs version control and rollback, with conflicting information explicitly retained. Fourth, perform classification and desensitization on privacy and sensitive information to prevent long-term memory from becoming a leakage surface. Fifth, evaluate memory benefits on a small-scale task first before gradually expanding the scope. Ultimately, the memory capability of long-term agents competes on consolidation quality and injection discipline, not storage limits.