Context Compression and Stable Resumption in Long-Horizon LLM Tasks

    Long-horizon tasks involving large language models inevitably require context compression as the message history grows. This article explores the risks associated with context compression, including information loss, state drift, continuation breaks, and attention dilution. It introduces a practical approach using process files to externalize task state and outlines a structured resumption protocol to ensure reliable execution. Additionally, it covers best practices for compression timing, evaluation methods for model reliability under long contexts, and common pitfalls to avoid.

    1. Why Long-Horizon Tasks Inevitably Trigger Context Compression

    Long-horizon tasks refer to tasks that require tens or even hundreds of tool calls and span a long period to complete, such as large-scale code migration, long document structured analysis, and multi-stage data cleaning with report generation. The message history of such tasks grows continuously. When the usage approaches the model’s context window limit, the execution framework typically summarizes and compresses the historical content, replacing early conversations with shorter summary text.

    Therefore, compression is a normal event in long-horizon tasks. If a task never triggers compression, it usually means it has not yet reached the “long-horizon” scale. When designing a system, you should assume that “compression will definitely happen” as a prerequisite, rather than treating it as an occasional failure.

    2. Four Risks Brought by Compression

    1. Information Loss. Summarization is lossy. Constraints written early on, confirmed decisions, and failed attempts may all be omitted in the summary. A typical manifestation is the model walking into the same dead end again later.

    2. State Drift. If the summary only states “data import completed” without specifying the scope and validation results, the model might misjudge the progress or repeatedly execute operations with side effects, such as writing or deleting.

    3. Continuation Breaks. If tool call requests and returns are truncated or mismatched during compression, the model might output malformed calls or continue reasoning on incorrect premises.

    4. Attention Dilution. Even without triggering compression, the longer the context, the lower the model’s utilization of early details. A long window can accommodate content, but that does not mean the model can stably use every part of it.

    These four risks indicate that relying solely on the model’s context window as “memory” is unreliable. A reliable approach is to move critical state outside the context and have the framework actively feed the state back to the model upon each resumption.

    3. Process Files: Moving State Out of Context

    The core idea is to write critical state to external files, keeping only pointers to these files and necessary recent information in the context. Process files typically include the following categories:

    File Content Purpose
    Task Spec Goals, constraints, I/O descriptions, acceptance criteria Re-injected as-is after compression, without relying on summaries
    Plan Checklist Phased steps, each with a status flag Locate current progress and next action
    Progress Log Timestamp, input, and result summary of each action Trace side effects that have occurred
    Decision Record Key choices and reasons, rejected alternatives Prevent the model from repeatedly discussing the same issue
    Artifact Index Intermediate file paths, generation time, checksum Verify output completeness during resumption

    When writing process files, three principles should be followed.

    First, write before doing. Before executing an action with side effects, record “what is about to be executed”; after the action is completed, record “what the actual result is.” This way, even if the process is interrupted midway, you can determine whether the action has taken effect.

    Second, idempotent operations. When writing or updating the status of a step, use overwrite or atomic replacement to avoid confusion caused by duplicate records.

    Third, human-readable. Process files serve as the model’s external memory and as a basis for developers to troubleshoot issues. Use clear structured text for easy manual inspection.

    Below is a sample code snippet demonstrating how to log before and after each action:

    1. import os
    2. import time
    3. STATE_DIR = "task_state"
    4. def _ensure_state_dir():
    5. os.makedirs(STATE_DIR, exist_ok=True)
    6. def log_event(stage: str, message: str):
    7. """Append a progress record with a timestamp for tracing during resumption."""
    8. _ensure_state_dir()
    9. ts = time.strftime("%Y-%m-%dT%H:%M:%S")
    10. path = os.path.join(STATE_DIR, "progress.log")
    11. with open(path, "a", encoding="utf-8") as f:
    12. f.write(f"[{ts}] [{stage}] {message}\n")
    13. def run_step(step_id: str, action):
    14. """Log intent first, execute action, then log the result."""
    15. log_event(step_id, "START")
    16. try:
    17. result = action()
    18. log_event(step_id, f"DONE result={result}")
    19. return result
    20. except Exception as e:
    21. log_event(step_id, f"FAILED error={e}")
    22. raise

    The key point of this code is that START and DONE/FAILED appear in pairs. During resumption, if a step only has a START without an end record, the framework knows that the step’s side effect status is uncertain and needs to be verified before deciding whether to redo it.

    4. Resumption Protocol After Compression

    Compression completion does not mean the task can simply continue. The framework needs to execute a fixed resumption protocol after compression to ensure the state received by the model is complete and accurate. The recommended steps are as follows:

    1. Re-inject the task spec. Read the original text of the spec file directly; do not rely on the paraphrase in the summary.
    2. Locate the current step. Read the plan checklist to find the first step that is incomplete or in a START state.
    3. Restore recent context. Read the last few records of the progress log as the basis for recent events.
    4. Verify artifacts. Check the artifact index to see if files exist and checksums match. If missing, mark the corresponding step for a redo.
    5. Explicitly declare the resumption state. Tell the model that it is in a resumption state, which steps are already completed, and forbid repeated execution.

    Sample code is as follows (helper functions like reading files or fetching the last lines are omitted):

    1. def build_resume_prompt(state_dir: str) -> str:
    2. spec = read_text(os.path.join(state_dir, "task_spec.md"))
    3. plan = read_text(os.path.join(state_dir, "plan.md"))
    4. recent = read_last_lines(os.path.join(state_dir, "progress.log"), n=50)
    5. next_step = find_first_pending_step(plan)
    6. return (
    7. "You are resuming an already started task. The following content is the authoritative state. Please rely on this.\n\n"
    8. f"[Task Spec]\n{spec}\n\n"
    9. f"[Current Plan]\n{plan}\n\n"
    10. f"[Recent Progress]\n{recent}\n\n"
    11. f"Please continue from step {next_step}. Do not repeat steps marked as completed."
    12. )

    The key design here is: the authoritative state comes from files, not from the compressed summary. The summary can only serve as auxiliary information.

    5. Compression Trigger Timing and Summary Quality

    Do not wait until the hard limit to compress. Compressing only when the window is near full capacity means the compression itself needs space, and the model’s performance is usually more unstable in the high-usage range. A soft threshold can be set to actively compress when usage reaches a certain proportion. The specific ratio should be adjusted based on your own task’s performance; the values below are just a starting point.

    Persist to disk before compressing. Before generating the summary, write the critical information that needs to be retained into process files. The summary is responsible for generalizing, while the files are responsible for fidelity.

    Structured summaries. Free-text summaries are prone to missing key points. It is recommended to use fixed fields, such as:

    • Task goals and core constraints
    • Completed steps and their results
    • Incomplete steps and the next action
    • Attempted and failed solutions, and the reasons for failure
    • Important file paths and identifiers

    Fixed fields force the summary to cover critical dimensions and make subsequent automated checks easier.

    6. How to Evaluate Your Reliable Range

    Different models vary greatly in their stable range under long contexts, and the same model may perform differently on different task types. Therefore, you should not rely on an empirical value to judge “how long is safe.” Instead, you should evaluate your own tasks.

    A viable method is as follows:

    1. Construct a probe set. Embed facts that the model needs to remember and use at different depth positions in the long text, such as a specific parameter value or an exclusion condition.
    2. Measure across intervals. Ask questions under different context lengths and calculate the recall accuracy.
    3. Test post-compression performance. After triggering compression, check the constraint adherence rate and the number of repeated executions.
    4. End-to-end run. Use the complete process of a real task to verify the resumption success rate.
    5. Continuous regression. Re-run the same evaluation suite after changing model versions or modifying the framework.

    Recommended metrics to monitor include:

    • Fact recall accuracy
    • Constraint adherence rate
    • Resumption success rate (the proportion of remaining steps completed after recovering from an interruption)
    • Repeated execution count (situations where the same side-effect action is executed multiple times)

    These metrics better reflect the true reliability of the system than a simple “can it run through” check.

    7. Common Pitfalls

    Treating context as storage. Context is the model’s workspace, not persistent storage. Process restarts, session switches, or compression will cause its contents to disappear or distort.

    Testing only with short tasks. Many issues only surface after compression. If the test set consists entirely of tasks that can be finished in a few rounds of conversation, you will not discover defects at the resumption level.

    Ignoring resumption costs. Each resumption requires re-injecting the spec and plan, which incurs extra token consumption. This cost needs to be factored into the task budget estimation.

    Using runtime duration to judge state continuity. A long task duration does not mean the model’s state is continuous. Continuity should be judged based on process files and validation results, not on whether the session was interrupted.

    8. Implementation Checklist

    • Are task spec, plan checklist, progress log, and artifact index all persisted to disk?
    • Do actions with side effects use paired start and end records?
    • Is critical information persisted to disk before compression?
    • Is the original spec re-injected during resumption, rather than just the summary?
    • Are artifacts verified and missing steps marked after resumption?
    • Has an evaluation set covering different intervals been established for your tasks?
    • Does the evaluation cover performance before and after compression, with regression after version changes?

    The stability of long-horizon tasks largely depends on the framework’s state management capabilities. Externalizing state, fixing the resumption process, and using real tasks for evaluation are the foundational work for running long-horizon tasks reliably.