Long-Context Long-Running Tasks: Stable Resumption Through State Persistence and Context Compression
This article explains why context compression is unavoidable in long-running tasks, identifies the key risks of extended context windows, and presents a practical engineering approach: moving state from the context window to persistent files. It covers file-based state management, pre-compression checkpointing, minimal resume prompts, post-resume validation, and verification methods for compression quality, along with common operational pitfalls and a checklist for production use.
Why Long-Running Tasks Cannot Avoid Context Compression
Long-running tasks typically require multiple rounds of reasoning, repeated tool invocations, and hours or even days to complete—examples include large-scale codebase migrations, multi-document analysis, and long-form data cleaning with report generation. The context of such tasks accumulates rapidly as tool outputs, intermediate reasoning, file contents, and error logs pile up, often exceeding the model’s context window.
Therefore, compression is not an anomaly but the norm for long-running tasks. If a task never triggers context compression from start to finish, its scope is probably not long-running. The real engineering challenge is whether the system can correctly resume after compression—not restart from scratch, lose constraints, or repeat already completed steps.
In practical scenarios, cumulative token consumption can reach hundreds of millions. This means compression and resumption are foundational requirements, not optional enhancements.
Two Major Risks with Long Contexts
Attention Degradation
Models are not equally sensitive to all positions in very long inputs. A common observation in industry research is that information in the middle of the context is more easily ignored, while content at the beginning and end is more readily used. When the context is filled with tool outputs and historical dialogues, key constraints set early may gradually lose influence.
Different models have vastly different effective context ranges, and these also depend on task type and information density. Therefore, it is unwise to treat a fixed token count as a “safe threshold.” Instead, construct evaluation samples specific to your tasks and observe performance changes across different lengths.
Information Loss During Compression
Compression is usually achieved through summarization. Summaries are inherently lossy, and common losses include:
- Specific file paths, function names, and parameter values;
- Explicit negative constraints from the user, such as “do not modify a certain directory”;
- Already attempted and failed approaches, causing repeated mistakes later;
- Incomplete subtasks and their dependency relationships.
Once this information is lost, the model may make decisions that contradict earlier choices in subsequent steps, and such mistakes are often hard to notice.
Error Accumulation
In long-running tasks, small early deviations can amplify along the chain of steps. If the state record of one step is wrong, all subsequent steps will be built on an incorrect premise. Therefore, persisting and validating key state is more reliable than relying solely on context memory.
Core Principle: Move State from Context to Files
The context window is suitable as temporary working memory, but it is volatile. Reliability in long-running tasks should be built on persistent state—in other words, “context handles reasoning, files handle memory.”
It is recommended to maintain the following types of files:
- Task specification file: Records goals, scope, constraints, and acceptance criteria. Written at task start, later only appended or explicitly revised.
- Progress state file: Records completed steps, current step, next action, and blockers.
- Decision log: Records key decisions and their rationale, preventing re-discussion of settled issues after compression.
- Artifact index: Records generated file paths, versions, and purposes, enabling quick recovery.
A workspace directory structure can be organized as follows:
workspace/├── spec/│ └── task_spec.md # Goal, constraints, acceptance criteria├── state/│ ├── state.json # Progress and next action│ └── decisions.md # Key decision log├── artifacts/ # Intermediate outputs└── logs/ # Tool call logs (truncated before saving)
A fundamental rule is “write before act”: before executing a step with side effects, write the plan into the state file; immediately after completion, update the state. This way, even if compression or process interruption occurs mid-way, the system can continue from the most recent record.
Resume Mechanism After Compression
Before Compression: Force State Flush
The runtime framework should complete two tasks before compression is triggered: first, ensure the current progress has been written to the state file; second, append any intermediate conclusions not yet persisted to the decision log. The compression process itself should not be responsible for saving state.
After Compression: Minimally Reconstruct Context
After compression, the framework needs to re-inject a compact resume prompt containing the task goal, key constraints, current progress, and the next action. The resume prompt should not contain the full history; instead, it should point to the state file and let the model read on demand.
Below is a simplified implementation example showing state read/write and resume prompt construction:
import jsonimport osfrom pathlib import PathSTATE_FILE = Path("workspace/state/state.json")def load_state() -> dict:"""Read progress state; return initial structure if file missing."""if STATE_FILE.exists():return json.loads(STATE_FILE.read_text(encoding="utf-8"))return {"goal": "", "done": [], "next_step": "", "blocked": []}def save_state(state: dict) -> None:"""Atomic write: write to temp file, then replace target to avoid corruption."""STATE_FILE.parent.mkdir(parents=True, exist_ok=True)tmp_path = STATE_FILE.with_suffix(".tmp")tmp_path.write_text(json.dumps(state, ensure_ascii=False, indent=2),encoding="utf-8",)os.replace(tmp_path, STATE_FILE)def build_resume_prompt(state: dict) -> str:"""Generate a resume prompt with minimal info needed to continue."""done_items = ";".join(state["done"]) or "None"blocked_items = ";".join(state["blocked"]) or "None"return (f"Task goal: {state['goal']}\n"f"Completed: {done_items}\n"f"Blockers: {blocked_items}\n"f"Next step: {state['next_step']}\n""First verify workspace files match the state record, then continue.")
Atomic replacement is the critical detail. If the target file is overwritten directly and the process is terminated mid-write, the state file may be corrupted, and resumption would read incomplete content.
After Resume: Validate First, Then Execute
The first step after resumption should not be to continue directly. Instead, verify that the state record matches the actual environment. For example, if the state file claims a file has been generated, check whether that file exists and its content meets expectations. If validation fails, mark the corresponding step as incomplete and record the reason.
Verifying Compression Quality
Whether the compression mechanism is reliable must be proven through testing, not subjective judgment. Establish validation methods from the following aspects.
Summary Fidelity Checks
At task start, extract the constraints that must be preserved into a checklist, such as “do not modify a directory” or “keep the interface backward-compatible.” After compression, check whether the resume prompt or summary still contains these items. Missing items should trigger an alert or be merged directly into the resume prompt.
Fixed Checkpoint Tests
Design several checkpoints for long-running tasks, each corresponding to a mechanically verifiable result, such as the number of passing tests, file count, or the hash of a specific output. Run the same checkpoint under different context lengths and compare results to quantify the impact of compression on task quality.
Resume Consistency Tests
Manually insert a compression in the middle of a task, resume from the state file, and then compare the final artifacts with those from a run without compression. If there are differences in key fields, it indicates that the state record has omissions that need to be supplemented.
Additional Common Engineering Issues
Oversized tool output. A single command output may contain tens of thousands of log lines, which would quickly exhaust the context window if placed directly into the context. The recommended approach is to write the full output to a log file and keep only a summary, key lines, and the file path in the context.
Context budget allocation. System instructions, tool definitions, memory content, conversation history, and current input all share the window. Set limits for each category, and when exceeded, prioritize compressing historical conversation rather than squeezing system instructions.
Write scope restrictions. State files and artifacts should be confined to a designated working directory to prevent the agent from writing to system paths. Credentials and personal information must not be written to state files; credentials should be injected via environment variables or a key management service.
Cost and latency. Long contexts increase both token consumption and response time. Estimate expected usage per task during design, and set abort or confirmation mechanisms for cases that exceed the budget.
Practical Checklist
- Write a specification file before task start, including goal, constraints, and acceptance criteria.
- Update the state file before and after every step with side effects.
- Force state flush before compression; compression itself does not save state.
- Use atomic writes for state files to avoid corruption due to interruption.
- Keep the resume prompt minimal and point to the state file.
- After resumption, verify environment and state consistency before continuing.
- Perform fidelity checks for critical constraints; alert on missing items.
- Validate the compression mechanism with fixed checkpoints and resume consistency tests.
- Write tool outputs to files and keep only summaries and paths in the context.
- Clearly define limits for context budget, write scope, and resource usage.
The stability of long-running tasks depends on the framework’s ability to resume correctly after compression. Keeping state in files, letting context focus on reasoning, and testing every recovery outcome is the currently most reliable engineering path.
