Long-Context LLMs in Practice: Real Behavior at 200K and a Guide to Long-Horizon Task Pitfalls
A practical look at how long-context models behave in coding-agent workflows, why a larger context window is not automatically better, and why journaling plus reliable compaction and recovery matter more than nominal window size for long-horizon tasks.
1. The Real Long-Context Experience: Less Dramatic Than Expected, and That Is a Good Thing
When people hear “long context,” many first think of whether a model can remember something from the beginning. In real coding scenarios, however, mature long-context behavior often feels unremarkable. You do not notice the model emphasizing or repeating earlier content, but when it needs to reference earlier rules, interface agreements, or code structure, it can still retrieve them correctly.
Take a common 200K context window as an example. In practical coding-agent scenarios, model behavior can be summarized at several levels.
1. Implicit memory rather than explicit repetition
A stable model does not repeatedly “recite” the system prompt or rule documents from the beginning, such as AGENTS-style configuration files. Instead, it invokes them accurately when needed. This “not emphasized, but remembered” state is ideal for long coding collaboration. By contrast, some models keep recalling initial rule information with high weight, which leads to redundant output and rigid style, making long conversations feel unnatural.
2. Style differences across models are obvious
Even with similar context lengths, different models can have very different “personalities” in long conversations. Some models use vivid and flexible language and may produce unexpected expressions in their replies. Others are specialized for code and lean toward a “tool-like” style: stable but less flexible. This does not affect functional correctness, but it significantly affects the user experience and is worth considering during model selection.
3. Versions iterate quickly, so conclusions age quickly
It is important to stress that any conclusion about usage experience with a specific version may remain valid for only one or two months. Models iterate very quickly, and perceivable changes in language style and chain-of-thought length can appear between versions in the same series. Therefore, evaluating long-context capability should be based on reproducible test methods rather than one-off subjective impressions.
2. A Common Misconception: “The Bigger the Window, the Better”
A long-standing mismatch exists in the industry. Some models did have leading context windows two or three years ago, so many articles described this as their killer feature. Those articles were absorbed into the training corpora of later models, and models learned to repeat the claim. When users ask AI, AI answers the same way, forming a self-reinforcing cycle in communication terms.
In reality, for genuine long-horizon coding tasks, a 100K–200K window range is already enough to cover the vast majority of scenarios. Within this range, mainstream models generally do not show obvious degradation. By comparison, the marginal benefit of pushing the window to the 1M level is far smaller than doing the following two things well:
- Writing process files: Have the agent continuously persist阶段性 outputs, key decisions, and task状态 to disk. [Note: This phrase should be fully English in final; see corrected below.]
- Reliable compaction and recovery mechanisms: Ensure that tasks can continue correctly after context compaction is triggered.
An inequality summarizes this mismatch: nominal window size ≠ practical long-horizon capability. A 262K window is not necessarily worse than a 1M window; what matters is the engineering quality of the harness.
3. The Real Bottleneck in Long-Horizon Tasks: Compaction and Recovery
For genuinely long-horizon tasks, such as automated engineering work that lasts several hours and consumes hundreds of millions of tokens, context compaction is inevitable. A task that never triggers compaction is not really a long-horizon task. Therefore, long-horizon stability depends not on window size but on harness-level design. The following are key practices.
1. Journaling process files
Write task goals, completed steps, pending items, and key interface agreements to disk files instead of relying on conversation history. For example:
# .task/state.md## GoalRefactor the auth module and ensure tests pass## Completed- [x] Review existing auth logic (see docs/auth-analysis.md)- [x] Split session management## TODO- [ ] Migrate token validation logic- [ ] Add unit tests
2. Correct continuation after compaction is triggered
The harness must ensure that after compaction: the task state file is reinjected, in-progress tool calls are not interrupted, and file-edit offsets are not invalidated. The compaction summary should retain “decisions and constraints” and discard “procedural dialogue.”
3. Layered context management
- System layer: stable rules and tool definitions;
- Project layer: static documents such as architecture descriptions and code standards;
- Session layer: the dynamic state of the current task.
During compaction, prioritize the first two layers and rebuild the session layer as needed.
4. Selection Advice
Overall, current mainstream coding models have been specifically optimized for long context and long-horizon tasks, and their practical performance is generally solid. There is no need to worry excessively about “pitfalls.” When selecting a model, consider three points:
- Test rather than just reading parameters: Use long tasks from your own scenarios for A/B validation; nominal window values have limited reference value.
- Check style fit: In long collaboration scenarios, a model’s language style and “proactiveness” matter just as much.
- Invest in your harness: Process file writing plus reliable compaction and recovery improve long-horizon task stability more than switching models.
Competition in long-context capability is shifting from a “window arms race” to a contest of engineering usability. For developers, building solid infrastructure means any 200K-level model can handle most long-horizon work.
