# Long Context in AI Coding: Real-World Experience, Limits, and Pitfalls to Avoid

    Long context is often marketed as the decisive advantage of AI coding tools. In practice, window size is only one of three separate capabilities, and for most coding work it is not the binding constraint. This article breaks down context capacity, information retention, and long-horizon planning; describes the three behavior patterns you can observe in coding agents today; explains why "bigger context wins" comparisons are methodologically weak; and lists the six places long-horizon tasks actually break. It then covers the engineering practices that keep long tasks reliable: progress files that move state out of the context, layered loading of project rules, compaction summaries designed as a resumable protocol, sub-agent dispatch with structured returns, machine-checkable rules, and context budgets with observability. Finally, it outlines how to evaluate tools with your own tasks instead of asking models to rank themselves.

    1. First, Clear Up a Confusion: Long Context Is Not the Same as Usable Long Context

    “A model’s long-text ability is its killer feature.” “The bigger the context window, the more reliable complex tasks become.” Both claims circulate widely among developers, and both blur three capabilities that are independent of each other, need to be measured separately, and have to be optimized separately.

    Capability Question it answers Common metric
    Context capacity How many tokens fit into a single request Context window size
    Information retention Can what was fed in be retrieved accurately and acted on correctly Key-fact recall, cross-turn consistency, instruction adherence
    Long-horizon planning Can multi-turn, multi-hour, even multi-day tasks keep advancing Task completion rate, number of human interventions, tokens per task

    Most everyday coding requests are only a few thousand to a few tens of thousands of tokens, so window capacity is rarely the constraint. What actually determines the experience are the other two, and they are what developers really mean by “long context” in coding:

    • Large projects with many files that depend on each other;
    • Long-horizon tasks that span hours in a single run;
    • Multi-turn iterative development that has to keep following project-level rules.

    A common misconception is “a big window means it definitely remembers and uses what it read.” In engineering practice, the more useful question is whether high-frequency, high-stakes information (project rules, the current goal, facts already verified, the next planned step) keeps being reinforced and preserved across repeated iteration and multiple turns.

    So evaluating a coding tool’s long-context ability really comes down to two questions:

    1. In the everyday 100K–200K token range, can the model “remember without reciting” — no redundancy, no repeated confirmations, but accurate citation when needed?
    2. When a longer context simply does not fit and compaction is guaranteed to kick in, can the task resume correctly?

    2. What You Actually Experience: Three Behaviors Worth Watching

    For a coding agent, the same context length can feel completely different depending on behavior. Based on hands-on use, three patterns stand out.

    2.1 Stable Retention: “It Remembers, but Doesn’t Belabor It”

    Observed behavior:

    • Does not keep restating early information, and rarely asks “what did you say before?”;
    • Still cites early agreements, the file layout, and completed work correctly;
    • Interaction rhythm is steady; the process feels unobtrusive.

    For long-horizon coding tasks this is the most comfortable pattern. It means retention is sufficient, context is not being occupied by pointless repetition, and the model is spending its token budget on continuing the work. In the 100K–200K token range, this is generally table stakes for mainstream coding models today.

    2.2 Rule-Heavy: “Terrified of Forgetting the Original Constraints”

    Observed behavior:

    • Repeatedly revisits project-level rule files (coding standards, directory conventions, build and test commands);
    • Keeps re-emphasizing initial constraints deep into a long context, with higher rule adherence.

    This cuts both ways:

    • Upside: across many iterations nothing drifts — no casual restructuring of directories, no quietly bypassing the agreed test commands;
    • Cost: repeated confirmation and verbose replies consume effective tokens, responses slow down, and the interaction feels chatty.

    By comparison, some general-purpose variants mention rules noticeably less often when facing a context of the same length. This may relate to targeted optimization for coding scenarios, but the exact cause cannot be confirmed from outside observation alone. Practical guidance:

    • If the task is highly sensitive to rule constraints, rule-heavy behavior is the safer choice;
    • If response pace and token efficiency matter more, stable-retention behavior is usually smoother;
    • Either way, key rules belong in mandatory harness checks, not in the hope that the model “remembers.”

    2.3 The Boundary Zone: 100K–200K Tokens Is Where “Usable” Meets “Compaction Is Inevitable”

    An easily overlooked reality: 100K–200K tokens is already long context for a good number of coding tools.

    • In this range, mainstream coding models typically show no obvious degradation; retention and instruction adherence remain broadly usable;
    • Beyond it, no matter how large the window, compaction will eventually trigger as long as the task is long enough. Tasks at the scale of hundreds of thousands of tokens or more are never “loaded in one go” — they are held together by a compaction-and-resume loop.

    In other words, the stability of a long-horizon task does not depend on how large a single window is, but on whether the task resumes correctly after compaction. That is the core issue of the next sections.

    3. Why “Long Context Is the Killer Feature” Doesn’t Hold Up

    Putting a 262K-class window next to a million-token-class window in the same ranking, and then treating the relationship as an inequality, is methodologically flawed. Three reasons:

    1. A single measurement dimension. Window size is only a capacity metric, while the real experience is the product of capacity, information retention, and task completion. No window replaces context engineering techniques such as retrieval, summarization, and externalizing state;
    2. Versions keep changing. Models iterate on a monthly or even weekly basis. Test conclusions from a few months ago may quickly become stale — conversational style and rule adherence both shift;
    3. Conclusions get self-reinforced by training data. Two or three years ago, a certain category of product did lead on long context, and articles at the time called it a “killer feature.” Later, new models trained on large volumes of that content, so when asked, they repeat the same “long-text ability is a killer feature” line. This is marketing language reinforcing itself and should not be used as technical evidence.

    Also worth watching: a coding tool’s performance on a specific project is influenced by three factors:

    • The context management mechanism;
    • Behavioral constraints in the system prompt;
    • Version differences, such as the stylistic gap between general and code-specialized variants.

    These factors can make the same underlying model behave very differently across products, which limits the value of comparing them out of context.

    4. Where Long-Horizon Tasks Actually Break

    There is no need to worry about “whether there are pitfalls,” but the pitfalls are not in “whether the context is long enough.” They are in how context is managed. Six typical problems:

    1. Broken links when compaction fires. Compaction summaries usually lose details such as file paths, command output, and trade-offs already confirmed. The model may then redo finished work, or “forget” which approach it already failed with;
    2. Large original files permanently occupy the window. Long source excerpts and error logs read early stay resident, and later iterations get drowned in noise;
    3. Early instructions get diluted. The farther away they are, the lower the probability that project rules and acceptance criteria are honored;
    4. State lives only in the context. Progress, to-dos, and verification results all depend on conversation history, so once compaction happens or the model is switched, they cannot be reliably reconstructed;
    5. Interruptions and quota limits break the flow. Timeouts, rate limiting, or an exhausted session quota can all stop a task. A task that relies only on conversation history has to be re-explained from scratch when it restarts;
    6. Looping trial and error. Without explicit state, the model keeps “trying once more.” Each attempt appends more logs and errors, the context inflates fast, and budget and wall-clock time go out of control together.

    5. Turning Long Context into a Manageable Resource: Core Engineering Practices

    5.1 Progress Files: Get State Out of the Context

    Keep one unified state file per task and update it as work proceeds. Key design points:

    • Atomic writes (write to a temp file, then replace), so a mid-way crash cannot corrupt the file;
    • Structured content, so a post-compaction model can read it directly;
    • A next field, so every resume has an explicit entry point.
    1. # progress.yaml — one per task, atomically updated as work proceeds
    2. task_id: idempotent-order
    3. goal: "Add idempotency to the order creation endpoint and pass integration tests"
    4. phase: implement # explore | design | implement | verify | done
    5. done:
    6. - "Located the creation entry point order/service.py:create_order()"
    7. - "Added a unique index on the idempotency key (migration applied)"
    8. next:
    9. - "Wire idempotency checks into create_order() and handle conflict branches"
    10. - "Run make test-integration and fix the failing cases"
    11. facts: # verified facts only
    12. - "Duplicate submissions raise a unique-key conflict in the DB instead of returning the existing order"
    13. artifacts: # deliverable paths and their status
    14. - "migrations/20260828_idem_key.sql (applied)"
    15. - "order/service.py (in progress)"
    16. verification:
    17. cmd: "make test-integration"
    18. last_result: "12 passed, 2 failed (duplicate-submission cases)"
    19. constraints: # only the few most critical constraints, to survive across turns
    20. - "Do not change the public request body structure"
    21. - "Failures return a unified error code; never throw exceptions at callers"

    5.2 Layered Loading: Put Rules at Every Critical Entry Point

    Borrow the idea of multi-level storage from operating systems: keep “project rules, task state, source detail, retrieval results” in separate layers. Treat the context as memory and the file system as disk — page things in on demand rather than loading everything at once. A context conventions file can be organized like this:

    1. # Project context conventions (excerpt)
    2. ## Required commands
    3. - Build: build ... / Test: test ...
    4. - Local tests must be fully green before any commit
    5. ## Directory conventions
    6. - Business logic goes in src/; tests go in tests/, mirroring the source layout
    7. - Shared error definitions live in src/common/errors; modules must not define their own
    8. ## Change discipline
    9. - Each round of changes focuses on a single task goal
    10. - After finishing each phase, update done / next / verification in progress.yaml
    11. - Before touching code, read progress.yaml and follow the resume protocol inside it

    To keep rules intact after compaction, critical constraints should also be enforced by the harness — for example, a commit hook that verifies tests pass, and static analysis that checks import paths and directory conventions.

    5.3 Compaction: Design It as a Resumable Protocol

    Compaction summaries should be templated and fielded, not free-form text:

    1. summary_schema:
    2. goal: "" # one-sentence goal, including acceptance criteria
    3. constraints: [] # constraints that must not be violated, at most 5
    4. state: "" # current phase
    5. facts: [] # verified facts, with evidence (commands, output summaries)
    6. assumptions: [] # unverified assumptions, marked "to be verified"
    7. next: [] # next actions, in order

    The matching resume protocol should be simple and executable:

    1. Resume steps:
    2. 1. Read progress.yaml and the project context conventions;
    3. 2. Summarize the goal and the next step in one sentence before touching code;
    4. 3. If the current state is uncertain, run make check or the relevant tests first and trust actual results;
    5. 4. Do not repeat completed steps; do not re-read archived log dumps.

    Compare the two and the difference is obvious:

    • Free-form summary: “Implemented order idempotency, ran into some issues, some tests still failing.”
    • Templated summary: names the failing cases, the migration already applied, the function still to be wired in, and the verification command.

    The information gap between the two is often exactly what decides whether the resume succeeds.

    5.4 Sub-Agent Dispatch: Not “Start Over,” but “Split the Work”

    Looking at how coding tools are designed for long-horizon tasks, one approach stands out: instead of forcing all the work into a single context, split exploration, implementation, review, and testing across sub-agents, with the parent agent holding only the plan, the state, and structured results. This is a general multi-agent pattern, and it has three benefits:

    • Noise isolation: the large logs and irrelevant code produced during exploration never enter the main context;
    • Focused capability: each sub-agent receives only the minimum context needed for its subtask;
    • Controllable budget: every sub-agent has a clearly defined task and output.

    The limitations are equally clear: coordination overhead is high, duplicated work is easy, and information can be lost during aggregation. So sub-agent results must meet a “verifiable, aggregatable, size-controlled” bar:

    1. # The parent agent keeps only structured results; raw process stays in each sub-agent's context
    2. def dispatch(task, role, tools):
    3. return agent.run(
    4. role=role, # e.g. explore / implement / review / test
    5. prompt=(
    6. f"Task: {task}\n"
    7. "Handle only the subtask you own; do not modify other modules.\n"
    8. "Your result must strictly follow this structure:\n"
    9. "- summary: no more than 5 sentences on what changed and why\n"
    10. "- files_changed: [{path, intent}], paths and intent only, do not paste whole files\n"
    11. "- decisions: [{question, choice, reason}]\n"
    12. "- open_risks: [{risk, evidence}]\n"
    13. "- next: 1-3 actions you recommend the parent agent take\n"
    14. "Mark unverifiable conclusions as 'to be verified'; speculation is forbidden."
    15. ),
    16. tools=tools, # grant only the minimum tool set the subtask needs
    17. read_only=role in ("review", "test"),
    18. max_turns=30, # per-task budget cap to prevent endless trial and error
    19. )

    Three lessons matter when putting this pattern into practice:

    • The parent agent receives “results,” not “process”;
    • Sub-agents get only the minimum tool set and minimum context needed for their subtask;
    • Every sub-agent has a turn or token cap, so a single branch cannot spiral out of control.

    5.5 Make Rules Machine-Checkable

    1. from pathlib import Path
    2. import re
    3. RULES_FILE = Path("docs/conventions.md")
    4. def lint_rules():
    5. text = RULES_FILE.read_text(encoding="utf-8")
    6. issues = []
    7. if len(text) > 4000: # heuristic: long rule files get diluted in long tasks
    8. issues.append("Rule file is too long; split it or reference by domain")
    9. for required in ("Required commands", "Directory conventions", "Change discipline"):
    10. if required not in text:
    11. issues.append(f"Missing required section: {required}")
    12. for line in text.splitlines():
    13. if line.startswith("- ") and "example" not in line:
    14. continue
    15. # Check that conventions are actually followed: import paths vs directory conventions
    16. for py in Path("src").rglob("*.py"):
    17. for module in re.findall(r"from\s+([\w.]+)\s+import", py.read_text(encoding="utf-8")):
    18. if module.startswith(("requests", "pandas")):
    19. issues.append(f"{py}: business code must not depend directly on third-party library {module}")
    20. return issues

    More importantly, turn the hard constraints in your rules into CI checks, static analysis, and commit hooks. Models forget; pipelines do not.

    5.6 Context Budgets and Observability

    Define and continuously monitor these budgets in your harness:

    • Upper bound on new context added per turn;
    • Cumulative token cap per task;
    • Threshold for consecutive turns without progress. Once exceeded, force a state cleanup and replan instead of continuing to “try once more.”

    On observability, record at least:

    • The summary before and after each compaction;
    • Time and token consumption per phase;
    • Number of failed retries and human interventions.

    On top of that, you can archive task definitions, progress, and summaries to object storage, route run logs to a logging service, and configure monitoring and alerts for metrics such as compaction frequency and failure rate. Whether you build your own harness or rely on general cloud capabilities from platforms such as Baidu AI Cloud, this observability setup makes the hidden costs of long-horizon tasks visible.

    6. Choosing and Evaluating: Don’t Ask the Model, Measure with Tasks

    Detached from a concrete task, any conclusion about “whose long context is stronger” fails to generalize. Use a benchmark set close to real projects:

    1. Prepare 3–5 real long-horizon tasks (for example, refactoring a module and adding tests, or tracking down an intermittent failure);
    2. Prepare a state file and an acceptance script for each task, and start every test subject from the same initial state;
    3. Record completion rate, token cost, number of human interventions, and resume success rate after compaction;
    4. Re-run the same benchmark periodically, because version iterations can change rule adherence and response style.

    7. Conclusion

    Back to the original question: what is the long-context experience of AI coding tools actually like, and are there pitfalls?

    • The experience: in the everyday 100K–200K token range, mainstream coding tools can basically “remember without belaboring it.” Behavior is converging and becomes fairly stable, and moving across tools is usually smooth too. Styles differ by product and version, though — some emphasize rules more, others are more terse — so no single generalization holds;
    • The real pitfall: not the context length itself, but the fact that compaction will inevitably trigger in long-horizon tasks. If state lives only in the context, there are no progress files, and the resume protocol is missing, compaction brings broken links, duplicated work, and budget overruns.

    The conclusion is simple: leave long context to the model, and leave continuity to the harness. Progress files guarantee “no amnesia,” the compaction protocol guarantees “picks up where it left off,” sub-agent dispatch guarantees “no runaway,” and externalized rules plus machine checks guarantee “no drift.” Get those four right, and context size stops being the deciding factor in whether long-horizon tasks succeed.