Million-Token Context Windows for Coding Agents: Do You Really Need Them?

When coding agents began supporting million-token context windows, many developers’ first reaction was: the bigger the window, the better. In practice, however, this capability usually requires manually adjusting settings such as the context window size and compaction thresholds — a sign that it is an advanced, task-specific capability rather than a universal default. The real question to ask is: given your task type, cost budget, and agent workflow, does million-token context deliver a clearly measurable benefit?

A context window is the maximum amount of input a model can accept in a single pass. It is neither the model’s effective comprehension ability nor the quality of the agent’s working memory. Stuffing more history into the window does not automatically improve code-editing accuracy. On the contrary, it can introduce stale information, duplicated logs, and already-rejected proposals, increasing attention dilution and reasoning noise.

1. The Question Isn’t Whether You Can Enable It, but Whether It Should Be On by Default

After a coding agent runs for a long time, its context tends to accumulate a lot of “historical garbage”:

• Analysis built on assumptions that were later proven wrong
• File contents retrieved early via text search that have since become outdated
• Dozens of command-line outputs, build logs, and failed test logs
• Code snippets that have already been edited away and no longer exist
• Large volumes of intermediate analysis returned by subagents
• Design proposals that were later rejected
• Long reasoning traces that may not be critical

What actually drives the next decision is usually much smaller: the current workspace changes, the relevant code, key architectural constraints, the most recent failure, acceptance criteria, and unresolved questions. The longer the context, the harder it becomes for the model to retrieve, correlate, and filter out irrelevant information. For everyday tasks — fixing bugs, adding small features, debugging a single module — carrying too much historical state does more harm than good.

2. Window Capacity Isn’t Effective Working Memory

The biggest benefit of million-token context is fewer compaction events, which means the agent loses less historical state during long sessions. It fits scenarios such as:

• Large-scale code migrations that require understanding many interfaces, call chains, and configurations at once
• Architecture audits of large projects that involve reading extensive documentation, modules, and change history
• Cross-module refactoring that requires repeatedly cross-checking multiple files and constraints
• Long-running batch tasks that need to preserve fairly complete task state and decision records

In ordinary coding scenarios, however, the costs are just as obvious. The cost of a single request is roughly determined by input tokens, output tokens, and the number of turns. If a session has already accumulated hundreds of thousands of tokens and the user simply says “fix this test,” the next request may still carry a massive historical context. A short message doesn’t mean a small bill — what gets billed is the working state carried along behind it. The larger the window and the higher the compaction threshold, the longer the context stays huge, and the more visible the cumulative overhead becomes.

Million-token context is therefore more like heavy machinery: suited to specific engineering tasks, not something to enable by default across the board.

3. The Real Benefits and Costs of Million-Token Context

Many developers treat context compaction as a “necessary evil,” but from an agent engineering perspective, compaction is actually an important capability. Proper compaction isn’t simple truncation — it reconstructs historical information into structured state, for example:

• Goals and acceptance criteria
• Key architectural constraints and interface contracts
• Completed changes and the current workspace state
• The most recent failure and its cause
• Open questions and next actions
• Rejected proposals and the reasons they were rejected

At the same time, the tool layer should take on filtering responsibilities instead of feeding every raw result straight to the model. For example: build logs keep only error lines and failed targets; test logs keep failing cases, assertion diffs, and the top of the stack; code search returns file paths, symbol signatures, and key snippets; subagents return conclusions, evidence locations, and recommendations rather than entire analysis processes.

A mature agent workflow is a combination of “default context + retrieval + compaction + subagents + tool-output filtering” — not a million-token window that carries every intermediate result forever.

4. Compaction Isn’t a Flaw — It’s Core Agent Engineering

A safer strategy is to make million-token context a manually enabled, per-task configuration rather than a global default. The following is an abstract configuration sketch that illustrates the approach only; it does not represent any specific platform’s parameters:

agent_profile:
  default:
    context_window_tokens: 200000
    auto_compact_threshold_tokens: 140000
    max_tool_output_tokens: 6000
    recent_turns_kept: 6
    summary_schema: goal_constraints_changes_tests_open_questions
  large_task:
    context_window_tokens: 1000000
    auto_compact_threshold_tokens: 850000
    manual_enable: true
    session_budget_alert_tokens: 300000
    max_tool_output_tokens: 12000

For day-to-day development, use the default context with a lower auto-compaction threshold so the system clears out stale history earlier. Manually enable the large window only for clearly large migrations, architecture audits, or cross-module refactoring, and set a session budget alert. If a task drifts off target within a few minutes, reset the session and restructure the task first — don’t keep enlarging the window.

5. Configuration Strategy: Small Windows by Default, Large Windows on Demand

Whether to enable million-token context shouldn’t be decided by gut feeling. Run an A/B comparison on the same class of tasks:

• Run a set of similar tasks with default context and with million-token context
• Record task success rate, number of human takeovers, and total elapsed time
• Record input tokens per turn, total tokens per task, and the number of compaction events
• Check whether the large window significantly reduces repeated retrieval and incorrect edits
• Calculate the overall cost per problem solved

If the large window only slightly reduces compaction events without improving success rates, while making per-request costs clearly higher, it isn’t worth enabling by default. Conversely, if tasks genuinely need to retain large amounts of historical state, cross-file evidence, and pending decisions over a long period, the large window has clear value.

6. Evaluation: Let Data Decide

Million-token context is a valuable capability, but it isn’t the default answer for coding agents. For everyday bug fixes, small iterations, and local refactoring, a default context combined with compaction, retrieval, and tool-output filtering is usually more economical and better at preserving information quality. For large migrations, architecture audits, and big cross-module tasks, million-token context works well as a manually enabled heavy-duty mode.

What really needs optimizing isn’t “how much context you can stuff in,” but “whether the information the model sees at each step is relevant, accurate, and actionable.” Treating the window as the only solution often trades higher costs for more noise. Getting context governance right is what keeps coding agents stable over the long run.

7. Conclusion