Million-Token Context Windows and Agentic Coding: How AI Assistants Evolve Into Digital Coworkers

1. The Shift Is in the Work Paradigm, Not the Parameter Sheet

Over the past two years, the mainstream way of working with large models has been one question, one answer. We would paste in code and ask for an explanation, drop in a block of copy and ask for a polish, or ask about an error and get a suggested fix. When the session ended, the task ended too, and context lived about as briefly as human attention.

The latest wave of capability upgrades changes something more fundamental: where the model sits in the task chain. Instead of just answering, it is now expected to take on the executor’s role. It receives a relatively vague goal and then breaks it into steps on its own. It calls tools, reads intermediate results, catches errors, and retries. It keeps going until it produces a verifiable result.

Three signals usually indicate whether a model release carries paradigm-level significance:

1. Context capacity has crossed an engineering threshold. The model can now hold not just a single function, but a whole module or even an entire code repository.
2. Multi-step autonomous execution is possible. The model can chain tool calls and maintain intermediate state instead of needing a human to re-explain the background at every step.
3. Reasoning cost is adjustable. Compute can be allocated according to task difficulty instead of giving every request the same level of deep thinking.

When all three signals appear together, the meaning of the term “AI assistant” is quietly rewritten. It starts to resemble a junior engineer you can delegate work to: you describe the intent and the acceptance criteria, and it handles the process.

2. Million-Token Context: Capacity Is the Entry Ticket, Not the Finish Line

A context window of around one million tokens is roughly equivalent to hundreds of thousands of lines of source code. It could also hold a hundred or more complete design documents, or a collection of written material amounting to tens of millions of characters. The practical impact for engineering work is direct: in many scenarios, the chunk-retrieve-inject pipeline that used to be mandatory can be simplified to injecting the whole context.

2.1 What It Actually Solves

• Cross-file reasoning: When refactoring a shared function used by 30 modules, the model can see both the callers and the implementation at the same time. That reduces locally optimal fixes that work in isolation but break something once they ship.
• Long-chain debugging: Put the full error logs, relevant config files, and dependency version declarations into the context together. The model is noticeably better at pinpointing the problem than when you paste in only the stack trace.
• Historical consistency: Across multiple rounds of iteration, you no longer have to keep restating constraints. Code style and naming conventions are also easier to keep consistent.

2.2 What It Does Not Solve

Three physical constraints still apply and need to be acknowledged:

• Information decay in the middle of the context: Attention is not distributed evenly across the entire window. Key information in the middle of the context is more likely to be overlooked. Placing the most important constraints at the beginning and the end is a low-cost practical optimization.
• Roughly linear cost growth: As the number of input tokens grows, both the bill and time to first token rise along with it. Stuffing everything into the context is an expensive form of laziness.
• Noise is pollution: Irrelevant, outdated code and duplicated logs dilute useful signals. The core of context engineering is not how much you pack in, but how accurately you pack it.

2.3 A Practical Layered Context Strategy

[L0] Stable layer: system prompt, coding standards, directory structure, dependency list      → always retained
[L1] Task layer: current request description, acceptance criteria, relevant interface definitions  → always retained
[L2] Evidence layer: retrieved source snippets, log excerpts, test results        → injected by relevance, trimmed when over budget
[L3] History layer: earlier conversation and completed intermediate artifacts      → kept as compressed summaries

The engineering rule is simple: L0/L1 are never dropped, L2 is ranked by score and kept as Top-K, and L3 is replaced by summaries rather than kept in full.

3. Agentic Coding: The Capability Splits Into Three Layers

“An agent that can write code on its own” is marketing language. In engineering terms, it breaks down into three separate things.

3.1 Planning Layer: Turning Goals Into Verifiable Subtasks

The difference shows in how the agent behaves when a requirement is vague. An inefficient approach starts writing the implementation immediately. An effective one first builds a task tree with acceptance conditions, then works through it step by step.

# Example: task decomposition with acceptance criteria
plan = agent.plan(
    goal="Add the ability to export order statements by time range to the order service",
    constraints=["Do not change the existing order table schema", "The export API must be idempotent"],
    acceptance=[
        "The new API does not produce duplicate files under concurrent calls",
        "Exporting 1 million rows stays under 512 MB of memory",
        "Unit test coverage is at least 80%",
    ],
)

Only with acceptance criteria does autonomy mean anything. Without them, autonomy is just a flattering name for loss of control.

3.2 Execution Layer: Tool Calls and Side-Effect Boundaries

The execution layer is where the real work happens: reading and writing files, running commands, running tests, and accessing the network. In engineering practice, side effects must be authorized by tier:

• Read-only operations (searching, viewing files, checking logs): fully allowed.
• Reversible operations (editing code on a branch, generating temporary files): allowed, but logged for traceability.
• Irreversible operations (deleting data, changing production config, sending external requests): require human confirmation.

3.3 Verification Layer: The Key to Closing the Loop

The gap between high-output and low-output coding agents is almost entirely at this layer. The core requirement is that the execute-observe-correct loop must have an objective source of signal: whether compilation succeeds, whether tests pass, whether static checks come back clean. Asking the model to judge whether it got it right is ineffective. Letting the test suite judge is reliable.

for step in range(max_steps):
    result = agent.act(current_state)
    report = run_lint_and_tests(result.changed_files)   # objective signal
    if report.passed and acceptance_check(result):
        return result
    current_state = agent.reflect(result, report)        # evidence-based correction
raise BudgetExceeded(max_steps)

4. Adaptive Reasoning: Manage Compute as a Budget

Deep reasoning is not free. In deep thinking mode, the same model often performs worse on simple problems. Responses are slower, answers are more verbose, and costs can multiply several times over. For that reason, mainstream solutions now offer reasoning effort tiers, usually four levels from lowest to highest (low / medium / high / max).

Here is how they can map to engineering work:

Tier	Typical tasks	Trade-off
Low	Formatting, renaming, unit test generation, copy rewriting	Latency and cost first
Medium	Routine feature development, initial log triage, code explanation	Balanced
High	Root-cause analysis of defects, cross-module refactoring plans	Quality first
Max	Complex architecture decisions, difficult performance problems, long-chain proofs	Willing to wait minutes

The recommended practice is task routing rather than manual switching. A lightweight model first classifies the difficulty of each request, and that classification decides which tier to call.

def choose_effort(task: dict) -> str:
    if task.files_touched <= 1 and task.has_clear_spec:
        return "low"
    if task.requires_cross_module_reasoning or task.is_perf_issue:
        return "max"
    if task.is_bug_root_cause:
        return "high"
    return "medium"

Two guardrails should go with this setup: a global quota cap on tokens and calls, and a per-task cap on reasoning steps. Without guardrails, an agent’s typical failure mode is correcting itself endlessly until it times out.

5. Engineering Long Sessions: Don’t Let Context Become a Single Point of Failure

Long-running sessions are the norm in agentic workflows, and they are also where things most easily go wrong. Three engineering techniques cover most of the problems.

First, externalize state. Do not expect conversation history to carry all of the state. Write progress, decision records, and the to-do list to a standalone task file, so the current situation can be rebuilt from disk at any time:

task: export-billing-sheet
status: in_progress
done: [schema-design, repository-layer, idem-key]
todo: [api-layer, integration-test, doc]
decisions:
  - id: D3
    note: Use object storage for export files and deliver presigned URLs to avoid memory pressure on the gateway

Second, compress the context instead of truncating it. Mechanical truncation cuts away the most critical constraints. The right approach is to produce structured summaries of completed history. Keep four kinds of information: goals, constraints, decisions already made, and unresolved problems. Discard the process details.

Third, use checkpoints and resumable runs. Take a snapshot after each step of changes (code branch, change list, test report). If a run fails, roll back to the most recent checkpoint that passed the tests instead of starting from scratch. This significantly improves the success rate and cost curve of long tasks.

6. Enterprise Adoption: Four Things Beyond the Technology

1. Data residency and compliance: If the business requires data to be processed within a specified region, confirm regional availability during vendor selection. Plan for the cost differences that follow (exact pricing is subject to official pricing), and build a region-specific deployment switch into the architecture.
2. Permissions and auditing: Credentials for agents should follow least privilege and be short-lived, and every side-effecting operation should produce a full audit log. For centralized collection and anomaly detection, use your cloud platform’s log service and monitoring and alerting capabilities. At runtime, a container platform can provide resource isolation and quota limits. Cold data and export artifacts should go to object storage with lifecycle policies configured. On Baidu AI Cloud, this combination of container platform, log service, monitoring and alerting, and object storage is a common deployment pattern for enterprise-grade agents.
3. Evaluation baseline: Before launch, you must build your own task set. It should cover 20–50 real requirements and record the first-pass rate, average steps, average cost, and number of human interventions. Without a baseline, model selection is just guesswork.
4. Cost observability: Instrument usage along three dimensions: task, user, and reasoning tier. Only then can you spot problems like one category of tasks that makes up 5% of the volume but consumes 60% of the budget.

7. A Phased Rollout Plan

• Phase 1 (1–2 weeks): read-only scenarios. Codebase Q&A, documentation generation, and log attribution, with humans performing all changes.
• Phase 2 (2–4 weeks): restricted write access. Complete small tasks on isolated branches with all test gates enabled, and have humans review before merging.
• Phase 3 (1–2 months): multi-step, long-running tasks. Add checkpoints, context compression, and task state files, and integrate CI as the objective verification step.
• Phase 4: batch and asynchronous execution. Move tasks to queues, run multiple instances concurrently, and assign reasoning tiers dynamically by difficulty.

8. Three Common Pitfalls

• Treating context as a database: Long context is suited to reasoning with evidence. It is not suited to serving as a persistent retrieval layer. Stable knowledge should still come from version control, documentation systems, and retrieval services.
• Using self-assessment instead of tests: A model’s self-evaluation cannot serve as a quality gate. Executable tests and static checks are what count.
• Granting write access from the very start: Human confirmation for irreversible operations is not an efficiency loss in the early stages. It is a form of cost control.

9. Conclusion

Million-token context solves the “can see it” problem, agentic execution solves the “can do it” problem, and reasoning effort tiers solve the “is it worth it” problem. Only when all three come together does an AI assistant evolve from a tool into something you can hand tasks to.

For engineering teams, however, what usually decides success or failure is not the model’s capabilities. It is the surrounding engineering: how tasks are decomposed, how acceptance is defined, how state is externalized, how side effects are authorized, and how cost is observed. Once this framework is in place, every model upgrade brings multiplied returns. Without it, even the most capable model is just a more expensive chat box.