Million-Token Context and Long-Horizon Agents: From Capability Boundaries to Engineering Practice

1. Background: Why Long Context and Agent Programming Matter

Over the past two years, the capability race among large language models has shifted from “being able to converse” to “being able to get work done.” Developers no longer expect just single-turn Q&A; they want a model that can read an entire codebase, follow tool-call chains spanning many turns, and close the loop on end-to-end tasks in a real terminal environment. Two developments underpin this shift.

The first is the expansion of the context window. Only when context grows from tens of thousands of tokens to the million-token scale can a model hold the complete source code of a large project, multiple technical documents, and historical conversation records in a single pass—freeing workflows from the information fragmentation caused by repeated chunk-and-retrieve cycles.

The second is the maturation of long-horizon agent capabilities. “Long-horizon” means a task can only be completed through dozens or even hundreds of tool calls, state checks, and error-recovery steps. This places far higher demands on instruction following, error introspection, and planning stability than single-turn generation ever did.

This article uses a model profile that combines million-token context with reinforcement-trained agent capabilities as its reference point, walking through its capability structure, typical application scenarios, cost model, and the key points of engineering adoption.

---

2. Capability Structure: From Code to Multimodal Toolchains

2.1 Code and Terminal Tasks: The Litmus Test for Real Work

If you want to judge whether a model has practical agent capability, terminal tasks are the most direct test. These tasks require the model to work inside a constrained command-line environment—reading and writing files, running scripts, and parsing error messages to move toward the goal—rather than simply outputting a piece of code that looks correct.

On this class of benchmarks, a model with genuine long-horizon ability typically shows:

• State tracking: remembering how earlier steps affected the file system or runtime environment;
• Error recovery: when a command exits with a non-zero code, reading stderr and adjusting strategy;
• Minimal side effects: avoiding destructive operations and preferring changes that can be rolled back.

As a reference from public data, on one long-horizon terminal benchmark a reinforcement-trained model scores around 65%, narrowing the gap with leading closed-source models to single-digit percentage points. On an internal long-horizon coding benchmark, the gain over the previous generation is roughly 5 percentage points. In long-horizon settings, that kind of improvement usually signals a significant drop in failure rate—because a long-horizon task’s overall success rate is the product of its per-step success rates, so a 5% gain at the single-step level gets amplified across the chain.

2.2 3D Modeling and Game Engines: A Canonical Tool-Calling Scenario

One high-value landing zone for agent capability is parametric 3D modeling and engine scripting. Concretely, the model needs to:

1. Understand the user’s geometry or scene description;
2. Call the modeling tool’s scripting interface to generate a parametric script;
3. Run the script and iteratively fix it based on the output;
4. Import the artifact into a game engine or rendering pipeline.

In evaluations of this “CAD via tools” style, strong models score above 90%, approaching the level of leading closed-source models. The technical key is whether the model correctly understands the parameter semantics of the modeling API—not whether it can pile up code.

A schematic structure for a parametric script looks like this (pseudocode, for structure only):

# Pseudocode: general structure of a parametric modeling task
params = parse_user_intent(intent)          # Parse user intent into a parameter dictionary
script = model.generate_script(params)      # Generate the modeling script
for attempt in range(MAX_RETRY):
    result = run_in_tool(script)            # Execute inside the modeling tool
    if result.ok:
        break
    script = model.repair(script, result.stderr)  # Repair based on the error output
export(result.artifact)                     # Export the artifact for downstream use

This structure highlights an important engineering principle: an agent’s value lies not in a single generation, but in the “generate–execute–repair” loop. When building such a system, an execution sandbox, timeout control, and artifact validation are all indispensable.

2.3 Finance and Knowledge Workflows: End-to-End Delivery

In scenarios such as financial modeling, investment analysis, and industry research reports, the model has to chain together multiple capabilities: reading structured tables, performing multi-step numerical reasoning, producing documents that follow industry conventions, and keeping data consistent across the entire text.

The hard part here is cross-section consistency. The same metric may appear repeatedly in different chapters of a report; without global planning, the model can easily contradict itself. This is where million-token context pays off: the model can hold the raw data tables, intermediate computation results, and the draft text in a single inference pass, reducing information loss between multiple calls.

On public benchmarks targeting office and knowledge work, models of this class score in the 64-point range, which is a strong level today.

---

3. Factuality and Safety: Two Unavoidable Engineering Constraints

3.1 Hallucination Control: From Aggregate Error Rates to Stress Tests

A common mistake when evaluating factual accuracy is looking only at the average. The more meaningful metric is the error rate under deliberately constructed, error-prone scenarios. For example, in a high-difficulty hallucination stress test, the error rate for individual factual claims can drop from roughly 23% in the previous generation to about 8%, while the share of answers containing at least one factual error falls to roughly 30–40% of the previous generation’s level.

It is worth stressing that stress-test samples deliberately focus on confusable, time-sensitive, or long-tail knowledge, so these error rates do not represent everyday usage levels—but they are an effective yardstick for stability under extreme conditions. In engineering practice, it is advisable to:

• Add an external retrieval verification step for factual outputs;
• Set human review thresholds for key figures and citations;
• Explicitly instruct the model, in the prompt, to flag content it is uncertain about.

3.2 Safety Alignment: Data Governance During Training

Safety is not built solely by a final filtering pass. A more systematic approach is to identify and remove high-risk domain data (such as corpora related to network attack and defense) at both the mid-training and reinforcement-learning stages, reducing the potential for misuse.

This kind of handling usually involves a capability trade-off. On one internal safety benchmark, for instance, the unfiltered version solves about 90% of the problems, while the final released version reaches about 81%—that gap is precisely the cost of the safety constraints. For enterprise users, understanding this trade-off helps set realistic expectations: there is inherent tension between safety compliance and peak capability, and the choice must be made according to your business risk level.

---

4. Cost Model: How to Read a Pricing Table

Million-token context and agent-style calls amplify token consumption dramatically, so cost structure is a core variable in model selection. Typical billing dimensions include:

Billing Dimension	Description
Input tokens	Everything sent to the model in each request
Output tokens	Content generated by the model, usually priced well above input
Cache hit	Reusing an already-cached prefix, at the lowest unit price
Cache write (short TTL)	Writing to the cache for short-term reuse
Cache write (long TTL)	Writing to the cache for longer-term reuse, at a higher price

Take the published prices of one model tier as an example: input is priced at roughly a dozen CNY per million tokens, output at just over sixty CNY per million tokens, and cache hits can cost as little as one-tenth of the input price. Cache writes come in 5-minute and 1-hour tiers, with the latter about twice the former. There is also a speed-optimized version, priced at roughly twice the standard version.

From an engineering standpoint, this structure sends clear optimization signals:

1. Maximize the cache hit rate. Place stable, unchanging content (system prompts, codebase skeleton, documentation) at the front of the prefix so the cache actually pays off.
2. Choose the right write TTL. Short-TTL cache suits multi-turn interaction within one session; long-TTL cache suits fixed knowledge reused across sessions.
3. Use the speed version sparingly. Enable it only on latency-sensitive paths (real-time interaction, inline CI checks); batch and offline jobs can run on the standard version.

One practical way to cut costs is tiered invocation: use a smaller model for intent recognition and routing, and hand only complex subtasks to the large model; at the same time, compress the context so only the fragments relevant to the current step are kept.

---

5. A Practical Checklist

Pulling the analysis together, here are recommendations for development teams:

Context management

• Use a “stable prefix + dynamic tail” assembly strategy to maximize cache benefits;
• For very large repositories, combine summarization with retrieval instead of stuffing everything into the context;
• Periodically prune stale history turns so the context is not diluted by noise.

Agent design

• Set timeouts and retry limits for every tool call to keep long-horizon tasks from stalling;
• Use a rollback-capable execution environment to reduce the risk of destructive operations;
• Add validation checkpoints at critical steps instead of fully trusting the model’s self-assessment.

Quality and compliance

• Add retrieval augmentation and human spot checks for factual outputs;
• Govern data sources carefully during training and fine-tuning;
• Define explicit criteria for the capability-versus-safety trade-off based on your business risk level.

Cost control

• Monitor cache hit rate, the average input/output ratio, and per-task token consumption;
• Replace per-item calls with batch processing;
• Route latency-insensitive tasks to the standard version.

---

6. Closing Thoughts

The combination of million-token context and long-horizon agent capability is pushing large language models from “text generators” toward “task executors.” For developers, the real challenge is not the model itself but designing engineering systems that can reliably carry long-horizon tasks: how context is organized, how tools are orchestrated, how errors are recovered, how costs are kept under control, and how compliance is guaranteed. Solve these problems well, and improvements in model capability will genuinely translate into business value.