Trillion-Parameter Models: Long-Horizon Agent Capability Leaps and the Inference Cost Challenge

1. Background: From Chat Models to Persistently Working Agents

Over the past two years, the evaluation criteria for large models have shifted noticeably. Early benchmarks centered on general QA, knowledge retrieval, and single-turn generation quality are gradually giving way to more engineering-oriented metrics: multi-step reasoning, tool calling, end-to-end task completion, and stable execution in resource-constrained environments.

Behind this shift is a change in application patterns. Enterprises and developers are no longer satisfied with “generate a piece of code” or “answer a question.” They want models that behave like a dependable digital employee, working continuously for hours or even days inside terminals, IDEs, browsers, or private toolchains. This demands three key capabilities:

1. Long-horizon planning and state retention: staying on target across hundreds of steps;
2. Tool calling and environment interaction: reading and writing files, executing commands, calling APIs;
3. Error recovery and self-correction: replanning after failures instead of crashing outright.

A recently released trillion-parameter model takes a systematic step in exactly these directions. Its core approach can be summarized as: shifting post-training compute from leaderboard-chasing to real task environments, and expanding reinforcement learning from a handful of skills to hundreds of downstream domains.

2. The Technical Path: Post-Training Scaling and RL at Scale

2.1 Parameter Scale and Context Window

The model features a 3-trillion-parameter scale and supports a million-token context window. From an engineering perspective, these two numbers mean different things:

• Parameter scale determines the model’s “knowledge capacity” and ceiling for generalization;
• Context window determines how much history the model can hold within a single task.

A million-token context is especially critical for long-horizon agents. Take code refactoring: the full codebase of a mid-sized project can exceed 500,000 tokens. If the model cannot fit the entire context into a single inference pass, it has to rely on retrieval augmentation or chunking—and chunking breaks the integrity of cross-file dependencies. A million-token window lets the model “see” the entire project structure in a single session and make more globally informed decisions.

2.2 More Post-Training Compute

The most significant change in this release compared with the previous generation is the increase in post-training compute, along with reinforcement learning extended to hundreds of downstream domains. Two concepts are worth distinguishing here:

• Pre-training: learning general representations from massive unlabeled data;
• Post-training: aligning general capabilities to specific tasks through supervised fine-tuning, reinforcement learning, and related techniques.

Post-training has traditionally focused on a few high-value domains (such as code and math), but real-world task distributions are long-tailed. An agent may need to handle financial data parsing, 3D file manipulation, command-line tool invocation, and API orchestration at the same time. If post-training covers only a few domains, the model shows lopsided performance when it encounters unfamiliar task types.

Extending RL to hundreds of downstream domains is essentially trading task diversity for generalization robustness. Each domain contributes different reward signals and failure modes. What the model learns through joint training is not any specific skill but a meta-capability: how to quickly adapt to a new environment.

2.3 Systems Upgrades Behind Large-Scale Joint Training

Supporting RL joint training across hundreds of domains requires parallel upgrades to the underlying infrastructure. From public information, several key engineering directions stand out:

(1) RL framework and scheduling

The core bottleneck of large-scale RL training is the asynchrony between sample generation and policy updates. Task environments vary widely across hundreds of domains—some tasks finish in seconds, others take minutes. The scheduler must allocate compute dynamically so that fast tasks are not blocked by slow ones.

(2) Long-context inference optimization

Inference cost for a million-token context grows superlinearly with sequence length. Common optimization techniques include:

• Blockwise attention and sparse attention;
• KV cache compression and tiered storage;
• Disaggregated scheduling of prefill and decode phases.

(3) Sandbox infrastructure and environment quality checks

When agents execute tasks in real environments, isolated sandboxes are needed to prevent unintended side effects. Environment quality also directly affects RL training outcomes—if reward signals are noisy, the model learns wrong policies. This calls for an automated environment quality-check platform that verifies each task environment for consistency, reproducibility, and reward correctness.

3. Performance: Real-World Numbers on Coding and Long-Horizon Agents

3.1 Terminal Coding Agent Benchmarks

On Terminal-Bench 4.0, the model scored 65.7%. Key characteristics of this benchmark:

• Tests run in real, resource-constrained terminal environments;
• Each task has dedicated test cases emphasizing end-to-end completion;
• Tasks cover system-level operations such as file manipulation, process management, and network configuration.

For comparison, two other leading closed-source models scored 58.2% and 66.4% on the same benchmark. This places the new model in the top tier for terminal coding agents, within one percentage point of the strongest closed-source model.

3.2 Long-Horizon Coding Tasks

On an internal long-horizon coding evaluation, the new model scored 61.2%, up from 55.95% for the previous generation. The evaluation spans multiple engineering domains and focuses on:

• Multi-step execution capability;
• Tool-calling accuracy;
• End-to-end problem resolution rate.

A gain of roughly 5 percentage points (55.95% → 61.2%) on long-horizon tasks usually translates into a significant extension of the maximum number of steps a task can run. For example, if the previous generation started losing track of the goal after about 20 steps, the new model can reliably execute 30 or more.

3.3 Remaining Concerns

Despite strong numbers, two issues deserve attention in practice:

Uneven domain coverage: RL now spans hundreds of domains, but data volume and task difficulty are not balanced across them. Some domains (such as terminal operations) perform outstandingly; others may still be weak spots.

Verbose output: On long-horizon tasks, the model tends to generate large amounts of intermediate reasoning text, consuming extra tokens and reducing effective output speed. For token-billed APIs, verbosity directly translates into cost.

4. Inference Cost Structure: Why Credits Burn Out in Ten Minutes

4.1 Pricing Breakdown

The model’s API pricing adopts a cache-tiered structure:

Billing item	Price (RMB per million tokens)
Cache-hit input	1.3
Cache-miss input	13
Output	68

The high-speed tier offers the same model capabilities at roughly twice the standard price.

This pricing structure conveys several signals:

• A 10x gap between cache-hit and cache-miss input: developers are incentivized to reuse system prompts and fixed context;
• Output costs more than 5x uncached input: long-form generation is the dominant cost driver;
• A 2x premium for the high-speed tier: suited to latency-sensitive, cost-insensitive interactive scenarios.

4.2 Token Consumption Patterns of Long-Horizon Agents

Why do some users report “credits burning out in ten minutes”? From an engineering standpoint, it breaks down like this:

A long-horizon agent task typically includes:

• System prompt and tool definitions (fixed, cacheable);
• Task description and initial context (different each time);
• Multiple rounds of tool calls and returned results;
• The model’s intermediate reasoning and final output.

Assume a task runs 30 steps, with each step consuming on average:

• Input: 8,000 tokens (6,000 of them cacheable)
• Output: 2,000 tokens

Per-step cost is then approximately:

• Cache-hit input: 6,000 × 1.3 / 1M = RMB 0.0078
• Cache-miss input: 2,000 × 13 / 1M = RMB 0.026
• Output: 2,000 × 68 / 1M = RMB 0.136
• About RMB 0.17 per step in total

A 30-step task costs around RMB 5.1. If tasks involve more steps or longer outputs (such as code generation), costs climb quickly. For developers running agents at high frequency, dozens of tasks a day can add up to hundreds of RMB.

4.3 Cost Optimization Recommendations

(1) Make full use of caching

Place system prompts, tool definitions, and fixed knowledge bases at the front of the context, and keep their content stable. Any tiny change invalidates the cache, sending input costs from 1.3 to 13 per million tokens.

(2) Control output length

Constrain output format explicitly in prompts—for example, “output only the final result, no intermediate reasoning.” For coding tasks, ask for “only the modified code blocks.”

(3) Use model tiers deliberately

• Use the standard tier for simple tasks;
• Use the high-speed tier for latency-sensitive interactive scenarios;
• Use the standard tier for batch offline jobs where longer latency is acceptable.

(4) Task splitting and timeout controls

Set a maximum step count and per-step timeout for agents to avoid infinite loops that burn tokens for nothing.

5. Managed Agents: From Model API to Infrastructure

Beyond the model itself, the vendor has also launched a managed agent service, offering the Harness that supports long-running agents as cloud infrastructure. This marks a change in how capabilities are delivered:

• Layer 1: Model API—provides inference capability; developers build the agent loop themselves;
• Layer 2: Managed agent service—provides 24/7 long-horizon task execution, multi-agent collaboration, state persistence, and more.

The managed service targets several engineering pain points:

1. State management for long-running work: after an agent runs for days, context may exceed the window and needs external memory storage;
2. Multi-agent collaboration: communication, task allocation, and result aggregation across agents;
3. Failure recovery: resuming from checkpoints after a crash instead of starting over.

Architecturally, such services typically include:

• Task queues and schedulers;
• Sandboxed execution environments;
• Memory storage and retrieval;
• Monitoring and logging.

6. Engineering Practices

6.1 Model Selection Criteria

Dimension	What to consider
Task type	For code/terminal operations, prioritize models strong at long-horizon agents
Context needs	Choose million-token-window models beyond 100K tokens
Latency requirements	High-speed tier for interactive use; standard tier for offline work
Cost budget	Estimate daily token consumption; pick cache-friendly designs

6.2 Pseudocode for an Agent Loop

def agent_loop(task, max_steps=30):
    context = build_system_prompt()  # fixed, cacheable
    context += task
    for step in range(max_steps):
        response = model.generate(
            context,
            max_tokens=2000,
            temperature=0.2
        )
        action = parse_action(response)
        if action.type == "finish":
            return action.result
        observation = sandbox.execute(action)
        context += format_observation(observation)
        if step > max_steps * 0.8:
            context = compress_context(context)  # compress history
    return "task timed out"

6.3 Monitoring Metrics

• Token consumption and cost per task;
• Average steps to task completion;
• Cache hit rate;
• Task success rate and distribution of failure causes.

7. Conclusion

Trillion-parameter models are shifting from competing on “conversation ability” to competing on “sustained work ability.” Through more post-training compute, RL extended to hundreds of domains, and rebuilt training and inference infrastructure, the new generation of models has reached near-frontier closed-source levels in terminal coding and long-horizon agent scenarios.

But capability gains come with cost pressure. High output prices and heavy token consumption on long-horizon tasks make “credits running out fast” a real pain point in daily use. When choosing a model, developers need to evaluate both the capability ceiling and the cost per task, and optimize total cost of ownership through cache reuse, output constraints, task splitting, and tiered usage.

The emergence of managed agent services signals that cloud providers are extending competition from the model layer to the infrastructure layer. For enterprise users, this means focusing on task definition and business processes while leaving long-running execution, state management, and failure recovery to the platform. Going forward, model capability, cost structure, and engineering support will together determine how well these applications land in production.