The Agentic AI Race: Long-Horizon Task Capabilities and the Cost Dilemma

1. From Conversation to Execution: Shifting Model Evaluation Focus

Over the past two years, the focus of large language model (LLM) evaluation has shifted significantly. Moving beyond general knowledge Q&A and single-turn reasoning, assessments now prioritize code generation, multi-step tool calling, and long-horizon task execution. This shift reflects a change in application paradigms—users no longer want models that simply answer isolated queries. Instead, they expect models to act as continuous engineering assistants capable of reading files, running commands, modifying code, searching for information, and validating results in real-world environments while maintaining goal consistency over hours or even days.

A recently released ultra-large parameter model exemplifies this trend. Featuring a parameter scale in the tens of billions and supporting a context window of one million tokens, the model specifically enhances capabilities in coding, 3D creation, financial research, and long-horizon agentic tasks. It also introduces a high-speed version that doubles or triples output speed. The core upgrade is not about being “better at conversation,” but about being able to “continuously execute work.”

2. Long-Horizon Agent Capabilities: The Engineering Meaning Behind Metrics

2.1 Terminal Environment Programming Agent Evaluation

In terminal environment programming agent evaluations, this model scored 65.7%, less than one percentage point behind a leading closed-source model at 66.4%, and surpassing another top-tier model’s 58.2%. Such evaluations require agents to complete complex tasks with specific tests in real, resource-constrained terminal environments.

The value lies in its authenticity. Rather than merely asking the model to complete a function, it requires the agent to autonomously plan steps, call tools, handle errors, and verify outputs under limited resources. Scoring this close to leading closed-source models indicates that the open-source and open-weight camp has entered the first tier in agentic engineering capabilities.

2.2 Long-Horizon Code Task Evaluation

In internal long-horizon code tests, the model scored 61.2%, an improvement from the previous generation’s 55.95%. This test spans multiple engineering domains, focusing on multi-step execution, tool calling, and end-to-end problem-solving.

“Long-horizon” is the critical factor. While short-horizon tasks test isolated capabilities, long-horizon tasks evaluate:

• State Maintenance: Remembering initial goals and intermediate constraints after dozens of operational steps.
• Error Recovery: Self-correcting when commands fail, dependencies are missing, or paths are incorrect.
• Context Compression: Deciding how to summarize, filter, and retrieve key information within a one-million-token context.
• Termination Judgment: Knowing when a task is complete or when to ask the user for help.

These capabilities cannot be achieved simply by scaling up parameters; they require systematic design during the post-training phase.

3. Post-Training Compute Expansion: Scaling Reinforcement Learning to Hundreds of Domains

A notable change in this model is the increased post-training compute and the expansion of reinforcement learning (RL) to hundreds of downstream domains. The technical implications of this decision are significant.

Traditional RLHF (Reinforcement Learning from Human Feedback) primarily focuses on alignment and preference within a relatively narrow domain. Expanding RL to hundreds of downstream domains entails:

1. Diverse Reward Signals: Code tasks use test pass rates, financial research uses data accuracy, and 3D creation uses geometric consistency. Different domains require different verifiable rewards.
2. Environment Scaling: Each domain requires an interactive sandbox environment, and the quality of this environment directly determines the signal-to-noise ratio of the training signal.
3. Training Stability: Joint training across hundreds of domains is prone to gradient conflicts and catastrophic forgetting, requiring coordination with scheduling and replay mechanisms.

To support this large-scale joint training, the provider systematically upgraded the RL framework, long-context inference, training efficiency, sandbox infrastructure, scheduling systems, and environment quality inspection platforms. This demonstrates that as model capability competition enters the agentic phase, the focus has shifted from model architecture to training infrastructure.

4. Managed Agents: Turning the Harness into Infrastructure

A easily overlooked yet significant release is the managed agent service. It offers the harness supporting long-running agents—task orchestration, tool calling, state management, error retry—as cloud infrastructure for developers and enterprises. It supports 24/7 long-horizon tasks and multi-agent collaboration.

This solves a core pain point in agent deployment: developers can write prompts and define tools, but building a stable runtime that operates for days, supports multi-agent collaboration, and provides observability is incredibly difficult. Abstracting the harness as a managed service essentially turns the “agent runtime” into a cloud service category, similar to how container orchestration, message queues, and logging services are treated as infrastructure.

Architecturally, such services typically need to include:

• Task Queuing and Scheduling: Supporting queuing, prioritization, timeout, and retry for long-running tasks.
• State Persistence: Ensuring intermediate states during agent execution are recoverable to prevent task loss during process restarts.
• Tool Gateway: Centrally managing permissions and auditing for tools like code execution, file read/write, and network access.
• Observability: Tracking tool calls, latency, token consumption, and errors at every step.
• Multi-Agent Communication: Supporting message passing and task decomposition between agents.

5. The Cost Dilemma: The New Challenge After Performance Convergence

The API pricing for this model is set at 1.3 for cached input, 13 for uncached input, and 68 for output per million tokens. The high-speed version is priced at approximately twice that of the standard version.

This pricing structure reveals two signals:

1. A 10x Price Difference Between Cache Hits and Misses: This indicates that reusing cached, repetitive content in long contexts is critical for cost control. In agentic scenarios, system prompts, tool definitions, and historical summaries are highly repetitive, meaning cache hit rates directly dictate the final bill.
2. Output Priced Significantly Higher Than Input: This aligns with industry norms. However, agent task outputs often contain massive amounts of tool calling parameters and intermediate reasoning, making output token consumption vastly higher than in standard conversations.

Therefore, despite performance approaching leading closed-source models, user feedback about “burning through quotas in ten minutes” is not an exaggeration. Token consumption in long-horizon agentic tasks grows non-linearly: each tool call can trigger new context concatenation, and a million-token context window fills up rapidly after multiple iterations.

Practical Recommendations for Developers:

• Design cache-friendly prompt structures by placing stable content at the forefront.
• Set token budgets and step limits for long-horizon tasks to prevent infinite loops.
• Use the high-speed version or smaller models on non-critical paths, reserving the most powerful models for critical decision-making.
• Leverage the observability features of managed agent services to identify token consumption hotspots.

6. Conclusion

The release of this ultra-large parameter model marks a new phase in competition. Parameter size and context length are gradually becoming entry tickets. The true differentiator lies in how post-training compute translates into long-horizon task capabilities and whether the infrastructure can support stable, economical agent operation. The full model weights are scheduled for release at a later date; if delivered as promised, this will provide developers with the possibility of deploying agentic capabilities within private environments.

While performance approaching leading closed-source models is commendable, the engineering challenges of cost and stability are just beginning. Enterprise users should look beyond benchmark scores during selection. They must evaluate the comprehensive cost and success rate of completing an end-to-end long-horizon task within their specific business scenarios. This is the true technical threshold of the Agent era.