Toward Long-Horizon Agents: Capability, Speed, and the Real Cost of Large Models

1. Introduction: Agents Are Becoming the Main Battleground for Large Models

Over the past year, the center of competition in the large model industry has quietly shifted. As the intelligence gains from pretraining slow down, leading vendors are directing more post-training compute toward the same goal: making models that can actually get work done. That means writing code, doing research, operating tools, and running long-horizon tasks that last hours or even days.

A recent flagship model with trillion-scale parameters from a major vendor captures this trend. It supports a context window of one million tokens and emphasizes code, 3D creation assistance, financial research, and long-horizon agent capabilities. The vendor also introduced a high-speed variant that delivers roughly 2–3x faster output while maintaining the same intelligence level. According to official statements, the model has reached or approached the strongest closed-source models on code, agent, and knowledge benchmarks.

Strip away the product packaging, and the technical logic, evaluation methods, and pricing strategy behind this upgrade deserve close attention from developers. They point to where large model services are heading over the next few years.

2. The Next Step in Post-Training: Scaling Reinforcement Learning Across Hundreds of Domains

2.1 From Alignment to Capability

Early post-training focused heavily on reinforcement learning from human feedback, with the goal of making models more helpful and better behaved. In this generation, the emphasis has shifted to extending reinforcement learning across hundreds of downstream domains, including code, terminal operations, financial analysis, and 3D tool invocation. Each domain is set up with task environments that provide verifiable rewards.

In other words, reinforcement learning is no longer just politeness training. It is capability training. Models repeatedly practice in verifiable task environments, turning occasional correct behavior into consistent correct behavior.

2.2 The Engineering Foundation for Large-Scale Joint Training

Running reinforcement learning across hundreds of domains at the same time is far harder than training in a single domain. Industry-wide systematic changes generally focus on six areas:

• Reinforcement learning framework rework: Training and inference alternate. Sample generation throughput directly determines iteration speed, so the framework must support large-scale parallel sampling and reliable reward propagation.
• Long-context inference: Million-token contexts sharply increase attention computation and cache management overhead. Sequence parallelism and efficient attention mechanisms are needed to preserve throughput.
• Training efficiency optimization: Compute budgets are limited. Effective training steps per unit of compute become a core competitive advantage.
• Sandbox infrastructure: Agent training requires models to take actions in real environments, such as terminals, compilers, and test suites. Sandboxes must support massive concurrency, rapid rebuilding, and fault isolation.
• Scheduling systems: Resource orchestration across thousands of concurrent training environments and inference instances determines both cost and cadence.
• Environment quality inspection platforms: The ceiling of reinforcement learning is set by environment quality. Tasks must be solvable, reward signals must be correct, and difficulty distribution must be reasonable. Continuous platform-based inspection is essential. Otherwise, models learn to game the rules, a problem known as reward hacking.

Taken together, these changes signal a broader shift: infrastructure competition in the agent era is moving from stacking compute to engineering environments.

3. How Agent Capability Is Measured: Terminal Benchmarks and Long-Horizon Code Tests

3.1 Terminal Environment Benchmarks: Realistic and Resource-Constrained

These benchmarks test coding agents in realistic, resource-constrained terminal environments and assign task-specific tests to each job. They do not only check whether the model can write a function. They check whether it can read an unfamiliar codebase, locate dependencies, run builds, execute tests, and iterate based on error messages. That is a normal day in real engineering work.

Public results suggest that the new generation of models has entered the top tier occupied by leading closed-source models on such benchmarks. The gap with the strongest systems has narrowed to single-digit percentage points. The signal is clear: in the specific scenario of working inside a terminal, the capability ceiling is rising fast.

3.2 Long-Horizon Code Tests: Multi-Step Execution and End-to-End Problem Solving

Another category of evaluation focuses on long-horizon capability. It spans multiple engineering domains and emphasizes multi-step execution, tool invocation, and end-to-end problem solving. Compared with one-shot generation, long-horizon tests are closer to the real-world pattern of taking a requirement and working on it for an entire day.

Improvements over the previous generation usually come from three sources: fewer mid-task detours, higher-quality self-correction, and more reliable orchestration of tool calls.

3.3 Three Key Points for Evaluation Design

For teams building their own evaluations, the industry consensus is clear. First, prevent data contamination. Tasks should be private and rotated regularly. Second, combine process scoring with outcome scoring to avoid rewarding accidental correct answers. Third, make environments reproducible. Dependency versions and resource limits must be fixed. Otherwise, scores are not comparable.

In addition, some vendors have committed to releasing full model weights alongside flagship releases. As the gap between open-source and closed-source capability narrows, the risk structure of enterprise model selection is changing.

4. The Inference Engineering Behind High-Speed Variants

Delivering the same intelligence with output speed improved by 2–3x usually does not come from distilling a smaller model. It comes from systematic optimization of the inference stack. Common industry approaches include:

• Prefix caching and KV cache reuse: In agent scenarios, system prompts, tool definitions, and codebase context are highly repetitive. Cache hits can lower both latency and cost.
• Speculative decoding: A small model drafts tokens and a large model verifies them, improving generation throughput without sacrificing quality.
• Quantization and operator optimization: Memory and compute overhead are reduced within an acceptable accuracy loss range.
• Continuous batching and parallelism strategies: These keep hardware utilization high under concurrent request load.

A simple selection principle applies: choose the high-speed variant for latency-sensitive scenarios where a human is waiting, such as interactive coding and high-frequency agent loops. Choose the standard variant for background batch analysis and overnight jobs where a machine is waiting. Note that high-speed variants usually carry a per-unit price premium. Only latency-sensitive workloads justify paying for speed.

5. Token Economics: Why Agent Tasks Burn Through Budget So Quickly

5.1 Three-Tier Pricing Structure

Mainstream large model services now commonly use three-tier billing: cache-hit input, cache-miss input, and output. Cache-hit pricing can be an order of magnitude cheaper than cache-miss pricing, while output tokens typically carry the highest rate. This structure itself pushes developers to organize contexts so they can be cached.

5.2 Token Consumption Patterns in Agent Tasks

Heavy agent users often find their quota burned in minutes. This is not a perception problem. It is a direct result of how agent tasks are structured.

1. System prompts and tool definitions enter the context on every task and are billed repeatedly across long tasks.
2. Context grows over time. Every tool result is appended to the conversation. The input at step 50 contains the full history of the previous 49 steps.
3. Multiple round trips are normal. A single bug fix can trigger dozens of model calls.
4. Reasoning output is invisible to the user but still billed as output tokens.
5. Compared with one-question-one-answer interactions, agent tasks can consume two to three orders of magnitude more cumulative tokens.

5.3 Illustrative Calculation and Code

# Illustrative cost estimate: values only demonstrate the billing method.
# Refer to each platform's official pricing page for actual rates.
PRICE = {  # per million tokens
    "input_hit": 1.0,    # cache-hit input
    "input_miss": 10.0,  # cache-miss input
    "output": 50.0,      # output
}
def estimate_cost(hit, miss, out):
    """hit/miss/out are token counts for each billing category."""
    cost = (hit * PRICE["input_hit"]
            + miss * PRICE["input_miss"]
            + out * PRICE["output"]) / 1_000_000
    return round(cost, 4)
# One round of a long-horizon coding task for illustration
print(estimate_cost(hit=400_000, miss=100_000, out=20_000))
# Roughly 3.4 per round; a 50-round task reaches the hundreds.
# This is the mathematical reason budgets disappear quickly.

5.4 Cache-Friendly Prompt Structure

The key to improving cache hit rates is placing stable content at the front and dynamic content at the end.

[Stable prefix] System role + tool list + coding standards    <- Nearly unchanged, consistently hits the cache
[Semi-stable layer] Project structure summary + long-term memory <- Updated infrequently
[Dynamic layer] Current file diff + tool results + user instruction <- Placed at the end

6. Managed Agents: Agent Runtime Is Becoming Cloud Infrastructure

Going a step beyond APIs, managed agent services turn the execution framework that supports long-running agents into cloud infrastructure. Enterprises are no longer just calling a model. They are renting digital labor that can work continuously. These services support 24/7 long-horizon tasks and multi-agent collaboration. Model capabilities are also being embedded across developer workflows, from web, mobile, and desktop clients to command-line coding tools.

When evaluating such services, four areas deserve attention: task isolation and security boundaries, including sandbox permissions and secret management; observability, including whether every tool call and token consumption can be audited; failure recovery, including checkpoint resumption, idempotent design, and compensation mechanisms; and cost governance, including per-task or per-agent budget controls.

Cloud providers are moving in the same direction. Platforms such as Baidu AI Cloud are investing in general capabilities around large model services, agent hosting, monitoring, and scheduling. In essence, they are turning the job of running agents into an engineering discipline.

7. Developer Practice: Account for Capability and Economics

7.1 Evaluation: Let Your Own Tasks Speak

Public benchmarks should only be used as an initial filter. A better approach is to extract 30–50 representative tasks from real business workflows, including failure cases, and build a private evaluation set. Score candidates using three indicators: completion rate, average number of steps, and cost per task.

7.2 Five Cost-Control Techniques

1. Context management: Summarize long histories in segments. Keep only tool result fields that are relevant to the current step.
2. Cache first: Keep stable prefixes at the front. Avoid inserting timestamps, random numbers, or other content in the middle that breaks cache consistency.
3. Tiered routing: Send simple steps to a smaller model or a high-speed variant. Reserve the flagship model for critical decisions.
4. Budget circuit breaker: Set a token budget for each agent run. When the threshold is reached, automatically degrade or escalate to human review:

class TokenBudget:
    """Set a budget and circuit breaker for a single agent run."""
    def __init__(self, max_total, warn_ratio=0.8):
        self.max_total, self.warn_ratio, self.used = max_total, warn_ratio, 0
    def consume(self, tokens):
        self.used += tokens
        if self.used >= self.max_total:
            return "STOP"       # Circuit breaker: escalate to human or switch to a lower-cost model
        if self.used >= self.max_total * self.warn_ratio:
            return "COMPACT"    # Trigger context compaction
        return "OK"

1. Result caching and idempotency: Reuse tool call results for identical inputs to avoid paying twice.

7.3 A Pragmatic Selection Process

Start by benchmarking candidate models and their high-speed variants on a private evaluation set. Then split traffic by scenario, separating interactive and background workloads. Finally, put budgets and monitoring in place and treat cost per task as an iteration metric. Model capabilities change every month. Selection should be a quarterly cycle, not a one-time decision.

8. Conclusion

The clearest signal from this round of upgrades is that large model competition is moving from smarter chat to more reliable work. Post-training compute is flowing into agent environments, evaluation is focusing on long-horizon execution, pricing is carefully layered around caching and output, and managed services are turning the agent runtime into infrastructure. Together, these four threads form the early shape of digital labor.

For developers, the task is twofold: track the capability leaps and build an evaluation system and cost governance framework early. In the agent era, whether tokens are spent where they matter will determine whether a project scales into production or stops at a demo.