LLM Agent Evolution: Post-Training, Evaluation, and Token Cost Optimization

1. Introduction: From “Single-Turn Intelligence” to “Persistent Execution”

The recent release of next-generation flagship large language models (LLMs) showcases several shared characteristics: parameter scales reaching the trillion level, context windows entering the million-token range, and a strong emphasis on coding and agentic capabilities. Many of these models also offer high-speed versions that promise “equivalent intelligence with 2x to 3x faster output.” Public evaluations indicate that top-tier models now match or closely approach the performance of the strongest closed-source models in coding, knowledge, and agent benchmarks.

However, user feedback reveals a contrasting reality: in heavy agent scenarios, token quotas can be depleted in just over ten minutes. The contradiction between “approaching top-tier capabilities” and “quotas running out quickly” actually points to the same underlying trend. The competitive focus of LLMs is shifting from pre-training scale to post-training depth and inference infrastructure. Understanding the technical logic behind this shift is essential for effective model selection and cost governance.

2. Scaling Post-Training: Reinforcement Learning Across Hundreds of Domains

1. From “Alignment Tool” to “Capability Engine.” Traditional post-training primarily relied on Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) to make the model “speak well.” The new approach centers on Reinforcement Learning with Verifiable Rewards (RLVR): whether code passes tests, whether tool calls complete tasks, and whether reasoning results can be verified. These objective signals allow rewards to no longer depend on expensive and noisy human annotations, enabling large-scale automated generation. As RL training domains expand from code and math to hundreds of downstream tasks, the model acquires the ability to “get things done.”

2. The True Bottleneck is Infrastructure. Conducting large-scale reinforcement learning across hundreds of domains requires far more than just a training script; it necessitates systematic engineering overhauls:

• RL Training Framework: Rollout generation and policy updates alternate, requiring a high-throughput sampling pipeline and stable update mechanisms to prevent gradient instability in long-sequence training.
• Long-Context Inference: During training loops, models repeatedly process ultra-long trajectories. Inference-side KV cache reuse and memory management directly determine training throughput.
• Training Efficiency: Increasing the proportion of valid samples through techniques like sample replay and difficulty curriculum learning.
• Sandbox Infrastructure: Agent training requires models to execute code and tool calls, demanding a safe, isolated, and reproducible execution environment.
• Scheduling System: Large-scale rollouts create a need for mixed orchestration of heterogeneous computing resources for both training and inference.
• Environment Quality Assurance Platform: Automatically verifying the consistency between task environments and reward signals to prevent reward hacking—where the model learns to exploit environment loopholes rather than genuinely completing the task.

The maturity of this training infrastructure is becoming the key variable widening the capability gap among models, occupying a role similar to that of large-scale compute clusters in the previous cycle.

3. Long-Horizon Agents: The Model is the “Brain,” the Harness is the “Body”

If the selling point of the previous generation was “answering well,” the direction for this generation is “working long.” The software scaffolding that supports the model’s continuous long-term work—often referred to in the industry as the Harness—includes tool registration and invocation protocols, task state and memory management, error recovery and retry strategies, and execution lifecycle management. The model determines the upper limit of intelligence, while the Harness determines whether that intelligence can be reliably realized.

This has also spawned the service model of “Managed Agents”: platform providers consolidate the Harness supporting long-running agents into a cloud infrastructure, opening it to enterprises and developers, supporting 24/7 long-horizon task execution and multi-agent collaboration. For enterprises, this transfers the engineering burden of “building a reliable agent runtime” to the platform, allowing them to focus on task definition and result verification. When evaluating such services, it is recommended to focus on three aspects: checkpoint recovery capabilities (whether tasks can resume from a checkpoint after failure), execution observability (whether every tool call leaves a trace), and sandbox security level.

4. Coding Capability Evaluation: Understanding “Approaching the Strongest Closed-Source”

1. Terminal Environment Benchmarks: These assign complex tasks to coding agents in real, resource-constrained terminal environments, pairing each task with specific test cases to comprehensively evaluate environment perception, command selection, multi-step execution, and self-correction. Compared to static code completion benchmarks, this is much closer to real-world engineering. Public data shows that top models have generally improved their pass rates on these benchmarks by tens of percentage points over the past year, though a clear gap remains before “reliable delivery” is achieved.

2. Long-Horizon Code Benchmarks: These simulate cross-file, cross-module real-world engineering changes, focusing on evaluating multi-step execution, tool call orchestration, and end-to-end problem-solving capabilities. The improvement of new-generation models on these tests is often greater than on single-turn benchmarks, directly reflecting the investment of post-training compute into agent scenarios.

3. Three Pitfalls in Evaluation: First, data contamination, where tasks or solutions may have already entered the training corpus; second, environment drift, where the benchmark environment differs vastly from your production environment; third, variance, where pass rates for small-sample tasks fluctuate heavily, and a single-digit percentage lead may not be statistically significant. A prudent approach is to use public benchmarks as an initial filter, and use a self-built business evaluation set (extracting real tickets, configuring reproducible environments) for final validation.

5. Speed and Cost: High-Speed Versions, Caching, and Token Economics

1. Understanding the Standard Pricing Structure. Mainstream LLM API pricing generally follows three rules: the price for cached input tokens is about one-tenth of uncached tokens; the unit price for output tokens is significantly higher than for input; and high-speed versions (equal intelligence, faster generation) cost about twice as much as standard versions. This structure profoundly impacts the total cost in agent scenarios.

2. Why “Quotas Burn Out in Minutes.” This is usually not because the model is “expensive,” but due to the compounding of three structural factors:

• Long Context Repeated Replay: The agent must carry its full history in every call, causing the context to expand linearly with turns.
• Multi-Turn Loop Amplification: A long-horizon task easily involves dozens of tool calls, multiplying token consumption.
• Verbose Output and High Throughput: Models tend to produce detailed explanations, and output unit prices are the highest. High-speed versions increase generation speed by 2x-3x; if quotas are measured by tokens, the quota consumption per unit time is likewise amplified. Burning through it faster is a physical inevitability.

Let’s do a schematic estimation (numbers are hypothetical, not representing any product data): A long-horizon task loops 20 times, carrying an accumulated context of about 80,000 tokens per turn, accumulating roughly 1.6 million input tokens. Output is calculated at 2,000 tokens per turn, totaling only 40,000 tokens. The output token volume accounts for less than 3%, but because its unit price is multiples of the input, its share of the total cost may exceed 30%. In other words, the benefit of controlling output length and call turns is often greater than obsessing over input unit prices.

3. Cost Optimization Checklist:

• Prefix-Cache Friendly Design: Place stable content like system instructions and tool definitions at the prompt prefix and keep them byte-level consistent to maximize cache hit rates.
• Context Compression: Summarize old trajectories, retaining only key states and the details of the recent K turns.
• Task Decomposition and Model Routing: Delegate planning steps to stronger models; assign format conversion and simple retrieval to lightweight models.
• Structured Output Constraints: Require compact formats like JSON and set length limits to suppress verbose output.
• Budget Circuit Breaker: Set a token budget for each task; degrade or abort if exceeded.

Schematic code (Python-style pseudocode):

class BudgetedAgent:
    """Agent main loop with token budget and context compression (schematic)"""
    def __init__(self, model, tools, budget=1_000_000, ctx_limit=80_000):
        self.model, self.tools = model, tools
        self.budget, self.ctx_limit = budget, ctx_limit
        self.history = []
    def run(self, task, max_turns=20):
        for _ in range(max_turns):
            # 1. Context exceeds limit: summarize old trajectories to avoid linear expansion
            if self._tokens(self.history) > self.ctx_limit:
                self.history = [self._summarize(self.history)]
            # 2. Stable prefix first to maximize cache hits
            prompt = self._stable_prefix() + self._render(self.history)
            # 3. Structured output, constrain length
            action = self.model.decide(prompt, fmt="json", max_tokens=2000)
            if action["type"] == "finish":
                return action["result"]
            # 4. Tools execute in sandbox, idempotent and replayable
            result = self.tools[action["name"]].run(action["args"])
            self.history.append((action, result))
            # 5. Budget circuit breaker: abort if overspent
            if self._spent() > self.budget:
                raise BudgetExceeded(self._spent())

6. Enterprise Implementation: A Reference Agent Platform Architecture

Synthesizing the above analysis, enterprises deploying long-horizon agents can refer to the following layered architecture:

• Access and Orchestration Layer: Task queues, schedulers, and multi-agent collaboration logic.
• Model Layer: Accessing standard/high-speed models via a unified gateway, routing by task type.
• Execution Layer: Containerized sandboxes running code and tool calls, ensuring isolation, reproducibility, and replayability.
• Data Layer: Object storage for task trajectories and artifacts, message queues for decoupling orchestration and execution.
• Observability Layer: Logs, monitoring, and alerts covering every model call and tool execution, supporting cost attribution.

These are general foundational capability categories in the industry. Taking Baidu AI Cloud as an example, one could combine its container platform (hosting sandboxes and executors), object storage (trajectory and artifact archiving), message queue (task decoupling), and logging and monitoring services (full-chain observability) to build the entire architecture. For selection, it is recommended to focus on SLA, network latency, sandbox security compliance, and overall cost as core considerations, rather than a single dimension.

Another factor worth incorporating into planning is the pace of model weight releases: top vendors generally adopt a strategy of “closed-source API first, open weights following.” The full weights of a new flagship are often released within weeks after launch. For scenarios with strict data compliance requirements, a two-stage route of “API effect validation → private deployment of open weights” can be adopted.

7. Conclusion

“Performance approaching top-tier” and “quotas depleting rapidly” are two sides of the same coin: the former is the return on post-training scaling and agent infrastructure investment, while the latter is the bill resulting from the overlay of long contexts, multi-turn loops, and high-priced outputs. For developers and enterprises, the rational stance is not to chase single-digit percentage leads on leaderboards, but to establish their own evaluation systems and cost models. Solidifying engineering practices like caching, compression, routing, and budget circuit breakers often yields a better return on investment than simply switching to a smarter model. The next leg of the LLM competition is not just about whose model is smarter, but about who can make intelligence work persistently in an affordable manner.