Why Long-Horizon Agents Burn Through Token Quotas in Minutes

1. The Paradox: Stronger Models, Faster-Burning Quotas

Flagship model releases have accelerated noticeably. Based on public information, this generation shares several traits: parameter counts in the trillions, context windows extended to millions of tokens, substantially increased compute investment in reinforcement learning post-training across hundreds of downstream domains, and code plus long-horizon agent capabilities as headline features. Some vendors also offer high-speed variants that match the standard version’s intelligence while delivering 2–3x faster output.

As the capability curve climbs steeply, a different signal has emerged from the developer community: run long-horizon agent tasks on these models and you can burn a full day’s quota in ten minutes. This isn’t simply “models got more expensive.” Long-horizon agents fundamentally change the structure of token consumption. This article unpacks three things: where the capability comes from, where the money goes, and what practical engineering countermeasures exist.

2. Post-Training and RL Scaling: The Foundation of Capability Jumps

Pre-training provides general language and world knowledge. What increasingly determines the ceiling of “how smart” a model is, however, is reinforcement learning during post-training. Scaling RL across hundreds of downstream domains essentially does three things:

1. Diversified reward signals. Code tasks use unit test pass/fail as verifiable reward; tool-calling tasks use whether the final task state is achieved as reward; reasoning tasks use process correctness and conclusion consistency as reward. Multi-domain rewards teach the model general problem-solving strategies rather than task-specific test-taking tricks.
2. Environment scaling. To learn in environments resembling the real world, you must build large numbers of executable, reproducible, scoreable sandbox environments covering terminal commands, file systems, network requests, browser operations, and more.
3. Training system rearchitecture. Supporting post-training at this scale typically requires systematic overhauls of the RL framework, long-context inference, training efficiency, sandbox infrastructure, scheduling systems, and environment quality-check platforms. This goes well beyond “algorithmic innovation”—it’s closer to building a complete engineering system.

One trend worth noting: the boundary of model capability increasingly depends on the quality of environment engineering. Non-reproducible environments, reward signal leakage, and distribution overlap between evaluation and training tasks all cause RL to learn “gaming the system” rather than genuine capability. As a result, environment quality checks—such as detecting whether rewards can be captured by speculative strategies—are becoming first-class citizens in the post-training pipeline.

3. Million-Token Context and Long-Horizon Agents: The Harness Is the Real Battleground

A million-token context is a nice-to-have for ordinary chat, but a hard requirement for agents: long-horizon tasks need to carry task plans, tool-call history, intermediate artifacts, and failure lessons. Context is the agent’s working memory.

The more critical insight is this: Agent capability = model + harness. The harness refers to the scaffolding outside the model, including tool sets, sandboxes, memory, scheduling, checkpoints, and retry policies. The same model attached to harnesses of different quality can differ several-fold in end-to-end task completion rate. Industry benchmarks have followed this trend: new coding benchmarks no longer test single-turn code completion but instead present a series of complex tasks in realistic, resource-constrained terminal environments, each with dedicated tests, evaluating whether the model can pass all tests after multiple rounds of tool calls and autonomous debugging. Such benchmarks measure the combined capability of “model + scaffolding.”

Complementing this is the rise of managed agent infrastructure: offering the harness that supports long-running agents as a cloud service, enabling 24/7 long-horizon tasks and multi-agent collaboration. For teams, this means handing the engineering burden of “keeping the model working” to infrastructure and focusing on task definition and result acceptance.

4. High-Speed Variants: The Speed-Cost Tradeoff

High-speed variants boost output speed 2–3x while maintaining intelligence. For long-horizon agents, the value shows up in two ways. First, shorter wall-clock time: for a task with hundreds of rounds, doubling output speed nearly halves delivery time. Second, better interaction experience: in streaming scenarios, first-token latency and throughput directly determine usability.

From a general technical perspective, common inference acceleration paths include prefix caching (reusing the KV cache of repeated prefixes), speculative decoding (a small model drafts, the large model verifies), and continuous batching (improving overall throughput). Note that high-speed variants usually carry a higher unit price. The sensible approach is to apply speed where it matters: latency-sensitive steps like planning and summarization go through the fast channel, while batch execution steps go through the standard channel.

5. Why Tokens Burn So Fast: A Clear Accounting

Long-horizon agent token consumption has three amplifiers:

1. Context accumulation. Agents are loops: each round resends the full history (system prompt, tool definitions, prior reasoning and observations) as input, so total consumption grows roughly quadratically.
2. Long reasoning chains. Reasoning models emit lengthy thinking before acting, all billed at output rates—and output rates are typically 5–20x input rates.
3. Tool observation bloat. Log files, web page bodies, directory listings, and other tool returns easily run into thousands of tokens, much of it noise.

A simple estimator makes this concrete:

def estimate_session_cost(
    rounds: int,            # number of loop rounds
    base_context: int,      # initial context (system prompt + tool definitions)
    avg_new_input: int,     # new observations per round (tool results, etc.)
    avg_output: int,        # output per round (thinking + action)
    price_in: float,        # input price: per million tokens
    price_out: float,       # output price: per million tokens
    cache_ratio: float = 0.0,     # prefix cache hit rate
    cache_discount: float = 0.1,  # cache hit price = input price x discount
) -> float:
    total_in = total_out = 0
    ctx = base_context
    for _ in range(rounds):
        total_in += ctx + avg_new_input
        total_out += avg_output
        ctx += avg_new_input + avg_output      # history keeps accumulating
    cached = total_in * cache_ratio
    return ((total_in - cached) * price_in
            + cached * price_in * cache_discount) / 1e6 \
           + total_out * price_out / 1e6

Plug in illustrative parameters: a 60-round task, 8,000 tokens of initial context, 3,000 tokens of new observations per round, 1,500 tokens of output per round. The calculation yields roughly 8.62 million input tokens and about 90,000 output tokens. At illustrative prices of “10 per million input, 60 per million output,” total cost is about 92; if the prefix cache hit rate reaches 90% (cache hits billed at one-tenth of input price), cost drops to about 22—a reduction of more than three-quarters.

The conclusion is clear: the dominant cost in long-horizon agents is “repeatedly resent historical input,” followed by “long reasoning output.” The former is addressed by caching; the latter by compression and output governance.

6. An Engineering Checklist for Cost and Efficiency

1. Design cache-friendly prompt structures. Place stable content (system prompt, tool schemas, examples) at the very front of the message sequence and dynamic content after it. Avoid inserting timestamps, random IDs, or other volatile fields into the system prompt—a single character change invalidates the entire prefix cache.

SYSTEM_PROMPT = "You are a task-execution agent..."   # stable segment: placed first
TOOL_BLOCK = json.dumps(tool_schemas)                 # stable segment: frozen tool definitions
def build_messages(task: str, scratchpad: list) -> list:
    return [
        {"role": "system", "content": f"{SYSTEM_PROMPT}\nTools: {TOOL_BLOCK}"},
        {"role": "user", "content": task},      # dynamic segment comes after
        *scratchpad,                            # history appended incrementally
    ]

2. Context governance. Sliding window plus rolling summarization: once a threshold is exceeded, compress early rounds into structured summaries. Filter tool output before it enters context (for example, keep only ERROR and key lines from logs). Keep large files out of context—externalize them to object storage and let the agent retrieve fragments on demand.

3. Task decomposition and sub-agent isolation. Split long tasks into several sub-agents, each holding its own context, with the parent agent receiving only conclusion summaries. Sub-task contexts don’t contaminate each other, and a single session’s context avoids linear bloat.

4. Output governance. Constrain action formats with structured output. Limit per-round reasoning length. Require “plan first, then execute step by step” to reduce back-and-forth trial and error.

5. Checkpoints and sandboxes. Persist intermediate state so failures resume from a breakpoint rather than rerunning from scratch. Sandboxes ensure tool calls can be safely retried. One successful breakpoint recovery saves all the tokens from prior rounds.

6. Budget guardrails. Set a token budget per session or task, triggering a circuit breaker or degradation when exceeded, so runaway tasks don’t burn through the quota:

class TokenBudget:
    def __init__(self, limit: int):
        self.limit, self.used = limit, 0
    def spend(self, n: int) -> None:
        if self.used + n > self.limit:
            raise RuntimeError(
                f"Budget exceeded: used {self.used}, this request needs {n}, limit {self.limit}")
        self.used += n
# Attach counting before and after each request; on breach, trip the breaker and persist a checkpoint

7. Tiered model routing. Route planning and complex reasoning to strong models; route formatting, information extraction, summarization, and other lightweight steps to smaller models or high-speed variants. Allocate compute by task value.

7. Managed Agent Infrastructure: Build or Buy

If you build the harness yourself, you need to cover several infrastructure pieces: containerized sandboxes (tool execution and isolation), object storage (artifacts and checkpoints), message queues (multi-agent task orchestration), logging and monitoring/alerting (observability), and scheduling (24/7 task queuing and resource allocation). General-purpose capabilities from cloud platforms—such as container services, object storage, log services, and monitoring/alerting like those offered by Baidu AI Cloud—can serve as the foundation for building this infrastructure, letting you focus on task orchestration and quality evaluation.

If you choose a managed agent service, evaluate four aspects: the persistence mechanism for context and state, the security and isolation level of the sandbox, the scheduling capability for multi-agent collaboration, and the metering methodology—long tasks are most prone to overspending on “invisible tokens.”

8. Conclusion

Investment in post-training compute makes models “better at doing work,” but being better at work inevitably means “burning more tokens.” This isn’t a defect; it’s an inherent property of the long-horizon agent paradigm. For engineering teams, the real competitive edge is no longer “using the strongest model” but maximizing output per million tokens: caching design determines input cost, context governance determines information density, task orchestration determines wasted rounds, and budget guardrails determine the floor before things spiral. Capability comes from the model; cost savings come from you.