Multi-Agent Coding: Why Token Consumption Runs Away and How to Govern the Cost

1. The Signal: The Slope of the Cost Curve Has Changed

Over the past year, AI coding tools shifted from completion to agency. Completion tools were billed close to linearly: the amount of code a user types and the number of completions generated made consumption roughly predictable. Agentic tools introduced an autonomy loop — the model reads context, decides which tool to call, executes the tool, reads the result, and decides the next step until the task is judged complete.

That change has two consequences:

1. The unit of consumption moved from the person to the loop. A human engineer’s output is bounded by attention and working hours; a model loop’s output is bounded by budget and tool permissions. As long as the task is not judged complete, the loop keeps spending.
2. Free and low-cost tiers are disappearing, and cost is moving from implicit to explicit. Early trial credits, gift credits, and bundled subscriptions hid the true unit cost. Once heavy users start being billed per token or per call, the cost structure surfaces suddenly as a “bill event” — by which point there is often no code anywhere in the system that is measuring it.

Token consumption governance is therefore not a procurement question but an engineering one: control points have to be established at the application, model, tool, and data layers at the same time.

2. Mechanism Breakdown: Why Consumption Amplifies Quadratically

2.1 A Stateless API Serving a Stateful Task

Mainstream LLM inference endpoints are stateless by design: the server keeps no session, and the client must resend the full conversation history every turn. Agentic tasks, by contrast, are inherently stateful and long-horizon.

Let a task contain n inference turns, where turn i has input context length L_i and output length O_i. Without summary compression:

L_i ≈ L_0 + Σ(j < i) O_j + Σ(j < i) ToolResult_j
    ≈ L_0 + i · δ          (δ = average per-turn increment)

Total input tokens therefore:

Total_in = Σ L_i ≈ n · L_0 + δ · n · (n - 1) / 2

The key conclusion is the second term: input tokens grow quadratically with the number of turns. Doubling the task duration quadruples the cost. A task that merely “took a few extra turns to converge” can cost several times more than a comparable one, and the two look identical on the invoice.

2.2 An Illustrative Quantitative Model

Assume an initial context L_0 = 8000 tokens (system prompt + repository structure summary + task description), a per-turn increment δ = 1200 tokens, n = 40 turns, and a per-turn output O = 600 tokens:

def estimate_task_tokens(L0=8000, delta=1200, n=40, out=600, fanout=1):
    """
    Estimate token consumption for a single agent task
    (illustrative model, not the billing basis of any specific product)
    L0    : initial context length
    delta : average per-turn context increment (tool results, code snippets, reasoning traces)
    n     : number of inference turns
    out   : per-turn output length
    fanout: number of parallel sub-agents
    """
    total_in = n * L0 + delta * n * (n - 1) / 2
    total_out = n * out
    return {
        "input_tokens": total_in * fanout,
        "output_tokens": total_out * fanout,
    }
# 8000 initial + 1200 increment, 40 turns
# input  ≈ 320,000 + 936,000 = 1,256,000
# output ≈ 24,000
# input/output ratio ≈ 52 : 1

This model explains three counterintuitive facts:

• Input tokens dominate. They run one to two orders of magnitude above output tokens, so any cost reduction that only tries to make output “shorter” buys very little.
• The turn count n is the most sensitive variable, because it is the only parameter that enters the quadratic term.
• Parallel fan-out is a linear multiplier. k sub-agents in parallel compress wall-clock time, but the token cost is multiplied by k — and if the sub-agents share the same large context, that waste is multiplied by k as well.

2.3 Four Common Amplifiers

Amplifier	Symptom	Governance direction
Full-context replay	Every turn resends the complete history with no summary compression	Segment summaries, rolling compression
Oversized tool payloads	One directory listing or log fetch returns tens of thousands of tokens	Pagination and truncation of tool output
Loops without a convergence criterion	The model repeatedly re-checks itself and re-reads the same file	Deterministic checks, a maximum turn limit
Failed retries and drift	After a failure the task restarts halfway through, stacking context	Checkpoint resume, isolated sessions

3. Measure First: No Governance Without Metrics

The first step is not optimization; it is making consumption visible and attributable. Four metrics form a minimal viable observability surface:

• Cost per unit task: total tokens and cost to complete one acceptance-testable task, rather than “average daily tokens per user.” Only the former is comparable across teams, languages, and projects.
• Cache hit rate: how much of the input consists of reusable common prefixes (system instructions, standards documents, repository summaries). This is the single metric with the largest cost-reduction headroom.
• Turn efficiency: productive turns / total turns. If a large share of turns are re-reading the same file or re-running the same command, the tool design or the convergence criterion is at fault.
• Rework rate: the share of output rejected by a human or failing validation. Rework means a full re-run, an invisible amplifier of cost.

A lightweight accounting middleware covers the first two:

class TokenLedger:
    """Record context composition before and after each inference call,
    attributing cost by tenant / project / task"""
    def __init__(self, sink):
        self.sink = sink          # any generic writer: log service, message queue, monitoring endpoint
    def record(self, *, task_id, tenant, model_tier,
               prompt_tokens, cached_tokens, completion_tokens,
               turns, tool_calls, tool_bytes):
        cache_ratio = cached_tokens / prompt_tokens if prompt_tokens else 0.0
        self.sink.write({
            "task_id": task_id,
            "tenant": tenant,            # attribution dimension: team / project / repository
            "model_tier": model_tier,    # routing tier: lightweight / standard / reasoning-enhanced
            "prompt_tokens": prompt_tokens,
            "cached_tokens": cached_tokens,
            "completion_tokens": completion_tokens,
            "cache_ratio": round(cache_ratio, 4),
            "turns": turns,
            "tool_calls": tool_calls,
            "tool_bytes": tool_bytes,
        })

In practice this maps onto generic cloud platform building blocks — log collection, log service, monitoring and alerting, object storage. Raw detail goes to object storage for long-term retention and offline analysis; aggregated metrics go to monitoring and alerting for real-time threshold alerts. On cloud platforms such as Baidu AI Cloud, this pipeline is assembled entirely from generic product categories and needs no specialized component.

4. Six Cost-Reduction Levers

4.1 Context Engineering: Load on Demand Instead of Filling Up

Context is the most expensive resource because it is re-paid on every turn. The effective approach is layering:

• L0 stable layer: system instructions, coding standards, tool definitions. These must be fully static — any timestamp, random ID, or change of the current working directory busts the cache.
• L1 retrieval layer: code snippets, docs, and interface definitions retrieved dynamically for the current task. Mix semantic and keyword retrieval, and return only signatures and relevant implementations, not whole files.
• L2 observation layer: tool returns, command output, error stacks. Size control is mandatory here.

L2 in particular needs hard constraints: truncate command output to the first few thousand characters by default, keeping head and tail with a summary marker in the middle; list one directory level by default; filter logs by level. Put these constraints in the tool implementation, not in a prompt that hopes to persuade the model.

4.2 Prompt Caching: Design a Cacheable Prompt

Most inference services reuse a cache for identical prefixes. The engineering point is to push the volatile parts as late as possible:

# Anti-pattern: volatile content first, so the common prefix is busted every time
[current time] [username] [5000-token system prompt] [repo summary] [task]
# Recommended: stable content first, volatile content last
[5000-token system prompt] [coding standards] [tool definitions] [repo summary]   <- reusable prefix
                                                                                     cache boundary
[current time] [user identity] [current task description] [retrieval results]

In practice, keep the prefix byte-for-byte identical within a session (whitespace and newlines included), and put cache hit rate on the top-level monitoring dashboard as a first-class metric.

4.3 Tiered Model Routing

Not every step needs the strongest model. Route by task type:

Step	Typical capability requirement	Suggested tier
Intent classification, field extraction, format conversion	Short in / short out, high fault tolerance	Lightweight model
Single-file local edits, test completion	Medium context	Standard model
Cross-module refactoring, architecture trade-offs, hard defect localization	Long context, strong reasoning	Reasoning-enhanced model

The routing layer also handles degradation: when budget approaches a threshold, non-critical tasks drop to the lightweight tier automatically, preserving availability of the core path.

4.4 Budgets and Admission Control: Make Cost a First-Class Concern

Budget control has to live inside the execution loop, not after the bill is issued:

class BudgetGuard:
    """Two-level (task + tenant) budget with a hard circuit breaker against runaway loops"""
    def __init__(self, quota, ledger):
        self.quota = quota            # {task_id: token limit}, {tenant: daily limit}
        self.ledger = ledger
    def check_before_call(self, ctx):
        used = self.ledger.used(ctx.task_id)
        tenant_used = self.ledger.used_today(ctx.tenant)
        if used >= self.quota["task"][ctx.task_id]:
            raise BudgetExceeded("task budget exhausted", hint=ctx.partial_result)
        if tenant_used >= self.quota["tenant"][ctx.tenant]:
            raise BudgetExceeded("tenant daily quota exhausted")
        return True
    def check_progress_efficiency(self, ctx):
        """Abort early when turn efficiency is too low: 20 turns and not one static check passed"""
        if ctx.turns > 20 and ctx.validations_passed == 0:
            raise BudgetExceeded("no deterministic progress, abort early")

Supporting policies: set a maximum turn limit; on failure, resume from a checkpoint rather than restarting the whole session; add a human approval gate for high-risk or high-cost tasks.

4.5 Tool Design: Cut the Number of Round Trips

Every agent turn is a full re-payment of context, so reducing turns is more effective than shortening each turn’s content.

• Batch beats fine-grained. Offer tools that read multiple files at once or run multiple checks at once, collapsing 10 round trips into 1.
• Return structured, truncatable results. Include a pagination cursor, a summary header, and statistics so the model does not retry repeatedly to see the full picture.
• Make failure semantics explicit. Error messages should contain an actionable next step to cut down exploratory retries.
• Cap single-response size. Truncate on overflow and annotate the header: “truncated — request a specific range on demand.”

4.6 Replace Model Self-Review with Deterministic Validation

A model repeatedly “checking once more” is a common cost sink, and the root cause is the absence of a trustworthy objective feedback signal. Wire compilation, type checking, static analysis, unit tests, and formatting into the agent loop so that “is it correct” is answered by a deterministic tool rather than weighed repeatedly by the model on feel.

This pays off twice: fewer turns, and better output quality — which lowers the rework rate, the largest hidden source of cost.

5. A Sustainable Architecture and Common Anti-Patterns

A durable governance architecture usually has five layers:

1. Access layer: a unified inference gateway that all model calls must pass through, hiding provider differences and making uniform changes easy.
2. Routing and degradation layer: pick a tier by task type, degrade by budget state.
3. Budget and admission layer: task-, tenant-, and project-level quotas with hard circuit breakers.
4. Observability layer: token attribution, cache hit rate, turn efficiency, rework rate, wired into log service and monitoring/alerting.
5. Data layer: detail retained in object storage for offline cost attribution and trend analysis; findings feed back into routing policies and prompt templates.

Anti-patterns to avoid actively:

• Treating “higher context window utilization” as the optimization goal and stuffing the whole repository in.
• Using a model to do parsing, formatting, and table lookups that deterministic code could do.
• Approving cost only at the monthly invoice, with no control point anywhere in the execution path.
• Mixing volatile content such as timestamps and random IDs into prompts, so the cache is continuously invalidated.
• Using “average daily tokens” as the only KPI while ignoring cost per unit task and rework rate — which incentivizes teams to write more verbose task descriptions.
• Scaling parallel sub-agents without limit, trading cost for wall-clock time with no overall budget cap.

6. Conclusion and Action Checklist

The consumption explosion of agentic coding tools is not an anomaly; it is the inevitable result of combining an autonomous loop with a stateless API. In priority order:

1. Instrument first: attribute token consumption to team, project, and task; build a cost-per-unit-task dashboard.
2. Control turns: set a maximum turn limit, add deterministic validation, tune tool granularity to compress round trips.
3. Chase cache hits: restructure prompts so stable prefixes come first, and monitor cache hit rate.
4. Route by tier: lightweight tasks on lightweight models, reserve strong reasoning for architecture-level decisions.
5. Enforce budgets: two-level quotas at task and tenant scope, plus hard circuit breakers and automatic degradation.
6. Watch quality: measure efficiency by rework rate and acceptance pass rate, not by how many tokens were spent.

Once cost per unit task is observable, attributable, and predictable, token spend stops being a surprise at the end of the month and becomes an engineering cost item that can be reduced continuously.