Compute Consumption and Cost Governance Practices in Multi-Agent Collaboration

    In multi-agent applications, a single user request often triggers a complex chain of internal model calls, leading to unpredictable compute costs and budget overruns. This article explores the sources of token and compute consumption in multi-agent systems, analyzes the typical mechanisms behind cost runaway—such as recursive fan-out and infinite validation loops—and outlines five core principles for cost governance. It further details engineering practices including tiered budgeting, derivation control, context compression, model tiered routing, and retry strategies to help developers build cost-effective and sustainable multi-agent architectures.

    1. Problem Background: The Multiple Calls Behind a Single Request

    In Agent-based applications, users typically submit a single instruction, such as “organize these materials and output a conclusion.” Internally, the system may break this down into multiple stages, including task planning, data retrieval, step-by-step analysis, result verification, and final summarization. Each stage may trigger one or more model calls, and subtasks often need to share context with one another. What the user perceives as a single task is actually a call chain from a billing and resource perspective.

    This cognitive gap leads to two direct consequences. First, there is a mismatch between the user’s expectation of quota consumption and the actual usage, often resulting in a poor experience where users feel their “quota disappears too quickly.” Second, the compute cost for the service provider grows with task complexity, and this growth rate often exceeds linear scaling. Without a constraint mechanism, the cost structure can quickly become unbalanced.

    2. What Constitutes the Cost?

    To govern costs, we must first clarify the cost sources. The expense of a single agent task can be estimated using the following approach:

    Single Task Cost ≈ Σ (Input Tokens per Call × Input Unit Price + Output Tokens × Output Unit Price) + Tool Call Cost + Retry-Induced Extra Cost

    Specifically, Token consumption mainly comes from four types of content:

    1. Instructions and System Prompts: These must be re-sent with every call. The more subtasks there are, the more apparent this repeated consumption becomes.
    2. Context History: In multi-turn interactions, existing conversation records and intermediate results are repeatedly fed into the model. The longer the context, the higher the input cost per call.
    3. Tool Returned Content: Data volume from retrieval results, file snippets, and API responses is usually large, accounting for a significant portion of total consumption.
    4. Model Outputs: This includes reasoning content and the final generated results. In many pricing schemes, the output unit price is higher than the input unit price, subject to the service provider’s specific rules.

    Beyond Tokens, there are two other pressures on compute resources. Long contexts increase the overhead of attention calculation, and parallel subtasks occupy more concurrent resources. Retries after failures also repeat the resources already spent.

    3. Typical Mechanisms of Cost Runaway

    Cost runaway is rarely caused by a single factor; rather, it is the result of overlapping mechanisms.

    Recursive Derivation Leading to Fan-Out Explosion. When subtasks spawn further subtasks, the number of calls grows exponentially. Assuming three branches are derived per layer, after three layers, a single root task will generate 39 sub-calls (3 + 9 + 27). If each sub-call carries the complete context, total consumption will far exceed expectations.

    Lack of Limits on Sibling Parallelism. To speed up processing, a system might simultaneously launch a dozen or more subtasks. This consumes quota in a short burst, causing users to feel their quota has “suddenly run out.” Parallelism itself isn’t the issue; the problem is the lack of concurrency limits and budget validation.

    Verification Loops Creating Cycles. Many systems set up verification steps, instructing the model to correct issues if found, followed by re-verification. Without a maximum round limit, this loop could run indefinitely.

    Context Being Fully Duplicated. Every subtask receives the complete history, even if it only needs a small portion of that information. The cost therefore scales linearly with the number of subtasks, and sometimes even higher.

    4. Basic Principles of Governance

    In response to the mechanisms above, five principles can be summarized:

    1. Upfront Budgeting: Establish a Token limit before the task starts, rather than throwing an error only after the quota is exhausted.
    2. Depth and Width Dual Constraints: Simultaneously constrain the number of derivation layers and the number of subtasks per node to prevent unbounded fan-out.
    3. Deterministic Steps to Code: Operations like format conversion, deduplication, counting, and sorting do not require model involvement. Completing these with standard code is cheaper and more stable.
    4. Full-Link Observability: Every call must be traceable to its parent task, making it easy to locate which step consumes the most.
    5. Cost-Value Alignment: Investment should be measured by task completion quality, not just by the number of calls.

    5. Engineering Practice Solutions

    5.1 Tiered Budgeting

    Budgets can be divided into three tiers. The first tier is the organizational or user’s periodic quota, determining the total available volume. The second tier is the task-level limit, capping how many Tokens a single task can consume. The third tier is the subtask quota, allocating a relatively fixed share to each subtask.

    When task consumption approaches the limit—for example, reaching 80%—the system should prompt the user to confirm whether to continue and provide an estimated remaining cost. This gives users control and prevents passive interruptions mid-task.

    5.2 Controlling Derivation Behavior

    Derivation control requires three parameters: maximum derivation depth, maximum subtasks per node, and global maximum concurrency. These three must be used together to restrict both vertical and horizontal expansion.

    The schematic code below demonstrates the basic logic of budget validation and derivation control:

    1. from dataclasses import dataclass
    2. class BudgetExceeded(Exception):
    3. """Exception raised when budget or derivation rules are triggered."""
    4. @dataclass
    5. class TaskBudget:
    6. max_tokens: int # Total token limit for the task
    7. max_depth: int = 3 # Maximum derivation depth
    8. max_fanout: int = 5 # Maximum number of subtasks per node
    9. used_tokens: int = 0 # Number of tokens consumed
    10. def precheck(self, estimated: int) -> None:
    11. """Pre-check using estimated value before the call to avoid discovering the limit only after exceeding it."""
    12. if self.used_tokens + estimated > self.max_tokens:
    13. raise BudgetExceeded(
    14. f"Estimated to exceed budget: used {self.used_tokens}, "
    15. f"estimated {estimated}, limit {self.max_tokens}"
    16. )
    17. def charge(self, actual: int) -> None:
    18. """Correct with actual usage after the call."""
    19. self.used_tokens += actual
    20. def check_spawn(self, depth: int, fanout: int) -> None:
    21. if depth >= self.max_depth:
    22. raise BudgetExceeded(f"Exceeded maximum derivation depth {self.max_depth}")
    23. if fanout > self.max_fanout:
    24. raise BudgetExceeded(f"Exceeded maximum subtasks per node {self.max_fanout}")
    25. def run_subtasks(budget, depth, subtasks, call_model, estimate_tokens):
    26. """Execute subtasks sequentially, pre-checking and charging for each step."""
    27. budget.check_spawn(depth, len(subtasks))
    28. results = []
    29. for task in subtasks:
    30. budget.precheck(estimate_tokens(task))
    31. text, used = call_model(task) # Returns result text and actual token usage
    32. budget.charge(used)
    33. results.append(text)
    34. return results

    There are two key points to this code. The pre-check phase uses estimated values, intercepting over-budget requests before the call. The charging phase uses actual values, ensuring the ledger is accurate. Together, they prevent obvious overspending while keeping budget data consistent with real consumption.

    5.3 Context Compression and Reuse

    Context management is the most direct way to reduce costs. In practice, follow these guidelines:

    • Pass Only Necessary Fields: Subtasks should only receive the information required to achieve their specific goals, not the entire history.
    • Stage Summaries: After a subtask completes, compress its result into a structured summary. Subsequent steps should reference the summary rather than the original text.
    • Prefix Stable Content: Place system prompts, fixed rules, and reference documents at the beginning of the prompt. If the server supports prefix caching, the stable portions may be reused, reducing repeated billing.

    The quality of the summary directly impacts subsequent results. When compressing, retain entities, numbers, conclusions, and action items, while discarding pleasantries, redundant descriptions, and intermediate reasoning processes.

    5.4 Model Tiered Routing

    Not all subtasks require the most powerful model. In practice, you can tier models by complexity:

    • Simple Extraction, Classification, Format Conversion: Assign to smaller models, or complete directly using rules.
    • Routine Analysis and Rewriting: Use medium-sized models.
    • Complex Reasoning, Solution Design, Key Decisions: Reserve for larger models, and strictly control the number of calls.

    The prerequisite for tiering is having an evaluation set. The team should use a batch of labeled samples to compare the accuracy of different models on various types of subtasks before deciding on routing rules. Without evaluation data, tiering easily degrades into subjective downgrading, ultimately compromising the quality of results.

    5.5 Failure and Retry Strategy

    Retries are a common source of cost amplification. Retry strategies should distinguish between two types of errors: Transient.