Context Bloat and Cost Tiering Practices in Multi-Agent Collaboration
This article examines how multi-agent (orchestrator-worker) architectures address context bloat in long-running LLM tasks. It covers task decomposition principles, four practical context management strategies, model tiering for cost control, evaluation methods for long-horizon tasks, and an implementation checklist for balancing quality, stability, and cost.
1. Background: From Single Conversations to Multi-Agent Collaboration
As large language models have evolved from single-turn Q&A into agents that continuously call tools, read and write files, and run tests, individual sessions now carry increasingly long tasks. A typical coding or research task may include requirement breakdown, code retrieval, solution design, implementation, testing, and documentation. Keeping all of these stages in a single context creates two immediate problems: the context grows rapidly with each step, and different stages have very different requirements for model capability and cost.
Multi-agent collaboration is an engineering pattern designed to address exactly these issues. It assigns tasks to multiple roles: an orchestrator agent handles planning and aggregation, while multiple worker agents execute specific subtasks. This structure improves parallelism but also introduces new complexity in scheduling, state synchronization, and cost accounting, which must be clearly bounded at design time.
2. Basic Decomposition in the Orchestrator-Worker Architecture
The core of the orchestrator-worker pattern is separation of responsibilities:
- Orchestrator agent: understands the overall goal, generates a subtask list, dispatches work to worker agents, checks whether results meet acceptance criteria, and decides whether rework is needed.
- Worker agents: receive only the minimal context relevant to their own subtask, and return structured results upon completion rather than passing back their full reasoning trace.
- Shared state: task lists, interface contracts, and completed artifacts are stored in external storage and maintained centrally by the orchestrator.
Three principles should guide decomposition. First, dependencies between subtasks should be minimal to reduce waiting and conflicts. Second, every subtask should have machine-checkable acceptance conditions, such as tests passing, files existing, or output matching a specified format. Third, subtask granularity should be moderate: too fine amplifies scheduling overhead, while too coarse eliminates the benefits of parallelism.
Below is a simplified scheduling example that illustrates the data flow; it is not the interface of any specific framework:
def run_task(goal: str):plan = planner_model.plan(goal) # Orchestrator decomposes the taskresults = {}for subtask in plan.subtasks:ctx = build_minimal_context(subtask, results) # Pass only necessary informationoutput = worker_model.execute(subtask, ctx)if not passes_acceptance(subtask, output):output = worker_model.retry(subtask, ctx, feedback=output.issues)results[subtask.id] = summarize(output) # Return a summary, not full textreturn planner_model.review(goal, results)
The key functions in this example are build_minimal_context and summarize. The former limits the scope of downstream input, while the latter prevents long upstream outputs from entering the orchestrator’s context verbatim.
3. Causes and Costs of Context Growth
Context growth comes mainly from three sources: conversation history, tool call returns, and intermediate reasoning text generated by the model. Tool outputs are the most easily overlooked. A single file read or log retrieval may return thousands of lines of text; if kept in the context unfiltered, every subsequent reasoning round repeatedly pays for that overhead.
The cost of context growth typically shows up in three ways:
- Cost: input tokens are billed cumulatively per turn, so the longer the context, the higher the per-turn cost.
- Latency: longer inputs generally mean longer time to first token.
- Attention dilution: when key constraints are buried under large amounts of irrelevant content, the model becomes more likely to miss early requirements—manifesting as conventions confirmed earlier being ignored later.
Context management is therefore a prerequisite for reliably completing long tasks, not an optional optimization.
4. Four Common Context Management Strategies
First, rolling summarization. When the context exceeds a threshold, compress earlier conversation into a structured summary that preserves four categories of information: goals, constraints, decisions made, and open items. The summary should be readable on its own, not just a vague description.
Second, external memory. Write requirement documents, design decisions, and interface definitions to files or a database, and retrieve them by keyword or version when needed, rather than keeping them resident in the context.
Third, tool output truncation and filtering. Set length limits on logs, search results, and similar content; return only matching fragments and necessary line numbers, with full content stored externally and read on demand.
Fourth, subtask isolation. Each worker agent uses an independent context and brings back only conclusions. This is one of the most direct benefits of the multi-agent pattern.
In practice, these four strategies are usually combined. Relying solely on rolling summarization risks losing detail; relying solely on external memory adds retrieval steps. Combining them balances completeness against overhead.
5. Model Tiering and Quota Management
Not every subtask requires the most capable model. A common approach is tiering by role:
| Role | Typical Tasks | Capability Requirements | Cost Strategy |
|---|---|---|---|
| Orchestrator agent | Planning, conflict arbitration, final review | Strong reasoning and judgment | Fewer calls; prioritize quality |
| Worker agents | Code modification, format conversion, document generation | Stable instruction following | Batch calls; choose cost-effective models |
| Verification | Test execution, rule checks | Can be done by scripts | Avoid model calls where possible |
The prerequisite for tiering is observability. At minimum, record the following metrics: input and output token counts per task, number of calls, number of retries, and acceptance pass rate. Without this data, there is no way to judge whether a given model is actually more cost-effective for a given task type.
Quota management requires two additional considerations. First, set budget thresholds within a cycle; when approaching the limit, automatically downgrade to a cheaper model or pause non-critical tasks. Second, set an upper limit on retries to prevent failure loops from continuously consuming quota.
6. Evaluating Model Performance on Long Tasks
Subjective impressions are easily influenced by usage phase. When quota is tight, people pay more attention to savings, and changes in output style may be misread as capability degradation. A more reliable approach is to build a fixed evaluation set:
- Select ten to twenty representative tasks covering code modification, troubleshooting, document writing, and similar types;
- Define clear acceptance criteria for each task, and use scripts to judge results wherever possible;
- Fix context length and tool configuration, changing only the model, then compare pass rate, number of turns, and total token consumption;
- Examine reasoning ability separately—for example, present problems requiring multi-step derivation and check whether intermediate steps are self-consistent.
Two phenomena are worth recording specifically. The first is premature task termination, where the model claims completion before acceptance conditions are met. This can be mitigated by adding explicit completion checks in the orchestrator. The second is declining adherence to early instructions as the context grows. This can be improved by injecting a summary of key constraints at each turn.
7. Implementation Checklist
- Can the task be decomposed into subtasks with few dependencies, each with machine-checkable acceptance conditions?
- Does each worker agent receive only the necessary context and return only a summary?
- Are length limits set on tool outputs?
- Are token consumption and retry counts recorded for each role?
- Are budget thresholds and retry limits configured?
- Is there a fixed evaluation set for comparing different models?
8. Conclusion
The value of multi-agent collaboration lies in breaking long tasks into manageable units, but decomposition itself introduces scheduling and cost challenges. Through clear responsibility boundaries, context control, model tiering, and observable evaluation, developers can find the balance between quality, stability, and cost that fits their own scenarios.
