How to Choose LLM Models by Tier for Multi-Agent Coding Assistants: A Budget-Sensitive Selection Guide
Choosing the right model for each role in a multi-agent coding system is what keeps token cost, latency, and retries under control. This guide explains how to divide a coding agent into single-responsibility roles, map those roles to five abstract model tiers, route tasks by difficulty, and manage everything through declarative configuration and runtime policies. You will learn context-compression, retrieval-reuse, parallel-dispatch, and failure-handling practices, plus a small evaluation-set approach and five common pitfalls to avoid. The goal is a stable, cost-efficient agent setup that avoids both "all-top-tier" waste and "over-compression" quality collapse.
1. Introduction: Why Tiered Selection Matters
AI-assisted coding has evolved from “single-turn completion” into coding agents capable of “autonomous planning plus multi-step execution.” Under complex requirements, a single agent often has to handle requirement decomposition, code retrieval, solution design, implementation, test-and-fix, and result review all at once. Because of context-window and attention-quality limits, a single model struggles to stay consistent across every one of these stages.
The common industry response is multi-agent collaboration: a coordinator agent that orchestrates, executor sub-agents that do deep work, and reviewer agents that check quality. Dividing labor lowers each agent’s cognitive load, but it also creates new problems:
- The number of calls multiplies;
- Context keeps accumulating as tasks pass between roles;
- A single failure can trigger retries or even task rollback.
In such a chain, if every role runs the strongest model, token consumption, response latency, and retry costs spiral out of control quickly. Conversely, if everything runs low-tier models, planning and review quality drop, rework increases, and total cost actually rises. So the core of a “budget-sensitive” multi-agent design is not simply pushing unit price down, but selecting models by tier according to the value of each role, so that expensive capability only appears at critical nodes.
This article starts with a cost-estimation example used throughout. Suppose a complex coding task needs 60 rounds of model calls, averaging 4,000 input-plus-output tokens per round, for a total of 240,000 tokens. If different roles use models at different price points, and you add retrieval reuse, context trimming, and downgrade strategies, the call count can be kept under 30 rounds. Roughly estimated, overall cost can drop to around one-tenth of the “all-premium-model” approach.
Note: All numbers here are illustrative values for explanation. Actual results depend on the specific models, business scenario, and implementation quality.
2. Role Division and Responsibility Boundaries in Multi-Agent Collaboration
Before configuring models, you should first identify which roles exist in the system and define the input/output boundary of each. Role division follows the “single responsibility” principle: each role handles one type of similar work, takes the leanest possible input, and produces structured output where feasible (for example, JSON).
| Role | Typical Responsibilities | Input Characteristics | Core Model Requirements |
|---|---|---|---|
| Master planner | Understand requirements, decompose tasks, decide scheduling order, aggregate results | Full requirement + task history, longest context | Long-context understanding, complex planning, instruction following |
| Deep executor | Code implementation, refactoring, bug fixing, running tests | Large code blocks + error logs | Code generation and debugging, tool-call reliability |
| Solution designer | Design module structure, define interfaces and data structures | Requirements + summary of existing architecture | Architectural reasoning, trade-off analysis |
| Reviewer/analyst | Review code diffs, analyze root causes, suggest improvements | Changed snippets + relevant context | Analytical accuracy, attention to detail |
| Lightweight subtask executor | Format conversion, boilerplate, simple edits | Small, well-defined instructions | Basic competence, low cost, low latency |
| Code retriever | Search symbols and definitions, locate related files | Keywords + retrieval results | Retrieval capability, no heavy reasoning needed |
| Multimodal interpreter | Read screenshots, prototypes, architecture diagrams | Image + a little text | Visual understanding |
| Auxiliary roles (verify/challenge/locate) | Cross-check, raise objections, pinpoint issues | Medium-length context | Medium reasoning ability |
Three points also deserve attention in role division:
- Overlapping responsibilities inflate cost: If a reviewer also takes on planning, it needs a stronger model, and call cost rises. Keeping boundaries clean lets lower-tier models take on more work.
- Auxiliary roles can be merged: Verification, challenge, and localization roles are similar in nature and can share the same tier, avoiding configuration fragmentation.
- Define clear success criteria for each role: For example, a retriever only needs to return correct file and symbol locations; an executor’s output must pass tests. Quantifiable success criteria form the basis for later evaluation and are also the trigger for downgrade strategies.
3. Model Capability Tiers: The Basis for Selection
3.1 Four Selection Dimensions
Setting aside specific models, you can evaluate whether a model fits a role along four dimensions:
- Context window: Can it hold the role’s maximum required input? A planner needs the full task description and history summary; an executor needs the target files and their dependency code.
- Reasoning depth: With multi-constraint, multi-branch problems, can it produce correct plans or judgments? This is the key distinction between “high-tier” and “low-tier.”
- Code ability: Correctness of generated code, debugging ability, and reliability in following structured output formats (such as tool-call protocols).
- Cost and latency: Look beyond per-token price to the “expected total cost of completing the same task,” that is, “cost per call × expected number of calls ÷ success rate.” A cheap model that fails easily is often more expensive because of repeated retries.
Two categories of “specialized capability” also matter:
- Long-context ability: Locating key information in very long input without losing track.
- Multimodal ability: Understanding non-text content such as images and charts.
3.2 Five Capability Tiers
Based on these dimensions, available models can be abstracted into five tiers. When implementing, pick a currently available model whose price and capability match each tier.
| Tier | Capability Profile | Typical Use |
|---|---|---|
| T1 flagship long-context | Very long context, strong complex planning and instruction following | Master planning, overall solution design |
| T2 balanced strong-reasoning | Balanced reasoning and code ability, medium per-call cost | Deep execution, solution design, auxiliary reasoning roles |
| T3 specialized analysis | Strong analytical judgment and logical rigor | Review/analysis, root-cause analysis |
| T4 lightweight code | Adequate code search and basic code generation, low cost, high throughput | Retrieval, lightweight subtasks |
| T5 fast low-cost | Fast responses, low price, limited complex reasoning | Quick-response tasks, fallback default |
3.3 Role-to-Tier Mapping
| Role | Preferred Tier | Alternative | Rationale | Tuning Tip |
|---|---|---|---|---|
| Master planner | T1 | T2 | Must read the full context and build a long-chain plan; long context and planning take priority | Summarize task history; keep only decision-relevant info |
| Deep executor | T2 | T1 | Code generation and debugging are the core need; balanced models offer better value | Limit the editable file scope; keep error logs and retry on failure |
| Solution designer | T1 | T2 | Same “design-intensive” nature as master planning; use the same model for consistency | Output structured design docs the executor can consume directly |
| Reviewer/analyst | T3 | T2 | Review and root-cause analysis hinge on accuracy; analysis models fit better | Define a review checklist; output issues graded by severity |
| Auxiliary roles (verify/challenge/locate) | T2 | T3 | Medium reasoning tasks, no flagship model needed | Run multiple auxiliary roles concurrently to cut chain latency |
| Code retriever | T4 | — | Retrieval is done by the search engine; the model only interprets and organizes results | Use hybrid retrieval to narrow candidates and send less code to the model |
| Lightweight subtask executor | T4 | T5 | Simple, clear tasks, cost-sensitive | Strictly validate output format; retry on invalid output |
| Multimodal interpreter | A vision-capable model | — | Screenshots and prototypes require multimodal understanding | Pass image and text together; require structured description output |
The mapping table should follow three principles:
- Vertical alignment: Similar design work (planning and solution design) should use the same tier to reduce style conflicts.
- Horizontal layering: Push execution, auxiliary, retrieval, and subtask work down to mid/low tiers; these steps have the highest call volume and are the main space for cost reduction.
- No capability downgrades: Multimodal roles cannot be swapped for text-only models, or they will fail outright.
4. Task Classification Routing: Match Difficulty to Tiers
Role mapping answers “who does the work”; task classification answers “how hard is the work.” Within the same role, differences in task difficulty are often larger than differences between roles. Editing one line of configuration and refactoring an entire module should clearly not run on the same tier.
Coding-agent tasks can be divided into five classes:
- Ultra-reasoning: multi-constraint proofs, complex algorithm design, architectural arguments with far-reaching consequences;
- Deep analysis: defect root-cause location, performance bottleneck analysis, code review;
- Creative writing: requirement docs, naming design, copy polishing, solution exposition;
- Quick response: single-file small edits, format adjustments, simple Q&A;
- Default low tier: routine tasks with no explicit difficulty signal.
4.1 Classification Principles
- Default low tier: When there is no clear complexity signal, route to a low-tier model and reserve high-tier models for genuinely hard tasks.
- Explicit upgrade: When users mark intent like “think deeply” or “full review,” or when a task touches a critical path (payment, auth, data migration), upgrade the tier directly.
- Result-driven implicit upgrade: When a low-tier model fails repeatedly (tests not passing, output-format validation failing), upgrade and retry automatically, and record the reason for later tuning.
- Cost control: Set a daily call cap for each task class; downgrade or queue as the cap approaches.
4.2 Classification Implementation
Classification does not need an extra large-model call. A hybrid approach of “rules first, model as fallback” avoids making classification itself a cost burden:
import re# Tier for each category; upgrade order is from low to highCATEGORY_TIER = {"ultra_reasoning": "T1","deep_analysis": "T3","creative_writing":"T1","quick_response": "T5","unspecified_low": "T5",}UPGRADE_PATH = ["T5", "T4", "T3", "T2", "T1"]# Rule layer: explicit intent and critical pathsEXPLICIT_SIGNALS = [(r"think deeply|full review|architecture review|algorithm proof", "ultra_reasoning"),(r"root cause|why.*(fail|crash)|performance bottleneck", "deep_analysis"),(r"write (a )?doc|polish|naming|explain", "creative_writing"),(r"change it|format|rename|add a line|simple", "quick_response"),]CRITICAL_PATHS = ("payment", "auth", "migration", "security")def classify(prompt: str, meta: dict) -> str:"""Run rules first; fall back to a lightweight model if no rule matches"""for pattern, category in EXPLICIT_SIGNALS:if re.search(pattern, prompt, re.IGNORECASE):return categoryif any(k in meta.get("changed_files", []) for k in CRITICAL_PATHS):return "ultra_reasoning"# Fallback: call a lightweight classifier that returns one of the five labelsreturn lightweight_classifier(prompt) # defaults to unspecified_low
4.3 Dynamic Upgrade at Runtime
def select_tier(category: str, attempt: int = 0) -> str:base = CATEGORY_TIER[category]# First attempt uses the category tier; on failure, step up the upgrade pathreturn base if attempt == 0 else \UPGRADE_PATH[min(UPGRADE_PATH.index(base) + attempt,len(UPGRADE_PATH) - 1)]# Example: quick_response starts at T5, then tries T4, T3 on failure...assert select_tier("quick_response", 0) == "T5"assert select_tier("quick_response", 1) == "T4"assert select_tier("ultra_reasoning", 2) == "T1" # already the top tier, no further upgrade
5. Configuration Templates and Runtime Strategy
5.1 Declarative Configuration
Role mapping and classification mapping should live in a single declarative config file, decoupled from business code for easy review, canary rollout, and rollback. The example below uses “model-a/b/c/d/e” to represent the concrete models bound to the five tiers; replace them with your chosen model service identifiers at deploy time. If you use a model-service platform such as Baidu AI Cloud Qianfan, you can pick the appropriate model by capability category from the platform’s model catalog and fill it into the config.
# Models bound to each tier (illustrative, replace with actually available models)tiers:T1: { model: "model-a", max_output_tokens: 8192 }T2: { model: "model-b", max_output_tokens: 8192 }T3: { model: "model-c", max_output_tokens: 4096 }T4: { model: "model-d", max_output_tokens: 2048 }T5: { model: "model-e", max_output_tokens: 1024 }# Role -> model tierroles:master_planner: T1 # master planningdesigner: T1 # solution designdeep_worker: T2 # deep executionreviewer: T3 # review/analysisverifier: T2 # verificationchallenger: T2 # challengelocator: T2 # localizationexplorer: T4 # code retrievalsubtask_worker: T4 # lightweight subtaskmultimodal_looker: { provider: "vision", model: "model-f" } # vision understanding# Task category -> tiercategories:ultra_reasoning: T1deep_analysis: T3creative_writing: T1quick_response: T5unspecified_low: T5# Runtime policypolicy:max_attempts: 3upgrade_on_failure: trueparallel_dispatch: true # schedule independent subtasks in parallelcontext_budget_tokens: 32768daily_budget_usd: 20on_budget_exceeded: downgrade # downgrade | queue | reject
5.2 Scheduling and Downgrade Flow
The processing flow of a typical call:
- Determine the initial tier from the role and category;
- Assemble context and apply budget trimming;
- Call the model and validate output structure and whether tests pass;
- On failure, upgrade the tier and retry until the max attempts is reached;
- Record usage and results throughout, for cost accounting and offline analysis.
from dataclasses import dataclass@dataclassclass CallResult:text: strtier: strattempts: intprompt_tokens: intcompletion_tokens: intdef run_task(role: str, category: str, messages: list,validate, cfg: dict, budget: "Budget") -> CallResult:tier, attempt = select_tier(category), 0while attempt < cfg["policy"]["max_attempts"]:tier = select_tier(category, attempt)spec = cfg["tiers"][tier]# Context budget trimming: keep system prompt + recent turns + task summarymessages = trim_context(messages, cfg["policy"]["context_budget_tokens"])resp = client.chat(model=spec["model"], messages=messages,max_tokens=spec["max_output_tokens"])budget.record(tier, resp.usage) # real-time accountingif validate(resp.text): # format / compile / unit-test validationreturn CallResult(resp.text, tier, attempt + 1, resp.usage)attempt += 1 # validation failed -> upgrade and retryraise RuntimeError(f"Task failed repeatedly on {role}/{category}")
6. Six Key Practices for a Budget-Sensitive Design
6.1 Context Engineering: Controlling Input Beats Switching Models
In most multi-agent chains, input tokens account for the bulk of consumption. You can compress input as follows:
- Roll a running summary over the history of completed stages;
- Send only code snippets directly related to the target file to the model;
- Do deterministic work like syntax parsing and symbol indexing locally instead of handing the whole codebase to the model to “read.”
6.2 Retrieval and Reuse: Reduce Repeated Computation
- Semantic cache: Reuse historical results for identical or highly similar requests. Fits high-frequency, read-only scenarios like code explanation and doc Q&A.
- Result reuse: Persist retriever and analyst outputs in structured form so later roles can reference them directly, avoiding duplicate calls.
- Hybrid retrieval: Keyword search handles precise location, vector search handles semantic recall; fuse both, then hand off to a lightweight model for organizing.
6.3 Parallel Dispatch: Cut Latency Without Adding Cost
In the task tree produced by the master planner, independent subtasks can be dispatched in parallel. Parallelism does not increase token consumption, yet it noticeably shortens overall response time, which matters especially for interactive coding.
6.4 Failure Handling: Combine Downgrade and Upgrade
- Transient errors (rate limits, timeouts): back off briefly and retry on the same tier;
- Output validation failures (format errors, tests failing): upgrade the tier and retry;
- Repeated failures: roll back to the last usable state, organize the error info, and hand it back to the master planner to re-plan.
Also set a timeout and max token count for each call to keep individual tasks from running away.
6.5 Cost Observability: Account by Role and Category
Each call should record the role, category, tier, token usage, latency, and validation result. Offline analysis can then answer three questions:
- Which roles consume the most?
- Which categories have the highest upgrade rate?
- In which scenarios are low-tier models already “good enough”?
Adjust the mapping table periodically based on this, forming a continuous-optimization loop. Common conclusions in practice:
- Subtasks like format conversion and boilerplate can be pushed down further to lower tiers;
- Planning and solution design should not be downgraded;
- Categories with persistently high upgrade rates should either adjust their default tier or optimize prompts.
6.6 Security and Compliance
- Inject secrets uniformly through a key-management service or environment variables; never write them into config files or code repositories;
- External code and docs may contain prompt-injection content, so treat them as “data” rather than “instructions”;
- Screenshots in multimodal input may contain sensitive information; desensitize them before upload;
- All model outputs should pass compile, static-analysis, and test validation before merging; never merge directly into the main branch.
7. Evaluating Selection Quality and Common Pitfalls
7.1 Validate the Mapping Table with a Small Evaluation Set
Build an evaluation set covering typical scenarios: planning tasks drawn from real requirements, defect-fix tasks with reference answers, and code-review samples. Metrics include task success rate, average number of calls, average number of upgrades, end-to-end latency, and cost per task.
When adjusting the mapping table, don’t just look at “did the model answer correctly,” but at “did the whole chain answer correctly at lower cost.” When swapping the model for a given tier, first regress on the evaluation set, then roll out gradually.
7.2 Five Common Pitfalls
- Parameter count only: A model with more parameters isn’t necessarily better at coding; domain-specific data and training methods matter just as much.
- Price only: Picking only the cheapest model increases retries and rework, raising total cost.
- Ignoring classification defaults: Defaulting to a high tier when classification confidence is low wastes budget, while defaulting to a low tier may hurt quality; adjust based on actual failure rates.
- Coupling config with code: Hardcoding the model mapping in code means every change requires a redeploy; keep configuration external.
- No fallback: Without downgrade and rollback plans, the whole chain breaks if a model service becomes unavailable.
8. Conclusion
At its core, a budget-sensitive multi-agent coding design treats “model capability” as a resource allocated on demand:
- Selection level: First divide roles, then match models by capability tier;
- Scheduling level: Use task classification for difficulty grading, with failure-driven upgrades and budget control as a safety net;
- Engineering level: Continuously lower cost per task through context compression, result reuse, parallel dispatch, and cost observability.
This method is not tied to any specific product. The mapping is maintained as configuration and can keep evolving as model services and business scenarios change. Once teams master this approach, they avoid both the waste of “everything at top tier” and the quality collapse caused by “over-compression,” gaining stable, controllable, continuously optimizable agentic coding capability within a limited budget.
