How to Choose LLM Models by Tier for Multi-Agent Coding Assistants: A Budget-Sensitive Selection Guide

    Choosing the right model for each role in a multi-agent coding system is what keeps token cost, latency, and retries under control. This guide explains how to divide a coding agent into single-responsibility roles, map those roles to five abstract model tiers, route tasks by difficulty, and manage everything through declarative configuration and runtime policies. You will learn context-compression, retrieval-reuse, parallel-dispatch, and failure-handling practices, plus a small evaluation-set approach and five common pitfalls to avoid. The goal is a stable, cost-efficient agent setup that avoids both "all-top-tier" waste and "over-compression" quality collapse.

    1. Introduction: Why Tiered Selection Matters

    AI-assisted coding has evolved from “single-turn completion” into coding agents capable of “autonomous planning plus multi-step execution.” Under complex requirements, a single agent often has to handle requirement decomposition, code retrieval, solution design, implementation, test-and-fix, and result review all at once. Because of context-window and attention-quality limits, a single model struggles to stay consistent across every one of these stages.

    The common industry response is multi-agent collaboration: a coordinator agent that orchestrates, executor sub-agents that do deep work, and reviewer agents that check quality. Dividing labor lowers each agent’s cognitive load, but it also creates new problems:

    • The number of calls multiplies;
    • Context keeps accumulating as tasks pass between roles;
    • A single failure can trigger retries or even task rollback.

    In such a chain, if every role runs the strongest model, token consumption, response latency, and retry costs spiral out of control quickly. Conversely, if everything runs low-tier models, planning and review quality drop, rework increases, and total cost actually rises. So the core of a “budget-sensitive” multi-agent design is not simply pushing unit price down, but selecting models by tier according to the value of each role, so that expensive capability only appears at critical nodes.

    This article starts with a cost-estimation example used throughout. Suppose a complex coding task needs 60 rounds of model calls, averaging 4,000 input-plus-output tokens per round, for a total of 240,000 tokens. If different roles use models at different price points, and you add retrieval reuse, context trimming, and downgrade strategies, the call count can be kept under 30 rounds. Roughly estimated, overall cost can drop to around one-tenth of the “all-premium-model” approach.

    Note: All numbers here are illustrative values for explanation. Actual results depend on the specific models, business scenario, and implementation quality.

    2. Role Division and Responsibility Boundaries in Multi-Agent Collaboration

    Before configuring models, you should first identify which roles exist in the system and define the input/output boundary of each. Role division follows the “single responsibility” principle: each role handles one type of similar work, takes the leanest possible input, and produces structured output where feasible (for example, JSON).

    Role Typical Responsibilities Input Characteristics Core Model Requirements
    Master planner Understand requirements, decompose tasks, decide scheduling order, aggregate results Full requirement + task history, longest context Long-context understanding, complex planning, instruction following
    Deep executor Code implementation, refactoring, bug fixing, running tests Large code blocks + error logs Code generation and debugging, tool-call reliability
    Solution designer Design module structure, define interfaces and data structures Requirements + summary of existing architecture Architectural reasoning, trade-off analysis
    Reviewer/analyst Review code diffs, analyze root causes, suggest improvements Changed snippets + relevant context Analytical accuracy, attention to detail
    Lightweight subtask executor Format conversion, boilerplate, simple edits Small, well-defined instructions Basic competence, low cost, low latency
    Code retriever Search symbols and definitions, locate related files Keywords + retrieval results Retrieval capability, no heavy reasoning needed
    Multimodal interpreter Read screenshots, prototypes, architecture diagrams Image + a little text Visual understanding
    Auxiliary roles (verify/challenge/locate) Cross-check, raise objections, pinpoint issues Medium-length context Medium reasoning ability

    Three points also deserve attention in role division:

    1. Overlapping responsibilities inflate cost: If a reviewer also takes on planning, it needs a stronger model, and call cost rises. Keeping boundaries clean lets lower-tier models take on more work.
    2. Auxiliary roles can be merged: Verification, challenge, and localization roles are similar in nature and can share the same tier, avoiding configuration fragmentation.
    3. Define clear success criteria for each role: For example, a retriever only needs to return correct file and symbol locations; an executor’s output must pass tests. Quantifiable success criteria form the basis for later evaluation and are also the trigger for downgrade strategies.

    3. Model Capability Tiers: The Basis for Selection

    3.1 Four Selection Dimensions

    Setting aside specific models, you can evaluate whether a model fits a role along four dimensions:

    • Context window: Can it hold the role’s maximum required input? A planner needs the full task description and history summary; an executor needs the target files and their dependency code.
    • Reasoning depth: With multi-constraint, multi-branch problems, can it produce correct plans or judgments? This is the key distinction between “high-tier” and “low-tier.”
    • Code ability: Correctness of generated code, debugging ability, and reliability in following structured output formats (such as tool-call protocols).
    • Cost and latency: Look beyond per-token price to the “expected total cost of completing the same task,” that is, “cost per call × expected number of calls ÷ success rate.” A cheap model that fails easily is often more expensive because of repeated retries.

    Two categories of “specialized capability” also matter:

    • Long-context ability: Locating key information in very long input without losing track.
    • Multimodal ability: Understanding non-text content such as images and charts.

    3.2 Five Capability Tiers

    Based on these dimensions, available models can be abstracted into five tiers. When implementing, pick a currently available model whose price and capability match each tier.

    Tier Capability Profile Typical Use
    T1 flagship long-context Very long context, strong complex planning and instruction following Master planning, overall solution design
    T2 balanced strong-reasoning Balanced reasoning and code ability, medium per-call cost Deep execution, solution design, auxiliary reasoning roles
    T3 specialized analysis Strong analytical judgment and logical rigor Review/analysis, root-cause analysis
    T4 lightweight code Adequate code search and basic code generation, low cost, high throughput Retrieval, lightweight subtasks
    T5 fast low-cost Fast responses, low price, limited complex reasoning Quick-response tasks, fallback default

    3.3 Role-to-Tier Mapping

    Role Preferred Tier Alternative Rationale Tuning Tip
    Master planner T1 T2 Must read the full context and build a long-chain plan; long context and planning take priority Summarize task history; keep only decision-relevant info
    Deep executor T2 T1 Code generation and debugging are the core need; balanced models offer better value Limit the editable file scope; keep error logs and retry on failure
    Solution designer T1 T2 Same “design-intensive” nature as master planning; use the same model for consistency Output structured design docs the executor can consume directly
    Reviewer/analyst T3 T2 Review and root-cause analysis hinge on accuracy; analysis models fit better Define a review checklist; output issues graded by severity
    Auxiliary roles (verify/challenge/locate) T2 T3 Medium reasoning tasks, no flagship model needed Run multiple auxiliary roles concurrently to cut chain latency
    Code retriever T4 — Retrieval is done by the search engine; the model only interprets and organizes results Use hybrid retrieval to narrow candidates and send less code to the model
    Lightweight subtask executor T4 T5 Simple, clear tasks, cost-sensitive Strictly validate output format; retry on invalid output
    Multimodal interpreter A vision-capable model — Screenshots and prototypes require multimodal understanding Pass image and text together; require structured description output

    The mapping table should follow three principles:

    • Vertical alignment: Similar design work (planning and solution design) should use the same tier to reduce style conflicts.
    • Horizontal layering: Push execution, auxiliary, retrieval, and subtask work down to mid/low tiers; these steps have the highest call volume and are the main space for cost reduction.
    • No capability downgrades: Multimodal roles cannot be swapped for text-only models, or they will fail outright.

    4. Task Classification Routing: Match Difficulty to Tiers

    Role mapping answers “who does the work”; task classification answers “how hard is the work.” Within the same role, differences in task difficulty are often larger than differences between roles. Editing one line of configuration and refactoring an entire module should clearly not run on the same tier.

    Coding-agent tasks can be divided into five classes:

    • Ultra-reasoning: multi-constraint proofs, complex algorithm design, architectural arguments with far-reaching consequences;
    • Deep analysis: defect root-cause location, performance bottleneck analysis, code review;
    • Creative writing: requirement docs, naming design, copy polishing, solution exposition;
    • Quick response: single-file small edits, format adjustments, simple Q&A;
    • Default low tier: routine tasks with no explicit difficulty signal.

    4.1 Classification Principles

    • Default low tier: When there is no clear complexity signal, route to a low-tier model and reserve high-tier models for genuinely hard tasks.
    • Explicit upgrade: When users mark intent like “think deeply” or “full review,” or when a task touches a critical path (payment, auth, data migration), upgrade the tier directly.
    • Result-driven implicit upgrade: When a low-tier model fails repeatedly (tests not passing, output-format validation failing), upgrade and retry automatically, and record the reason for later tuning.
    • Cost control: Set a daily call cap for each task class; downgrade or queue as the cap approaches.

    4.2 Classification Implementation

    Classification does not need an extra large-model call. A hybrid approach of “rules first, model as fallback” avoids making classification itself a cost burden:

    1. import re
    2. # Tier for each category; upgrade order is from low to high
    3. CATEGORY_TIER = {
    4. "ultra_reasoning": "T1",
    5. "deep_analysis": "T3",
    6. "creative_writing":"T1",
    7. "quick_response": "T5",
    8. "unspecified_low": "T5",
    9. }
    10. UPGRADE_PATH = ["T5", "T4", "T3", "T2", "T1"]
    11. # Rule layer: explicit intent and critical paths
    12. EXPLICIT_SIGNALS = [
    13. (r"think deeply|full review|architecture review|algorithm proof", "ultra_reasoning"),
    14. (r"root cause|why.*(fail|crash)|performance bottleneck", "deep_analysis"),
    15. (r"write (a )?doc|polish|naming|explain", "creative_writing"),
    16. (r"change it|format|rename|add a line|simple", "quick_response"),
    17. ]
    18. CRITICAL_PATHS = ("payment", "auth", "migration", "security")
    19. def classify(prompt: str, meta: dict) -> str:
    20. """Run rules first; fall back to a lightweight model if no rule matches"""
    21. for pattern, category in EXPLICIT_SIGNALS:
    22. if re.search(pattern, prompt, re.IGNORECASE):
    23. return category
    24. if any(k in meta.get("changed_files", []) for k in CRITICAL_PATHS):
    25. return "ultra_reasoning"
    26. # Fallback: call a lightweight classifier that returns one of the five labels
    27. return lightweight_classifier(prompt) # defaults to unspecified_low

    4.3 Dynamic Upgrade at Runtime

    1. def select_tier(category: str, attempt: int = 0) -> str:
    2. base = CATEGORY_TIER[category]
    3. # First attempt uses the category tier; on failure, step up the upgrade path
    4. return base if attempt == 0 else \
    5. UPGRADE_PATH[min(UPGRADE_PATH.index(base) + attempt,
    6. len(UPGRADE_PATH) - 1)]
    7. # Example: quick_response starts at T5, then tries T4, T3 on failure...
    8. assert select_tier("quick_response", 0) == "T5"
    9. assert select_tier("quick_response", 1) == "T4"
    10. assert select_tier("ultra_reasoning", 2) == "T1" # already the top tier, no further upgrade

    5. Configuration Templates and Runtime Strategy

    5.1 Declarative Configuration

    Role mapping and classification mapping should live in a single declarative config file, decoupled from business code for easy review, canary rollout, and rollback. The example below uses “model-a/b/c/d/e” to represent the concrete models bound to the five tiers; replace them with your chosen model service identifiers at deploy time. If you use a model-service platform such as Baidu AI Cloud Qianfan, you can pick the appropriate model by capability category from the platform’s model catalog and fill it into the config.

    1. # Models bound to each tier (illustrative, replace with actually available models)
    2. tiers:
    3. T1: { model: "model-a", max_output_tokens: 8192 }
    4. T2: { model: "model-b", max_output_tokens: 8192 }
    5. T3: { model: "model-c", max_output_tokens: 4096 }
    6. T4: { model: "model-d", max_output_tokens: 2048 }
    7. T5: { model: "model-e", max_output_tokens: 1024 }
    8. # Role -> model tier
    9. roles:
    10. master_planner: T1 # master planning
    11. designer: T1 # solution design
    12. deep_worker: T2 # deep execution
    13. reviewer: T3 # review/analysis
    14. verifier: T2 # verification
    15. challenger: T2 # challenge
    16. locator: T2 # localization
    17. explorer: T4 # code retrieval
    18. subtask_worker: T4 # lightweight subtask
    19. multimodal_looker: { provider: "vision", model: "model-f" } # vision understanding
    20. # Task category -> tier
    21. categories:
    22. ultra_reasoning: T1
    23. deep_analysis: T3
    24. creative_writing: T1
    25. quick_response: T5
    26. unspecified_low: T5
    27. # Runtime policy
    28. policy:
    29. max_attempts: 3
    30. upgrade_on_failure: true
    31. parallel_dispatch: true # schedule independent subtasks in parallel
    32. context_budget_tokens: 32768
    33. daily_budget_usd: 20
    34. on_budget_exceeded: downgrade # downgrade | queue | reject

    5.2 Scheduling and Downgrade Flow

    The processing flow of a typical call:

    1. Determine the initial tier from the role and category;
    2. Assemble context and apply budget trimming;
    3. Call the model and validate output structure and whether tests pass;
    4. On failure, upgrade the tier and retry until the max attempts is reached;
    5. Record usage and results throughout, for cost accounting and offline analysis.
    1. from dataclasses import dataclass
    2. @dataclass
    3. class CallResult:
    4. text: str
    5. tier: str
    6. attempts: int
    7. prompt_tokens: int
    8. completion_tokens: int
    9. def run_task(role: str, category: str, messages: list,
    10. validate, cfg: dict, budget: "Budget") -> CallResult:
    11. tier, attempt = select_tier(category), 0
    12. while attempt < cfg["policy"]["max_attempts"]:
    13. tier = select_tier(category, attempt)
    14. spec = cfg["tiers"][tier]
    15. # Context budget trimming: keep system prompt + recent turns + task summary
    16. messages = trim_context(messages, cfg["policy"]["context_budget_tokens"])
    17. resp = client.chat(model=spec["model"], messages=messages,
    18. max_tokens=spec["max_output_tokens"])
    19. budget.record(tier, resp.usage) # real-time accounting
    20. if validate(resp.text): # format / compile / unit-test validation
    21. return CallResult(resp.text, tier, attempt + 1, resp.usage)
    22. attempt += 1 # validation failed -> upgrade and retry
    23. raise RuntimeError(f"Task failed repeatedly on {role}/{category}")

    6. Six Key Practices for a Budget-Sensitive Design

    6.1 Context Engineering: Controlling Input Beats Switching Models

    In most multi-agent chains, input tokens account for the bulk of consumption. You can compress input as follows:

    • Roll a running summary over the history of completed stages;
    • Send only code snippets directly related to the target file to the model;
    • Do deterministic work like syntax parsing and symbol indexing locally instead of handing the whole codebase to the model to “read.”

    6.2 Retrieval and Reuse: Reduce Repeated Computation

    • Semantic cache: Reuse historical results for identical or highly similar requests. Fits high-frequency, read-only scenarios like code explanation and doc Q&A.
    • Result reuse: Persist retriever and analyst outputs in structured form so later roles can reference them directly, avoiding duplicate calls.
    • Hybrid retrieval: Keyword search handles precise location, vector search handles semantic recall; fuse both, then hand off to a lightweight model for organizing.

    6.3 Parallel Dispatch: Cut Latency Without Adding Cost

    In the task tree produced by the master planner, independent subtasks can be dispatched in parallel. Parallelism does not increase token consumption, yet it noticeably shortens overall response time, which matters especially for interactive coding.

    6.4 Failure Handling: Combine Downgrade and Upgrade

    • Transient errors (rate limits, timeouts): back off briefly and retry on the same tier;
    • Output validation failures (format errors, tests failing): upgrade the tier and retry;
    • Repeated failures: roll back to the last usable state, organize the error info, and hand it back to the master planner to re-plan.

    Also set a timeout and max token count for each call to keep individual tasks from running away.

    6.5 Cost Observability: Account by Role and Category

    Each call should record the role, category, tier, token usage, latency, and validation result. Offline analysis can then answer three questions:

    • Which roles consume the most?
    • Which categories have the highest upgrade rate?
    • In which scenarios are low-tier models already “good enough”?

    Adjust the mapping table periodically based on this, forming a continuous-optimization loop. Common conclusions in practice:

    • Subtasks like format conversion and boilerplate can be pushed down further to lower tiers;
    • Planning and solution design should not be downgraded;
    • Categories with persistently high upgrade rates should either adjust their default tier or optimize prompts.

    6.6 Security and Compliance

    • Inject secrets uniformly through a key-management service or environment variables; never write them into config files or code repositories;
    • External code and docs may contain prompt-injection content, so treat them as “data” rather than “instructions”;
    • Screenshots in multimodal input may contain sensitive information; desensitize them before upload;
    • All model outputs should pass compile, static-analysis, and test validation before merging; never merge directly into the main branch.

    7. Evaluating Selection Quality and Common Pitfalls

    7.1 Validate the Mapping Table with a Small Evaluation Set

    Build an evaluation set covering typical scenarios: planning tasks drawn from real requirements, defect-fix tasks with reference answers, and code-review samples. Metrics include task success rate, average number of calls, average number of upgrades, end-to-end latency, and cost per task.

    When adjusting the mapping table, don’t just look at “did the model answer correctly,” but at “did the whole chain answer correctly at lower cost.” When swapping the model for a given tier, first regress on the evaluation set, then roll out gradually.

    7.2 Five Common Pitfalls

    1. Parameter count only: A model with more parameters isn’t necessarily better at coding; domain-specific data and training methods matter just as much.
    2. Price only: Picking only the cheapest model increases retries and rework, raising total cost.
    3. Ignoring classification defaults: Defaulting to a high tier when classification confidence is low wastes budget, while defaulting to a low tier may hurt quality; adjust based on actual failure rates.
    4. Coupling config with code: Hardcoding the model mapping in code means every change requires a redeploy; keep configuration external.
    5. No fallback: Without downgrade and rollback plans, the whole chain breaks if a model service becomes unavailable.

    8. Conclusion

    At its core, a budget-sensitive multi-agent coding design treats “model capability” as a resource allocated on demand:

    • Selection level: First divide roles, then match models by capability tier;
    • Scheduling level: Use task classification for difficulty grading, with failure-driven upgrades and budget control as a safety net;
    • Engineering level: Continuously lower cost per task through context compression, result reuse, parallel dispatch, and cost observability.

    This method is not tied to any specific product. The mapping is maintained as configuration and can keep evolving as model services and business scenarios change. Once teams master this approach, they avoid both the waste of “everything at top tier” and the quality collapse caused by “over-compression,” gaining stable, controllable, continuously optimizable agentic coding capability within a limited budget.