Tiered LLM Selection and Implementation Guide for Complex Engineering Scenarios

    Development teams often use a single highly-rated LLM for all tasks, which wastes costs on simple tasks and risks rework on complex ones due to insufficient capabilities. This article introduces a tiered approach to LLM selection based on task complexity, fault tolerance, and cost constraints. It covers how to categorize tasks, the characteristics of each model tier, smart routing rules, cost calculation, safety checklists for high-risk refactoring, and best practices for CI/CD integration.

    When selecting models, R&D teams often pick a single highly-rated model for all use cases. This approach wastes costs on simple tasks and may cause rework on complex tasks due to insufficient capability. A more reliable approach is to categorize tasks by complexity, fault tolerance, and cost constraints, assigning a specific tier of model to each category, and using unified rules to determine which tier handles which task.

    The following content answers four questions: How to select LLMs, how to calculate costs, what common mistakes exist, and what to note during implementation.

    1. Determine the Task Tier First

    Selection starts with the task itself. The table below outlines a typical three-tier division, which teams can adjust based on their own business thresholds.

    Task Type Context Requirements Fault Tolerance Cost Constraints Suggested Tier Primary Verification Method
    Cross-module refactoring, underlying protocol migration Requires reading multiple modules and their dependencies Extremely low; errors affect production Accepts higher unit prices Flagship Tier Compilation, full regression testing, dual-person review
    Cryptographic algorithm implementation, mathematical derivation Medium; mainly relies on preconditions Extremely low Accepts higher unit prices Flagship Tier Test vectors, independent derivation review
    CRUD, business logic, technical documentation Single module or file Medium; reviews usually catch issues Medium Main Tier Unit tests, code review
    Data cleaning, crawler scripts, prototype experiments Small; single inputs are independent High; allows multiple retries Sensitive to per-call cost Low-cost Tier Sampling inspection, automated validation

    2. What Each Model Tier is Best For

    2.1 Flagship Tier: Complex Refactoring and Zero-Fault-Tolerance Tasks

    Flagship models can handle longer contexts and read multiple related files in a single task, map out call relationships, and provide a modification plan. When using them, you can ask the model to first output the scope of impact and the modification plan, then generate code file by file for easier review.

    The dependency analysis and interface judgments provided by the model are still hypotheses that need verification. They cannot replace compilers, test suites, and static analysis. Outputting large chunks of code at once also increases review difficulty. Therefore, it is recommended to deliver in split modules so each segment can be compiled and tested independently.

    2.2 Main Tier: Daily Development and Technical Documentation

    The main tier is suitable for well-structured tasks with limited impact, such as CRUD APIs, standard business logic, boilerplate code, and initial drafts of technical solutions. It responds quickly and serves well as a pair programming assistant. When integrated into CI pipelines, generated code should pass static analysis and unit tests before entering code review, and must not be merged directly.

    2.3 Low-cost Tier: Batch Scripts and Prototype Experiments

    The low-cost tier processes large batches of independently verifiable tasks, such as web data extraction, log cleaning, and rapid prototyping. In these tasks, per-call accuracy is less important than overall throughput and unit cost. By executing concurrently with local scripts and container platforms, and using automated validation to filter out incorrect results, total costs can be reduced. The prerequisite is that validation rules are strict enough; otherwise, erroneous data will silently flow downstream.

    3. Smart Routing Rules

    The role of the routing layer is to decide which tier of model to use when a request comes in. The rules need to be explainable and adjustable.

    The first step is to define complexity labels. It is recommended to include at least four dimensions:

    1. Impact scope: Number of files and modules involved, whether public APIs are modified.
    2. Risk level: Whether it touches payment, permissions, data migration, or encryption logic.
    3. Verifiability: Whether existing test cases or runnable validation scripts are available.
    4. Batch attribute: Whether it belongs to a homogeneous large-batch task.

    The second step is to set routing thresholds. Below is a starting example, which should be calibrated based on the team’s historical task data:

    • Touches public APIs or high-risk modules, or affects more than 5 modules: Flagship Tier.
    • Standard development and documentation within a single module: Main Tier.
    • Batch, homogeneous, automatically verifiable tasks: Low-cost Tier.

    The third step is to define fallback rules for failures. If the same task fails twice consecutively in the current tier, it is escalated to the tier above. If it fails three times consecutively in the low-cost tier, it is transferred to manual processing. If code generated by the flagship tier fails tests, it is not allowed to be merged.

    The fourth step is logging. Each call should at least log the model name and version, assigned tier, input/output token usage, test results, and final reviewer. Without logs, it is impossible to determine if the routing rules are effective.

    Finally, periodically sample and review routing results. If the first-time pass rate of a certain tier is persistently low, adjust the thresholds or replace the model.

    4. How to Calculate Costs

    Model pricing is subject to the provider’s current price list, which may change, and optional features have different billing methods. Before budgeting, the team should verify the model name, version, input and output unit prices, as well as the trigger conditions and surcharge rules for options like fast mode.

    The cost of a single call can be estimated as follows:

    Single Call Cost = Input Token Count × Input Unit Price + Output Token Count × Output Unit Price

    Actual expenditure must also account for rework. The expected cost of a task is:

    Expected Cost = Single Call Cost ÷ First-time Pass Rate

    This formula shows that a model with a lower unit price is not necessarily cheaper. If the first-time pass rate of a low-price model is only half that of a flagship model, and each rework requires manual review, the total cost may actually be higher. Therefore, when selecting models, compare the total cost per passed task and factor in manual review time.

    5. Safety Checklist for High-Risk Refactoring

    The risks of cross-module refactoring and protocol migration mainly come from the refactoring itself. Before the model generates code, the team should complete the following preparations:

    • Perform refactoring in an isolated branch or workspace; do not directly modify the main branch.
    • Back up code and related data before refactoring. There must be a recoverable snapshot before data migration.
    • Use gradual rollout (canary release), validating first in a small percentage of traffic or a test environment.
    • Prepare a rollback plan, clearly defining rollback steps, responsible persons, and trigger conditions.
    • Run automated tests and supplement regression test cases for the modified points.
    • Have an engineer familiar with the module conduct a manual review; critical paths require dual-person confirmation.

    6. Verification Steps for Zero-Fault-Tolerance Tasks

    For cryptographic algorithms and mathematical derivation tasks, the model’s conclusions must be independently verified. It is recommended to execute in the following order:

    1. Prepare test vectors covering normal inputs, boundary values, and abnormal inputs.
    2. Check formalizable properties, such as input/output ranges, invariants, and boundary conditions.
    3. Have another engineer derive independently, then compare with the model’s conclusion. Do not look at the conclusion first and then reverse-engineer the process.
    4. Conduct comparative testing between the new implementation and an existing trusted implementation.
    5. Include the results in regression test cases. If it fails, restore according to the rollback plan.

    7. Notes on Integrating with CI/CD

    • Keys and Permissions: Store API keys in a key management service; do not write them into code or logs. Grant only minimum permissions to model accounts, allowing access only to necessary repositories and environments.
    • Code and Data Protection: Verify the provider’s data retention and training usage policies. Ensure compliance with data protection policies for sensitive information.