Multi-Model Routing in Practice: Reducing Cost Without Sacrificing Quality

    This article explains why different large language models can complete the same spreadsheet task at different costs. It presents a framework for measuring cost per successful delivery, defining quality acceptance criteria, and routing tasks across models based on complexity, failure impact, and latency requirements. It also covers how to verify that routing actually saves money and how to operate the system safely after launch.

    1. Same Task, Different Costs: Why Does Cost Still Vary?

    When large language models generate spreadsheets, a common pattern appears. Different models can all produce files that meet the requirements, yet the quota consumed, the processing time, and the final bill differ.

    This shows that “whether a task can be completed” and “how many resources it takes to complete the task” are two different questions.

    Suppose a task requires organizing raw data, calculating category subtotals, setting currency formats, and producing a spreadsheet that opens correctly. If several execution approaches pass acceptance, they have similar usability under this set of acceptance conditions. However, that does not mean their reasoning processes, output lengths, tool call counts, and retry probabilities are identical.

    For developers and enterprise users, the goal to optimize is:

    Lower the total cost of each successful delivery while meeting quality, security, and latency requirements.

    This observation comes from one task in which several results met the requirements. No complete task set, billing breakdown, or repeated test records are included. Therefore, no general savings percentage can be derived from it, and a single result should not be treated as a performance conclusion for all scenarios. A more useful approach is to turn this observation into a verifiable, continuously improvable engineering method.

    2. Define Cost First: Quota, Fees, and Resource Consumption Are Different

    Many teams evaluate models by looking only at how much quota a single call deducts. However, quota is often an internal metering unit on the platform. Its relationship to actual fees depends on the specific billing rules and should be verified.

    Different operating modes may also include different service overheads. Therefore, a reduction in quota does not necessarily mean cash spending drops by the same proportion. It also does not necessarily mean that computing resource consumption drops proportionally.

    A more complete cost framework can be written as:

    Total cost = model inference cost + tool execution cost + storage and transfer cost + retry cost + manual review cost

    Each component can affect the final conclusion.

    1. Model Inference Cost

    This cost usually depends on input size, output size, the number of calls, and the service’s pricing model.

    If an execution path repeatedly sends the full conversation history, or generates a large amount of unnecessary explanatory text, its cost can rise significantly. In contrast, a clear input structure and constrained output length usually help reduce wasted consumption.

    However, shortening output must not sacrifice necessary information. For tasks that require a complete audit trail, omitting calculation notes or data sources may make the result impossible to accept.

    2. Tool and Execution Environment Cost

    Spreadsheet tasks may involve reading files, transforming data, computing formulas, applying formats, and writing files. The model is only one step in this process.

    If an approach starts the execution environment many times, repeatedly reads large files, or splits operations that could be batched into many small steps, the end-to-end cost may not be lower, even if the model call price is low.

    3. Failure and Manual Handling Cost

    A low-cost model that often misses fields, generates incorrect formulas, or outputs unusable files under boundary conditions will require retries and manual corrections.

    For this reason, use the following metric instead of comparing only the average price per request:

    Cost per successful delivery = total execution cost during the evaluation period ÷ number of deliveries that pass acceptance

    Total execution cost should include failed requests, retries, and escalations. Manual review costs should also be included wherever possible. This prevents failed attempts from being excluded from the statistics.

    3. Quality Acceptance Must Come Before Model Selection

    Before asking which model is more cost-effective, first define what counts as completing the task.

    If the acceptance standard is only “the file was generated,” a model may pass surface checks while leaving serious problems at the business level. For spreadsheet generation, at least four layers of acceptance are recommended.

    Layer 1: File Usability

    Check whether the file exists, can be opened, contains all required worksheets, and is free of corruption or abnormal content.

    This layer confirms that the deliverable is basically usable. It cannot prove that the business results are correct.

    Layer 2: Data Integrity

    Check record counts, field sets, field types, and key identifiers against the requirements.

    For example, if the input contains duplicate records, the business rules must decide whether to keep, merge, or delete them. The model should not guess. Missing values should also be clearly classified as empty, zero, unknown, or not applicable.

    Layer 3: Business Correctness

    Check whether category summaries, amount calculations, date ranges, and business rules are accurate.

    For calculations that a deterministic program can perform, use the program to recalculate and compare the results with the output. Do not rely on another model to judge from text alone whether the result “looks correct.”

    For amounts, agree in advance on precision, rounding rules, and currency. This avoids misjudgments caused by inconsistent calculation conventions.

    Layer 4: Presentation and Maintainability

    Check column widths, headers, number formats, filter ranges, and formula references against the requirements.

    If users need to modify the original data later, fixed values instead of formulas may not meet the delivery goal. If users only need an immutable archived result, however, fixed values may make cross-environment verification easier.

    The clearer the acceptance criteria, the easier it becomes to turn model selection from a subjective experience into an engineering decision.

    4. The Core of Multi-Model Routing: Assign Capability by Task Risk

    Multi-model routing does not mean picking models at random, and it does not always mean choosing the cheapest model. It means selecting an execution path based on task characteristics and constraints.

    A practical routing system can assess tasks along three dimensions.

    1. Task Complexity

    Complexity depends mainly on the number of rules, reasoning depth, data scale, and degree of ambiguity.

    A format conversion with clear fields and fixed rules usually does not need the same capability as open-ended analysis. Tasks that involve linking across worksheets, calculations with exception conditions, or business rules written in natural language require more careful handling.

    Long input does not necessarily mean a task is complex, and short input does not mean it is simple. Implicit constraints in a few sentences can sometimes be harder to handle than a large set of structured records.

    2. Impact of Failure

    An internal draft and a formal financial document may have similar structures, but they carry different levels of risk.

    For low-risk tasks, automatic retries or escalation may be acceptable. For tas