Multi-Model Routing Guide: How to Choose Models, Estimate Cost, and Control Risk
A practical guide to grading LLM tasks, routing them to the right model, and estimating total cost from multipliers, unit prices, and first-pass quality—without treating all models as interchangeable.
1. Grade the task first, then pick a model
The first step in multi-model routing is deciding how capable a model the task actually needs. Too little capability causes rework; too much wastes quota. Use the table below to assign tasks to three levels.
| Level | Typical tasks | Quality bar |
|---|---|---|
| L1 Light | Format conversion, rewriting, short summaries, field extraction | Valid format; no obvious content errors |
| L2 Standard | Briefings, multi-paragraph summaries, single-function code | Key facts consistent; no fabrication |
| L3 High capability | Long-document analysis, annual-report number checks, cross-module code, multi-step planning | All veto criteria pass |
Task level is determined by five dimensions together. If task risk is high, the task goes to L3; other dimensions cannot downgrade it.
| Dimension | Question | Bias toward lower level | Bias toward higher level |
|---|---|---|---|
| Task risk | Would errors cause financial, legal, or customer harm? | Internal drafts | External disclosure, contracts, financial data |
| Input length | Share of the chosen model’s context limit | Under 30% | Over 60%; split the input or use a long-context model |
| Output format | Strict structured output required? | Free text | Must pass JSON Schema validation |
| Latency | Immediate response required? | Batch is acceptable | Real-time interaction |
| Quality bar | Lowest acceptable accuracy | Quick human skim is enough | Item-by-item verification required |
The 30% and 60% figures are starting points; calibrate them on your own test set.
A single entry point does not make models equivalent. Before use, verify model version, context limit, max output length, and billing (input/output tokens, per call, or quota deduction). The same prompt can behave differently across versions; rerun the test set after every version change.
2. Cost: multiplier, unit price, and pass rate must be counted together
Looking only at the multiplier leads to the wrong conclusion. A cheap model that fails often and needs retries can cost more. Use:
Platform quota cost = base quota per call × model multiplier × average submissionsBase quota per call = input size × input unit price + output size × output unit priceAverage submissions ≈ 1 ÷ first-pass pass rateHuman cost = revision minutes × cost per minuteTotal task cost = platform quota cost + human cost (track separately; do not fold human cost into the multiplier)
Three notes on the formula:
First, input and output unit prices are the platform’s standard prices at multiplier 1. Apply the multiplier once at the end; do not bake it into unit prices.
Second, human revision time is not part of the multiplier. Book it as human cost, separate from platform quota.
Third, average submissions assumes independent attempts with a constant pass probability. That is an approximation; if logs disagree, use the logs.
The numbers below are illustrative only and do not represent any real prices or model quality. Assume base quota per call = 10, light model multiplier 0.5 with 50% first-pass rate; high-capability model multiplier 1.5 with 90% first-pass rate.
- Light model: average submissions = 2; platform cost = 10 × 0.5 × 2 = 10.
- High-capability model: average submissions ≈ 1.11; platform cost = 10 × 1.5 × 1.11 ≈ 16.7.
If the light model’s first-pass rate falls to 30%, average submissions ≈ 3.33 and platform cost ≈ 16.7—matching the high-capability model. Pass rate is what actually ranks cost.
Collect these metrics:
- First-pass pass rate: tasks whose first output passes all checks, divided by total tasks. Count only the first submission per task.
- Average submissions: mean number of submissions until checks pass, from call logs.
- Format compliance rate: share of outputs that pass JSON Schema or table-field checks, measured automatically.
- Human revision time: minutes to make the output usable, recorded by reviewers.
- Long-document stability: error rate on key numbers after the same material is processed in chunks.
- Fact-check extra cost: items requiring human verification × time per item.
Quota usage comes from the platform bill; latency from logs. Put all metrics in one table so comparisons are grounded.
Sample size matters. Pass rate is a binomial proportion. With n = 50 and a 50% rate, a 95% confidence interval is about ±14 percentage points—direction only. For formal comparison, use at least 200 samples per task type and resample after model or prompt changes.
Discounted quota windows can be used for non-urgent work (batch summaries, meeting notes, table conversion). Do not delay real-time decisions, customer communication, or production incidents to save quota. Tasks with unredacted personal data must use authorized endpoints only; do not move them to unevaluated services to save quota.
3. Scoring and validation: shared rules and a test-set template
Each check is pass or fail only.
