Tiered Model Routing: How Multi-Model Strategies Reduce LLM Token Costs
This article explains how to reduce LLM token costs under fixed credit-based or plan-based subscriptions by estimating each model’s input and output consumption rates from usage logs, comparing model tiers, and designing a four-layer routing strategy. It also covers throughput estimation, caching, quota monitoring, output-length control, and fallback planning so that expensive models are reserved for high-value work while cheaper models handle high-volume execution.
1. Why Token Consumption Rates Matter
As large language models become part of everyday development workflows—code generation, task planning, bulk refactoring, log analysis, and more—many developers use credit-based or plan-based subscriptions offered by cloud providers. These plans provide a fixed credit allowance per month or per week, and model calls consume credits based on the number of input and output tokens.
In this model, a seemingly minor issue can quickly become significant:
The credit consumption rate per token can differ by more than two orders of magnitude across model tiers.
If every task uses the same flagship model without differentiation, a fixed quota may be exhausted within hours. If tasks are layered appropriately and expensive models are used only where their cost is justified, the same quota can support days or even weeks of intensive use. This article discusses that cost-optimization methodology.
2. Inferring Consumption Rates from Usage Logs
Subscription plans usually do not publish the exact conversion ratio for every model, but that does not mean allocation has to rely on guesswork. One practical approach is to fit a model using your own usage records:
- Collect samples: Record a number of calls. For each call, capture the model used, input token count, output token count, and the credits actually deducted. Some platforms provide these fields in session details or log services.
- Build a linear model: Assume credit consumption follows
C ≈ a × input tokens + b × output tokens, whereaandbare the input and output consumption rates to be estimated. - Fit with least squares: Use Excel’s
LINESTfunction or any linear regression implementation in Python. A few lines of code are enough:
import numpy as np# X: each row is [input tokens, output tokens]; Y: credits actually deductedcoef, *_ = np.linalg.lstsq(X, Y, rcond=None)a, b = coef# Evaluate fit qualitypred = X @ coefr2 = 1 - ((Y - pred)**2).sum() / ((Y - Y.mean())**2).sum()
When the samples cover multiple models and the data is clean, the goodness of fit (R²) can be very close to 1.0, with near-zero residuals. This indicates that the conversion rule is essentially linear and that the inferred consumption rates are highly credible. Cache hits often appear as a significantly lower conversion rate, so a third column for “cache-hit tokens” can be added to the model and fitted together.
Note: These inferred results are based on personal measurements. Platform rules may change at any time, so they should be used only as a reference for resource allocation, not as an official billing basis.
3. How Large the Differences Can Be
Exact values vary by platform and model, but the industry-wide tier pattern generally follows these rules. The following are illustrative magnitude descriptions, not data for any specific product:
- Flagship chat/reasoning models: Output consumption rates can reach the level of hundreds of credits per million tokens, tens or even hundreds of times that of entry-level models.
- Mid-tier general models: Input and output consumption rates are moderate, roughly one-fifth to one-third of flagship models.
- High-speed/lightweight models: Input and output rates are both very low, making them suitable for massive volumes of small tasks.
- Cache hits: These can reduce input cost to a very low percentage of the normal value, delivering major benefits for repeated-prefix scenarios such as long system prompts or long-context continuation.
Take output-cost comparison as an example. It is not uncommon for the output consumption ratio between a flagship model and an entry-level model to reach the hundredfold level. This means that for the same 1 million output tokens, a flagship model may consume a substantial portion of a weekly quota, while a lightweight model makes the cost almost negligible.
4. Estimating Capacity Under a Fixed Quota
Once consumption rates are known, you can estimate theoretical throughput under a given quota. Let the quota be Q credits, the input share of the workload be p (for example, 80%), and the output share be 1-p (for example, 20%). Then:
Total processable tokens ≈ Q / (p × input consumption rate + (1-p) × output consumption rate)
For example, in a typical coding scenario with a weekly quota, 80% input and 20% output, and no cache accounting, the processable token volume across model tiers usually shows a stepwise gap: lightweight models can handle tens of times more tokens than flagship models. This number directly determines workflow design—whether a flagship model runs out after a day, or a lightweight model can steadily support an entire week.
5. Tiered Routing Strategy: Expensive Models Think, Cheap Models Execute
Based on this cost structure, a four-layer division of labor has proven effective in practice:
- Flagship models—planning and final review: Use them in only two stages. First, at task startup, to break down requirements, design solutions, and generate an execution plan. Second, at task completion, to review outputs and gate quality. These stages involve few calls but have very high value per call, making them the most worthwhile place to spend money.
- Mid-tier models—complex reasoning: Handle intermediate steps that require stronger logical capability, such as algorithm selection, difficult bug localization, and reasoning about cross-file refactoring plans.
- High-speed models—main execution: Handle more than 80% of daily coding calls, such as function implementation, unit test generation, and code-completion iterations. They are cheap and fast, and quota consumption is almost imperceptible.
- Lightweight models—batch simple tasks: Handle trivial tasks that can be parallelized, such as format conversion, bulk renaming, document summarization, and simple script generation.
This structure is consistent with cost management in human teams: architects and junior engineers should not be paid the same hourly rate to do the same work.
6. Engineering Implementation Recommendations
- Routing-layer abstraction: Encapsulate a unified model-call entry point on the application side and automatically route to different tiers based on task type or token estimates. Avoid hardcoding model names in business code.
- Use caching for repeated prefixes: Place stable system prompts and long context at the request prefix and try to hit the cache. This can compress input cost to a very low level.
- Quota monitoring and alerting: Pull usage data periodically, rerun the fit, monitor whether consumption rates have changed, and set threshold alerts for remaining quota.
- Output-length control: Output consumption rates are generally higher than input rates. Limiting maximum output length and requiring concise answers can significantly extend quota life.
- Fallback plan: When the main model’s quota is tight, automatically downgrade to a lower-tier model while preserving a high-tier channel for critical tasks.
7. Conclusion
Token quota is fundamentally a resource-allocation problem. By fitting usage logs to infer consumption rates—where an R² close to 1.0 is enough to confirm that a linear rule holds—you can design a tiered strategy in which high-cost models think and low-cost models execute. Combined with caching and routing engineering practices, this can amplify the actual output of a fixed quota by an order of magnitude without increasing budget. The key to cost optimization is not choosing the cheapest model, but matching each type of task to the tier with the right cost-performance balance.
