Token Cost Analysis and Tiered Model Routing for Multi-Model LLM Applications
Token pricing is not a single per-million-token number. In real workloads, uncached input, output, and cache-hit tokens have different costs, and model selection changes total spend. This article abstracts a usage log into four model cost tiers, shows how cache hits affect effective cost, calculates token throughput under a fixed credit budget, and outlines task-based routing and validation methods.
Why a Single Token Price Is Not Enough
When LLMs are used for code generation, document processing, data analysis, or automated execution, model capability is not the only decision factor. For continuously running applications, the biggest cost drivers are input tokens, output tokens, cache-hit ratio, task complexity, and model scheduling.
Many teams compare only the “per million tokens” price. But in practice, input and output prices differ, cached input can be much cheaper than regular input, and complex contexts or repeated calls make simple unit-price comparisons misleading.
A typical model call has at least three billable categories:
- Uncached input tokens: system prompt, user question, conversation history, tool results, repository content, and so on.
- Output tokens: answers, code, reasoning output, structured data, execution plans, and so on.
- Cache-hit tokens: repeated context that the service bills at a lower rate.
These price differences can reach an order of magnitude or more. Output tokens are usually more expensive than input tokens, while cache-hit prices can be only a small fraction of the regular input price.
Therefore, the real cost should be calculated as a weighted cost rather than by reading a single published input price:
Total cost =(uncached input tokens × normal input price)+ (output tokens × output price)+ (cache-hit tokens × cache price)
When expressed as an average cost per token:
Average cost =(uncached input share × normal input price)+ (output share × output price)+ (cache-hit share × cache price)
The input, output, and cache-hit shares should come from actual business statistics, not from rough estimates.
What a Measured Usage Log Shows
The following analysis is based on a continuous set of usage records, but it is not tied to a specific product or vendor. To keep the results general, the observed model tiers are named by their role:
- High-level planning model
- Complex reasoning model
- General execution model
- Lightweight batch model
Without considering cache hits, the inferred reference consumption rates are roughly as follows:
| Model tier | Input cost | Output cost | Typical role |
|---|---|---|---|
| High-level planning | 80,000 credits / million tokens | 400,000 credits / million tokens | Complex planning, final review |
| Complex reasoning | 32,000 credits / million tokens | 112,000 credits / million tokens | Difficult analysis, solution derivation |
| General execution | 4,000 credits / million tokens | 8,000 credits / million tokens | Daily generation, code execution |
| Lightweight batch | 2,000 credits / million tokens | 2,000 credits / million tokens | Classification, rewriting, extraction |
The most important pattern is not the exact numbers, but the cost gradient. The output cost of the high-level planning tier is about 200 times that of the lightweight batch tier. If the expensive model is used for large-scale format conversion, text cleaning, or simple code completion, the budget will disappear quickly. Conversely, using a lightweight model for complex architecture design or critical code review can create quality and stability risks.
How Cache Hits Change Effective Cost
Cache hits are especially valuable for long-context tasks.
For example, the general execution tier may have a cache-hit unit cost that drops to about 2% of the normal input cost. If an application sends the same large context repeatedly, cache hits can significantly reduce the cost of repeated input.
Suppose every request carries 1 million tokens of fixed context, with 900,000 tokens hitting the cache and 100,000 tokens being new content. The real input cost is not based on 1 million normal input tokens:
Total input cost =900,000 × cache price+ 100,000 × normal input price
If the cache price is 2% of the normal input price:
Total input cost≈ 900,000 × 2% × normal input price+ 100,000 × normal input price= 118,000 normal-input-token equivalents
From a cost perspective, a 1 million token context can behave like about 118,000 normal input tokens.
Cache hits are not guaranteed just because the content is similar. Actual hit rate depends on prefix ordering, stable context placement, request interval, cache lifetime, and server-side policies. In engineering practice, keep stable content near the front of the prompt and reduce unnecessary dynamic field insertion.
Token Volume Under a Fixed Credit Budget
Assume an account has about 60,000 credits per 5-hour period, with an additional weekly planning figure of about 600,000 credits. Assume further that the workload is 80% input and 20% output, with cache hits temporarily ignored.
Using the reference cost model, the approximate weighted average costs and 5-hour token capacities are:
| Model tier | Weighted average cost | 5-hour processing capacity |
|---|---|---|
| High-level planning | 144,000 credits / million tokens | About 417,000 tokens |
| Complex reasoning | 48,000 credits / million tokens | About 1.25 million tokens |
| General execution | 4,800 credits / million tokens | About 12.5 million tokens |
| Lightweight batch | 2,000 credits / million tokens | About 30 million tokens |
For the high-level planning tier:
Average cost =80% × 80,000+ 20% × 400,000= 144,000 credits per million tokens
With a 60,000-credit quota:
Token capacity =60,000 / 144,000 × 1,000,000≈ 417,000 tokens
For the general execution tier:
Average cost =80% × 4,000+ 20% × 8,000= 4,800 credits per million tokens
Token capacity:
60,000 / 4,800 × 1,000,000≈ 12,500,000 tokens
This shows a clear budget tradeoff between model capability and processing scale. High-end models are suitable for small numbers of critical tasks, while lower-cost models are better for high-frequency, repetitive, and large-volume work.
Recommended Multi-Model Workflow
Reasonable model routing should not send every request to the strongest model. Instead, tasks should be divided by value and complexity.
1. High-Level Planning Model: Think It Through
This tier is best suited for:
- System architecture design
- Complex requirement decomposition
- Multi-option tradeoff analysis
- Risk identification
- Final result review
- Key technical documentation
Its strength is handling complex constraints and long-chain decisions, but its cost is high. It should not continuously participate in every small implementation step. A better approach is to let it generate the task plan, acceptance criteria, and key constraints first, then hand execution to a lower-cost model.
2. Complex Reasoning Model: Solve the Difficult Parts
This tier is suitable for:
- Algorithm derivation
- Anomaly diagnosis
- Complex query design
- Multi-step logical analysis
- Code refactoring suggestions
- Execution result interpretation
It can serve as a middle layer between the high-level planning model and the general execution model. When a problem requires meaningful reasoning but does not justify the highest-cost model, this tier often provides better value.
3. General Execution Model: Handle Mainstream Production Work
This tier is suitable for:
- Generating code from a plan
- Writing test cases
- Generating API documentation
- General question answering
- Text organization
- Data format conversion
- Repeated operations with clear rules
In most automation workflows, the real token consumption comes from execution steps, not from planning. Using a general execution model as the main workhorse can significantly reduce total budget.
4. Lightweight Batch Model: Run Large-Scale Simple Tasks
This tier is enough for:
- Text classification
- Keyword extraction
- Tag generation
- Simple summarization
- Format validation
- Field completion
- Template-based rewriting
- Batch content cleaning
These tasks have clear objectives, stable rules, and relatively large tolerance for variation. Using a high-cost model here usually does not produce benefits proportional to the cost.
How to Verify the Credit Consumption Model
A single call record is not enough to infer reliable unit costs. A better approach is to run controlled experiments.
Step 1: Fix Output Length
Set a uniform maximum output length to reduce output token variance. Otherwise, input changes and output changes are mixed together and cannot be separated.
Step 2: Increase Input Size Gradually
Prepare inputs of different lengths with similar structure. Record the credit balance before and after each request, then estimate the marginal input cost through the difference.
Step 3: Test Output Cost Separately
Keep the input almost unchanged and ask the model to generate different output lengths. Observe the credit changes to estimate output cost.
Step 4: Test Cache Hits
Send multiple requests with the same prefix and change only the tail content. Compare the first request with subsequent requests to measure the cache-hit saving. Repeat the test multiple times to avoid accidental conclusions.
Step 5: Perform Linear Fitting
Model the consumption as:
Credit consumption =a × input tokens+ b × output tokens+ c × cache-hit tokens+ d
Where:
ais the normal input unit costbis the output unit costcis the cache-hit unit costdis possible fixed overhead
If multiple experiment sets produce high R-squared values and near-zero residuals, the model explains the observed data well. However, a high R-squared only proves consistency within the sample. Future pricing policies, context handling behavior, and account limits may change.
Cost Control Recommendations for Production Applications
In a real system, monitor at least the following:
- Input tokens and output tokens per request
- Cache hit rate
- Average credit consumption per task
- Model distribution by task type
- Extra consumption caused by retries
- Share of requests with long contexts
- Business value produced by high-cost model calls
Budget protection mechanisms are also useful:
- Set separate quotas for different task types
- Cap unusually long outputs
- Limit automatic retries for repeated failures
- Alert when the cache hit rate drops
- Require manual confirmation or approval for high-cost model calls
- Automatically downgrade low-cost models for non-critical tasks
For development teams, the most important architectural decision is to wrap model calls in a unified routing layer instead of hardcoding a model in business logic. The routing layer can select the model dynamically based on task type, context length, priority, remaining budget, and historical success rate.
Conclusion: Let Expensive Models Decide, Let Cheap Models Execute
The core principle of multi-model collaboration is:
High-cost models should be responsible for understanding, planning, and review; low-cost models should be responsible for execution, expansion, and batch processing.
This is not simply about finding the lowest price. It is about dividing work by capability, stability, and budget. High-end models reduce directional errors, while lightweight models expand processing scale. Only when each model handles tasks matching its ability and price can a sustainable intelligent application architecture be formed.
The cost data in this article is reverse-estimated from actual usage records and should not be treated as any vendor’s official price list. Different accounts, time periods, cache policies, and service rules can change the results. Before going live, re-measure with repeatable experiments and include cost monitoring in operational and budget management.
