LLM Inference Cost Benchmark: Token Consumption Across Model Tiers

When building LLM-based agent applications, cost control is often the decisive factor in whether a project can keep running sustainably. Models of different sizes differ significantly in reasoning capability, but their token consumption rates can also differ by two orders of magnitude. Based on actual usage records, this article reverse-engineers the credit consumption patterns of several typical model tiers and provides an actionable tiered usage strategy.

Note that the following data comes from personal measurements and curve fitting, not from any vendor’s official price list, and the underlying rules may change over time. That said, the fit produced R² = 1.0 with residuals near zero, matching actual credit consumption almost exactly, so it remains useful as a reference for cost estimation.

1. Why Token Consumption Rates Matter

In large-scale agent scenarios, a single task often requires multiple model calls: planning, reasoning, execution, and review, each with different capability requirements. If every step uses the same flagship model, costs quickly spiral out of control. If every step uses a lightweight model instead, complex reasoning may fail frequently, and retry costs can end up being higher.

Understanding the token consumption rates of different models is therefore a prerequisite for designing a cost-effective agent pipeline.

2. Reverse-Engineered Token Consumption Rates

By fitting actual usage records, the credit consumption rates of several model tiers can be derived (measured in credits per million tokens):

Model tier	Input cost / 1M	Output cost / 1M	Cache hit / 1M
Flagship reasoning model	80,000	400,000	—
Mid-tier reasoning model	32,000	112,000	—
Lightweight execution model	4,000	8,000	80
Ultra-light batch model	2,000	2,000	—

Several key conclusions stand out from the table:

1. Output costs far more than input: For the flagship model, output costs 5x as much as input, which means the longer the generation, the faster costs grow.
2. The gap between flagship and ultra-light is enormous: The flagship model’s output cost is 200x that of the ultra-light model, a difference of two orders of magnitude.
3. Cache hits deliver significant savings: When a cache hit occurs, the lightweight execution model costs only 2% of its normal input cost, making it well suited to scenarios with repeated context.

3. Estimating Token Capacity by Model Tier

Assuming a credit budget of 60,000 credits per 5 hours and 600,000 credits per week, and estimating at a ratio of 80% input and 20% output (excluding caching for now), the approximate token capacity of each model is as follows:

Model tier	Approximate tokens processed / 5 hours
Flagship reasoning model	~417K
Mid-tier reasoning model	~1.25M
Lightweight execution model	~12.5M
Ultra-light batch model	~30M

This comparison makes it clear: high-cost models are suited to “thinking,” low-cost models to “doing.” If every task is handed to the flagship model, the token capacity is only about 1/70 that of the ultra-light model.

4. Tiered Usage Strategy

Based on the cost structure above, the following tiered approach can be designed:

1. Flagship Reasoning Model: Planning and Final Review

Flagship models excel at complex planning, long-chain reasoning, and final quality review. These tasks involve few calls but demand very high result quality. Using the flagship model at key decision points avoids large amounts of wasted execution caused by planning errors.

2. Mid-Tier Reasoning Model: Complex Reasoning

Mid-tier models strike a balance between reasoning capability and cost, making them suitable for tasks that require a certain depth of reasoning but not flagship-level capability, such as multi-step logical deduction and structured information extraction.

3. Lightweight Execution Model: Primary Execution

Lightweight models are the workhorses of the agent pipeline, handling most tool calls, code generation, data transformation, and similar tasks. Their cost is only a few dozenths of the flagship model’s, and cache hits reduce it further, making them ideal for high-frequency calls.

4. Ultra-Light Batch Model: Bulk Simple Tasks

Ultra-light models are suitable for bulk tasks such as classification, tagging, formatting, and simple Q&A. Their cost is extremely low, and they can be called in large parallel batches, making them the first choice for handling massive volumes of simple requests.

5. The Optimization Value of Caching

When a cache hit occurs, the input cost of the lightweight execution model drops to 2% of its normal level. This means caching can deliver substantial benefits in the following scenarios:

• System prompt reuse: An agent’s system prompt is usually fixed; caching it can greatly reduce repeated input costs.
• Multi-turn conversation context: Historical context within the same session can be cached, avoiding re-billing on every call.
• Templated tasks: If batch tasks share the same template, the cache hit rate will be very high.

6. Engineering Best Practices

In practical engineering, cost can be optimized along the following dimensions:

Task layering:
  Planning layer -> flagship model
  Reasoning layer -> mid-tier model
  Execution layer -> lightweight model
  Batch processing -> ultra-light model
Caching strategy:
  Fixed prompts -> enable caching
  Multi-turn conversations -> reuse context
  Templated tasks -> pre-set cache keys
Monitoring metrics:
  Token consumption per task
  Cache hit rate
  Retry rate and failure rate

It is also advisable to establish a cost monitoring mechanism that records each task’s token consumption, cache hit rate, and retry count. When a certain type of task shows abnormal costs, the model tier can be adjusted or the prompt optimized in a timely manner.

7. Conclusion

Token consumption rates vary enormously across model sizes; the flagship model’s output cost can reach 200x that of the ultra-light model. With a layered strategy in which “high-cost models think and low-cost models do,” costs can be kept within a reasonable range while maintaining quality. Caching further reduces the cost of repeated context and is an optimization that should not be overlooked in agent engineering.

The data above is reverse-engineered from personal measurements and is for reference only. Actual costs vary with usage patterns, cache hit rates, and task structure, so continuous monitoring and tuning for your own scenarios is recommended.