Large Language Model API Cost Comparison: Pricing Mechanics and Optimization Strategies
LLM API costs depend on far more than the advertised price per million tokens. This guide explains the main pricing variables—including input and output rates, prompt caching, time-based pricing, batch discounts, and storage fees—then provides a practical cost model, a worked example, and five strategies for reducing spend.
1. Why LLM API Pricing Is More Complex Than It Looks
Large language model API pricing involves much more than a simple “price per million tokens” calculation. The final bill may depend on the model tier, whether input tokens are served from cache, the time of day when a request is completed, whether the request uses a batch channel, and how long cached context is stored. Each variable can materially change the effective unit price.
Many teams focus only on the standard list price during model selection and discover after launch that their actual bill is much higher than expected. The problem is often an incomplete understanding of the less visible billing dimensions.
Before comparing providers or models, it is therefore more useful to break pricing down into its common components than to compare headline prices directly.
2. The Core Dimensions of API Pricing
2.1 Input and Output Prices Are Asymmetric
Almost all major LLM APIs price input tokens and output tokens separately. Output tokens are typically two to eight times more expensive than input tokens because generation requires autoregressive decoding one token at a time, while input processing can be performed more efficiently in parallel.
This has an important implication for prompt and application design: controlling prompt length is not enough. Teams should also reduce unnecessary output, for example by setting a suitable max_tokens limit to constrain generation length.
2.2 Discounts for Cached Context
When multi-turn conversations or batch workloads repeatedly use the same prefix—such as a system prompt or a knowledge-base excerpt—the provider may cache that context. The input price for cache hits can be one-tenth of the uncached price or even lower.
For example, one major model may charge CNY 0.02 per million cached input tokens compared with CNY 1 per million uncached input tokens, a 50-fold difference.
Caching is not necessarily permanent, however. Some providers charge separately for cache storage, usually based on CNY per million tokens per hour. If requests are spaced too far apart, the cache may expire before it is reused, turning the storage fee into an additional cost without a corresponding benefit.
2.3 Time-Based Pricing
Some providers use a pricing model similar to peak and off-peak electricity rates. Requests completed during weekday business hours may be charged at the standard rate, while requests completed overnight, on weekends, or during holidays may receive discounts of 50% or more.
The billing period may be determined by the request completion time rather than the submission time. As a result, a long-running batch job submitted near the end of a peak period may finish during an off-peak period and qualify for the lower rate. This creates a potentially useful scheduling window.
2.4 Batch Processing Discounts
For offline workloads, many platforms offer a Batch mode in which jobs are submitted asynchronously and executed when capacity is available. Batch pricing is commonly 50% to 70% of the standard price.
However, batch discounts may be mutually exclusive with prompt-cache discounts. In that case, a workload must use one or the other. Before optimizing costs, calculate which option fits the request pattern:
- Short requests with highly repetitive prefixes are often better suited to cache discounts.
- Long requests with little or no repeated prefix content are often better suited to Batch processing.
3. Representative Pricing Tiers
The following ranges summarize several representative pricing tiers from publicly available information and are intended to provide a sense of scale. Prices are shown in CNY per million tokens.
| Model tier | Uncached input | Output | Cached input | Typical use |
|---|---|---|---|---|
| Lightweight models | 0.8–1.0 | 2.7–4.0 | 0.02–0.16 | High-concurrency, simple tasks |
| Mid-range balanced models | 2.4–6.0 | 9.6–30 | 0.6–1.2 | General conversations and summarization |
| Flagship reasoning models | 12–20 | 36–100 | 1.3–2.0 | Complex reasoning and long-form generation |
| High-speed dedicated models | 26+ | 136+ | 1.3+ | Extremely latency-sensitive workloads |
Several conclusions are immediately apparent:
- The output price difference between lightweight and flagship models can reach 25 to 30 times. Model selection should match task complexity rather than defaulting to the most powerful model.
- Cached input is often an order of magnitude cheaper than uncached input, making caching one of the largest cost-optimization opportunities.
- Flagship models may charge a cache-write fee when previously uncached content is written to the cache. This fee can sometimes be comparable to the output price and should not be overlooked.
4. A Practical Cost Estimation Process
Step 1: Measure Request Characteristics
Before estimating costs, answer the following questions:
- Average prompt length: Use the mean or P50 input-token count.
- Average output length: This depends on the task. Summaries are usually shorter than article generation.
- Repeated-prefix ratio: Are system instructions fixed? Are the same knowledge-base sections referenced repeatedly?
- Request volume and time distribution: Are requests evenly distributed throughout the day or concentrated during business hours?
- Latency requirements: Can the application accept asynchronous batch processing?
Step 2: Build the Pricing Formula
Total cost can be estimated as follows:
Total cost = Request volume × [Uncached input price × (1 - Cache hit rate)+ Cached input price × Cache hit rate] × Average input tokens / 1,000,000+ Request volume × Output price × Average output tokens / 1,000,000+ Cache storage cost, if applicable
When using a batch channel, apply the batch discount factor to the relevant costs. Because batch processing may eliminate cache discounts, calculate both options separately and choose the lower total cost.
Step 3: Run a Scenario-Based Calculation
Consider a knowledge-base question-answering application with the following profile:
- 100,000 requests per day
- Average input length of 2,000 tokens, including an 800-token fixed system prompt
- Average output length of 300 tokens
- Expected repeated-prefix cache hit rate of 70%
- Moderate latency requirements, allowing batch processing
Assume a mid-range balanced model with the following prices:
- Uncached input: CNY 2.4 per million tokens
- Cached input: CNY 0.6 per million tokens
- Output: CNY 9.6 per million tokens
- Batch price: 70% of the standard price
Using cache discounts:
Input cost = 100,000 × (2.4 × 0.3 + 0.6 × 0.7) × 2,000 / 1,000,000= 100,000 × 1.14 × 0.002= CNY 228Output cost = 100,000 × 9.6 × 300 / 1,000,000= CNY 288Total ≈ CNY 516 per day
Using Batch mode while losing the cache discount:
Input cost = 100,000 × 2.4 × 2,000 / 1,000,000 × 0.7= CNY 336Output cost = 100,000 × 9.6 × 300 / 1,000,000 × 0.7= CNY 201.6Total ≈ CNY 537.6 per day
In this scenario, the cache discount is slightly more economical. If output tokens account for a larger share of total usage, Batch mode may become the better option. The calculation should therefore be repeated using real production data.
5. Five Cost Optimization Strategies
5.1 Optimize Prompt Structure to Improve Cache Hit Rates
Separate dynamic instructions from fixed system instructions, and make the stable prefix as long and consistent as possible. In multi-turn conversations, reuse the same system context whenever possible. Rebuilding the prefix differently for every request can invalidate the cache.
5.2 Schedule Non-Real-Time Workloads During Off-Peak Periods
If the platform supports time-based pricing, schedule video summarization, bulk document processing, and offline analysis for overnight hours or weekends. This can reduce costs by 40% to 50%.
A scheduler, timer, or message queue can be used to automate this workload placement.
5.3 Add a Model Routing Layer
Create a request-classification layer:
- Route simple intents, such as classification, extraction, and casual conversation, to lightweight models.
- Route complex reasoning tasks, such as code generation and logical analysis, to flagship models.
In practice, even a keyword-based routing rule can filter out more than 30% of requests that do not require a powerful model. Overall costs may fall by as much as 50%.
5.4 Limit Output Length and Reduce Unproductive Generation
Output tokens are often the most expensive part of an API call because they are multiplied by the highest unit price. Use the max_tokens parameter to prevent excessively long responses, and use structured prompts to constrain the output format and reduce unnecessary text.
5.5 Evaluate Application-Level Caching
If the workload has a very high repeated-prefix ratio—for example, a customer-service bot or legal document assistant—consider implementing semantic or exact-match caching at the application layer. Reusing previous answers can avoid API calls entirely.
For high-concurrency workloads, application-level caching can save significantly more than relying on provider discounts alone.
6. A Framework for Model Selection
Final model selection should not be based solely on a price table. Consider the following factors together:
- Task complexity: Use lightweight models for simple tasks and reserve flagship models for complex reasoning.
- Latency sensitivity: Strict real-time interaction requirements may rule out batch processing and off-peak pricing strategies.
- Request volume: When daily volume is below 10,000 requests, absolute price differences may be relatively small, making model quality the higher priority. Above 100,000 requests per day, scheduling and cache-hit improvements become much more significant.
- Repeated-prefix ratio: Workloads with a high ratio should prioritize platforms with robust caching mechanisms. Workloads with little repetition should focus on base prices and batch discounts.
- Pricing transparency: Prefer providers that clearly document all billing dimensions, including cache writes, cache storage, and rules governing the interaction between Batch processing and caching. This reduces the risk of unexpected charges after deployment.
7. Conclusion
LLM API pricing has become increasingly granular. The era of comparing only published list prices is over. Effective cost management depends on understanding the billing model and accurately measuring the characteristics of actual requests.
A useful practice is to review costs every quarter. Recalculate cache hit rates, output-token ratios, and time-of-day distributions, then compare those measurements with the provider’s current pricing rules. Adjust the model mix and request strategy accordingly.
A practical first step is to retrieve the last seven days of API logs and calculate the average input length, average output length, and repeated-prefix ratio. Then apply the formulas above to determine your actual effective unit cost.
That number may be substantially higher than the list price used in your original estimate.
