LLM API Pricing Strategies and Cost Optimization Practices

I. Background: Inference Cost as a Core Engineering Variable

As Large Language Models (LLMs) penetrate various business systems, API invocation costs have evolved from a negligible operational overhead into an engineering expense that requires meticulous management. For high-frequency invocation scenarios—such as intelligent customer service, code assistance, document analysis, and batch content generation—seemingly minor differences in token unit prices can scale into significant cost gaps.

Therefore, understanding the pricing structure of a model service is not merely a financial concern but an architectural design issue. It directly impacts decisions such as whether to introduce a caching layer, whether to split long and short context requests, and whether to pay a premium for latency-sensitive tasks. This article will use a typical model pricing scheme as a sample to break down its billing dimensions and provide actionable cost optimization strategies.

II. Billing Structure Breakdown: Four Dimensions Determine Actual Costs

Overall, the billing for current mainstream LLM APIs consists of the following components:

1. Base Input Tokens: Prompts, context, and system instructions sent to the model by the user.
2. Base Output Tokens: The response content generated by the model.
3. Context Cache Hit: When the prefix of a request matches a historical request, the precomputed intermediate state can be reused.
4. Context Cache Write: The cost incurred by writing context into the cache, usually tiered based on the cache’s Time-To-Live (TTL).

The unit price for output tokens is generally significantly higher than that of input tokens. This is because autoregressive generation requires decoding token by token, resulting in higher computational density. Conversely, the existence of caching mechanisms provides substantial cost reduction potential for scenarios with high-frequency, repetitive prefixes.

2.1 A Concrete Pricing Sample

Taking a certain model series as an example, the pricing comparison between its standard and high-speed versions is as follows (Unit: CNY / Million Tokens):

Model Type	Input	Output	Cache Hit	Cache Write (5min)	Cache Write (1h)
Standard	13.00	68.00	1.30	13.00	26.00
High-Speed	26.00	136.00	1.30	26.00	52.00

Several key patterns can be extracted from this data:

• The high-speed version is priced at exactly 2x the standard version. This multiplier remains consistent across input, output, and cache write dimensions, indicating a unified premium coefficient rather than differentiated pricing by dimension.
• Cache hit pricing is extremely low, at only 1/10th of the input price. This means that once a cache is hit, the processing cost for repetitive context becomes almost negligible.
• Cache writes are tiered by duration: The price of a 1-hour cache is 2x that of a 5-minute cache, aligning with the intuition that the longer resources are occupied, the higher the cost.

2.2 Comparison with the Previous Generation Model

Comparing the current version with the previous generation of the same model series reveals a general price reduction:

Model Type	Input	Output	Cache Hit
Current Standard	13.00	68.00	1.30
Previous Generation	20.00	100.00	2.00

Input prices dropped by approximately 35%, output prices by 32%, and cache hit prices by 35%. This trend indicates that with the maturation and scaling of inference infrastructure, the unit cost per token is steadily declining. For developers, this means that migrating to a new version usually yields direct cost benefits without requiring changes to the business logic.

III. Five Practical Cost Optimization Strategies

Understanding the pricing structure is only the first step; the real value lies in controlling expenditures in actual engineering. The following strategies are ranked by their return-on-investment (ROI) relative to their implementation cost.

3.1 Maximize Cache Hit Rate

With cache hit prices at a mere tenth of standard input prices, this is the most cost-effective optimization point. The core to improving hit rates is maintaining the stability of the request prefix:

• Place fixed system instructions, role definitions, and few-shot examples at the very beginning of the prompt.
• Place dynamic user inputs at the end.
• Avoid inserting timestamps, random IDs, or other variable elements into the system prompt.

# Recommended: Stable prefix + Dynamic suffix
[system] You are a code review assistant, follow these rules... (Fixed)
[few-shot] Example 1... Example 2... (Fixed)
[user] Please review this code: {{dynamic_content}}

Conversely, placing a timestamp at the very beginning will cause the request prefix to differ every time, meaning the cache will never be hit.

3.2 Select Cache Duration Based on Scenario

The price difference between a 5-minute cache and a 1-hour cache is 2x. The selection should be based on the temporal distribution characteristics of the requests:

• For scenarios with dense, short-term repetitions (e.g., multi-turn dialogue within a single session), a 5-minute cache is sufficient.
• For cross-session reuse scenarios with longer intervals (e.g., multiple users sharing the same knowledge base prefix), a 1-hour cache is more cost-effective.

Avoid blindly selecting long cache durations, as you will end up paying for idle Time-To-Live (TTL) that goes unused.

3.3 Carefully Evaluate the Necessity of High-Speed Versions

The high-speed version costs 2x the standard version. Whether it is worth it depends on the business’s latency sensitivity:

• Suitable for High-Speed: Real-time interactions, user-perceivable conversations, and Agent loops requiring rapid responses.
• Suitable for Standard: Offline batch processing, document summarization, data analysis, and non-real-time content generation.

A common practice is hybrid usage: routing latency-sensitive online paths to the high-speed version, and throughput-sensitive offline tasks to the standard version, managed unifiedly through a routing layer.

3.4 Control Output Length

The unit price for output is over 5x that of input, making waste on the output side significantly more expensive. Measures that can be taken include:

• Explicitly requesting “concise answers” or “output only the result, without explanations” in the prompt.
• Setting a reasonable max_tokens limit to prevent the model from generating redundant content.
• For structured tasks, requiring the model to output JSON or tables directly to reduce natural language preambles.

3.5 Batch Processing and Request Merging

When dealing with a large number of small requests, merging them into fewer large requests is often more economical. For example, combining 100 independent short-text classification tasks into a single batch request containing 100 samples reduces request overhead and allows the model to accomplish more work in a single inference pass. Of course, this requires balancing the context length limit of a single request against the granularity of failure retries.

IV. Building Cost Observability

Optimization cannot rely on guesswork. It is recommended to establish cost monitoring at the system level:

• Log token usage by invocation dimension: Track input, output, cache hits, and cache writes separately.
• Aggregate by business module: Identify the most expensive functional paths.
• Set budget alerts: Trigger notifications when a module’s daily consumption exceeds a defined threshold.

A simplified logging structure example:

{
  "model": "standard",
  "input_tokens": 1200,
  "output_tokens": 350,
  "cache_hit_tokens": 800,
  "cache_write_tokens": 400,
  "estimated_cost": 0.041
}

With this data, you can accurately determine whether to migrate to a new version, whether caching strategies are effective, and whether the high-speed version is being utilized in the right places.

V. Conclusion

Managing LLM API costs is essentially the process of mapping billing dimensions to architectural decisions. The core takeaways can be summarized as follows:

1. Output is more expensive than input: Controlling generation length is the highest priority.
2. Cache hits are extremely cheap: A stable prefix is the key to improving hit rates.
3. High-speed versions carry a flat 2x premium: Differentiate usage based on latency sensitivity.
4. New generation models are generally cheaper: Migration is typically a straightforward decision.
5. Observability is the prerequisite for optimization: No data means no optimization.

By implementing these principles into your prompt design, request routing, and monitoring systems, you can keep inference costs within a reasonable range while ensuring a high-quality user experience.