How to Choose Between Standard and High-Speed Large Language Model APIs

    This guide explains how modern LLM APIs use tiered pricing, breaking down input, output, cache-hit, and cache-write costs. It compares standard and high-speed service tiers, outlines suitable use cases, and presents routing and prompt-caching strategies for balancing latency, availability, and cost.

    1. From Single-Rate Pricing to Tiered Service Plans

    In the early days of large language model APIs, pricing was relatively simple: users paid according to the number of input and output tokens, and all requests shared the same service resources. As model capabilities improved and use cases diversified, this single-rate approach became less practical. Real-time conversations require very low response latency, while batch-processing workloads are usually more concerned with unit cost.

    To address these different requirements, major LLM services have increasingly adopted tiered pricing. The same model capability may be offered through standard and high-speed service tiers, allowing customers to choose according to their workload.

    A recent next-generation model illustrates this approach. Its base prices are lower than those of the previous generation, while a high-speed tier has been introduced at approximately twice the price of the standard tier. This article uses that pricing structure as a reference to examine LLM API billing dimensions, service-tier differences, and cost-optimization strategies from a general technical perspective.

    2. Five Key Dimensions of LLM API Pricing

    Modern LLM API pricing typically involves the following dimensions. Understanding them is essential for accurate cost planning.

    2.1 Input Token Costs

    Input tokens include everything sent to the model: system prompts, conversation history, retrieved context, and the user’s request.

    In multi-turn conversations, historical messages accumulate and may be billed repeatedly with each request. This is the fundamental reason context caching can provide significant value.

    2.2 Output Token Costs

    Output tokens are usually priced substantially higher than input tokens. Under the newer pricing structure, output tokens cost approximately five times as much as input tokens.

    The main reason is that output generation uses autoregressive decoding, producing tokens sequentially. Compared with the parallel prefill stage used for input processing, generation has higher computational density and occupies service resources for a longer period.

    As a result, output length is often the primary cost driver in practical estimates.

    2.3 Cache-Hit Costs

    Context caching allows the service to store a long prefix from a previous request, such as a system prompt or document segment. When a later request contains exactly the same prefix, the cached portion is billed at a cache-hit rate.

    This rate is typically around 10% of the regular input price. For example, if standard input costs ¥13.00 per million tokens (MTok), a cache hit costs ¥1.30 per MTok.

    For applications with long system prompts or many conversation turns, the cache-hit rate directly affects the slope of the actual cost curve.

    2.4 Cache-Write Costs

    A prefix that appears for the first time must be written to the cache. The write price is generally similar to, or slightly higher than, the regular input price.

    Caches typically offer multiple time-to-live options, such as five minutes and one hour. The write price for the one-hour option is approximately twice that of the five-minute option.

    A longer lifetime increases the probability of cross-request cache hits and is suitable for intermittent requests with a stable prefix. A shorter lifetime is more appropriate for high-frequency, continuous calls.

    2.5 Tier Difference: High-Speed Costs Approximately Twice as Much

    For the same model capability, the high-speed tier provides lower queueing latency and stronger throughput guarantees, but its unit prices are doubled.

    Using the reference pricing structure:

    Item Standard Tier High-Speed Tier
    Input ¥13.00/MTok ¥26.00/MTok
    Output ¥68.00/MTok ¥136.00/MTok
    Cache hit ¥1.30/MTok ¥1.30/MTok

    Cache-hit pricing is the same for both tiers, while cache-write pricing doubles with the service tier.

    3. How to Choose Between Standard and High-Speed Tiers

    Tier selection should not be based on intuition alone. A simple cost-versus-latency model provides a more reliable basis for decision-making.

    3.1 When the High-Speed Tier Makes Sense

    • Real-time online interaction: Customer-support bots, coding assistants, and similar applications are sensitive to time to first token and overall response latency.
    • Peak traffic periods: During high-demand periods, routing part of the traffic to the high-speed tier can help avoid queueing delays and timeouts.
    • Multi-step Agent workflows: When a task requires several sequential model calls, the latency of each step accumulates. The high-speed tier can significantly reduce end-to-end execution time.

    3.2 When the Standard Tier Is More Suitable

    • Offline batch processing: Document summarization, data labeling, and corpus cleaning are generally less sensitive to response time.
    • Asynchronous pipelines: Generation tasks decoupled through a message queue can usually tolerate additional latency.
    • Cost-sensitive long-form workloads: When output volume and request volume are high, the twofold price difference can substantially increase total costs.

    3.3 Hybrid Routing

    In practice, dynamic routing is usually the preferred approach. Requests can be labeled by workload type, with real-time interactions sent to the high-speed tier and batch jobs sent to the standard tier.

    1. def route(request):
    2. if request.latency_sensitive: # Online conversations and Agent workflows
    3. return "highspeed"
    4. else: # Offline batch processing and async tasks
    5. return "standard"

    A fallback policy can also improve resilience. For example, when the high-speed tier is rate-limited, non-critical requests can automatically fall back to the standard tier. This helps balance availability and cost.

    4. Context Caching: An Often-Underestimated Cost Lever

    Because cache-hit pricing is only one-tenth of the regular input price, caching can have a substantial impact on total costs.

    The following practices are particularly useful:

    1. Place stable content at the beginning of the prompt. Put system prompts, tool definitions, and long document summaries first. The prefix must remain exactly identical for a cache hit; even a one-character change can invalidate the cache.

    2. Preserve the conversation prefix across turns. In a multi-turn conversation, the history naturally forms an expanding prefix. As long as messages are not reordered or removed, the cache-hit rate can remain high and stable.

    3. Select the cache lifetime based on traffic patterns. For high-frequency calls, such as several requests per second, the short-lifetime option may be sufficient. If requests can be more than five minutes apart but use the same long prefix—for example, scheduled jobs that repeatedly process the same document—the one-hour option may be worthwhile. Its higher write cost can be exchanged for cross-period cache hits.

    4. Monitor the hit rate and refine prompt structure. Cache-hit rate should be treated as an observable metric and included in cost dashboards. If the rate is low, first check whether the prefix contains frequently changing elements such as timestamps or random IDs.

    5. Cost Estimation Example

    Assume an application processes 10 million input tokens and 2 million output tokens per day. Suppose 80% of the input tokens are served through cache hits and all requests use the standard tier.

    • Input cost: 2 million × ¥13/MTok + 8 million × ¥1.3/MTok
      ≈ ¥26 + ¥10.4 = ¥36.4
    • Output cost: 2 million × ¥68/MTok = ¥136
    • Total: approximately ¥172 per day

    If all requests use the high-speed tier, the total would double to approximately ¥344 per day.

    This example shows that output tokens are the dominant cost component. Controlling output length—for example, by constraining response size and setting an appropriate max_tokens value—together with intelligent tier routing, provides two of the most cost-effective optimization opportunities.

    6. Conclusion

    Next-generation LLM API pricing is moving in three major directions:

    1. Base prices are gradually decreasing.
    2. Service tiers are becoming more differentiated, typically including standard and high-speed options.
    3. Context caching is becoming more granular, with separate cache-hit, cache-write, and multi-lifetime pricing.

    For developers, the practical approach is to first assess how sensitive each workload is to latency and use that assessment to design routing rules. Next, improve cache-hit rates through stable prefix design and appropriate cache-lifetime selection. Finally, continuously monitor token consumption and cache performance.

    Treating pricing as a first-class consideration in system architecture creates a feedback loop between architecture, billing, and observability. This is essential for maintaining a sustainable balance between model capability and operating cost.