Large Model Pricing Divergence: Scale, Cost, and Model Selection Logic

1. The Phenomenon: Parameters Head Toward Trillions, Pricing Diverge

Over the past year, it has become a clear industry trend for flagship large models to move from hundreds of billions of total parameters into the trillion range. With Mixture-of-Experts (MoE) architectures, a model can greatly increase total parameter count while keeping the fraction of parameters actually activated per token relatively small, balancing capability against inference cost. Yet pricing strategies have diverged sharply: one class of new models lowers input and output unit prices even as parameter scale grows, while another class of next-generation flagships raises prices significantly, with output unit prices reaching several times those of the previous generation—a striking contrast of “same generation, higher price.”

This divergence is not accidental. Behind it lies a three-way interplay of inference cost structure, the evolution of technical efficiency, and business strategy. Understanding these three forces is more valuable than chasing any single price adjustment.

2. Where Inference Costs Come From

To understand pricing, you first need to understand the cost structure of a single large model call.

1. Compute cost: The computation required to generate each token is roughly proportional to the number of activated parameters. No matter how large the total parameter count, if the activation ratio is low, the increase in floating-point operations per token is limited; conversely, if activated parameters rise noticeably, or if longer reasoning chains are introduced (such as chain-of-thought or multi-step tool calls), compute overhead can rise severalfold.
2. Memory and bandwidth cost: The decoding phase is a classic memory-bound process—each token generated requires reading the model weights and the growing key-value (KV) cache. The longer the context and the higher the concurrency, the greater the pressure on high-bandwidth memory and interconnect networks, which are often the most expensive resources in an inference cluster.
3. Cluster and availability cost: To guarantee low latency and high availability, providers must reserve compute capacity to handle peaks; idle rates are themselves a cost. Data center, network, and operations amortization are also factored into the per-token price.
4. Training cost amortization: Training a trillion-parameter model requires enormous investment, and this cost is amortized into API pricing at varying ratios—one reason frontier flagships often carry higher prices.

3. Input, Output, and Cache: The Economics of Three-Tier Pricing

Mainstream model services generally bill per million tokens and distinguish three price tiers—input, output, and cache hits—with differences between tiers reaching severalfold or even an order of magnitude. This is not arbitrary pricing:

• Input is cheap: The prefill stage processes the entire prompt in parallel in one pass, with high arithmetic intensity and full batching, resulting in low marginal cost.
• Output is expensive: Autoregressive decoding generates token by token; each step must fully read weights and cache, bandwidth is constrained, parallelism is low, and output length is unpredictable, making capacity planning costly. In practice, output unit prices are typically several times input prices.
• Cache hits are extremely cheap: Prefix caching stores and reuses the KV cache of repeated prompt prefixes (such as system prompts and fixed examples). The hit portion requires almost only minimal compute and storage overhead, so it can be priced far below regular input.

With this pricing structure understood, the optimization direction becomes clear: make the expensive part (output) shorter, and hand the reusable part (stable prefixes) to the cache.

4. Why Some Get Cheaper as They Grow, and Others More Expensive

The divergence between parameter scale and price depends on the relative speed of three curves:

1. Efficiency curve: If optimizations such as architecture sparsity, KV cache compression, low-precision quantization, speculative decoding, and better parallel scheduling move fast enough, per-token cost can actually fall as scale grows—producing “double the parameters, discounted the price.”
2. Capability premium curve: Next-generation flagships often bring stronger reasoning, longer context, or more inference-time compute (the model generates many hidden reasoning tokens in the background). Actual resource consumption far exceeds the nominal increase in parameters, and the cost naturally passes through to price.
3. Business strategy curve: Low prices are a lever for customer acquisition and ecosystem building; high prices are pricing for frontier capability and scarce supply. The coexistence of both strategies itself shows the industry is still evolving rapidly.

For observers, rather than obsessing over any single price change, it is better to treat price as “a snapshot of current supply capability and the competitive landscape”—it will keep changing.

5. How Developers Should Reasonably Estimate and Select Models

Facing fluctuating prices and rapidly iterating models, it is advisable to establish a lightweight but rigorous cost evaluation process.

Step 1: Build a cost model. The cost of a single call can be approximated as:

Cost ≈ (uncached input tokens × input price + cached input tokens × cache price + output tokens × output price) × call volume

A few lines of code can convert business traffic into a monthly bill (illustrative):

def monthly_cost(daily_req, in_tok, out_tok, cache_ratio,
                 p_in, p_out, p_cache):
    """Estimate monthly cost using per-million-token prices (illustrative)"""
    M = 1_000_000
    per_req = (in_tok * (1 - cache_ratio) / M * p_in
               + in_tok * cache_ratio / M * p_cache
               + out_tok / M * p_out)
    return per_req * daily_req * 30

Step 2: Compare “effective capability per dollar” instead of unit price. Run small-scale evaluations on your own real task set, recording quality scores, latency, and token consumption, and compute output per unit cost. This is far more reliable than looking only at a price sheet.

Step 3: Tiered routing. Assign simple tasks (classification, extraction, format conversion) to small models, and escalate to flagship models only for complex reasoning. This can significantly dilute overall cost:

def route(prompt):
    if estimate_complexity(prompt) < THRESHOLD:
        return SMALL_MODEL      # low-cost tier
    return FLAGSHIP_MODEL       # capability tier

Step 4: Engineering-side cost reduction. This includes constraining output length and requiring structured output; placing stable prefixes such as system prompts and few-shot examples at the beginning of the prompt to improve cache hit rates; using batch interfaces’ lower-price windows for offline batch jobs; and trimming context properly for multi-turn conversations.

Step 5: Preserve substitutability. Abstract model calls at the gateway layer, record token usage and cost per call, and attribute them by business line. Current mainstream cloud platforms (such as Baidu AI Cloud) generally offer model services along with monitoring and routing tools, which can be included in a unified evaluation during model selection. When prices or capability landscapes change, you can switch at low cost and avoid being locked into a single supply.

6. Conclusion

The sharp fluctuations in large model pricing reflect a tug-of-war among scale competition, inference efficiency, and business strategy. For developers, rather than predicting price trends, it is better to keep certainty in your own hands: understand your bill structure, choose models with data rather than intuition, use engineering to lower unit output cost, and treat models as components that can be replaced at any time. When price is no longer the only decision variable, “effective capability per dollar” becomes the stable yardstick that carries you through model iteration cycles.