LLM Price Increases Explained: Parameter Scale, Inference Costs, and Engineering Trade-offs

From the Parameter Race to Visible Inference Costs

Over the past two years, the parameter counts of foundation models have kept climbing—from hundreds of billions to trillions, and on to even larger mixture-of-experts (MoE) architectures. As model capability improves, the compute consumed by a single inference request has grown substantially. With larger sparse-activation structures, each request touches more expert networks, more attention computation, and more communication overhead.

When the industry discusses “price increases,” the conversation usually stops at comparing input and output unit prices, ignoring the fact that the underlying inference cost structure has changed. A more direct way to see it: when parameter counts grow several times over but per-token inference latency and hardware footprint do not fall proportionally, a provider that keeps the old price must absorb a significantly larger compute bill. A price increase, in essence, is the rebalancing of inference resource consumption on the pricing side.

So when a new-generation model is priced at roughly CNY 100 per million output tokens, CNY 20 per million input tokens, and CNY 1 per million cache-hit tokens, the more valuable question is not “is this expensive or cheap,” but what engineering problem each of the three price tiers actually corresponds to.

What Input, Output, and Cache-Hit Pricing Actually Mean

1. Input pricing: longer context means more compute

In Transformer-style architectures, input tokens pass through multiple layers of attention. For long-context workloads in particular, attention complexity can grow roughly quadratically with sequence length. Even with sparse attention, sliding-window attention, or mixed-precision strategies, long inputs still create significant pressure on GPU memory, the KV cache, and compute time.

From an engineering standpoint, input tokens are not read-once data. At every layer they interact with existing state, and their key-value pairs must be maintained throughout inference. If a model is hundreds of layers deep, the total compute contributed by input tokens can be approximated as:

input_cost ≈ num_layers × attention_cost_per_token × seq_len

So when a new model charges CNY 20 per million input tokens, the core message is this: long-context inference is no longer just “reading a bit more text”—it materially raises the provider’s compute and memory costs.

2. Output pricing: generation costs more than comprehension

Output tokens cost more than input tokens—this is broadly accepted across the industry. The reason is that output tokens must be generated one at a time, with each step depending on the previous one, which limits parallelism. For large models, generating a single token triggers a full forward pass, tightly constrained by memory bandwidth and communication latency.

With autoregressive inference, the computation for each output token can be sketched roughly as:

for step in range(max_output_len):
    logits = model.decode(step)
    next_token = sample(logits)

When output length grows from a few hundred tokens to several thousand, the provider’s continuous decoding time multiplies accordingly. For requests that demand detailed code explanations, long-form writing, or elaborate reasoning chains, output-side latency and hardware occupancy can exceed input-side costs by several times. Output pricing above input pricing is therefore not a marketing tactic; it reflects the real consumption of inference resources.

3. Cache hits: from recomputation to state reuse

Cache-hit pricing is dramatically lower than input or output pricing, which signals that the provider now reuses state from part of the prefix computation. In multi-turn conversations and fixed system-prompt scenarios, if the KV state of earlier tokens can be retained, subsequent requests no longer need to recompute the same prefix.

Technically, this kind of optimization relies on key-value caching. The basic idea:

if prefix_cache_hit:
    reuse_kv(prefix)
else:
    compute_kv(prefix)

When users repeatedly send the same persona setup, conversation history, or tool descriptions, the server only runs inference on the new content, cutting costs dramatically. Pricing cache hits separately at CNY 1 per million tokens effectively re-bills requests that do not re-consume compute resources, bringing prices closer to actual consumption.

Note that cache hits do not apply in every scenario. If the context changes frequently, prefixes differ between requests, or the inference cluster reschedules instances, the cache reuse rate drops noticeably. When estimating costs, factor in whether your workload has a stable prefix and long fixed prompts.

Parameters Double, Prices Rise Fivefold: Is There a Linear Relationship?

Judging by parameter counts alone—scaling from one trillion to nearly three trillion—several-fold price increases might seem reasonable. Strictly speaking, though, LLM inference cost is not simply proportional to total parameters.

With mixture-of-experts architectures, each token activates only a subset of experts, so in theory compute grows more slowly than total parameters. In practice, several factors offset this:

1. More layers: every additional layer adds activation reads/writes and intermediate state storage;
2. Routing overhead: more experts mean more routing computation, load balancing, and communication;
3. Tensor and pipeline parallelism: larger models are split across more devices, sharply increasing communication volume;
4. Service stability requirements: controlling first-token latency and concurrent throughput demands more redundant compute;
5. Training and alignment investment: training, reward modeling, safety alignment, and evaluation for larger models also feed indirectly into pricing.

Even with sparse activation trimming part of the compute, aggregate engineering costs can climb fast. Whether a several-fold price increase is justified depends on how much effective capability the provider actually delivers—not on parameter counts alone.

Can Capability Gains Cover the Price Hike? Evaluate Against Task Value

For developers and enterprises, deciding whether a new model is worth adopting should not hinge on “is it expensive,” but on the cost per unit of effective output. Effective output includes:

• Accuracy on complex reasoning tasks;
• Factual consistency and structural stability in long-form generation;
• Recall capability over long contexts;
• The reduction in retries and manual review within a specific business pipeline;
• Total time and resource consumption per task.

For example, a model with a higher output price may still lower total spend if it completes in one call what previously took three attempts. Conversely, if the price increase mainly reflects parameter scale and brings no stable gains on your target tasks, adopt with caution.

A pragmatic evaluation method is to build an internal benchmark set and log every call along these dimensions:

effective_cost = total_api_cost / success_tasks
latency_per_task = end_time - start_time
retry_rate = retry_count / total_requests

Only by comparing these metrics across models can you tell whether “doubled prices” actually buy “doubled productivity.”

Different Teams Will Respond in Different Ways

Price changes naturally push engineering approaches apart. Budget-sensitive teams with stable input prefixes will lean hard on cache hits—front-loading system prompts, tool definitions, and fixed context to avoid recomputation. Teams with output-heavy workloads can constrain output length, adopt structured return formats, and cache intermediate results to curb generation costs.

Another group will keep more compute in-house. For fixed tasks or latency-insensitive offline batch processing, running inference on self-hosted or managed clusters sidesteps per-token billing. But this requires model deployment, memory optimization, batch scheduling, and operations expertise—a nontrivial investment.

A third approach is tiered invocation by capability: route complex reasoning to premium models, and handle simple extraction, classification, or format conversion with low-cost models or rule-based templates. This keeps overall pipeline costs under control while reserving stronger capability for critical nodes.

Beyond Price: Watch the Engineering Evolution of Inference Efficiency

LLM pricing is not an isolated market move; it reflects the maturity of inference infrastructure. As techniques like KV caching, speculative decoding, quantized inference, continuous batching, and dynamic batch sizing become widespread, the cost per token tends to fall over time. In other words, today’s high prices for a given model generation are not necessarily the final steady state.

The thing to keep watching: after parameter expansion, can providers bring prices back down while maintaining output quality—by compressing inference paths, raising cache hit rates, and reducing per-forward overhead? If optimization moves fast enough, the current price bump is a short-term cost of a capability leap; if optimization chronically lags parameter growth, the market will eventually reassess what larger models are actually worth.

For developers, the goal is not to pick between “price-sensitive” and “capability-first,” but to build the ability to decompose model usage costs. Understanding what input, output, and cache-hit pricing each correspond to in today’s inference systems is what enables sound trade-offs between business growth and budget control.

Conclusion

Behind the price increases of new-generation LLMs lie parameter expansion, the surfacing of real inference resource consumption, and a redistribution of engineering optimization headroom. Generating output tokens costs more than processing input, cache hits offer a significant cost reduction opportunity, and together the three form a finer-grained billing structure.

For enterprise users and developers, rather than simply calling it “expensive,” return to your actual business pipeline and answer three questions: Does the task truly need this capability level? Is there a cacheable, stable prefix? Is the effective cost per unit of output acceptable? Only after evaluating along these dimensions does model selection stop being a reaction to price tags and become a genuinely measurable engineering decision.