Cost Structure and Model Selection Strategy for Pay-As-You-Go LLMs

I. Price Adjustments: Do Higher Capabilities Inevitably Lead to Higher Prices?

A recent price adjustment for a next-generation Large Language Model (LLM) inference service has sparked heated debates within the developer community: the output price is roughly five times higher than the previous generation, input prices have increased concurrently, and the cache hit price remains relatively low. The focus of this discussion is not a simple “is it too expensive?” judgment, but rather whether the unit price increase matches the capability improvements, and where this service stands among similar offerings.

To answer this question, we cannot merely look at the absolute values on the pricing table. We must return to three fundamental dimensions: the actual consumption per unit task, the proportion of billing components, and the conversion relationship between subscription quotas and actual usage. This article attempts to clarify this calculation from an engineering perspective.

II. Basic Components of the Billing Model: Input, Output, and Cache

Currently, mainstream LLM inference billing widely adopts a three-tier structure:

Billing Component	Definition	Typical Characteristics
Input	All tokens sent to the model in a request, including system prompts, historical context, and user queries.	Usually has the lowest unit price, but accounts for a massive proportion in long-context scenarios.
Output	Tokens generated by the model.	Has the highest unit price, typically several times that of input.
Cache Hit	The portion overlapping with historical request prefixes that can reuse the KV cache.	Has the lowest unit price, typically at the order of one-tenth of the input price.

The engineering logic behind this structure is clear: output tokens require token-by-token autoregressive decoding, occupying the most intensive compute power; input tokens only require a single forward pass and can be processed in parallel; while cache hit portions can skip the forward pass entirely, naturally resulting in the lowest cost.

Therefore, evaluating whether a model is “expensive” essentially means evaluating the weighted cost of these three components under your actual business workload. Simply comparing output unit prices can easily lead to misleading conclusions.

III. Beyond Unit Price: Recalibrating with “Cost Per Unit Task”

A key piece of information mentioned earlier is that in a specific authoritative benchmark test, the average token consumption per task has approached that of industry-leading models. The engineering implication here is that token consumption at the task granularity is increasing.

Why is it increasing? There are usually two reasons:

1. Longer reasoning chains. To improve accuracy on complex tasks, models tend to generate longer intermediate reasoning steps, naturally increasing output tokens.
2. Heavier context. Scenarios like multi-turn tool calling, codebase retrieval, and document analysis require repeatedly injecting large chunks of context, significantly amplifying input tokens.

The combination of these two factors means that “unit price × consumption volume” represents the true cost. A model with a higher output unit price but more concise reasoning might actually cost less than a cheaper model that is overly “verbose”—and vice versa. When making selection decisions, it is recommended to estimate using the following formula:

Cost per unit task ≈ Average input tokens × Input unit price
                   + Average output tokens × Output unit price
                   + Cache hit tokens × Cache unit price

In scenarios with long conversations or fixed system prompts, the cache unit price component will significantly reduce the total cost. This is an optimization point many teams easily overlook.

IV. Subscription Quota Conversion: An Underestimated Decision Variable

Besides pay-as-you-go, subscription plans are another important option. The core contradiction discussed here is that for plans at the same price point, the available quotas provided by different services vary drastically.

This involves a conversion issue. Assuming a subscription plan has a monthly fee of P, its “leverage ratio” can be defined as:

Leverage ratio = Plan available quota (converted to pay-as-you-go pricing) / Plan monthly fee

For industry-leading subscription services, the leverage ratio generally starts at 20x, with some premium plans reaching 50x or higher. This means that for every single subscription fee paid, the user actually gets dozens of times the usage volume compared to pay-as-you-go billing. The business logic behind this design is: locking in long-term users through prepayment, diluting marginal costs through economies of scale, and simultaneously constraining extreme usage with quota limits.

For developers, evaluating a subscription plan should not focus solely on the monthly fee number, but rather on three metrics:

• Leverage ratio: Quota / Monthly fee; the higher, the more cost-effective.
• Quota structure: Whether it distinguishes between input, output, and cache, and whether there are caps on individual items.
• Overage strategy: Whether exceeding the quota results in downgrading, rate limiting, or additional pay-as-you-go charges.

If a plan’s leverage ratio is significantly lower than its peers, even if its absolute price is not high, its “product value” will be discounted in the eyes of heavy users.

V. Cache Hit Pricing: The Deciding Factor in Long-Session Scenarios

Maintaining the cache hit unit price at a low level is of great significance for specific scenarios. Typical benefiting scenarios include:

1. Agent Applications with Fixed System Prompts

Many agent applications carry a lengthy system prompt in every request (role definition, tool definitions, output format constraints). If this prefix is cached, it will only be billed at the cache price in each conversation turn, reducing costs by an order of magnitude.

2. Codebase Q&A and Document Analysis

When developers repeatedly ask questions using an entire code repository or technical documentation as context, the prefix is highly overlapping. The caching mechanism makes “repeatedly reading the same material” inexpensive.

3. Multi-turn Long Conversations

As conversation turns increase, historical context continuously accumulates. If everything were billed at the input price, the cost would grow linearly or even super-linearly with the number of turns; cache hits flatten this growth curve.

Therefore, although the cache unit price is not the most prominent number on the pricing table, it is the most decisive variable in high-reuse, long-session scenarios. When evaluating any model service, the cache hit rate must be incorporated into the cost model.

VI. A Developer’s Framework for Model Selection

Synthesizing the above analysis, we can outline an actionable evaluation process:

Step 1: Quantify the Business Workload Profile

Gather statistics on real traffic, including average input length, average output length, distribution of conversation turns, and prefix overlap rate. This step is the foundation of all estimations; avoid replacing data with “feelings.”

Step 2: Build a Cost Model

Use the formula from Section III to calculate the cost per task under pay-as-you-go billing, and simulate sensitivity under different cache hit rates.

Step 3: Calculate Subscription Leverage Ratios

If anticipated usage is high, multiply the pay-as-you-go cost by the monthly usage volume, compare it with various subscription quotas, and calculate the actual leverage ratio.

Step 4: Stress Test Capability Boundaries

Beyond cost, it is essential to verify the model’s accuracy, stability, and latency on target tasks. A cheap model that requires multiple retries might actually incur a higher comprehensive cost.

Step 5: Design Fallback and Degradation Strategies

Architecturally reserve multi-model routing capabilities, assigning simple tasks to low-cost models and complex tasks to high-capability models, while using a caching layer to uniformly manage prefix reuse.

VII. Conclusion

The discussion triggered by a price adjustment appears to be a dispute over numbers on the surface, but in essence, it is a debate over the methodology of capability pricing. A unit price increase itself is not a problem; the problem lies in whether the increase is offset by capability improvements, cache optimizations, and subscription leverage.

For developers and enterprise users, the most pragmatic approach is not to agonize over the高低 of a specific number, but to establish a cost evaluation system based on real workloads: speak with data, measure subscriptions with leverage ratios, and optimize architectures with cache hit rates. Only by doing so can one continuously make rational technical selection decisions during a cycle of rapid model capability iteration.

After all, true product competitiveness is never the cheapest line item on the pricing table, but rather the effective intelligence delivered per unit of budget.