Engineering Evaluation: Pay-As-You-Go vs. Subscription Pricing for LLMs

1. The Core Issue: Higher Unit Prices Do Not Equal Higher Costs

When a recent generation of LLM inference services announced new pricing, output-side unit prices surged several times higher than the previous generation, with input and cache-hit prices also rising proportionally. Developers often react by thinking it is too expensive. However, from an engineering perspective, the unit price alone does not provide a valid basis for decision-making. The real question to ask is: Does the total cost of completing the same business task increase or decrease?

This question is complex because it involves three layers of variables simultaneously:

• Unit Price Layer: The respective prices for input, output, and cache hits.
• Consumption Layer: How many tokens a single task consumes, and the ratio of input to output.
• Efficiency Layer: Whether the model can complete the task in fewer rounds and with shorter context.

Making judgments based solely on the unit price layer will almost inevitably lead to the wrong conclusion.

2. Cost per Task: The Only Meaningful Benchmark

When comparing costs across models, the most reliable metric is Cost per Task, rather than the price per million tokens. Its definition can be written as:

Cost per Task = Σ(Input tokens × Input unit price)
             + Σ(Output tokens × Output unit price)
             + Σ(Cache hit tokens × Cache unit price)
             + Extra consumption from retries and failures

This formula looks simple, but each item requires empirical data. A common evaluation method in the industry is to use a standardized set of tasks, record the average number of input, output, and cache hit tokens consumed per task, and multiply them by their respective unit prices.

The key insight is that models with stronger reasoning capabilities often consume fewer tokens per task. Reasons include:

1. Fewer clarification rounds, providing usable results in a single pass.
2. Fewer self-correction loops.
3. A higher success rate of getting it right the first time in scenarios like code generation and structured output.

Therefore, if the unit price increases several times while the token consumption per task drops to a fraction of the original, the total cost might actually decrease. This is the technical meaning behind a model being “justifiably expensive.”

3. Subscription Conversion: Calculating Equivalent Usage

Besides pay-as-you-go, subscription plans are another main pricing track. Developers should not focus on the subscription price itself, but rather on the equivalent monetary value of the subscription quota when converted to pay-as-you-go billing.

A practical conversion method is:

Equivalent usage = Subscription quota ÷ Average blended unit price (pay-as-you-go)
Cost-effectiveness multiplier = Equivalent usage ÷ Subscription price

Consider an illustrative calculation. Assuming a model has a high output unit price and a low input unit price, and the output accounts for about 30% of actual business operations, the blended unit price can be approximated as:

Blended unit price ≈ Input unit price × 0.7 + Output unit price × 0.3

Dividing the total tokens included in the subscription plan by the blended unit price yields the equivalent usage. Dividing this value by the subscription price provides a rough cost-effectiveness multiplier.

In engineering practice, the multipliers of subscription plans vary greatly across providers. Some plans offer equivalent usage equivalent to several or even dozens of times the sticker price, while others are close to 1:1. The core metric for judging a subscription plan’s value is this multiplier, not the nominal price.

4. Cache Hits: An Underestimated Cost Lever

In scenarios like multi-turn conversations, long-document QA, and codebase comprehension, the cache hit rate can impact total cost more than the unit price itself.

The basic logic of the caching mechanism is that when a request’s prefix highly overlaps with historical requests, this portion is billed at the cache unit price, which is typically much lower than the standard input unit price. Therefore:

Effective input cost = Cache hit tokens × Cache unit price + Cache miss tokens × Input unit price

To improve the cache hit rate, several actionable engineering methods exist:

• Stable Prefixes: Place system prompts, tool definitions, and fixed context at the very beginning of the request to avoid changes every time.
• Reduce Prefix Perturbation: Do not insert volatile fields like timestamps or random IDs into the system prompt.
• Control Context Growth: Regularly prune or summarize historical messages to prevent the continuous accumulation of invalid content.

In long-context scenarios, increasing the cache hit rate from 30% to 70% often yields a more significant cost reduction than switching to a cheaper model.

5. Input-Output Ratio: The Key Parameter Defining Cost Structure

The input-output ratio varies drastically across different business types, directly determining which pricing structure is most favorable.

Business Type	Approx. Input:Output Ratio	Cost Sensitivity
Long-document summarization	20:1 or higher	Input unit price, cache hits
Code generation & completion	3:1 to 10:1	Input unit price, first-pass success rate
Open-ended conversation	2:1 to 5:1	Output unit price, round control
Structured extraction	10:1 or higher	Input unit price, output length constraints

As shown above, for input-intensive businesses, the impact of an output unit price increase is relatively limited. Conversely, for conversational and creative businesses, the output unit price is almost decisive.

Therefore, the first step in evaluating whether a price adjustment negatively impacts your operations is to measure the true input-output ratio of your business, rather than applying conclusions from others.

6. Actionable Cost Evaluation Process

Integrating the dimensions above forms a standardized evaluation process:

Step 1: Collect Baseline Data

Run real traffic on the current model for a week. Record the input tokens, output tokens, cache hit tokens, retry counts, and failure rates for each task type.

Step 2: Build a Cost Model

def task_cost(input_tokens, output_tokens, cached_tokens,
              in_price, out_price, cache_price, retry_rate=0.0):
    base = (input_tokens * in_price
            + output_tokens * out_price
            + cached_tokens * cache_price)
    return base * (1 + retry_rate)

Substitute the actual parameters of each task to obtain the current Cost per Task.

Step 3: Recalculate with New Model Parameters

Note that you cannot simply replace the unit price; you must also replace the consumption volume. If the new model has stronger reasoning capabilities, the number of input and output tokens will likely change, requiring empirical testing through a small-scale A/B test.

Step 4: Calculate Subscription Equivalent Value

Compare the total pay-as-you-go cost against the subscription price and its included quota to obtain the cost-effectiveness multiplier.

Step 5: Perform Sensitivity Analysis

Conduct interval testing on three variables: cache hit rate, output length, and retry rate to see if the conclusion holds steady. If the conclusion reverses within a reasonable fluctuation range, the basis for the decision is insufficient.

7. Common Pitfalls

Pitfall 1: Sorting solely by unit price. If the cheapest model consumes three times as many tokens, the total cost will actually be higher.

Pitfall 2: Comparing sticker prices with subscriptions. There is no fixed relationship between a subscription’s equivalent usage and its sticker price; it must be calculated item by item.

Pitfall 3: Ignoring caching. In long-context scenarios, the optimization potential of caching strategies is often greater than switching models.

Pitfall 4: Using a single task for judgment. Cost structures vary greatly across task types; they need to be weighted by the true distribution of your business.

Pitfall 5: Ignoring failure and retry costs. A failed request still consumes tokens; a high failure rate significantly inflates the effective cost.

8. Conclusion

Model pricing adjustments are the norm, as are unit price increases. For developers, the key is not to memorize a specific price figure, but to master a reusable evaluation method: Use Cost per Task as the core metric, measure subscription value via equivalent usage, leverage cache hit rates and input-output ratios for optimization, and ensure robust conclusions through sensitivity analysis.

By adhering to these practices, you can select the truly suitable solution within budget constraints regardless of how pricing changes, rather than being led astray by raw unit prices.