Why Aren’t Open-Source LLM APIs Cheaper? A Cost Breakdown

    Open model weights do not eliminate the cost of delivering reliable, low-latency inference. This article breaks down usage-based API pricing, prompt caching, subscription plans, utilization thresholds, and self-hosting costs. It also provides a practical Python calculator and a decision framework for choosing between pay-as-you-go APIs, monthly plans, and self-deployment.

    1. If the Model Is Open Source, Why Does the API Still Cost So Much?

    As more high-quality large language models release their weights, a common question has emerged: if the code and weights are publicly available, why do official APIs still charge significant per-token fees? Is the pricing inflated?

    The key distinction is that open-source distribution covers the model itself, not the inference service. Running a model as a stable, low-latency, highly available online service requires GPU clusters, inference-framework optimization, traffic scheduling, monitoring, and ongoing operations. Most of the provider’s charges reflect these continuing costs.

    Once this distinction is clear, two questions become easier to analyze:

    • How is the weighted average cost of a pay-as-you-go API calculated?
    • Why can monthly plans offer a much lower effective per-token price?

    2. Usage-Based Pricing: Calculate the Weighted Average First

    2.1 Cost structure

    Most mainstream APIs charge per million tokens, or MTokens. Pricing usually distinguishes between input tokens, output tokens, and input tokens that benefit from prompt caching.

    The cost of a single request can be expressed as:

    1. Cost = uncached input tokens × input price
    2. + cache-hit input tokens × cache-hit price
    3. + cache-write tokens × cache-write price
    4. + output tokens × output price

    Here, “input tokens” refers to the portion that does not hit the cache. Cached input tokens are charged at the cache-hit price and are not charged again at the standard input price. Whether cache writes are billed separately depends on the platform’s pricing rules, so the official API documentation should be used as the source of truth.

    2.2 Why caching affects the average price

    In coding-assistance scenarios, system prompts and repository context may be repeated throughout the same session. The cache-hit ratio can therefore be relatively high. The actual ratio depends on the context structure and session length, so it should be measured using your own workload rather than copied from someone else’s estimate.

    A high cache-hit ratio can significantly reduce the average cost of input tokens. However, output tokens are often much more expensive than input tokens. As the output share increases, the overall average price rises again.

    2.3 A hypothetical calculation

    The following figures are assumptions for illustration only and do not represent any specific product:

    • Input price: ¥4 per MTokens
    • Cache-hit price: ¥0.8 per MTokens
    • Output price: ¥16 per MTokens
    • Total input: 0.8 MTokens
    • Output: 0.2 MTokens, producing a 4:1 input-to-output ratio
    • Cache-hit ratio: 98%

    The calculation is:

    • Uncached input: 0.016 M × ¥4 = ¥0.064
    • Cached input: 0.784 M × ¥0.8 = ¥0.627
    • Output: 0.2 M × ¥16 = ¥3.2

    The total cost is approximately ¥3.89. The total token volume is 1.0 MTokens, so the weighted average price is approximately ¥3.89 per MTokens.

    Output tokens account for most of the cost. Even with a 98% cache-hit ratio, the average price is still driven primarily by output generation.

    3. Monthly Plans: The Average Price Depends on Utilization

    3.1 Calculation method

    The effective plan price can be calculated as:

    1. Plan average price = monthly fee ÷ total included token volume

    This is the price obtained when the entire allowance is used.

    For example, suppose a plan costs ¥469 per month and includes approximately 7,800 MTokens. If the allowance is fully used, the average price is about ¥0.06 per MTokens, which is far below the hypothetical pay-as-you-go average of ¥3.89.

    Subscription plans often include restrictions, such as peak-hour rate limits or quota deductions based on usage periods. If each token used during peak hours consumes three times the normal quota, and all usage occurs during peak hours, the effective price becomes:

    1. ¥0.06 × 3 = ¥0.18 per MTokens

    Even this figure remains well below the usage-based average. However, the conclusion only holds when the plan is used close to its full allowance and no other deductions apply.

    A unified formula for the effective plan price is:

    1. Effective price = monthly fee × deduction multiplier
    2. ÷ (included volume × utilization rate)

    The deduction multiplier is the weighted average of the multipliers applied during different usage periods.

    3.2 Why can monthly plans be so much cheaper?

    Providers generally do not disclose their detailed cost structures. The following are common industry explanations and should be treated as hypotheses that cannot be externally verified:

    1. Risk transfer: With usage-based billing, the provider must maintain capacity for traffic spikes and include that capacity risk in the unit price. With prepaid subscription users, demand can be planned more accurately.
    2. Off-peak capacity reuse: Plans may include peak-hour throttling and faster service during off-peak periods. These rules can improve infrastructure utilization and spread hardware and operations costs across more requests.
    3. Economies of scale: When a provider serves a large concentration of users, fixed costs per inference can be distributed across a greater volume of usage.

    3.3 Break-even point: When is a plan cheaper?

    A subscription becomes more economical than pay-as-you-go usage when its effective price falls below the weighted API average.

    The break-even utilization rate can be calculated as:

    1. Break-even utilization =
    2. monthly fee × deduction multiplier
    3. ÷ (included volume × usage-based weighted average price)

    Using the assumptions above:

    • Without peak-hour deductions, the break-even utilization is approximately 1.5%.
    • If all usage occurs during peak hours with a 3× quota deduction, it is approximately 4.6%.

    These figures are only illustrative calculations. Real prices and terms may produce very different results, so they must be replaced with actual pricing parameters.

    3.4 Subscription plan conditions to check

    Before purchasing a plan, verify the following:

    • Rate limits and concurrency limits
    • Quota validity period and whether unused quota expires
    • Pricing after the included quota is exceeded
    • Peak-hour deduction multipliers
    • Available model versions and supported usage scopes

    4. Use Code to Calculate the Real Cost

    When evaluating a technology option, it is more useful to calculate the effective price using your own workload than to rely on promotional pricing claims.

    1. def api_avg_price(input_mtok, output_mtok, cache_hit_ratio,
    2. input_price, output_price, cache_price):
    3. """Weighted average price for usage-based billing (CNY/MTokens).
    4. input_mtok is the total input volume, including cache-hit tokens,
    5. measured in millions of tokens.
    6. """
    7. hit_mtok = input_mtok * cache_hit_ratio
    8. miss_mtok = input_mtok - hit_mtok
    9. cost = (miss_mtok * input_price
    10. + hit_mtok * cache_price
    11. + output_mtok * output_price)
    12. return cost / (input_mtok + output_mtok)
    13. def plan_effective_price(monthly_fee, included_mtok,
    14. utilization=1.0, peak_share=0.0,
    15. peak_multiplier=1.0):
    16. """Effective average price for a monthly plan (CNY/MTokens).
    17. utilization: quota utilization rate, from 0 to 1
    18. peak_share: share of usage during peak hours
    19. peak_multiplier: quota deduction multiplier during peak hours
    20. """
    21. k = (1 - peak_share) + peak_share * peak_multiplier
    22. return monthly_fee * k / (included_mtok * utilization)
    23. api_price = api_avg_price(
    24. input_mtok=0.8, output_mtok=0.2, cache_hit_ratio=0.98,
    25. input_price=4.0, output_price=16.0, cache_price=0.8)
    26. print(f"API weighted average price: ¥{api_price:.2f}/MTokens")
    27. print(f"Plan price (fully used, no peak deduction): "
    28. f"¥{plan_effective_price(469, 7800):.2f}/MTokens")
    29. print(f"Plan price (fully used, all peak usage with 3× deduction): "
    30. f"¥{plan_effective_price(469, 7800, peak_share=1.0, peak_multiplier=3.0):.2f}/MTokens")
    31. print(f"Break-even utilization (no peak deduction): "
    32. f"{plan_effective_price(469, 7800) / api_price:.1%}")

    All parameters above are hypothetical. Replace them with actual unit prices, measured cache-hit ratios, and real plan terms. The results are intended for estimation only.

    Three points are particularly important when performing the calculation:

    • The plan is cheap only when sufficiently utilized: If usage is low, the effective unit price rises quickly.
    • Include peak-hour terms: Model quota deduction multipliers and calculate a worst-case scenario.
    • Account for input and output structure: Coding workloads often have large inputs, smaller outputs, and favorable caching patterns. The same assumptions should not be applied directly to chat or content-generation workloads.

    5. A Decision Framework for Enterprise Adoption

    The analysis above suggests the following selection process:

    1. Measure the workload first: Track the team’s monthly token consumption, input-to-output ratio, and cache-hit ratio. These measurements are the foundation of any cost estimate.
    2. Match the billing model to the workload: High-frequency coding assistance with stable context may justify a monthly plan. Low-frequency or bursty online workloads that require flexible scaling are often better suited to a usage-based API.
    3. Calculate the full cost of self-hosting: GPU procurement or rental, inference optimization, high availability, monitoring, and model upgrade work are all part of the total cost. When daily usage is limited or there is no dedicated operations team, using a cloud inference platform is often simpler.
    4. Apply standard cost-control techniques: Shorten system prompts, improve cache-hit rates, use smaller models for non-critical tasks, and batch non-real-time workloads.

    The three approaches can be compared as follows:

    Approach Cost structure Elasticity Operations burden Main risks
    Usage-based API Pay per token with no fixed commitment High; scales with usage Low Bills can rise quickly as usage grows, and unit prices may change
    Monthly plan Fixed monthly fee; most economical when fully used Medium; subject to rate limits and peak-hour terms Low Low utilization increases the effective price, and terms may change
    Self-hosting Hardware or rental, personnel, and electricity Depends on internal capacity planning High Hidden costs are significant, and the organization owns upgrades and failures

    6. Conclusion

    The idea that an “open-source model is still expensive to use” reflects the cost structure of inference services. Usage-based API prices account for the marginal cost of providing elastic service, while monthly plans reduce the average price through committed usage and capacity scheduling.

    For developers and enterprises, the key is to calculate the real effective price using their own workload characteristics. The best choice among usage-based APIs, monthly plans, and self-hosting depends on which option delivers the lowest total cost for the required level of flexibility and operational effort.