Low-Cost LLM Inference and the Intelligence Ceiling: A New Divide in the Model Race

    This article explains how to evaluate low-priced large language models. It covers how to calculate inference costs from token pricing, how to estimate the cost of each correct result after accounting for failures, a checklist for model selection, and why post-training still matters after price cuts.

    1. How Do You Calculate Inference Cost?

    Large model APIs usually charge by tokens, with separate prices for input and output. The cost of a single call is calculated as follows:

    Cost per call = average input tokens ÷ 1,000,000 × input price + average output tokens ÷ 1,000,000 × output price

    Monthly cost = average daily calls × 30 × cost per call

    Here is an example with assumed numbers. A customer service system handles 1 million calls per day. Each call averages 1,500 input tokens and 500 output tokens. The input price is 1 CNY per million tokens, and the output price is 2 CNY per million tokens. The cost per call is 0.0015 CNY for input plus 0.001 CNY for output, or 0.0025 CNY in total. That comes to about 2,500 CNY per day, or about 75,000 CNY per month. These prices are only examples and do not reflect any vendor’s actual pricing. Use the current prices on the official website when you run your own calculations.

    Actual bills also depend on cache hits, pricing tiers for long contexts, and whether the internal output tokens of reasoning models are billed. Use your real call logs as the basis for cost estimates.

    2. Does a Lower Unit Price Always Mean a Lower Cost per Correct Result?

    Not necessarily. The unit price reflects only what a single call costs. What the business cares about is the total cost of completing one task. A common formula is:

    Cost per correct result = (cost per call + failure rate × manual handling cost per failure) ÷ task pass rate

    Using the same example numbers, assume manual handling costs 0.01 CNY per failure. Model A costs 0.0025 CNY per call and has an 80% pass rate, so each correct result costs about 0.0056 CNY. Model B costs 0.0012 CNY per call and has a 55% pass rate, so each correct result costs about 0.0104 CNY. Model B has the lower unit price, but it fails more often, so its overall cost is higher.

    The pass rate must come from tests on your own business samples. Do not use public leaderboard scores in its place.

    3. How Do You Choose a Low-Cost Model?

    Use the table below to check each candidate model item by item.

    Check item What to confirm Why it matters
    Unit price Input and output prices listed separately, and whether cache-hit pricing is itemized Determines the structure of the monthly bill
    P95 latency Measured values during peak hours, not only the average figure on the official website Determines the real-time interaction experience
    Rate limits QPS, TPM, and concurrency caps, and how the service responds when limits are exceeded Determines whether the service can run stably during peak periods
    Context length Stated maximum and effective length, and whether accuracy drops on long texts Determines whether long-text scenarios are viable
    Post-training evaluation Results and test conditions for instruction following, math, and code Determines whether the low price comes with usable quality
    SLA and fallback Availability commitments, compensation for timeouts, and how to switch to a backup model Determines whether the business can recover during outages

    Test each candidate model on a sample set drawn from real business traffic. Run at least several hundred requests per model, and record the latency distribution and pass rate. Include peak hours in the tests, because latency measured during quiet periods is usually optimistic.

    4. Does Post-Training Still Matter After Price Cuts?

    Yes. When unit prices fall but model capability stalls, the savings are quickly consumed by debugging and rework. Post-training, including instruction fine-tuning and reinforcement learning from human feedback (RLHF), is one of the main ways to improve model usability. It mainly affects instruction following, mathematical reasoning, and code generation.

    Verify post-training results yourself instead of relying on vendor claims. A more reliable approach combines public benchmarks with your own evaluation set. Public benchmarks make cross-model comparison easier, while a custom evaluation set reflects your specific business. When evaluating, focus on three points: whether instruction following is stable, whether the output format is easy for programs to parse, and whether answers stay consistent when the same question is asked repeatedly.