Multi-Model LLM API Cost Structure and Selection: A Practical Guide

    This article examines the cost structure of multi-model LLM APIs and presents a reusable framework for model selection. It covers input/output pricing asymmetry, cache-hit discounts, and cache storage costs; quantifies the impact of output tokens and cache usage with illustrative examples; and proposes tiered routing across lightweight, balanced, and flagship models. It also includes a Python cost-estimation function and engineering practices for token metering, budget circuit breakers, context compression, output constraints, and periodic price and usage reviews, helping teams make cost-effective production choices without sacrificing quality.

    As LLM applications move from prototype validation to production, API call costs have become a core engineering constraint alongside quality and latency. Many teams focus only on the single metric of “price per million tokens” during model selection, while overlooking asymmetric input and output pricing, discounts from caching mechanisms, and the hidden overhead of cache storage. Starting from the billing structure, this article establishes a reusable cost analysis framework to help developers make rational choices among multiple model tiers.

    1. Basic Dimensions of API Billing

    Mainstream LLM APIs generally bill along the following dimensions:

    • Input token unit price: prompts, context, retrieved snippets, and other content submitted by the user are counted on the input side.
    • Output token unit price: content generated by the model is counted on the output side and is usually significantly higher than the input price.
    • Cache hit unit price: when the context prefix of a request matches a previous request, the computed intermediate state can be reused and billed at a discounted price.
    • Cache storage fee: storage resources occupied to keep the cache valid are usually measured in “CNY per million tokens per hour.”

    Understanding how these four dimensions combine is a prerequisite for cost estimation. The following illustrative price list (CNY per million tokens) shows pricing patterns across model tiers:

    Model tier Input Output Cache hit Cache storage (CNY per million tokens/hour)
    Flagship reasoning 6.00 30.00 1.20 0.017
    Lightweight economy 0.80 2.70 0.16 0.017
    Balanced general-purpose 3.00 15.00 0.60 0.017

    Two key facts can be read from the table: first, output unit prices are generally 3–5 times input unit prices, so the generation side is the main cost driver; second, the cache-hit price is about 20% of the input price, making caching a core lever for cost optimization in long-context and repeated-prefix scenarios.

    2. Underestimated Cost Variables: Output Side and Cache

    2.1 Amplification Effect of Output Tokens

    Assume an application has 2,000 input tokens and 800 output tokens per request. At the flagship tier:

    1. Single-request cost = 2000/1e6 * 6.00 + 800/1e6 * 30.00
    2. = 0.012 + 0.024
    3. = 0.036 CNY

    Output accounts for 67% of the cost. This means cost savings from compressing output length and constraining response format—such as requiring structured JSON or limiting maximum generated tokens—are often more direct than trimming input. In engineering practice, set max_tokens explicitly and use prompts to prevent the model from producing redundant elaboration.

    2.2 Correct Use of Cache Hits

    The caching mechanism is most friendly to request patterns with a “fixed prefix + variable suffix.” Typical scenarios include fixed system prompts, fixed few-shot example sets, and fixed head segments of a knowledge base. If the prefix is 4,000 tokens and hits the cache, compared with a miss:

    1. Input cost on miss = 4000/1e6 * 6.00 = 0.024 CNY
    2. Input cost on hit = 4000/1e6 * 1.20 = 0.0048 CNY

    Single-request savings are about 80%. However, cache storage fees need attention: if the cache occupies 4,000 tokens and is retained for 24 hours, the storage cost is 4000/1e6 * 0.017 * 24 ≈ 0.0016 CNY, which is still worthwhile relative to the input cost saved. Engineering advice: place stable prefixes at the very beginning of the request, put variable content at the end, and match cache lifetime to business access frequency to avoid waste from “cached but unused” content.

    3. Quantitative Method for Tiered Model Selection

    Using a single model for everything is not economical. A more reasonable approach is tiered routing by task complexity:

    1. Lightweight economy: for short-output tasks such as intent recognition, text classification, simple extraction, and format conversion. Its input price is only about 1/7 of the flagship tier, making it suitable for high-concurrency, low-latency scenarios.
    2. Balanced general-purpose: for routine Q&A, summarization, and medium-complexity reasoning, balancing quality and cost.
    3. Flagship reasoning: only for quality-sensitive tasks such as complex reasoning, long-chain planning, and high-quality creation.

    A practical cost-estimation function is shown below:

    1. def estimate_cost(input_tokens, output_tokens, tier, cache_hit=False):
    2. price = {
    3. "lite": {"in": 0.80, "out": 2.70, "hit": 0.16},
    4. "turbo": {"in": 3.00, "out": 15.00, "hit": 0.60},
    5. "pro": {"in": 6.00, "out": 30.00, "hit": 1.20},
    6. }[tier]
    7. in_price = price["hit"] if cache_hit else price["in"]
    8. return input_tokens/1e6 * in_price + output_tokens/1e6 * price["out"]

    Before integration, run the tiered strategy against the real token distribution of online requests (rather than a guessed average). This usually reveals 30%–50% of optimization headroom.

    4. Engineering Practices for Cost Governance

    • Establish token metering instrumentation: record input, output, and cache-hit status for each request at the call layer, aggregate by business dimension, and build a cost dashboard.
    • Set budget circuit breakers: set daily/monthly quotas by tenant or feature module; when limits are exceeded, downgrade to the economy tier or queue requests to avoid runaway costs from abnormal traffic.
    • Compress context: use rolling summaries for historical conversations and relevance truncation for retrieved snippets to reduce ineffective input.
    • Constrain output format: use structured output and length caps to control generation-side overhead; this is one of the highest-ROI optimizations.
    • Periodically review pricing and usage mix: model tiers and pricing evolve with technology, so selection conclusions should be revalidated regularly with the latest prices and real traffic.

    5. Conclusion

    Cost optimization for multi-model APIs is not simply “choose the cheap one.” It is a systematic engineering effort built on a clear billing model: understand input/output asymmetry, use cache discounts well, route by task tier, and continuously tune with data. Only by making these four parts solid can unit business costs be controllable, predictable, and optimizable while ensuring quality.