LLM Selection on a Cost-per-Value Basis: A Practical Cost Modeling Methodology
Picking a large language model by comparing the headline "per million tokens" price on each vendor's pricing page is one of the most common and most costly mistakes teams make. Input and output prices are asymmetric, the same model can be listed at several times different prices across official endpoints, cloud marketplaces, and aggregation platforms, and hidden factors such as prompt caching, concurrency quotas, and rate limits reshape the final bill. This article lays out a repeatable cost model: how to weight input and output by real workload ratios, how to factor in cache hit rates, how to normalize currencies, why channel price differences appear, and a short Python implementation you can drop into your own selection tooling. It closes with a broader evaluation framework covering capability fit, stability, data compliance, and migration cost, plus a checklist for reproducible comparisons.
1. The Selection Trap: Why “Price per Million Tokens” Is Not Enough
When teams integrate an LLM provider, it is common to compare the “input price” or “output price” listed on each vendor’s pricing page and simply pick the cheapest one. In practice this approach tends to backfire, for three reasons:
- Input and output prices are usually asymmetric. Output tokens typically cost several times more than input tokens. If your workload is dominated by long-form generation, comparing only the input price will badly understate the real cost.
- The same model is listed at very different prices across channels. Official direct access, cloud-hosted offerings, and aggregation or proxy platforms can quote prices that differ by several multiples for the exact same model. Intuitive assumptions such as “third parties are always cheaper” or “the official endpoint is always the most expensive” are routinely disproved in real testing.
- Hidden factors shift the actual bill. Cache hits, concurrency quotas, and rate-limiting policies all change what you actually pay. Identical unit prices do not mean identical total spend.
The first step toward a sound selection, therefore, is to replace “look at the unit price” with “calculate the total cost.”
2. Establishing a Consistent Cost Model
Reliable side-by-side comparison requires a shared measurement basis first. Without one, the numbers are simply not comparable.
2.1 Weighting input and output
Model invocation cost comes mainly from two components: input tokens and output tokens. For a typical scenario you should weight them by real consumption ratio rather than taking a single unit price. Take agentic coding — read-heavy, long context, structured output — as an example: consumption roughly splits into 30% input / 70% output. The weighted unit cost can then be written as:
weighted unit cost = input price × input share + output price × output share
Prices are only comparable once every channel is evaluated with the same share ratios and the same currency conversion.
2.2 The effect of cache hits
Most providers offer some form of context caching: frequently reused prefixes, such as system prompts or long documents, are billed at a lower rate once they hit the cache. For multi-turn conversations or long-context reuse, the cache hit rate directly lowers actual cost. When modeling cost, maintain an additional “cached input price” field and estimate a hit ratio:
effective cost = (1 - hit rate) × total input tokens × input price+ hit rate × total input tokens × cached input price+ total output tokens × output price
2.3 Normalizing currency and conversion
Different channels may quote in different currencies. A cross-channel comparison must convert everything into a single currency using one fixed exchange rate, so that differing conversion assumptions do not distort the conclusion. Record both the exchange rate and the time the prices were captured in your comparison sheet so the results can be reproduced.
3. Why Prices Differ Across Channels
Once the measurement basis is fixed, it becomes easier to see where the differences come from. They fall into three main categories.
First, official direct access is not necessarily more or less expensive than third-party distribution. Intuitively one might assume that a proxy platform charges a premium for convenience, but the opposite is often observed in practice: some official direct prices are higher than promotional rates from certain cloud-hosted or aggregation platforms, while in other scenarios the official endpoint is the cheapest. There is no stable mapping between “channel type” and “price level.” The only reliable approach is to verify each item individually rather than rely on experience-based assumptions.
Second, flagship versions are not listed on every channel. When a new flagship model is released, third-party platforms often list it late, or offer only a lightweight or coding-specialized variant as a substitute. Comparing prices across different versions at that point produces meaningless conclusions. The correct approach is to explicitly mark the version as “not available” rather than fill the gap with numbers from a similar version — otherwise the credibility of the whole comparison table collapses.
Third, discount structures and tiered pricing differ. Discount levels, volume-based tiering, and free quotas vary widely between channels. Discount ratios themselves change with release cycles, so your comparison sheet should record, in addition to the unit price, the “discount tier relative to the official price” to make later re-checks fast.
4. A Cost Model You Can Actually Run
Turning the model above into code avoids manual calculation errors. The following illustrative implementation can be embedded directly into a selection tool:
def estimate_monthly_cost(pricing, request_count, avg_input_tokens, avg_output_tokens):"""Estimate monthly invocation cost (CNY) from the business workload.pricing: {'input': price, 'output': price, 'cache': cached price} # CNY per million tokensrequest_count: number of requests per monthavg_input_tokens / avg_output_tokens: average tokens per request"""input_total = request_count * avg_input_tokensoutput_total = request_count * avg_output_tokenscost = (input_total * pricing['input'] + output_total * pricing['output']) / 1_000_000return costdef estimate_with_cache(pricing, input_total, output_total, cache_hit_ratio):"""Cost after accounting for cache hits (CNY).cache_hit_ratio: share of input tokens served from cache, in [0, 1]"""normal_input = input_total * (1 - cache_hit_ratio)cached_input = input_total * cache_hit_ratiocost = (normal_input * pricing['input']+ cached_input * pricing['cache']+ output_total * pricing['output']) / 1_000_000return costdef normalized_unit_cost(pricing, in_ratio=0.3, out_ratio=0.7):"""Compute the weighted unit cost (per million tokens) used forcross-channel comparison. All channels must pass the samein_ratio / out_ratio for the results to be comparable."""return pricing['input'] * in_ratio + pricing['output'] * out_ratio
Suggested workflow: collect prices from every channel into a single pricing structure, call normalized_unit_cost with the same ratios for all of them to obtain comparable weighted unit prices, then use estimate_monthly_cost together with real workload numbers to compute total spend. Channels with a missing version should be recorded as None and excluded from ranking.
5. From Cost Model to Decision: An Evaluation Framework
Calculating cost is only the first step. A real selection decision needs to weigh other factors as well. A reusable evaluation framework should include these dimensions:
- Cost: the weighted unit cost plus a monthly total estimate derived from your actual workload.
- Capability fit: whether the required version (flagship / lightweight / specialized) is available, and whether context length and structured output meet your needs.
- Stability: rate-limiting policies, concurrency quotas, and SLA commitments — to avoid a service that is cheap but frequently throttled.
- Data compliance: where data is stored, whether it is used for training, and what logging and audit capabilities exist. This is especially critical for enterprise scenarios.
- Migration cost: whether the API is compatible with mainstream protocols and whether switching channels requires code changes.
Treating weighted cost as one cell in this grid rather than the sole criterion is what prevents decisions that trade stability for a few saved dollars.
6. A Practical Decision Checklist
- Does every channel use the same input/output ratio and the same exchange rate?
- Are missing versions explicitly marked as “not available” instead of substituted with a similar version?
- Is the impact of the cache hit rate on total cost included?
- Have rate limits and concurrency quotas been verified, not just the unit price?
- Have data compliance and migration cost been assessed?
- Does the comparison sheet record the capture time and price source so results are reproducible?
7. Summary
The core tension in LLM selection has never been “which provider is cheapest” but “how to calculate cost on a consistent basis and assess risk comprehensively.” Channel type does not map reliably to price level, flagship availability varies, and discount structures keep changing — which means intuition is not dependable. Only a standardized, reproducible modeling process leads to decisions that hold up under review in high-consumption scenarios such as agentic coding. Bake the model and script above into your own selection toolchain, so every price comparison is made with the same ruler and your limited budget goes to the service that best matches your requirements.
