How to Choose an LLM: Quantifying Usage Costs and a Practical Guide to Cutting Costs

    This article presents a data-driven framework for choosing large language models based on usage cost rather than intuition. It defines three core metrics — usage volume (token throughput), cost, and cost-performance ratio — and explains how to break down input versus output tokens, apply tiered pricing, and interpret the usage-to-cost share ratio under consistent calibers. It then lays out a three-step quantitative evaluation framework (data collection, layered statistics and concentration analysis, and cross-comparison), a pre-scale selection checklist, a three-layer cost monitoring system, four cost-optimization strategies (model tiering and routing, caching, prompt trimming, and batching), guidance on balancing quality against cost, and five common mistakes to avoid. Readers will learn to turn model selection into a reproducible, auditable process that stably meets needs at a lower unit cost.

    1. Turning “It Just Works” Into Metrics

    In production, “which large model is the most convenient to use” is usually a blend of subjective impressions: some people look at output quality, some at time-to-first-token and throughput, and some at unit price. But as an application moves from a demo to scale, small differences in unit price that look negligible get amplified by huge call volumes into a significant cost gap.

    Models at the same capability tier can have unit token costs that differ by several times depending on each vendor’s pricing strategy. And the ratio of “share of call volume” to “share of cost” often reflects the economics of a solution more truthfully than unit price alone. This article makes no product recommendations; it offers a general framework for making decisions with data rather than intuition.

    2. Three Core Metrics

    2.1 Usage Volume (Token Throughput)

    A token is the basic unit by which large models measure and process text. As a rough estimate: in Chinese, each character corresponds to roughly 1 to 2 tokens; in English, about 4 characters or 0.75 of a word equals about 1 token. It must be emphasized that this is only an order-of-magnitude reference — different models’ tokenizers segment Chinese, code, and mixed multilingual text quite differently, so actual figures should be measured with the model’s official tokenizer. Don’t budget using someone else’s conversion factor.

    Usage is typically counted as “input tokens + output tokens,” and the two are handled separately in both metering and billing. Long-context scenarios (document summarization, codebase understanding, multi-turn conversations) are dominated by input tokens, while generation scenarios (content creation, marketing copy) are dominated by output tokens. So evaluating usage can’t stop at the total; you also need to break down the structure.

    2.2 Cost

    Cost is usage multiplied by unit price — the hard metric that actually affects the budget. The industry generally bills by token, with several common characteristics:

    • Input and output unit prices differ, with output usually more expensive;
    • Most vendors offer tiered pricing, with larger call volumes earning bigger discounts;
    • Capabilities like context caching and batching can significantly reduce the cost of repeated computation;
    • High-capability models (complex reasoning, long-chain tasks) generally cost more per token than lightweight models.
    1. def estimate_cost(input_tokens, output_tokens, price_table):
    2. """Illustrative cost estimate; unit prices and discounts are subject to the actual terms of service"""
    3. in_unit = price_table["input"] # input unit price per 1K tokens
    4. out_unit = price_table["output"] # output unit price per 1K tokens
    5. return (input_tokens / 1000.0) * in_unit + \
    6. (output_tokens / 1000.0) * out_unit

    An example cost calculation table (unit prices and cache discounts are illustrative, not any vendor’s actual quote):

    Option Input tokens Output tokens Cache hit rate Cost per call Monthly calls Monthly cost
    Full direct calls 2000 300 0% 2000/1000×0.008 + 300/1000×0.032 = 0.0256 CNY 1 million 25,600 CNY
    Prefix cache + output trimming 2000 220 50% (hit portion billed at 0.25× input unit price) 0.0100 + 0.0070 = 0.0170 CNY 1 million 17,040 CNY

    Under the same workload, the monthly cost drops by about 33%. The point of this table is not to copy the numbers, but to replace them with your own unit prices and real token distribution and recompute.

    2.3 Cost-Performance Ratio

    An intuitive definition:

    1. Cost-performance ratio = usage share ÷ cost share
    2. where: a solution's usage share = its usage ÷ total usage of all solutions within the same statistical scope
    3. a solution's cost share = its cost ÷ total cost of all solutions within the same statistical scope

    Caliber requirements (otherwise the ratio is neither reproducible nor comparable): the same time window (e.g., the same calendar month), the same statistical scope (all models under the same business line or account), the same price snapshot, and the same currency. Data across different months, business lines, or price versions cannot be divided directly.

    • Ratio > 1: “used a lot, spent little” — high unit-cost efficiency;
    • Ratio = 1: usage and spend are roughly in sync;
    • Ratio < 1: “not used much, yet spent a fair amount” — usually corresponds to a premium solution with high quality and high unit price.

    Example: A business line’s models together process 1 billion tokens and cost 200,000 yuan in a given month. Solution A calls 300 million tokens at a cost of 30,000 yuan — shares of 30% and 15%, ratio 2.0; Solution B calls 50 million tokens at a cost of 60,000 yuan — shares of 5% and 30%, ratio about 0.17. The two have completely different positioning.

    Two clarifications: First, this ratio is not financial ROI — it only describes the relative relationship between the usage share and the cost share, and doesn’t include business returns like revenue or conversion. Second, some low-price, high-throughput solutions may show a usage share higher than their cost share, while some high-end solutions may show the opposite; but any specific conclusion must be validated against the actual rankings you pull yourself — don’t apply outside impressions.

    3. A Three-Step Quantitative Evaluation Framework

    3.1 Data Collection

    Build a trustworthy data source. A recommended unified set of fields to collect:

    Field group Fields
    Identification request_id, timestamp, model name, vendor, version
    Business scenario tag, tenant/business line, whether critical path
    Usage input tokens, output tokens, cache-hit tokens, whether retried and retry sequence
    Quality & performance status code, time-to-first-token, total latency, eval-set score (sampled)
    Cost per-call cost converted by price snapshot

    3.2 Layered Statistics and Concentration Analysis

    First tally by single model, then aggregate by vendor, then observe the share distribution across the Top 10, Top 20, and Top 50 bands. Concentration analysis usually reveals one pattern: a handful of top models carry most of the call volume and also contribute most of the cost. Note that this is an illustrative observation — the specific numbers must be based on your own usage rankings computed under the caliber above. Don’t cite unsourced proportions. The conclusion is at the methodology level: cost and risk are highly concentrated, so optimization gains are concentrated too; capturing the head captures the main issue.

    3.3 Cross-Comparison

    Cross-reference the usage ranking against the cost ranking. The greater the difference in rankings, the more pronounced the cost-performance divergence. The misalignments hide optimization opportunities: options with high usage but a lower cost rank are high-cost-performance choices you can safely scale up; options with a high cost rank but average usage need evaluation of whether the premium is worth it.

    4. Selection Checklist

    Check each item before formally scaling up, rather than deciding a solution by feel:

    Check item Question to answer Example pass criterion
    Quality floor Does the pass rate on your own eval set meet the bar? Accuracy on critical paths not lower than the baseline model
    Effective unit price Input/output unit prices and tiered discounts Effective unit price = input share × input unit price + output share × output unit price
    Peak throughput Does the rate-limit quota cover peak demand? Peak demand ≤ 70% of quota
    Failed retries Are the retry rate and retry costs included in the budget? Retry rate < 2%, 5% budget buffer
    Monitoring fields Are all Section 3 fields collectible? No gaps, joinable to request_id
    Alert thresholds Trigger conditions and owner Day-over-day cost increase > 30% triggers an alert

    5. Building a Cost Monitoring System

    Metrics only produce lasting value once they land in a system. A recommended three-layer setup:

    1. Instrumentation layer: record usage and metadata at the call entry/exit;
    2. Aggregation layer: asynchronously funnel into unified storage via log services and message queues to avoid affecting main-path performance;
    3. Visualization and alerting layer: set thresholds for per-model daily cost, day-over-day growth rate, and abnormal spikes.
    1. # Aggregation-layer pseudocode: batch reporting to reduce reporting frequency and overhead
    2. def report_usage(buffer):
    3. if len(buffer) >= BATCH_SIZE:
    4. aggregated = aggregate_by_model(buffer)
    5. log_service.push(aggregated) # unified logging/monitoring backend
    6. buffer.clear()

    Compliance risk warning (must be implemented): Instrumentation and logging will touch user inputs and prompts, so you must — desensitize or hash prompts and user inputs before writing them to disk, and never store sensitive fields verbatim; follow the data-minimization principle, recording only the fields necessary for cost accounting; define retention periods (e.g., 30–90 days for details, long-term retention for aggregated metrics) and auto-purge on expiry; apply role-based access control and access auditing, and don’t open cost-accounting data to everyone.

    6. Four Cost-Optimization Strategies

    • Model tiering and intelligent routing: send simple tasks (classification, extraction, formatting) to lightweight models, and only route complex tasks (reasoning, long-form generation) to high-capability models.
    • Context and result caching: cache results for high-frequency repeated requests, and reuse prefix caches for long system prompts. Compliance boundary for caching: caches must not cross tenant and permission-isolation boundaries (namespaces split by tenant/permission), must set a TTL and support active invalidation, must not hold sensitive data, and cache hits and invalidations must have audit records.
    • Prompt trimming: redundant prompts both increase input tokens and can interfere with output quality, so periodic trimming is a low-cost, high-return measure.
    • Batching and asynchronization: route non-real-time tasks through batch processing, which usually earns a better billing tier and improves throughput.

    Priority recommendation: first do routing tiering and prompt trimming (deployable same day, zero compliance risk), then prefix caching (needs evaluation of isolation and invalidation), and finally the batch-processing overhaul (involves pipeline changes).

    7. Balancing Quality and Cost

    Premium solutions with a ratio below 1 shouldn’t be eliminated outright. In scenarios where the cost of error is high — financial risk control, medical assistance, complex code generation — the accuracy gains from high-quality models may far exceed the cost premium. The rational approach is to bound the premium: identify the critical paths where “quality directly determines success or failure,” enable high-end models only on those paths, and revert everything else to high-cost-performance solutions — spending the limited budget where the marginal return is highest.

    8. Common Mistakes and Implementation Advice

    Five common mistakes:

    1. Comparing only unit prices while ignoring differences in input/output token structure;
    2. Using inconsistent time windows or statistical scopes when calculating the cost-performance ratio, rendering the ratio meaningless;
    3. Failing to include the cost of failed and retried calls in the budget;
    4. Looking only at averages, not P95/P99 latency and the long-context tail;
    5. Linearly extrapolating demo-stage usage to production scale, ignoring the non-linearity introduced by tiered discounts and caching.

    Implementation advice: For enterprises, build a standing “usage — cost — cost-performance” dashboard to turn selection from a one-time decision into continuous optimization; for developers, first build solid usage instrumentation and structured logging on the client side, so every call is traceable and auditable.

    Back to the original question: “Which large model is the most convenient to use?” Once the framework reconstructs it, the answer is no longer a vague preference but a reproducible conclusion — on top of a clear quality floor, compare the cost-performance ratios of each solution under the same load and the same caliber, and pick the combination that stably meets needs at a lower unit cost.