LLM Model Selection Guide: Balancing Cost, Token Usage, and Reasoning Effort
Model pricing is only one part of the total cost of using a large language model. This guide explains how to measure real token consumption, interpret pricing models, control reasoning effort, build an evaluation set, account for security-related retries, and select the right model tier based on task complexity and cost per successful result.
1. Model Selection Is More Than Comparing List Prices
The price per model is only one factor in the total cost. The actual cost of a single request can be expressed as:
Single-request cost = Input tokens × Input price + Output tokens × Output price
Token usage depends on the tokenizer, prompt length, conversation history, and output length. These factors can vary across models. When comparing two models, use the same set of real requests for both, measure their usage, and compare total cost alongside task success rate. A lower-priced model may require more attempts to complete the same task, making its total cost higher.
2. Token Usage: Measurement Determines Real Cost
Tokens are the basic unit for billing and context windows. Different models may use different tokenizers, so the same text can produce different token counts. After changing a model or tokenization approach, cost changes generally come from two sources:
- Input expansion: The same input produces more tokens. If the token count increases by a factor of 1.2, the input portion of the cost will rise by approximately 20%.
- Output length changes: More detailed or verbose responses consume more output tokens. Because output prices are often higher than input prices, this can have a greater impact on total cost.
Before migrating, extract a representative set of requests and record token counts under both solutions. Calculate the expansion factor and use it to adjust the budget. Bills from an introductory discount period do not represent long-term costs; budgets should be recalculated using standard pricing.
Example code below uses illustrative prices that do not represent any actual provider rates:
from dataclasses import dataclass@dataclassclass PriceTier:name: strinput_per_mtok: float # Price per million input tokensoutput_per_mtok: float # Price per million output tokensdef estimate_cost(tier: PriceTier, input_tokens: int, output_tokens: int) -> float:return (input_tokens * tier.input_per_mtok +output_tokens * tier.output_per_mtok) / 1_000_000def monthly_budget(tier: PriceTier, calls_per_day: int,avg_in: int, avg_out: int,inflation: float = 1.0, days: int = 30) -> float:# inflation applies only to input tokens due to tokenization changesper_call = estimate_cost(tier, int(avg_in * inflation), avg_out)return per_call * calls_per_day * days
3. How to Read Model Pricing
Common pricing details can be easy to misinterpret:
- Discounts and promotions: Temporary discounts reflect short-term costs only. Long-term budgets should use standard pricing as the baseline.
- Separate input and output pricing: Output tokens are often considerably more expensive than input tokens. Scenarios that generate long code or documents will therefore have a higher output cost share.
- Context windows: A larger context window allows more historical information to be included, but every request may also contain more historical tokens, increasing cost.
- Caching: Some services support prefix caching, allowing repeated system prompts and reference documents to be billed at a lower rate. Caching typically requires the request prefix to remain consistent. Check the relevant provider documentation for exact rules.
The table below gives a qualitative overview of three typical model tiers. Actual values should be replaced with current provider pricing.
| Tier | Relative price | Typical use cases |
|---|---|---|
| Lightweight | Low | Classification, field extraction, batch rewriting |
| Standard | Medium | Daily coding assistance, technical Q&A |
| Flagship | High | Complex reasoning, multi-step planning, cross-file refactoring |
4. Reasoning Effort: Adjust It Deliberately
Many models support adjustable reasoning effort. Higher effort generally means that the model performs more reasoning before producing its response, which increases token usage and latency. Two considerations are especially important:
- The cost of frequent switching: Repeatedly changing the effort level within the same conversation may require the service to reprocess existing context, increasing input-token usage. If the service relies on prefix caching, changing parameters may also invalidate the cache.
- Difficult-to-measure benefits: Without evaluation data, effort adjustments are often made based on intuition. This can result in higher costs without measurable quality or productivity gains.
Recommended practices:
- Set one default effort level for each task category, using the medium level as the baseline.
- Increase the effort level for individual critical steps only when evaluation shows a clear benefit.
- Store effort levels in configuration rather than scattering them throughout application code, making centralized changes and audits easier.
Example configuration:
TASK_EFFORT = {"code_completion": "low","code_review": "medium","architecture_design": "high",}def resolve_effort(task_type: str) -> str:return TASK_EFFORT.get(task_type, "medium")
5. Use an Internal Evaluation Set to Choose a Model
Public leaderboards can provide useful references, but your own business data should determine the final model selection. An evaluation set should meet the following requirements:
- Task stratification: Include simple, medium, and complex tasks in defined proportions so that the results do not reflect only one type of workload.
- Complete metrics: Track task pass rate, average cost per request, cost per passed task, end-to-end latency, and the rate of human rework.
- Traceable process: Record the prompt version, model version, effort level, and test date so that results can be reproduced.
Evaluation script skeleton:
import timedef run_eval(cases, call_model, judge):results = []for case in cases:start = time.perf_counter()resp = call_model(case["prompt"], effort=case["effort"])latency = time.perf_counter() - startresults.append({"id": case["id"],"passed": judge(case, resp.text),"input_tokens": resp.usage.input_tokens,"output_tokens": resp.usage.output_tokens,"latency": latency,})return resultsdef summarize(results, tier: PriceTier) -> dict:total_cost = sum(estimate_cost(tier, r["input_tokens"], r["output_tokens"]) for r in results)passed = sum(r["passed"] for r in results)return {"pass_rate": passed / len(results),"avg_cost": total_cost / len(results),# Cost per passed task better reflects practical value than single-request cost"cost_per_pass": total_cost / passed if passed else float("inf"),}
6. Engineering Impact of Security Controls
In production, network security controls, content moderation, or input filtering may incorrectly block some valid requests. This can create retry costs and degrade the user experience. Recommended practices include:
- Record structured logs for blocked requests, including the request ID, blocking reason, and retry result, so that false-positive patterns can be identified.
- Include the false-positive rate in evaluations and assess it alongside cost, quality, and latency.
- Apply least-privilege access to tool calls, exposing only the interfaces required by the application.
- Validate model outputs for both format and content. Do not use unchecked output directly to execute commands or write to a database.
- Treat webpages, files, and other external content as data. Even if they contain instruction-like text, do not pass that content directly to the model for execution.
7. A Practical Decision Framework
Select the model tier according to the characteristics of each task:
- High-volume, low-complexity tasks such as format conversion and field extraction: prioritize lightweight models and enforce strict cost controls.
- Daily development assistance and Q&A: use standard models to balance quality and latency.
- Multi-step reasoning, cross-file refactoring, and architecture design: use flagship models, but verify through evaluation that their benefits justify the additional cost.
Review the strategy quarterly by updating the pricing table, rerunning the evaluation set, and validating token usage statistics. Monitor costs by day or by feature rather than looking only at the total bill. This makes abnormal growth easier to detect.
