LLM Model Selection: From Token Pricing to Task-Level Cost Accounting

The pricing table is a misleading cost lens

Vendor price updates are easy to obtain and easy to misinterpret. A headline such as “20% price cut” only translates into a 20% cost reduction if token consumption, retries, and manual intervention remain unchanged. In practice, all three variables move significantly as model capability changes.

First, look at the unit-price structure. Mainstream model services usually bill input tokens and output tokens separately, and divide models into capability tiers such as light, main, and flagship. Using illustrative relative numbers: if the main tier is the baseline at 1× input price and 1× output price, the light tier is about 0.5×, while the flagship tier is about 2×. Even if a flagship model cuts its price by 20%, its unit price may still be roughly twice that of the main tier.

Now consider one typical task. Suppose a code-understanding task requires reading 100,000 input tokens and generating 10,000 output tokens. Using illustrative unit prices, the cost per run would look like this. The values are only for demonstrating relative relationships:

Tier	Input unit price	Output unit price	Cost per run (illustrative)	Relative multiple
Light	1	5	0.30	0.25×
Main	2	10	0.60	1.0×
Flagship	4	20	1.20	2.0×

This table only shows one thing: if task consumption is assumed to be the same, unit price directly determines the bill. That assumption is often the least realistic part of real-world usage.

Task-level cost accounting: put pass rate into the formula

A more realistic cost model measures cost by completed task rather than by token:

Effective cost per task = (Σ each round (input tokens × input unit price + output tokens × output unit price)
                  + retry cost + manual rework cost) / first-attempt pass rate

This formula has three key amplifiers:

1. Reasoning rounds. Weaker models often need more tool calls and more self-checks to reach the right answer. Every round re-carries context, so input tokens accumulate rather than grow linearly. A task that takes six rounds to finish may consume four to eight times the input tokens of a single-round approach.

2. Retries and manual rework. If the first-attempt pass rate drops from 70% to 50%, effective cost rises by about 40%. This 40% is invisible in a bill calculated only by unit price, but it shows up in engineers’ waiting time and review time.

3. Uncontrolled output length. Lower-capability models are more likely to produce redundant explanations, repeated reasoning, or off-format content. Output tokens are often 30% to 100% higher than with more efficient models. Since output unit prices are usually two to five times higher than input unit prices, this waste has a large leverage effect.

The conclusion also reverses: cheaper models can indeed save money for simple tasks that finish in one round. But for long-chain tasks, tool-heavy workflows, and low-tolerance scenarios, a more expensive model may be cheaper overall because it takes fewer detours and has a higher first-attempt pass rate.

Vendor benchmark data can be useful because it compares cost per task rather than unit price. However, such data is produced by vendors and may contain selection bias and test-set adaptation bias, so it should be treated as a hypothesis to be validated with your own data.

Do not pay a capability premium for everyday tasks

Even vendors that release flagship models usually provide conservative official guidance: most coding and knowledge-QA tasks should start with the main tier; only move upward when the task is large-scale, long-chain, and genuinely beyond the main tier’s capability. The economic logic is direct: capability premiums pay off only for tasks that are likely to take detours.

A practical tier-mapping suggestion:

Task type	Typical characteristics	Recommended tier
Text rewriting, summarization, weekly reports, PPT outlines	Single round, low tolerance, short context	Light
Retrieval QA, information extraction, classification labeling	Batchable, results can be automatically validated	Light / Main
Routine business code generation, unit tests, SQL writing	Requires some reasoning but can be validated	Main
Cross-file refactoring, hard defect localization	Multi-round tool calls, long context	Start with main; escalate on failure
Long-running agent tasks, large-scale codebase migration	Dozens of rounds or more, subtask decomposition, hours of execution	Flagship

Using the flagship tier to write a weekly report is a common waste. Conversely, using a light-tier model to force through complex refactoring may lose the saved unit price to repeated retries and manual backup, while also adding latency and uncertainty.

Routing layer: turn selection into a runtime decision

Model selection should not be a one-time static configuration. It should be an explicit component of application architecture. A minimal routing layer includes three parts: a task classifier, a tier-mapping table, and a failure-escalation chain.

# Illustrative routing based on task complexity and budget
ROUTING_TABLE = {
    "rewrite":   {"tier": "light", "max_rounds": 1},
    "qa":        {"tier": "light", "max_rounds": 2},
    "codegen":   {"tier": "main",  "max_rounds": 4},
    "refactor":  {"tier": "main",  "max_rounds": 8, "escalate_to": "flagship"},
    "agent":     {"tier": "flagship", "max_rounds": 64},
}
def route(task_type: str, budget_tokens: int, policy: dict):
    conf = ROUTING_TABLE.get(task_type, ROUTING_TABLE["qa"])
    # If budget is tight, fall back to a lower tier and mark the response as degraded
    if budget_tokens < conf["tier_min_budget"]:
        return {**conf, "tier": "light", "degraded": True}
    return conf
def run_with_escalation(task_type, payload, client):
    conf = route(task_type, payload["budget_tokens"], {})
    for attempt, tier in enumerate(escalation_chain(conf)):
        result = client.invoke(tier=tier, messages=build_messages(payload))
        if result.passed_validation and result.rounds <= conf["max_rounds"]:
            return result
        if attempt == len(escalation_chain(conf)) - 1:
            return result  # Reached the limit; return the last result and alert

The value of an escalation chain is that you pay the main-tier price by default and only pay the flagship-tier price when validation fails. This turns “is capability worth paying for?” from an intuition-based pre-launch decision into a data-driven runtime policy.

Cost governance: several actions that work better than switching models

Before changing tiers, it is usually better to start with architecture-level optimizations because they often produce faster and more certain savings:

• Context trimming. In retrieval-augmented scenarios, reducing top-k from 20 to 5 and using a reranker to filter low-relevance snippets can often cut input tokens by more than 60%, with little impact on answer quality. Long tasks should especially watch for linear context accumulation and compress context by round with summaries.

• Prompt and result caching. Services with prefix caching can significantly reduce the input cost weight for repeated system prompts, codebase descriptions, and document snippets. Putting stable content at the beginning of the prompt is a low-cost, high-return practice.

• Structured output constraints. Ask the model to return only required fields and limit length and format. This directly reduces the most expensive output tokens. An illustrative approach:

SYSTEM = """
You are a code-analysis assistant. Output JSON only, no explanations, no Markdown code blocks.
schema: {"files": [str], "risk": "low|mid|high", "reason": "<=80 characters"}
"""

• Batching and asynchronous processing. For tasks that can be completed offline, such as batch labeling, nightly refactoring, and log attribution, asynchronous batch processing interfaces can often provide lower unit prices while moving latency pressure away from the critical path.

• Local small-model fallback. For high-frequency, patterned tasks that can be validated by rules, such as intent classification, field extraction, and format conversion, small models or pure rule-based implementations are often an order of magnitude cheaper than calling a general cloud model.

Let selection return to data: a minimal evaluation process

Any selection conclusion should be reproducible. Teams should maintain an internal evaluation set of 50 to 200 samples that covers the real distribution. For each sample, record whether it passed, rounds, input/output tokens, end-to-end latency, and whether manual intervention was triggered.

1. Freeze the evaluation set and validation scripts (automated checks first; subjective items use dual-blind review)
2. Run all candidate tiers on the full set and record quality / cost / latency triples
3. Calculate "effective cost per unit" = total spend / number of passed samples
4. Draw the quality-cost curve and look for inflection points rather than a single optimum
5. Set the inflection-point tier as default and route extreme samples through the escalation chain
6. Re-run monthly; re-run immediately after price changes or version upgrades

This process answers three key questions: which task types actually turn capability premiums into cost advantages; whether quality loss from downgrading to the light tier is acceptable; and whether the default tier needs to move when vendors change prices again.

Common pitfalls

• Treating a price-cut announcement as cost reduction. Real bill = unit price × consumption × retries. The latter two often change more.

• Replacing internal evaluation with public benchmarks. Test sets and business distributions differ, so conclusions cannot be directly transferred.

• Defaulting to the highest tier. For short-chain tasks, this is pure spending.

• Looking only at token costs and ignoring labor. One hour of engineering rework often costs more than the model invocation fee for the entire task.

• Ignoring latency cost. Putting tasks into an asynchronous queue to save money can degrade the interactive experience and cause indirect business loss.

Conclusion

Model selection is shifting from “which model is strongest/cheapest” to “which model has the lowest effective cost per unit for this task.” The former can be answered with price lists and leaderboards. The latter must be answered with evaluation sets, round statistics, and routing strategies.

For developers, the three capabilities to build are: task-level cost observability, switchable tier routing, and a continuously repeatable evaluation baseline. With these three practices in place, your bill will be more predictable than your peers’, regardless of how vendors adjust prices.