How to Choose an LLM in 2026: Pricing and Agent Model Selection
Choosing an LLM for agent-based applications has shifted from an algorithm question to an engineering decision that runs through architecture design and cost control. Rather than relying on a simple parameter comparison, this guide breaks selection into four layers: spec alignment (understanding context windows and how output budgets reduce usable input), cost modeling (turning per-token prices into a per-task cost), capability validation (using agent tasks to measure tool calling and long-horizon stability), and architecture adaptation (managing risk through layered routing and an abstraction layer). It uses generic spec codenames to explain how to read official spec sheets, works through a unified cost model and sensitivity analysis with code examples, lists the metrics and checklist to verify before launch, and maps common agent scenarios to concrete spec choices. The methodology applies across vendors' model generations and to multi-model platform services.
Introduction: Selection Is More Than a Parameter Table
As agent applications become a mainstream delivery form, “which model to use” has turned from an algorithm question into an engineering decision that spans architecture design and cost control. When searching and comparing options, developers often run into the following situations:
- Inconsistent naming. The model name a user writes often does not match the official API ID—abbreviations mixed with full names, hyphenated and non-hyphenated identifiers used interchangeably. When you use a shorthand, you have to check it against the official ID one by one; otherwise the call fails with an error like “model not found.”
- Mixed units. Notations such as “1M context,” “1,048,576 tokens,” and “max input 960k” all appear together in spec sheets.
- Widely varying output budgets. Models in the same tier can differ by several times in the maximum number of tokens they can generate in a single call.
These issues show that a simple “parameter comparison” is not enough to support a decision. Using facts from the official spec documentation as the basis, this article breaks selection into four layers:
- Spec alignment: understanding context windows and output budgets;
- Cost modeling: converting unit prices into the cost of a single task;
- Capability validation: using agent tasks to measure tool calling and long-horizon stability;
- Architecture adaptation: managing risk through layered routing and an abstraction layer.
What are the common mistakes in selecting models for agents? How do you estimate cost? What should you watch out for before launch? The sections below address these practical questions. The same methodology applies to the various model generations sold by different providers, and to platform services that support multiple-model access.
1. Spec Alignment: First, Understand What the Documentation Is Saying
To avoid recommending or evaluating any specific product, the sections below use codenames to refer to a few common spec categories in today’s mainstream services. The specific values come from public documentation and are used only to illustrate how to read the table. This article is compiled from public documentation as of June 2026; since providers iterate frequently, re-check the latest docs before any actual integration.
| Spec | Positioning | Input types | Context | Max output budget |
|---|---|---|---|---|
| Spec A (flagship, omni-modal) | Text, image, and video input; two output tiers (non-thinking / thinking) | Text, image, video | 1,000,000 tokens | Non-thinking 131,072; thinking 262,144 |
| Spec B (lightweight multimodal) | Positioned for speed; independent limits on input and output | Text, image | 1M context; max input 960k | Max 64k |
| Spec C (fast, long-output) | Built for single-pass long-text generation | Text, image | 1M | Max 384K |
| Spec D (long-context flagship) | Context counted as 2^20; large default output | Text, image, video | 1,048,576 tokens | Default 131,072, capped by remaining context |
When reading this table, the following four points are where selection most often goes wrong.
Before formal integration, it is worth adding a checklist and confirming each item one by one:
- API ID: whether the model ID used in code exactly matches the official documentation;
- Billing unit: whether both input and output are priced per “USD per million tokens”;
- Thinking billing basis: whether thinking tokens are billed separately and whether they count toward output;
- Caching strategy: whether the system prompt and tool definitions support prefix caching;
- Rate-limit quotas: whether rate limits, concurrency quotas, and peak hours affect task completion.
1.1 Context Is Not Equal to Usable Input: See How the Output Budget Is Deducted
The context window holds both the prompt and the generated content, so the effective input limit = context window − the output space reserved for this call. Take Spec B as an example: it labels its context as 1M, max input as 960k, and max output as 64k. This notation may mean the max input already has the output reserve deducted, but 960k + 64k does not exactly equal 1M or 1,048,576, so you should rely on the vendor’s stated basis and measured token counts. Spec C also has a 1M context, but its max output reaches 384K. When the output is maxed out, only about 0.64M (roughly 640K) of usable input remains.
For agents that do “long-document analysis + long-report generation,” this difference directly determines whether the task can be completed in a single call. In system design, compute usable input against a conservative value:
def usable_input_tokens(context_window: int, max_output: int, safety_ratio: float = 0.9) -> int:"""Estimate a safe usable input budget.context_window: the context window limit stated in the documentationmax_output: max output tokens requested for a single callsafety_ratio: safety factor reserved for tokenization error, system prompt, tool definitions, etc."""reserve = max_output + int(context_window * (1 - safety_ratio))return context_window - reserve
1.2 The “Thinking / Non-Thinking” Tiers: Watch the Switching Conditions
Spec A splits its output limit into two tiers: 131,072 in non-thinking mode and 262,144 in thinking mode. This shows that reasoning depth is not just a capability switch but also a switch for output resources and cost.
What matters more in engineering is the switching trigger condition: thinking mode may be a parameter that is enabled manually, or the server may enter it automatically based on question complexity or default configuration. These two conditions directly affect cost and latency. Providers also differ in how they bill the thinking process—some bill it separately, others count it toward output tokens—so you must go by the documentation.
1.3 1M vs. 1,048,576: The Trap in Unit Conversion
Spec D writes its context as 1,048,576, which is exactly 2^20 and follows binary conversion; Specs A and C write 1,000,000 directly. Both are nominally “1M,” but the difference is about 4.9%. In ultra-long tasks that run close to the limit, this can decide whether a single call succeeds. Therefore:
- Configuration and verification: configure and verify uniformly in terms of “token count,” and avoid rounding on abbreviations like “1M”;
- Approximate relationships: as a rough guide, in some tokenization schemes Chinese may be close to 1 token per character and English words are usually split into 1–2 tokens, but you must verify with real corpus measurements;
- Measured consumption: Chinese tokenization, code, tables, and special characters can make the actual token count deviate noticeably; before launch, estimate token consumption with real corpora and do not simply multiply character count by a factor.
1.4 Input Modality: Do You Really Need Video?
Specs A and D support video input, while Specs B and C support only text and images. Video understanding adds significant value in some scenarios:
- UI automation agents need to understand operation trajectories in screen recordings;
- Ops agents need to parse screen recordings of incident scenes;
- Content moderation needs to judge violating frames one by one.
In these scenarios, “understanding a video” is closer to human intuition than “extracting frames and understanding each image one by one.” For agents that are mainly code- and text-centric, multimodality is more of a fallback capability and should not be a primary basis for the decision.
2. Cost Estimation: Turning Unit Prices Back Into “the Cost of a Single Task”
2.1 Pricing Structure: High Price Does Not Mean High Cost
Large-model APIs generally bill by token, with input and output priced separately. Most providers publish unit prices in “USD per million tokens” or an equivalent unit, but there are usually two caveats: unit prices change with model version and tier; and limited-time promotional prices can differ significantly from regular prices. Since there is no unified public quote in the spec sheet, a more reliable approach is to build a cost model first and then plug in each provider’s actual prices.
When evaluating a quote, the following questions must be answered beyond the price sheet:
- What is the ratio of the output price to the input price? In generation-heavy services, output tokens cost more to compute than input tokens, and many providers price output at several times the input price. So if a spec with strong output capability also has a high unit price, long-text tasks will significantly amplify the cost.
- Is the thinking process billed, and by what standard? If reasoning tokens are priced separately, the cost of long-chain reasoning tasks rises sharply with reasoning depth.
- Is context caching or prompt caching available? For agents with a fixed system prompt and unchanged tool definitions, the cache hit rate is often high, which can greatly reduce the cost of repeated prefixes.
- How are free tiers, tiered pricing, and quotas set up? Do batch offline tasks support cheaper asynchronous calls; what are the rate limits during testing versus production.
- Does a “large context window” mean long context is billed at the standard price? Some providers price ultra-long context separately or apply a lower rate limit.
2.2 Cost Model: Three Variables That Directly Determine Cost
Setting aside brand differences, the per-turn call cost can be abstracted as:
Per-turn cost = input tokens × input price + (generated tokens + thinking tokens) × output pricePer-task cost = per-turn cost × average number of interaction turns
Among these, the average number of interaction turns is the most easily underestimated. Each time an agent runs a tool call or reads a long document, it usually corresponds to a new model call, and the message history is sent again as input in the next turn. So the total input tokens for a single task are often several times the length of the user’s original question.
The following code demonstrates, without involving any specific provider, how to measure the cost of different specs on a unified basis. It uses a “constant table + parameter substitution” style that makes it easy to compare across models:
# Prices are illustrative only, in USD per million tokens; replace with the official pricing when integrating a specific service.PRICING = {"A": {"in": 10.0, "out": 40.0},"B": {"in": 2.0, "out": 8.0},"C": {"in": 1.0, "out": 16.0},"D": {"in": 21.0, "out": 84.0},}def estimate_cost(model: str, in_tokens: int, out_tokens: int,thinking_tokens: int = 0, cache_hit_ratio: float = 0.0) -> float:"""Estimate single-turn cost in USD using a unified token measure.For comparison, cached input is treated as free here;in a real project, recalculate using the vendor's cache discount, billing rules, and measured hit rate."""p = PRICING[model]fresh_in = in_tokens * (1 - cache_hit_ratio)cost_in = fresh_in * p["in"]cost_out = (out_tokens + thinking_tokens) * p["out"]return (cost_in + cost_out) / 1_000_000
With a unified measure in place, a parameter sensitivity analysis yields far clearer conclusions than “whose unit price is lower”:
- Under this illustrative price set, if a task has equal input and output tokens and caching is ignored, Spec A is about 5.0× Spec B and about 2.9× Spec C; Spec D is about 10.5× Spec B and about 6.2× Spec C.
- If the output-to-input ratio drops to about 1:4 (output is one-quarter of input), then Spec A is about 5.0× Spec B, Spec D is about 10.5× Spec B, and Spec D is about 8.4× Spec C.
- This shows that the cost gap between specs depends on the input-output ratio, the number of task turns, and the cache hit rate—not simply on the input unit price.
- If the system prompt and tool definitions together total 4,000 tokens sent repeatedly on every call, and Spec D’s input price is USD 21 per million tokens, then at a 90% cache hit rate, estimating the savings at the upper bound gives roughly per call: 0.9 × 4000 × 21 ÷ 1,000,000 = 0.0756 USD, i.e. about 0.076 USD. The actual savings depend on the cache billing basis and hit rate.
This shows: the first principle of selection is “match the spec to the task shape,” not “find the model with the lowest unit price.”
3. Capability Evaluation: Key Metrics for Agent Scenarios
The spec sheet can tell you “how long it can process, how long it can output, and which modalities it can see,” but it cannot directly tell you “whether it is suited to being an agent.” The latter needs to be answered through targeted evaluation.
3.1 Tool Calling and Structured Output
When an agent interacts with a model, it generally needs to provide three parts at once: the role and task description, the available tools (names, purposes, parameter structures), and the current conversation and execution state. The model needs to select the correct tool from these and generate a call request that follows the parameter structure. Evaluation should focus on:
- Tool selection accuracy: whether the model can choose correctly among several functionally similar tools;
- Parameter completeness and validity: whether required parameters are present and types are correct;
- Stability as the number of tools grows: selection accuracy drops noticeably when tools increase from 10 to 50 or even over a hundred, and this is not fully correlated with context length;
- Multi-turn error recovery: after a tool returns an error or an empty result, whether the model can read the error and adjust the call rather than repeating the same failed operation.
3.2 Effective Use of Long Context
“1M context” is often misunderstood as the model being able to use all 1M tokens evenly and accurately. Actual performance depends on the task:
- Retrieval tasks (locating a single fact within a long text): most mainstream models perform well;
- Summarization tasks (structured synthesis of the full text): omissions and biases increase as the input grows longer;
- Multi-hop reasoning (the answer requires linking several pieces of information far apart in the document): this places the highest demands on long-context usage.
For agents, what matters more is “state preservation over long conversations.” When a single task involves dozens of tool calls, whether earlier constraints (such as “do not modify the config file”) are still respected determines whether the task can wrap up reliably.
3.3 How the Output Budget and Reasoning Mode Affect Task Completion
The output budget not only limits the length of a single response but also affects whether a task can be “closed in one pass”:
- A spec with a 64k output cap, when generating a complete code repository or a long technical proposal, may be truncated into incomplete JSON or code, forcing the system to split it across multiple calls. Splitting adds turns and the repeated consumption of input tokens, so you should look at the total task cost, not the per-call unit price.
- A spec with a higher output cap is more comfortable on single-pass long-output tasks; but high-output specs often come with a higher output price, and the output price is usually higher than the input price.
- Thinking mode can improve accuracy on complex reasoning tasks, but it simultaneously increases generation time and output token consumption. When comparing costs, count thinking tokens as well.
3.4 Latency and Stability
An agent’s interaction experience is determined by both time to first token (TTFT) and end-to-end latency. The latter can be approximated as:
End-to-end latency ≈ TTFT + output tokens ÷ output rate (tokens/s)
During selection, watch the server’s rate limits, concurrency quotas, and availability at peak times. Even if a model is more capable, if it frequently returns rate-limit errors at peak, the agent’s reliability drops accordingly.
3.5 Evaluation Method: Select on Task Metrics, Not Subjective Feelings
It is worth building an evaluation set that covers real scenarios, where each case includes an input, an expected result, and automatically checkable criteria. The evaluation set fields should include at least:
- Input: the user question, context, and tool definitions or task description;
- Expected tools: the tools that should be called and their key parameters;
- Max allowed turns: the maximum number of interaction turns the task allows;
- Cost threshold: the acceptable upper bound on the cost of a single task;
- P95 latency threshold: the latency requirement for user experience or automation.
In the end, you should uniformly track the following metrics:
- Pass rate: the proportion of tasks that complete successfully;
- Tool call accuracy: the proportion where both the tool and parameters are correct;
- Average interaction turns: the number of model calls needed to complete a single task;
- Per-task token consumption: input, output, and thinking counted separately;
- Per-task cost and P95 latency.
Only when capability metrics and cost metrics are placed on the same table does the selection conclusion become comparable.
4. Allocation by Agent Scenario: A Decision Framework
Different scenarios have different constraints, and the optimal solution often differs too. The table below gives general guidance and summarizes the conclusions above:
| Scenario | Key constraints | Recommended focus |
|---|---|---|
| Code generation and refactoring | Long output, cross-file understanding, multiple tool calls | Specs with context ≥ 128k and larger single-call output; lightweight specs for non-core modules |
| Document analysis and report generation | Long input, structured and lengthy output | Evaluate usable input after “context − output reserve”; long-output specs beat specs that only emphasize context length |
| UI automation and visual tasks | Relies on screenshot or screen-recording understanding | Multimodal specs; lightweight specs for state checks between tasks |
| High-concurrency lightweight tasks (classification, extraction, routing) | Highly sensitive to latency and cost | Lightweight, low-price specs combined with caching |
| Long-horizon autonomous tasks (multi-step planning, continuous execution) | State preservation, tool error recovery, cost control | First compare the “accuracy gain under thinking mode” against the “extra token cost” before deciding whether to enable reasoning |
4.1 Layered Routing: Let the Right Spec Do the Right Job
A single-model approach often makes trade-offs between cost and effectiveness, so the more common practice is layered routing:
- Routing layer: use a lightweight spec or rules for intent recognition and difficulty grading;
- Execution layer: hand simple tasks to the lightweight spec and complex tasks to the more capable flagship spec;
- Review layer: use a stronger spec to validate critical steps (such as write operations and external publishing).
from dataclasses import dataclass@dataclassclass Task:need_vision: bool = False # whether the task includes image or video inputoutput_tokens: int = 0 # estimated output sizesteps: int = 1 # estimated number of reasoning stepsinput_tokens: int = 0 # estimated input sizedef route(task: Task, cfg: dict) -> str:"""Pick a spec by task features; all thresholds come from offline evaluation results, not fixed rules of thumb."""if task.input_tokens > cfg["light_context_limit"] or task.steps >= cfg["complex_steps"]:return "A" # flagship spec: long context + long outputif task.need_vision or task.output_tokens > cfg["light_output_limit"]:return "C" # image-capable with a generous output budgetreturn "B" # lightweight spec: high-frequency, low-complexity tasks
It is worth noting that the thresholds in the routing logic (the fields in cfg) cannot be set arbitrarily; they should be determined by offline evaluation results: sweep different thresholds on the evaluation set, plot an “accuracy–cost” curve, and pick values near the inflection point.
4.2 Abstraction Layer: Isolating Model Differences
Different providers’ APIs differ in message structure, how tools are defined, streaming response formats, and error codes. If business code depends directly on one provider’s interface, switching models requires large-scale changes. It is worth designing a unified model adapter interface:
from typing import Any, AsyncIterator, Protocolclass ChatMessage(Protocol):role: strcontent: Any # text, image, or video referenceclass ModelAdapter(Protocol):model_id: strcontext_limit: int # context window limitmax_output: int # max output per callsupports_vision: bool # whether image/video input is supportedasync def chat(self, messages: list[ChatMessage], tools: list[dict],max_output: int) -> dict: ...async def stream(self, messages: list[ChatMessage], tools: list[dict]) -> AsyncIterator[dict]: ...
The adapter layer should unify at least four things:
- format conversion of messages and tool definitions;
- parsing and assembly of streaming output;
- error classification and retry strategy (distinguishing rate limiting, timeouts, content-moderation blocks, and parameter errors);
- token usage accounting.
Only when these data definitions are unified can you make fair cost-and-effectiveness comparisons across different specs. When a spec experiences consecutive call failures or its P95 latency exceeds a threshold, circuit-breaker logic can automatically switch traffic to a backup spec, preventing fluctuations in a single spec from dragging down the whole chain.
4.3 Context Governance: A Systematic Way to Reduce Token Consumption
Token cost mainly comes from repeatedly sent context. Governance measures include:
- History summarization: compress earlier turns with summaries, keeping only key decisions and state;
- Retrieval-based injection: use vector retrieval to load relevant snippets on demand rather than stuffing the entire knowledge base into the prompt;
- Tool result truncation and structuring: compress long logs and large JSON with rules or a lightweight model before sending them back;
- Prefix caching: keep the system prompt and tool definitions at the front of the message and keep them stable to raise the cache hit rate;
- Batch async: use asynchronous batch interfaces for offline tasks to lower the unit price.
5. Continuous Evaluation After Launch
Models iterate quickly, and selection conclusions have a clear shelf life, so evaluation should be built into a repeatable process:
- Version locking and logging: log the model ID, version, parameters, and token usage on every call for traceability;
- Canary comparison: run the new version on a small fraction of traffic first, comparing pass rate, cost, and latency against the old version;
- Regular regression: run regular regressions on a fixed evaluation set to guard against changes in capability or output format after a model upgrade;
- Cost dashboard: track cost by scenario and by model, and catch abnormal growth early—for example, a sudden rise in the average number of turns for a category of tasks.
Conclusion
Back to the original question—“how to choose”—the more accurate answer is not “which one to choose” but “how to choose the right spec for each category of task and keep flexibility through architecture.”
- Spec level: understand context, output budgets, and unit conventions to avoid configuration mistakes;
- Cost level: use a unified cost model to measure the real impact of different unit prices, and factor interaction turns into the estimate;
- Capability level: do real measurements with tool-call accuracy, long-context utilization, and stability metrics;
- Architecture level: isolate differences through layered routing and an adapter layer, leaving room for model iteration.
If you carry out similar work on a platform that supports multi-model access, you can combine general model services, a knowledge base, vector retrieval, and monitoring capabilities with this article’s methodology to gradually form a “evaluate–route–monitor–iterate” loop. For most teams, the safest path is: first validate the workflow with a lightweight spec, then progressively introduce stronger specs in high-value stages, so that every upgrade is backed by data.
