# LLM Agent Selection Guide: Pricing, Capabilities, and the Real Cost of a Task
This guide walks through how to choose a large language model for agent workloads by measuring cost per successful task rather than list price per token. It explains why model prices that differ by 100x can produce bills that differ only a few times over, lists six hard constraints you can verify from vendor documentation before comparing capabilities, describes how to build a business-derived evaluation set and which task-level metrics to track, derives the cost formula for multi-step agent runs, and finishes with a four-step selection process, six cost-optimization tactics, and five common mistakes. Code samples show evaluation, cost estimation, and model selection logic.
1. The Problem: Unit Prices Differ 100x, Bills Differ by Only a Few x
After integrating large language models, many teams run into the same kind of bill: output prices on a platform can differ by a factor of several, or even a hundred, from model to model, yet total monthly spend does not move proportionally. There are four main reasons.
- Usage concentrates in a few tasks. High-volume jobs such as online customer support and batch document extraction account for most calls, and a single complex agent invocation often costs tens of times more than a simple classification request.
- Context is carried forward repeatedly. Every extra step an agent takes re-sends the previous round’s reasoning and tool results to the model. Token consumption grows roughly quadratically with the number of rounds.
- Reasoning models “think” more. With extended thinking enabled, generation covers not just the final answer but the thinking process, so output cost rises noticeably.
- Failures amplify cost. Malformed output, wrong tool parameters, and off-track steps all trigger retries or rollbacks. When actual spend is divided by the number of successful tasks, the per-task figure inflates several-fold.
The core selection principle follows from this: measure a model by cost per unit of business output, not by unit price. That is, given a defined task distribution and acceptance criteria, how much does it cost to complete one acceptable task? This cost is determined jointly by unit price, token usage, and first-pass rate.
2. Step One: Use Hard Constraints to Eliminate Unusable Models
Before asking “how good is it,” answer “can we use it at all.” The six categories below do not depend on subjective impressions and can be verified directly in platform documentation, so they narrow the candidate set quickly.
1. Input modality. Tasks that read screenshots, table images, or screen recordings require a model with image input (video input is even better). Text-only tasks do not need to pay a premium for multimodality. Also confirm how modalities can be combined: whether images can only appear in user messages or may also appear in tool results — this directly affects prompt design for vision-based agents.
2. Context window. The number that matters is not “how much fits at maximum” but “how much input still fits once output space is reserved.” Input and generation usually share one window, and many models cap single-turn input below the total window length.
3. Output budget and thinking mechanism. The per-call output limit determines whether you can generate long code or long reports in one shot. Pay particular attention to:
- whether enabling reasoning mode expands or shrinks that limit;
- whether the chain of thought and the final answer draw on the same budget;
- whether thinking effort can be adjusted or switched off.
Prompts for long-output tasks should explicitly ask for the complete content in a single response, to avoid truncation followed by a continuation retry.
4. Tool calling and structured output. Agent reliability depends largely on whether the platform can do three things: constrain output to formats such as JSON Schema so downstream code never parses free text; reliably produce function-call arguments that match the given definition; and return multiple tool calls from one request for parallel execution. At minimum, confirm in the docs whether output format is constrained and whether parallel tool calls are supported.
5. Pricing basis and billing units. Whether input and output are billed separately, whether reasoning tokens are billed, and whether prompt caching and batch channels are available should all be taken from the official pricing page. The meaning of “K” and “M” units and the billing granularity differ across platforms — do not convert and compare figures that use different conventions.
6. Quotas, rate limits, and stability. Concurrency limits, tokens per minute (TPM), requests per minute (RPM), and rate limiting determine whether a model can absorb traffic peaks. Service level agreements (SLAs) and data residency requirements bear on compliance.
| Hard constraint | What to confirm with the platform | Consequence of failing |
|---|---|---|
| Input modality | Image/video support; whether it can appear in tool results | Vision tasks are ruled out entirely |
| Context window | Total window vs. single-turn input cap; whether input and output share it | Long documents are truncated or rejected |
| Output budget | Per-call output cap; whether reasoning mode changes it; whether thinking can be disabled | Long code is cut off, or simple tasks are over-reasoned |
| Tool calling | Format constraints, parallel calls, tool-result length limits | Argument parsing fails, the flow stalls |
| Pricing and quotas | Input/output unit prices, billing of reasoning tokens, caching and batch availability | Costs spiral, or peak traffic is throttled |
| Stability | Rate limits and SLA | Requests fail at peak, degrading the experience |
Put this step into practice with a candidate matrix: rows are your business tasks, columns are candidate models, and each cell reads supported / not supported / to be verified. Once the matrix is full, keep only models that cover every task, and move on to evaluation.
3. Capability Comparison: Specs Are Not Performance
Models that survive the hard-constraint filter tend to look alike on paper: all support image-and-text understanding, all support tool calling, all offer context windows in the millions, and naming conventions may carry suffixes signaling different positioning. But “can call tools” and “can reliably orchestrate multi-step tool use” can be worlds apart. Judging real performance takes a consistent task set, not a spec sheet.
1. Build the evaluation set from real business traffic. Sample from production logs, stratify by task type, and construct 100–300 representative cases spanning easy, medium, and hard. Every case needs explicit acceptance criteria — JSON Schema validation, code unit tests, rule matching, or human scoring. Keep the prompt, tool definitions, and maximum allowed steps identical across models.
2. Measure tasks, not single turns. Track at least these six:
| Metric | Definition | Question it answers |
|---|---|---|
| Format compliance rate | Share of outputs that pass structural validation | Can it feed a downstream system directly |
| Task first-pass rate | Share meeting acceptance criteria on the first attempt | Is human rework needed |
| Eventual success rate | Share achieved after retries or extra steps | How high is the capability ceiling |
| Average rounds and tokens | Steps and tokens consumed to finish a task | What the bill is made of |
| Latency | Average and P95 latency | Whether the user experience is acceptable |
| Stability | Consistency of results across repeated runs on the same input | Whether it can drive high-risk decisions |
3. Score each task type separately. Different tasks weight different metrics, so a single aggregate score is a poor ranking mechanism:
- Batch image/text extraction: field-level accuracy and format compliance rate;
- Code generation and repair: unit-test pass rate, with compilation or static analysis as a supporting signal;
- Long-document analysis: citation accuracy and factual consistency;
- Multi-step agents: task first-pass rate, average number of rounds, and the “unsafe action rate” — the frequency of overreach such as unauthorized calls or skipping user confirmation.
A sketch of the evaluation harness:
import jsonimport statisticsdef evaluate(model, task_set):records = []for case in task_set:result = run_agent(model, case.prompt, tools=case.tools,max_steps=case.max_steps)records.append({"case_id": case.id,"schema_ok": validate_schema(result.final_output, case.schema),"passed": check_acceptance(result.final_output, case.rule),"steps": result.steps,"tokens": result.input_tokens + result.output_tokens,"latency_ms": result.latency_ms,"unsafe_action": result.unsafe_action, # Unauthorized calls, skipped confirmations, etc.})n = len(records)passed = [r for r in records if r["passed"]]return {"schema_rate": sum(r["schema_ok"] for r in records) / n,"pass_rate": len(passed) / n,"avg_steps": statistics.mean(r["steps"] for r in records),"avg_tokens": statistics.mean(r["tokens"] for r in records),"p95_latency_ms": sorted(r["latency_ms"] for r in records)[int(n * 0.95)],# Use successful tasks as the denominator to capture the real cost of one acceptable output"cost_per_pass": estimate_cost(records) / max(len(passed), 1),"unsafe_action_rate": sum(r["unsafe_action"] for r in records) / n,}
4. Test whether long context is actually used. A million-token window does not mean a million tokens are used reliably. Plant key information at the beginning, middle, and end of a long document, then ask about it and observe whether accuracy depends on position — the “needle in a haystack” style is a useful template. Business-side tests matter too, for example “every clause in this 200-page contract where the amount exceeds X.” If localization accuracy drops clearly as documents grow, switch to retrieval-augmented generation (RAG): retrieve the relevant passages first, then hand them to the model.
4. Cost Comparison: The Real Cost of One Task
4.1 The basic structure of cost
Mainstream platforms bill tokens with input and output priced separately, and some models also support prompt caching. The cost of a single round can be written as:
C_round = (L_in × P_in + L_out × P_out) / 1,000,000
where L_in and L_out are input and output token counts, and P_in and P_out are the input and output prices per million tokens.
An agent task spans N rounds, and round i sends the system prompt, task instructions, conversation history, and tool results from earlier rounds. Total cost is therefore approximately:
C_task ≈ N × [ (L_sys + L_task) × P_in + L_resp × P_out ]+ P_in × L_tool × N × (N + 1) / 2
This explains why “the same one-line question” can cost a fraction of a cent for one task and several dollars for another:
- Rounds are a multiplier. The system prompt and task instructions are paid for again every round — double the rounds, double that cost.
- Tool results accumulate. They grow quadratically with rounds. A tool that returns tens of thousands of tokens of page content or logs can consume several hundred thousand input tokens over ten rounds on tool results alone.
- Unit price is only one factor. A cheap model with many rounds, or a low-priced model with a poor first-pass rate, can both end up more expensive.
4.2 Compare models on cost per pass
So compare on C_per_pass = C_task / P_task, where P_task is the task first-pass rate. Suppose Model A has a lower unit price and half the per-task cost of Model B, but passes only 50% of tasks on the first attempt, while Model B completes 90% in one go. Their cost per pass is roughly equal — and once human review time is counted, Model B is the better deal.
A rough estimate:
def estimate_task_cost(model, workload, prices):"""Estimate per-task cost from workload structure. Prices are per million tokens; demo only."""p_in, p_out = prices[model]["input"], prices[model]["output"]n = workload["avg_steps"] # Average number of roundssys_task = workload["sys_tokens"] + workload["task_tokens"]tool_avg = workload["avg_tool_tokens"] # Average tokens returned by tools per roundresp = workload["avg_response_tokens"] # Tokens generated by the model per round (including thinking)input_tokens = n * sys_task + tool_avg * n * (n + 1) / 2total = (input_tokens * p_in + n * resp * p_out) / 1_000_000return {"cost_per_call": total / n,"cost_per_task": total,"cost_per_pass": total / workload["pass_rate"], # Adjusted for first-pass rate}
In production, log input and output token counts and the current step after each call, collect them centrally through a logging service, and wire them into monitoring and alerting with a per-task cost threshold. These are standard cloud service categories, and every major platform offers equivalents.
4.3 Other factors that move cost
- Reasoning overhead. Extended thinking consumes extra output tokens, and some platforms bill that portion separately. Whether thinking effort is adjustable and can be turned off per task is the key question for cost-sensitive scenarios.
- Cache hits. Repeated content — system prompts, tool descriptions, knowledge-base snippets — gets a lower input price on platforms that support prompt caching. Putting stable instructions at the start of the prompt helps the hit rate.
- Batch channels. Offline batch jobs can use the platform’s batch channel, which is usually cheaper at the cost of response times measured in hours.
- Output amplification. Requesting N candidates and letting your code pick one multiplies output cost. Use it only where it is genuinely needed.
5. Scenario Fit: Which Models for Which Tasks
| Task type | Key constraint | Model traits to prioritize | What to test |
|---|---|---|---|
| High-volume image/text extraction | Short per-call output, high call volume | Input price, throughput and rate limits, JSON Schema constraints | Field accuracy, format compliance rate |
| Batch long-document processing | Long input, medium output | Single-turn input cap, prompt caching, batch channel | Long-document localization accuracy, cost per document |
| Code generation and repair | Long output, must run as delivered | Per-call output budget, reasoning and coding ability | Unit-test pass rate, truncation rate |
| Long-horizon agents | Many rounds, many tools, complex state | Tool-calling reliability, context management, stability | Task first-pass rate, average rounds, unsafe action rate |
| Real-time interaction | Response speed first | Output speed, whether reasoning can be disabled | P95 latency, time to first token |
Take the task “understand user steps from a screen recording and generate a script”:
- Hard-constraint filter. Rule out text-only models first. If the recording is supplied as video, also confirm video input support, how much video fits in a single-turn input, and whether the output budget can hold a complete script.
- Cost baseline. Use comparable general-purpose tasks to establish a per-token cost baseline.
- Accuracy comparison. Compare step-recognition accuracy across a set of real recordings.
- Long-horizon validation. Run one task with five to eight tool nodes and compare final completion rate and average rounds.
6. The Selection Process: Four Steps
- Draw a task profile. Nail down input modality, typical input length, expected output length, maximum rounds, latency requirements, and compliance requirements.
- Filter on hard constraints. Work through the candidate matrix from Section 2, drop unusable models, and keep two or three candidates.
- Evaluate and compare. Use one shared task set and the methods in Sections 3 and 4 to compare quality and cost.
- Route and roll out gradually. Split traffic across models by task type, launch at a small traffic share, and ramp up once production metrics stabilize.
A code sketch:
def select_model(tasks, candidates):# 1. Hard constraints: input modality, single-turn input cap (with output headroom), per-call output capusable = [m for m in candidatesif tasks.input_modality in m.input_modalitiesand m.max_input_tokens >= tasks.p95_input_tokens * 1.2and m.max_output_tokens >= tasks.p95_output_tokens]# 2. Soft constraints: latency and costshortlist = [m for m in usable if m.avg_latency_ms <= tasks.latency_budget]scored = []for m in shortlist:metrics = evaluate(m, tasks.eval_set)meets_quality = (metrics["schema_rate"] >= tasks.min_schema_rateand metrics["pass_rate"] >= tasks.min_pass_rateand metrics["unsafe_action_rate"] <= tasks.max_unsafe_rate)if meets_quality:scored.append((m, metrics["cost_per_pass"]))# 3. Among models that clear the quality bar, pick the lowest cost per passscored.sort(key=lambda x: x[1])return scored[0][0] if scored else None
7. Cost Optimization: Six Ways to Bring the Bill Down
- Tiered routing. Send simple classification and field extraction to a lightweight model with extended thinking off; send complex reasoning and code generation to a model with stronger reasoning. A small model can judge difficulty first and decide who handles the rest.
- Context compression. Summarize or truncate tool results, replace whole-document input with retrieval-augmented generation, and compress multi-turn conversations into “task state + key conclusions.” This pushes down both the linear and the quadratic term.
- Use caching. Keep fixed instructions at the start of the prompt, take advantage of the platform’s prompt caching, and add an application-level result cache for requests with stable outputs.
- Batch the offline work. Send non-real-time batch jobs through the batch channel and keep only online calls in the real-time path.
- Reason only when needed. Turn off or dial down extended thinking for simple tasks and reserve it for genuinely complex ones.
- Reinforce structured output. Output-format constraints plus well-chosen few-shot examples raise first-attempt compliance and cut retries at the source.
8. Common Mistakes
- Treating the context window as output length. Input and generation share the same window, and output is further capped by the per-call output limit.
- Treating a reasoning model as a conversational model. The thinking process consumes output tokens and raises latency. Simple Q&A should default to models where thinking can be switched off.
- Looking at unit price but not first-pass rate. Retries and human rework usually cost more than tokens.
- Judging on single-turn behavior. Agent capability must be measured on complete tasks; a good single answer does not mean error-free multi-step execution.
- Ignoring rate limits and SLAs. Throttling at peak makes tasks fail, and that impact can exceed the cost difference between models.
Conclusion
There is no “strongest model” in LLM selection, only the most suitable combination. The workable path: use hard constraints to confirm what is usable, a business task set to measure quality, cost per pass to compare price, and then routing plus context management to improve continuously. For high-volume work such as batch image/text extraction, start with a cheaper model to set a baseline and escalate difficult cases to a stronger one. For tasks that require understanding video directly, confirm candidate modality support first. For long-horizon agents, let one shared multi-step task set decide the final choice. Specs only describe what is possible; what actually determines success is repeated validation on your own business data.
