AI Agent Selection and Cost Control After Mid- and Low-Tier Model Price Cuts
This article explains how to evaluate AI agents and control costs when mid- and low-tier model prices trend downward. It shows why lower token prices do not necessarily lower cost per task, provides a worked cost example, lists seven selection metrics, describes five agent capability layers with engineering practices, and summarizes cost-control, risk, compliance, and rollout checklists. Readers can apply the formulas and metrics to compare models on real tasks and base decisions on monitoring data.
1. Start With the Conclusions
Public information suggests that prices for mid- and low-tier models are trending downward, while agent capabilities in multi-step reasoning and tool calling are improving at the same time. Three points matter in production.
First, a lower unit price does not mean a lower cost per task. The number of calls, retries, and human review all affect final spending. Second, the stability of an agent’s tool calls must be evaluated separately; general leaderboards cannot replace this test. Third, model versions and prices will change, so model selection must be retested continuously.
This article does not provide specific price-cut percentages. Prices should be based on the provider’s current price list.
2. The Cost Logic Behind Price Cuts
Model invocation cost is usually determined by input tokens, output tokens, and the corresponding unit prices. The cost per call can be written as:
Cost per call = input tokens × input unit price + output tokens × output unit price
If the provider supports prompt caching, input tokens that hit the cache are usually billed at a lower unit price. When calculating costs, cached and uncached portions must be counted separately; otherwise, the cost may be overestimated or underestimated.
To judge whether a model is cost-effective, look at the cost per task:
Cost per task = cost per call × average number of calls + failed retry cost + human review cost
The terms are defined as follows.
- Average number of calls: the average number of model requests used to complete one task, including planning, continuation after tool results are returned, and final output. The longer the context and the more content tools return, the larger this number usually is.
- Failed retry cost: the proportion of failed tasks multiplied by the cost of one complete task.
- Human review cost: the proportion of tasks that require human review multiplied by the labor cost of one review, then amortized across tasks.
The following worked example uses assumptions. Assume a small model has an input price of CNY 0.5 per million tokens and an output price of CNY 2 per million tokens. A large model has an input price of CNY 3 per million tokens and an output price of CNY 12 per million tokens. Both handle the same type of task, with an average of 4,000 input tokens and 500 output tokens per call.
The cost per call for the small model is 4,000 × 0.5 / 1,000,000 + 500 × 2 / 1,000,000 = CNY 0.003. The task uses an average of 6 calls, for a total call cost of CNY 0.018. The task failure rate is 20%, so the failed retry cost is about CNY 0.0036. The human review rate is 3%, each review takes 3 minutes, and labor is CNY 120 per hour. Each review costs CNY 6, which is about CNY 0.18 per task after amortization. The total is about CNY 0.2016.
The cost per call for the large model is CNY 0.018. It uses an average of 2 calls, for a total call cost of CNY 0.036. The failure rate is 3%, so the retry cost is about CNY 0.0011. The human review rate is 1%, which is about CNY 0.06 per task after amortization. The total is about CNY 0.0971.
In this example, the small model’s unit price is only about one-sixth that of the large model, but its cost per task is actually higher. The reason is that multi-turn calls and human review are amplified. To determine whether a price cut is beneficial, use real tasks to produce these numbers.
3. Selection and Evaluation Dimensions
At minimum, evaluate the following seven metrics during model selection. Each metric must be measured on the same evaluation set for all candidate models so that the results are comparable.
- Task completion rate: the proportion of tasks completed end to end and passing acceptance.
- Tool-call success rate: the proportion of calls with correctly formatted parameters and successful tool execution.
- P95 latency: the time within which 95% of requests return. It reflects user experience better than average latency.
- Cost per task: calculated according to the definitions in Section 2.
- Failed retry rate: the proportion of tasks that must be executed again.
- Human takeover rate: the proportion of tasks that require human intervention.
- Data compliance and observability: whether data processing terms meet requirements, and whether the input, output, latency, and cost of each call can be recorded.
Thresholds should be set by the business side, not copied from others. One approach is to define a baseline and a target threshold first. Models below the baseline are excluded directly, and only models that meet the target threshold enter canary release. The evaluation set should cover normal samples, boundary samples, and abnormal samples. The more samples, the more stable the conclusions. After a model version update, prompt change, or price adjustment, run the evaluation again.
4. Five Capability Layers of AI Agents and Engineering Practices
An AI agent is a system that breaks down tasks around a goal, calls external tools, and adjusts actions based on feedback. Its capabilities can be divided into five layers, each with corresponding engineering requirements.
- Tool calling. The model generates structured parameters according to predefined function descriptions, and the program executes the call and returns the result. Engineering practices include writing clear parameter descriptions for each tool, validating parameters with a schema, and returning validation errors to the model when validation fails. The core metric is the tool-call success rate.
- Planning and decomposition. The model breaks a goal into ordered steps. Engineering practices include setting a maximum number of steps and recording the plan for troubleshooting. Core metrics are step completion rate and invalid loop rate.
- Memory and context. The model relies on historical information to decide the next action. Engineering practices include generating rolling summaries, keeping only fields relevant to the current step, and limiting the length of tool responses. Core metrics are the number of input tokens per task and the retention rate of key information.
- Execution feedback. The model reads tool results and corrects its actions. Engineering practices include returning errors in a structured form, such as error code, field name, and reason, rather than only a free-text message. The core metric is the success rate of self-correction.
- Safety constraints. The model’s actions must stay within authorized scope. Engineering practices include least privilege, human confirmation before sensitive operations, and rule-based validation of outputs. The core metric is the number of unauthorized calls, which should normally be zero.
During implementation, check the following six items one by one.
- Least privilege: expose only the tools required for the task, and authorize write operations separately from read-only operations.
- Timeouts: set time limits for each model call and for the entire task, with a clear degradation path after a timeout.
- Retries and idempotency: set an upper limit on retries and use backoff intervals. Write operations should carry an idempotency key to avoid executing the same action twice.
- Monitoring and alerting: record the token count, latency, tool results, and cost of each call, and set alerts for failure rates and unit costs.
- Canary release and rollback: run new models on a small share of traffic first, and keep a switch to roll back to the old model.
- Human review: high-risk results should be confirmed by a person before being sent externally or written to a system.
Applicability boundary: AI agents are suitable for tasks with many steps, verifiable results, and rollback capability. If a task only needs one call to produce a conclusion, directly calling the model with rule-based validation is usually simpler.
5. Cost Control Checklist
The following methods can be used separately or in combination. Each method should be validated with monitoring data.
- Prompt caching: place fixed system prompts and documentation at the beginning of the request to use the provider’s caching mechanism. Track the cache hit rate, that is, the proportion of cached input tokens among total input tokens.
- Result caching: reuse historical results for identical or semantically equivalent requests. Caches should have an expiration time, and results involving personal information should not be cached.
- Tiered model routing: route simple classification and field extraction to small models, and complex reasoning to large models. Routing rules should be based on task type and checked regularly with the evaluation set.
- Small model as primary with large model fallback: validate small-model outputs first, and escalate to the large model when validation fails. Track the fallback ratio; if it is too high, the small model is not suitable for the task.
- Batching: combine requests for non-real-time tasks to reduce fixed call overhead.
- Context compression: remove history unrelated to the current step and limit the length of tool responses.
- Budget alerts: set spending limits by day and by business line, send reminders as limits approach, and pause non-critical tasks when limits are exceeded.
Commonly used measurement metrics include cost per 1,000 tasks, that is, cost per task multiplied by 1,000, as well as cache hit rate and fallback ratio. These numbers should be reviewed regularly.
6. Risks and Compliance
- Price and version instability. Providers may adjust prices or retire old versions. The system should be able to switch to an alternative model and rerun evaluations after the switch.
- Unit price advantages may be offset. Multi-turn correction, retries, and human review all increase costs, so cost per task must be calculated during model selection.
- Data compliance. Personal information, trade secrets, and unpublished business data should be reviewed before being sent to external services, and whether the provider uses data for training should be confirmed.
- High-risk scenarios. Outputs in fields such as healthcare, legal, and finance should not be used directly for decisions and require review by professionals.
- Log security. Logs are used for troubleshooting, but sensitive information should not be stored in plain text; it should be masked before storage.
7. Steps to Get Started
- Choose a high-frequency task with verifiable results as a pilot.
- Build an evaluation set covering normal, boundary, and abnormal samples.
- Run the same evaluation on both model tiers and calculate cost per task and the seven metrics.
- Decide whether to enter canary release based on thresholds, and enable monitoring and alerting during the canary period.
- After stabilization, expand to other tasks and retest regularly.
