Chinese LLM Price Cuts and Agent Upgrades: How to Choose the Right Model
This article explains why prices for mid- and low-tier Chinese LLMs are falling, how newer models perform better at task planning and tool calling, and how to match models to specific workloads. It covers hybrid model routing, core Agent architecture components, the role of RAG, and how to evaluate models with your own data, including accuracy, task success rate, and latency. It also addresses the risks of multimodal Agents.
Price Cuts and Agent Upgrades Are Happening at the Same Time
In recent weeks, several Chinese vendors have released new model versions in quick succession. Two trends have appeared together. Prices for mid- and low-tier models have dropped, and new models have improved noticeably at task planning and tool calling. For enterprises and developers, the barrier to building AI applications is falling, but the complexity of model selection and architecture is rising.
1. Why Prices Are Falling
Price cuts for mid- and low-tier models come mainly from three kinds of engineering improvements, not only from price competition between vendors.
- Inference engine optimization: Techniques such as operator fusion, quantization, and speculative decoding can increase throughput per accelerator. The size of the gain depends on the model, hardware, and batching configuration. Use the vendor’s published benchmarks or your own measurements as the reference.
- Lighter architectures: Some models use a Mixture-of-Experts (MoE) architecture, which activates only a small number of parameters for each inference. This reduces GPU memory usage while keeping quality acceptable.
- Infrastructure scale: As AI computing centers and inference hardware deployments grow, the cost of each unit of compute goes down.
The size of the price reduction depends on the vendor’s pricing strategy, input and output length, batch discounts, and cache hit rate. The actual cost of the same model can differ several times between short-text and long-text scenarios, so looking only at the unit price can lead to wrong conclusions.
2. Changes in Agent Capabilities
Earlier models mainly handled question answering. Newer models are more practical in three areas:
- Instruction decomposition: Given a vague instruction, such as “organize last week’s sales data and generate a line chart,” the model can split it into steps: query the data, clean the fields, call a plotting function, and produce conclusions.
- Tool calling: In scenarios that involve multiple tools, such as search, code execution, and structured queries, the accuracy of function call parameters is higher than before. Invalid calls and fabricated results have decreased.
- Long-horizon state retention: Through mechanisms such as context compression, the model keeps key information across multiple turns, which supports tasks that run for tens of minutes.
These capabilities shift models from answering questions to executing tasks. As capabilities improve, permission controls and rollback mechanisms become equally necessary. These are covered below.
3. Where Mid- and Low-Tier Models Fit
Mid- and low-tier models are versions optimized for specific tasks, and their capabilities are not weak. When choosing a model, divide your workloads by task type.
| Scenario | Input/Output Length | Cost Sensitivity | Recommended Tier | Acceptance Metric |
|---|---|---|---|---|
| Support ticket urgency classification | Short | High | Mid- or low-tier | Classification accuracy |
| Field extraction from unstructured documents | Medium | High | Mid- or low-tier | Field recall and precision |
| Batch text rewriting | Medium | Medium | Mid- or low-tier | Manual spot-check pass rate |
| Multi-condition business decisions | Medium | Low | High tier | Consistency with business rules |
| Codebase-level analysis | Long | Low | High tier or long-context model | Accuracy of issue localization |
To put this into practice, set up hybrid routing. First, use a lightweight router model to assess the complexity of each request. Then send the request to the model tier that fits. Simple requests go to low-cost models, and complex requests go to higher-tier models. When a model version is updated, the routing boundaries may shift, so regularly re-run your evaluation set to check for regressions.
4. Basic Components of an Agent Architecture
As Agent capabilities improve, the surrounding system design matters more. Most Agents follow the ReAct (Reasoning + Acting) pattern: the model reasons, calls a tool, and then continues reasoning based on the result. In practice, an Agent usually needs three components: a planner, a tool registry, and a memory cache.
The code below is pseudocode that illustrates the scheduling logic. It is not meant to be run directly.
# Pseudocode: Agent task schedulingALLOWED_TOOLS = {"database_query", "http_request"}def execute_agent_task(instruction: str, max_steps: int = 10):# 1. Planning stage: have the model output a structured list of stepsplan = model.plan(instruction)if len(plan) > max_steps:raise ValueError("Plan has too many steps. Execution aborted.")result_accumulator = [] # Collect the result of each step# 2. Execution stage: call pre-registered tools step by stepfor step in plan:if step.type not in ALLOWED_TOOLS:raise PermissionError(f"Unauthorized tool call: {step.type}")try:if step.type == "database_query":# Only allow pre-registered query functions with validated parametersresult = query_mysql(step.parameters)elif step.type == "http_request":result = call_external_api(step.parameters)result_accumulator.append({"step": step.type, "result": result})except Exception as e:# Log the error and stop, so later steps do not run on bad datalog_error(step, e)raise# 3. Summary stage: pass each step's result to the model to generate the final answerreturn model.finalize(instruction, result_accumulator)
A few points in this code need attention. Tools must be registered on an allowlist, and any parameters generated by the model must be validated before use. Database queries should use parameterized statements instead of string concatenation. Operations that write, send, or delete data should require manual confirmation.
5. External Memory Still Matters
Context windows are getting larger, but in real business systems, retrieval-augmented generation (RAG) remains the main way to reduce hallucinations and connect models to real-time private data. In Agent scenarios, RAG should be part of the planning process: retrieve relevant information first, then build the plan, and then execute. Treating RAG as a simple retrieve-then-append step usually produces limited results.
6. Evaluating Models with Your Own Data
Benchmark scores published by vendors are only a reference. Model selection should be based on an evaluation set that reflects your own business.
- Building the evaluation set: Prepare about 100 prompts, stratified by type. Include routine requests, domain terminology, distracting information, and long-tail questions. Label each sample with a reference answer or a scoring rule.
- Task accuracy: Measure the proportion of model outputs that match the reference answers. The business team sets the pass threshold based on the cost of errors. For example, support ticket classification should reach a level where the cost of manual review is acceptable.
- Agent task success rate: Design a multi-step task, such as querying across tables, calculating a metric, and drafting an email. Measure the end-to-end completion rate and record the reason for each failed step.
- Latency: Track time to first token (TTFT) and time per output token (TPOT). These two metrics mainly determine how fast a response feels to users. Measure them at your target concurrency, not just for a single request.
Each time you change the model version or edit prompts, re-run the evaluation set to confirm that performance has not regressed.
7. The Limits of Multimodal Agents
Future competition will extend from text to image understanding, interface operation, and even robot control. Agents will gradually shift from digital assistants to executors that can operate systems.
These capabilities also carry higher risk. Interface operations involve account permissions, and a wrong action can cause financial or data loss. When deploying such Agents, limit the interface scope they can operate on, keep complete audit logs, require manual confirmation for critical steps, and design a rollback plan for each operation.
8. The Next Step for Inference Costs
After the price cuts, competition may shift toward inference speed. Models that are deeply optimized for specific hardware may deliver higher throughput at the same cost. However, the benefit of these optimizations depends on the specific hardware and deployment environment, so enterprises should run their own measurements before investing.
Conclusion
The recent price cuts have lowered the barrier for enterprises to use large models, and Agent upgrades allow models to take on more execution tasks. Cost risks remain. Enterprises still need their own data, engineering capability, and compute planning.
To put this into practice, start with three steps:
- Set up model routing to assign tiers based on task complexity, and re-test the routing regularly.
- Add caching, rate limiting, and budget alerts to API calls to prevent runaway costs.
- Measure accuracy, task success rate, and latency with your own evaluation set before deciding how far to replace existing models.
