Chinese LLMs: Code Capabilities and Cost Optimization in Practice

As large language models enter the engineering deployment phase, the focus of development teams has shifted from “can the model answer” to “can the model reliably complete tasks with controllable costs.” Recent Chinese models have shown clear progress in three areas: terminal-style coding tasks, 3D content creation, and hallucination suppression. This article examines the technical logic and implementation approaches behind these capabilities from an engineering perspective, helping developers establish a reusable evaluation and integration framework.

1. Terminal-Style Coding Tasks: From Completion to Agent Execution

Traditional code assistants mostly operate in a “completion + Q&A” mode. Models output suggested snippets, and humans copy and paste them into place. Terminal-style coding benchmarks test a different capability: the model must autonomously plan steps in a real command-line environment, invoke tools, read execution feedback, correct errors, and ultimately complete a verifiable engineering goal. This places significantly higher demands on reasoning chain length, tool invocation discipline, and error recovery.

Recent public evaluations show Chinese models entering the top tier on such tasks, gradually narrowing the gap with leading international models. What deserves more attention from developers is the metric of “cost per completed task” — under equivalent success rates, the inference tokens and tool calls consumed per task directly determine the feasibility of large-scale adoption. Some models achieve strong cost control by compressing redundant reasoning during task planning and reducing ineffective retries during tool invocation. While they may not always lead in absolute scores, they demonstrate competitiveness in the performance-to-cost ratio.

Practical recommendations for development teams:

1. Differentiate task types before selecting a model. Short-chain completion tasks are latency-sensitive; long-chain agent tasks are sensitive to success rate and cost. These two categories should not rely on the same evaluation conclusions.
2. Build an internal task set. Public leaderboards reflect general capabilities. Teams should construct 50–200 real tasks based on their own codebase — including dependency installation, build failure recovery, and test case completion — as a selection baseline.
3. Track failure modes, not just scores. Record model behavior in scenarios such as “tool returns an exception” or “multi-file dependency conflicts.” This information is more actionable for engineering integration than an aggregate score.

A typical terminal agent invocation flow can be abstracted as the following pseudocode, which developers can use to design their own wrapper layer:

def run_terminal_task(goal, max_steps=20):
    history = []
    for step in range(max_steps):
        plan = model.plan(goal, history)          # Plan the next step
        if plan.is_final:
            return plan.answer
        result = sandbox.exec(plan.command)        # Execute in sandbox for isolation
        history.append({"cmd": plan.command, "out": result})
        if result.failed:
            history.append({"hint": "retry with fix"})
    return {"status": "timeout", "history": history}

The critical engineering points are sandbox isolation, a step limit, and execution log retention. Together, these three determine the controllability of agent tasks.

2. Cost Control: Balancing Frontier Capabilities and Inference Overhead

Cost issues are significantly amplified in agent scenarios. A single long-chain task may trigger dozens of model calls. If each call consumes excessive tokens, total cost grows linearly or even super-linearly. The industry is currently pursuing three main optimization directions:

• Inference-stage compression: Using more compact reasoning formats and on-demand reasoning depth to reduce wasted tokens.
• Tool call convergence: Optimizing planning strategies to reduce repeated attempts and ineffective commands.
• Caching and reuse: Applying prefix caching to stable system prompts and tool descriptions to lower repeat billing.

It is important to emphasize that low cost does not equal high cost-effectiveness. Some models achieve lower prices by simplifying reasoning, but their success rates on complex tasks drop noticeably, and the total cost actually rises. Developers should evaluate using the metric of “cost per successful task”: total invocation cost divided by the number of successful tasks. This metric more accurately reflects real-world production economics.

3. Script-Based 3D Modeling: A Direct Path That Bypasses Proprietary Protocols

In the 3D content creation domain, a significant capability leap is the ability to “directly generate executable modeling scripts.” Previously, having models operate 3D software typically relied on proprietary protocols or plugin layers for tool registration, resulting in long pipelines and high debugging costs. A newer approach is to have the model directly write scripts targeting modeling software — covering scene setup, geometry generation, material assignment, and layout adjustment — which the runtime environment then executes and reads back.

The advantages of this approach:

1. Shorter pipeline: Eliminating the intermediate protocol layer means the model output is directly executable, making version management and reproducibility easier.
2. Better debuggability: Scripts are plain text. When errors occur, you can pinpoint the exact statement rather than debugging an opaque tool call.
3. Easier batch production: Parameterized scripts are naturally suited for generating variants at scale, useful for asset library construction and rapid scene prototyping.

In public CAD benchmarks, models relying on tool invocation are already approaching top-tier scores, demonstrating that the “script + execution feedback” loop is sufficient to support relatively complex spatial reasoning tasks. Recommended practices include: setting resource and timeout limits for script execution, running static checks on generated scripts (e.g., blocking dangerous system calls), and retaining before-and-after scene snapshots for traceability.

4. Hallucination Suppression: From Evaluation Metrics to Engineering Guardrails

Model hallucinations are especially dangerous in coding and modeling scenarios because erroneous outputs often “look plausible” yet fail to run. Recent internal stress tests show that some models have reduced error rates on high-difficulty hallucination test sets from over 20% down to single-digit percentages — a notable improvement. However, for engineering teams, relying on model capability alone is insufficient. System-level guardrails remain necessary:

• Prioritize verifiable outputs: Code must pass compilation and tests; scripts must execute; geometric results must satisfy constraint checks.
• Cross-source validation: For critical conclusions, require the model to provide supporting sources, or run a consistency check with a second model.
• Rollback capability: All automatically generated artifacts should enter version control so anomalies can be quickly reverted.

5. Model Selection and Implementation Recommendations

Taken together, current Chinese models have reached production viability on terminal-style coding tasks, offer clear engineering value in cost control, and show substantive progress in script-based 3D creation and hallucination suppression. When selecting a model, teams should follow this sequence: first define task types and success criteria, then build an internal evaluation set, and finally compare models horizontally using “cost per successful task.” On the integration side, prioritize building three infrastructure components: sandboxed execution, log retention, and artifact versioning. These are the prerequisites for converting model capability into actual productivity. As model iteration accelerates, continuously maintaining evaluation sets and guardrails provides more long-term value than chasing a single leaderboard ranking.