How Domestic Large Models Are Closing the Gap in Code Execution and Spatial Reasoning

    A technical analysis of how post-training optimization, tool-free script generation, and long-chain document reasoning are driving capability leaps in Chinese frontier models, and what engineering teams should consider when evaluating them.

    From Post-Training Optimization to Multimodal Spatial Reasoning: A Technical Path Analysis of Capability Leaps in Domestic Frontier Models

    1. Background: Post-Training as the Primary Lever for Capability Leaps

    As large model iteration cycles accelerate, the capability boundaries of base models are gradually stabilizing, and the real differentiation increasingly comes from the design quality of the post-training stage. Post-training typically encompasses supervised fine-tuning, preference alignment, rejection sampling, reinforcement learning, and tool-calling trajectory distillation. One domestic model series recently chose to perform systematic post-training reinforcement on top of an existing base model rather than re-pretraining. This choice itself is worth discussing: it achieved significant gains on specific high-value tasks at lower compute cost.

    From publicly available evaluation results, the gains from this iteration are not evenly distributed but concentrated in three task categories: code execution in terminal environments, 3D modeling and spatial reasoning, and multi-step numerical reasoning over real documents. What these three categories share is that they all require the model to possess a composite capability of “long-horizon planning + precise execution + error self-correction,” rather than mere text generation fluency. Therefore, understanding this capability leap should start from task characteristics rather than looking only at scores.

    2. Code Execution: Optimizing Both Performance and Cost on Terminal Tasks

    2.1 Why Terminal-Style Benchmarks Are Closer to Engineering Reality

    Terminal-style benchmarks differ fundamentally from traditional code completion or function-level unit tests. They simulate the complete workflow of a real developer in a command-line environment: reading error messages, locating files, modifying code, running tests, and iterating based on output. The requirements these tasks impose on models include:

    • Environment awareness: understanding the current working directory structure, dependency state, and available commands;
    • Multi-step planning: decomposing a vague goal into an executable command sequence;
    • Feedback-driven correction: adjusting the next action based on command output (especially error messages);
    • Side-effect control: avoiding destructive operations and ensuring actions are rollback-safe.

    Because of this, terminal-style benchmark scores tend to better reflect a model’s usability in real engineering scenarios than pure code generation benchmarks. One domestic model achieved 65.7 points on this type of benchmark, ranking among the top and second only to a tier of leading models, while surpassing some international competitors from the same period. The significance of this result lies in the fact that a domestic model has entered the first tier on code tasks requiring continuous environment interaction — not merely performing well on static code generation.

    2.2 Cost Control: An Underrated Engineering Metric

    What engineering teams should pay more attention to than scores is cost. In public statistics, this model is the most cost-effective among comparable frontier models on terminal-style tasks. This point deserves elaboration:

    Token consumption for terminal-style tasks has a significant long-tail characteristic. A single task often involves dozens of interaction rounds, each requiring historical context to be carried, causing input tokens to grow at an approximately quadratic rate. If the model frequently makes trial-and-error attempts in early rounds, costs escalate rapidly. Therefore, low cost typically implies two optimizations:

    1. Higher first-attempt hit rate: the model’s inference about the environment before generating commands is more accurate, reducing invalid attempts;
    2. More concise interaction trajectories: single-round output is more focused, avoiding lengthy explanatory text that consumes context budget.

    By comparison, some models that emphasize lightweight low pricing may have a lower per-token price, but due to insufficient task completion rates, the cumulative overhead from retries is actually higher. This suggests that when evaluating model cost, one should adopt “task-level total cost” rather than “per-token price” — the former reflects real engineering expenditure.

    1. # Illustrative logic for task-level cost estimation (not a real API)
    2. def task_cost(model, task, max_turns=30):
    3. total_tokens = 0
    4. context = task.initial_context
    5. for turn in range(max_turns):
    6. output = model.generate(context)
    7. total_tokens += count_tokens(context) + count_tokens(output)
    8. if task.is_solved(output):
    9. return total_tokens, True
    10. context = task.step(output)
    11. return total_tokens, False
    12. # A low-price model with a low solve_rate may have a higher expected cost than
    13. # a higher-price model with a high hit rate

    3. 3D Modeling and Spatial Reasoning: From Tool Dependency to Direct Script Authoring

    3.1 Why Spatial Reasoning Is Difficult

    3D creation tasks pose challenges to models that are entirely different from traditional text tasks. They require models to simultaneously possess:

    • Geometric intuition: understanding object dimensions, proportions, relative positions, and occlusion relationships;
    • Structured representation capability: translating natural language descriptions into executable modeling instruction sequences;
    • Tool semantic understanding: mastering the semantics and side effects of various operations in modeling software (extrusion, boolean operations, material assignment, layout adjustment).

    In approaches that rely on external tool protocols (such as calling modeling software through standardized plugins), the model only needs to output structured parameters, and the tool handles execution. This approach reduces the model’s learning difficulty but introduces deployment and maintenance costs for the toolchain, and flexibility is limited by the interfaces the tool exposes.

    3.2 Engineering Value of the Direct Script Authoring Approach

    A notable change in this iteration is that the model no longer needs to rely on external tool protocols but can directly write scripts for modeling tools to accomplish asset construction, detail refinement, layout adjustment, and material and geometric parameter configuration. The engineering value of this shift is reflected in three aspects:

    1. Simplified deployment: no need to deploy additional tool services in the runtime environment, reducing the operational surface;
    2. Enhanced expressiveness: scripts can express complex composite operations not exposed by tool interfaces, such as procedural generation and parametric layout;
    3. Improved reproducibility: the script itself is a versionable, reviewable record of the modeling process.

    On a modeling benchmark under tool-dependent conditions, this model scored 92.7, slightly below the leading models’ 95.9 and 96.2, but it is already the best result among comparable open-source approaches. This gap indicates that the direct script authoring approach still has room for improvement in complex scenarios but is already practically useful for medium-complexity tasks.

    3.3 Online Preview and Export Loop

    Supporting online features extend modeling capabilities to the user side: users can preview and edit generated models directly on web and mobile, with support for export to printable formats, video, or dynamic photos. The significance of this loop is that it shortens the “generate—verify—share” path, moving 3D creation from professional tools toward lightweight collaboration.

    4. Document Reasoning and Hallucination Suppression: Reliability Assurance in Real Scenarios

    4.1 Challenges of Multi-Step Reasoning over Financial Documents

    Complex analysis tasks based on real financial documents typically involve retrieval, cross-document verification, and multi-step numerical reasoning. The challenges include:

    • Document heterogeneity: key data scattered across tables, footnotes, and appendices in inconsistent formats;
    • Numerical consistency checking: the same metric may have different definitions across documents, requiring cross-validation;
    • Reasoning chain length: going from raw data to conclusions often requires multiple calculation steps, and any single error can cause the final answer to deviate.

    Scoring 64.4 on this type of benchmark and leading all current models indicates that the model has advantages in long-chain numerical reasoning and cross-document alignment. On another benchmark examining full investment workflow support capability, the model scored 58.6, a substantial improvement over the previous generation and ranking second. This suggests that model capabilities are evolving from “single-point Q&A” toward “process-level support.”

    4.2 Quantified Progress in Hallucination Suppression

    Hallucination is the biggest obstacle to deploying models in high-value scenarios. In internal stress tests, this model’s error rate dropped from 23.4% in the previous generation to 8.3%, ranking second. This magnitude of improvement is noteworthy because it was not achieved through a simple refusal strategy (refusals reduce the effective answer rate) but by lowering the error rate while maintaining task completion. Common engineering approaches include:

    • Retrieval augmentation with citation constraints: requiring the model to anchor conclusions to document fragments;
    • Self-consistency sampling: performing multiple independent derivations of key numerical values and comparing them;
    • Explicit uncertainty: outputting confidence levels rather than fabricating answers when evidence is insufficient.

    5. Practical Recommendations: How to Evaluate and Select Such Models

    Based on the above analysis, here are several recommendations for engineering teams:

    1. Evaluate by task family: do not rely solely on composite leaderboards. Build small-scale evaluation sets targeting your own scenarios (terminal operations, 3D content production, document analysis), focusing on task-level success rates and total cost.
    2. Prioritize interactive tasks: score differences on terminal-style and modeling tasks tend to be larger than on text tasks. It is advisable to run end-to-end workflows in real environments before making selection decisions.
    3. Consider the maintainability of scripted capabilities: if adopting the direct script authoring approach, generated scripts should be brought under version control, and regression tests should be established to avoid output drift when models are upgraded.
    4. Include hallucination metrics in acceptance criteria: in high-value decision scenarios, a reduction in error rate has more business significance than a score improvement.

    6. Summary

    The core message of this iteration can be summarized in three points: the post-training approach can still deliver significant gains on specific high-value tasks; cost control should be treated as a first-class metric alongside performance; and direct script authoring and long-chain reasoning capabilities are pushing models from “assistive tools” toward “process participants.” For developers, the real value lies not in the scores themselves but in whether these capabilities can be reliably embedded into existing engineering systems — this requires parallel investment in evaluation methodology, cost accounting, and reliability assurance.