The Evolution of LLM Post-Training: From Code Generation to 3D Scripting Capabilities

1. The Value of Post-Training Paradigms: Why Iterations Matter

The growth of large language model capabilities has gradually shifted from “scaling pre-training parameters” to “refining post-training.” Post-training typically refers to the process of strengthening a foundation model’s capabilities for specific task distributions through supervised fine-tuning, preference alignment, reinforcement learning, and tool-use trajectory training. Its core value lies in exchanging relatively controllable compute costs for significant leaps in vertical tasks.

Recently, a domestic model team released a post-training update on top of its foundation model. Official benchmark results indicate significant improvements in coding, 3D modeling, financial document analysis, hallucination control, and safety. Regardless of the specific brand, the technical route revealed by this update offers several noteworthy takeaways for developers:

• As post-training scales, model stability in long-horizon tool-use tasks improves significantly.
• Script generation capability has become a critical bridge connecting language models with professional software (such as modeling, CAD, and rendering tools).
• Cost control and performance enhancement can proceed in parallel; cutting-edge capabilities no longer necessarily equate to the highest inference costs.
• Hallucination and security vulnerability rates show a declining trend with training scale, though not linearly.

Let’s break this down by capability dimension.

2. Coding Capabilities: Engineering Tests on Terminal Tasks

2.1 What Terminal Benchmarks Evaluate

Benchmarks like Terminal Bench essentially require models to complete multi-step tasks in real terminal environments: reading files, executing commands, parsing errors, iterating modifications, and final verification. This is fundamentally different from simple “function completion” or “algorithmic challenges”—it tests the model’s planning and error recovery capabilities in uncertain environments.

Official data shows this post-trained version scored 65.7 on Terminal Bench 4.0, leading among domestic models, approaching top-tier closed-source models, and surpassing some overseas models of the same generation. The significance of this score lies not in the ranking itself, but in demonstrating that:

1. Multi-turn tool-use trajectory training is showing results. The model can read stderr, adjust parameters, and retry after a command failure, rather than giving up after a single generation.
2. Context management capabilities have improved. Terminal tasks often involve massive log outputs, requiring the model to filter out key error lines. This demands high information compression capability over long contexts.

2.2 Engineering Significance of the Cost Dimension

Official statistics mention that this model is one of the best-performing frontier models in terms of cost control on this benchmark. This is critical for developers: inference costs directly determine whether Agent-based applications can scale.

Here is an illustrative cost estimation framework (not real pricing data, just for explaining the methodology):

# Pseudo-code: Cost estimation framework for a single terminal Agent task
task_steps = 12          # Average interaction turns
tokens_per_step_in = 3000   # Input per turn (including history)
tokens_per_step_out = 400   # Output per turn
unit_price_in = 0.000001    # Placeholder unit price
unit_price_out = 0.000004
cost = task_steps * (tokens_per_step_in * unit_price_in +
                     tokens_per_step_out * unit_price_out)

As seen above, the number of turns and the history length per turn are the two multipliers of cost. If post-training enables the model to complete tasks in fewer turns (i.e., a higher “first-time success rate”), costs will drop multiplicatively. This is why the combination of “similar performance + lower cost” is often more attractive in actual engineering than “slightly higher performance + doubled cost”.

3. 3D Modeling: The Significance of Direct Script Writing Bypassing the Middle Layer

3.1 From “MCP Calls” to “Direct Script Writing”

The original text mentions a key technical shift: the model can directly write scripts for modeling tools to build assets, refine details, handle layout, and adjust materials and geometry, without relying on middle layers like the Model Context Protocol (MCP).

The difference here is worth expanding upon. The essence of MCP-like solutions is standardizing tool calls: the model outputs structured parameters, which are executed by external services. Its advantage is safety and controllability; its disadvantage is that expressiveness is limited by the interface granularity exposed by the tool. Directly generating scripts (e.g., Python API scripts for common modeling software) hands full programming capability to the model. While the upper limit of expressiveness is vastly increased, it also demands higher accuracy in API knowledge and geometric reasoning from the model.

A simplified script generation illustration:

# Illustration: Programmatic generation of a parametric geometry layout via script
def build_scene(objects):
    for obj in objects:
        mesh = create_primitive(obj["type"], obj["params"])
        apply_material(mesh, obj["material"])
        set_transform(mesh, obj["location"], obj["rotation"], obj["scale"])
    return export_scene()

The model needs to understand not just syntax, but also coordinate system conventions, units, parent-child hierarchies, material node connections, and other domain semantics. This is the true challenge behind improving scores in 3D modeling benchmarks (like CAD tests).

3.2 Benchmark Performance and Capability Positioning

Official data shows that, under tool-dependent conditions, the model scored 92.7 on a certain CAD benchmark, slightly lower than two top-tier closed-source models (95.9 and 96.2), but achieving the best result among open-source models. The interpretation of this position should be objective:

• A high score range (90+) indicates the model can stably complete standard modeling script tasks.
• The gap with top models (about 2-3 points) typically manifests in edge cases like complex assemblies, constraint solving, and abnormal geometry processing.
• The “tool-dependent” premise is crucial—meaning the evaluation environment provides an execution and feedback loop, allowing the model to iterate through trial and error, rather than blind writing in one shot.

3.3 Potential Impact on the Creative Workflow

Whether scripted 3D generation will “break out” into the creative field depends on three conditions:

Condition	Current Status	Impact
Script Executability	Stable for routine tasks	Lowers the barrier to entry
Result Editability	Generates parametric scripts instead of baked models	Facilitates secondary modifications
Platform Integration	Online preview, editing, and export pipelines are gradually improving	Shortens the path from idea to delivery

It is worth noting that parametric scripts offer better maintainability compared to directly generating mesh models: changing a single parameter adjusts size or layout, which is highly valuable in product iteration scenarios. Therefore, the direct script writing route might be more significant for independent creators and small teams than for large industrial pipelines—the latter’s requirements for precision, topology quality, and process constraints far exceed current model capabilities.

4. Financial Domain: Retrieval, Reconciliation, and Multi-step Numerical Reasoning

The difficulty of financial document analysis benchmarks (such as Q&A tasks based on real financial reports and announcements) lies in the combination of three factors:

1. Retrieval: Locating relevant sections from long documents.
2. Cross-document Reconciliation: Aligning the same metric across different documents with varying calculation standards.
3. Multi-step Numerical Reasoning: Calculating ratios, year-over-year growth, compound growth, etc.

Official data shows the model achieved a leading score on this type of benchmark, with significant improvements over its predecessor on another benchmark covering the entire investment workflow. The engineering value of these capabilities lies in: pushing “document understanding” from single-turn Q&A to auditable multi-step processes. For developers, the key design during implementation is to retain intermediate reasoning steps and citation sources for manual review.

Meanwhile, in internal hallucination stress tests disclosed officially, the error rate dropped from 23.4% in the previous generation to 8.3%. This drop indicates post-training has a substantial effect on “curbing fabrication,” but 8.3% is still not zero—in high-risk scenarios, citation verification and rule-based fallbacks are indispensable.

5. Security Capabilities and Post-Training Scale

The original text mentions that as post-training scales, the model’s capabilities related to cybersecurity have changed. This requires careful phrasing: security capability improvements usually encompass two aspects—defensive capabilities (identifying vulnerabilities, providing remediation advice) and potential misuse risks. Common industry practice involves adding safety alignment data and red teaming during post-training, alongside enforcing usage policy constraints.

Practical recommendations for enterprises include:

• Treat the model’s security outputs as auxiliary signals, not final verdicts.
• Always run code execution capabilities within a sandbox environment.
• Establish output audit logs for traceability.

6. Developer Selection Recommendations

Synthesizing the dimensions above, here are a few actionable criteria:

1. Look at task distribution fit, not overall leaderboard rankings. A model excelling at terminal tasks may not be suitable for document analysis.
2. Factor cost into evaluations. Measure the “completion rate / average turns / average token consumption” triplet using a fixed task set.
3. Prioritize capabilities that support scripted output. Whether for coding, modeling, or data processing, executable scripts are easier to integrate than natural language descriptions.
4. Retain manual review steps. No matter how low the hallucination rate is, it isn’t zero; high-risk pipelines must have a verification layer.
5. Focus on platform-side integration capabilities. Whether the online preview, editing, and export pipelines are complete directly determines the implementation cost.

Conclusion

The trend reflected by this post-training update is clear: model capabilities are shifting from “able to answer” to “able to execute”, and script generation is the core carrier of this transition. For developers, what truly matters is not the ranking on a specific benchmark, but whether these capabilities can be reliably orchestrated into their own engineering workflows. The balance between performance, cost, and controllability remains the ultimate basis for model selection.