LLMs Writing 3D Scripts Directly: From Benchmarks to Real-World Creation

1. A Paradigm Shift: From “Conversational” to “Operational”

Over the past two years, the coordinate system for evaluating large language model capabilities has fundamentally changed. Early benchmarks focused on knowledge Q&A, reading comprehension, and mathematical reasoning, measuring what could be called the model’s “static intelligence.” Today, dynamic execution tasks — terminal automation, code engineering, and script generation for professional software — have become the new focal point. The logic is straightforward: conversational ability is approaching its ceiling, and what truly determines a model’s production value is whether it can complete tasks in real environments — configuring environments, modifying code, running tests, self-correcting based on errors, and ultimately delivering results.

Recent public benchmark results clearly reflect this trend. On terminal automation benchmarks, the strongest closed-source flagship model scores around 66 points, while a new generation of post-trained open-source models has reached 65.7 points — compressing the gap to nearly negligible levels for the first time. More important is the cost dimension: these open-source post-trained models incur significantly lower costs than some high-consumption flagship models when completing equivalent tasks, placing them among the most cost-efficient frontier models. By contrast, some lightweight models known for low pricing remain inexpensive but show clear gaps in task completion quality. “Cheap” and “cost-effective” are not the same thing.

It is worth emphasizing that these leaps typically do not result from retraining base models. They come from “strong base model + systematic post-training”: through reinforcement learning aligned with tool use, long-horizon planning, and error recovery, models achieve generational improvements on multi-step execution tasks. This also explains why each post-training update brings a stepwise rise in benchmark scores.

2. Terminal and Coding Tasks: The Litmus Test for Agent Capabilities

Why are terminal tasks considered the litmus test for agent capabilities? Because they replicate an engineer’s daily workflow almost completely:

• Long task chains: A single task is often broken down into a dozen or more stages — environment setup, dependency installation, multi-file modification, build and run, test verification — where any single error causes downstream failure.
• State management and error recovery: The model must remember previously executed commands and their outputs, and determine whether to retry, switch approaches, or roll back when errors occur. This tests long-context utilization and reasoning ability.
• Verifiable results: Passing tests is an objective standard, avoiding the bias of subjective scoring, which makes these benchmarks more credible.

For enterprise developers, these capabilities map directly to high-frequency scenarios such as agentic programming, automated operations, and batch script generation. Cost is equally real: an agent task often involves dozens of tool-call rounds, with token consumption potentially exceeding ordinary conversations by two orders of magnitude. As a result, “cost per completed task” is becoming an evaluation metric as important as accuracy. If open-source models can approach flagship precision at lower inference cost, the savings for large-scale deployment are measured in orders of magnitude.

3. Direct 3D Script Writing: Generative AI Enters Professional Creative Pipelines

This is currently the direction with the greatest potential to break into mainstream creative workflows. Previously, the dominant approach for model-driven modeling software involved inserting a tool-calling protocol or plugin middleware layer between the model and the software: the model outputs structured instructions, and the middleware translates them into software API calls. The problem with this chain is that the middleware must be developed and maintained separately for each software package, the instruction coverage is limited, and multi-hop translation tends to accumulate errors.

“Direct script writing” means having the model generate executable scripts (such as Python) for modeling software directly — completing asset construction and refinement in a single step. The model is no longer “instructing someone else to operate the software” but “writing the program that operates the software itself.” This places higher demands on model capabilities:

• Spatial reasoning: Understanding 3D coordinate systems, transformation matrices, and parent-child hierarchies, and accurately determining object position and orientation in a scene.
• Geometry and topology: Generating base primitives, then performing Boolean operations, chamfering, and subdivision to ensure mesh structures are valid.
• Materials and lighting: Writing material parameters (metalness, roughness, texture channels) and lighting configurations to achieve visual realism.

Public benchmarks show that on CAD modeling benchmarks relying on toolchains, the top-tier models cluster in the 92–96 range. Among them, a post-trained open-source model has achieved 92.7 points, narrowing the gap with the strongest closed-source flagship to roughly 3 points — the best result for an open-source model on this dimension.

A typical direct-written script looks something like this (pseudocode illustration):

# Modeling script written directly by the model (illustrative)
scene = Scene.new()
desk = scene.add_primitive("cube", size=(1.2, 0.6, 0.75), pos=(0, 0, 0.375))
lamp = scene.import_model("desk_lamp.glb", pos=(0.45, 0.15, 0.75))
lamp.rotate(axis="z", degrees=35)
lamp.set_material(metalness=0.85, roughness=0.25)
scene.add_light(type="area", pos=(1.5, -1.0, 2.0), energy=800)
render(scene, camera=scene.add_camera(pos=(2.5, -2.5, 1.8)), samples=256)

This script covers the core stages of modeling: creating geometry, importing models, adjusting poses, setting materials, arranging lighting, and rendering output. The model can generate and iterate on such scripts based on natural language requirements. Public demonstrations have produced works of varying complexity, including interior scenes, complex mechanical structures, and 3D-printable desk lamps.

The accompanying online capabilities are equally critical: generated 3D models can be previewed and edited in a web browser, exported directly as 3D printing files, and rendered as videos or GIFs for easy sharing. This means the pipeline from “a one-sentence idea” to “a deliverable 3D asset” has been dramatically shortened.

The impact on creative communities can be understood at three levels. First, barrier collapse: people who cannot model can produce initial assets using natural language. Second, efficiency gains: professional artists can delegate repetitive scene construction to models and focus on creative judgment. Third, pipeline restructuring: AI-generated content must be integrated into existing professional workflows such as asset review, topology optimization, and rigging and animation. In the short term, this is more likely to be “human-machine collaboration” than replacement.

4. Financial Document Analysis: A Compound Test of Retrieval, Verification, and Multi-Step Reasoning

Finance is a natural proving ground for model rigor. Analysis tasks based on real earnings reports and research notes simultaneously test four capabilities:

1. Document parsing: Financial reports have complex layouts. Nested tables, footnote references, and multi-column formatting pose serious parsing challenges.
2. Retrieval and localization: Finding the location of key data points across hundreds of pages of documents.
3. Cross-document verification: The same metric may have inconsistent definitions or units across different documents, requiring identification and normalization.
4. Multi-step numerical reasoning: Unit conversions, year-over-year and quarter-over-quarter comparisons, exchange rate adjustments, and profit breakdowns all require multi-step calculations where any intermediate error propagates to the final conclusion.

Public benchmarks show that a frontier model achieves approximately 64 points on a Q&A benchmark built from real financial documents, leading all current models. On a benchmark covering the complete investment workflow of “target screening — due diligence — profitability analysis,” it also ranks in the first tier. The value of these benchmarks is that they simulate real analyst workflows rather than isolated calculation problems, making them directly relevant to quantitative investment research, risk control, and auditing scenarios.

5. Hallucination Suppression: Reliability Is the Entry Ticket to Production

Hallucination is an unavoidable obstacle for deployment in serious scenarios. Its sources fall into three main categories: fabrication caused by knowledge gaps, incentive bias in training objectives that prioritizes fluency, and overconfidence when no external factual anchor exists. Engineering approaches to suppression include:

• Retrieval augmentation: Anchoring conclusions in traceable document evidence.
• Self-verification: Performing consistency checks after generation and recomputing key numerical values.
• Abstention calibration: Training models to explicitly say “I don’t know” when evidence is insufficient, rather than guessing.
• Structural constraints: Using controlled output formats to limit free-form generation.

Public data shows that a frontier model reduced its error rate on a high-difficulty hallucination stress test from over 23% in the previous generation to around 8%, entering single-digit territory. The significance of this magnitude of change: a 23% error rate means one out of every five conclusions is unreliable, suitable only for auxiliary reference. At 8%, combined with citation traceability and human spot checks, the output can be deployed in production across a considerable range of business processes using a “human-in-the-loop” approach. Of course, in high-risk decision scenarios such as investment advice or compliance judgment, human review remains indispensable.

6. Cost, Open Source, and Model Selection: How Developers Can Evaluate Rationally

Faced with rapidly refreshing benchmark leaderboards, developers need a disciplined evaluation methodology:

1. Benchmark scores are not your scenario’s performance. The task distribution covered by public benchmarks may differ significantly from your business needs. Extract 50–200 tasks from real requirements to build an internal evaluation set and score it with automated scripts.
2. Measure performance and cost together. Record token consumption and call rounds for each task, calculate “cost per unit of effective output,” and avoid being misled by low per-call pricing.
3. Keep human review for high-risk scenarios. In finance, legal, and medical domains, model output should be positioned as “draft + recommendation,” with final decisions left to humans.
4. Pay attention to deployment and toolchain ecosystems. The advantages of open-source models lie in transparent weights, private deployment options, and the ability to fine-tune on vertical data — but you need to evaluate your own inference resources and operational capabilities.

For engineering implementation, it is advisable to leverage mature cloud infrastructure to lower the barrier to entry. Cloud platforms such as Baidu AI Cloud provide managed large model services, object storage, container platforms, logging services, and monitoring and alerting capabilities. Developers can build a complete “evaluation — canary release — production — monitoring” pipeline on top of these services, focusing their energy on business logic and workflow optimization rather than underlying resource operations.

7. Conclusion

From terminal coding to direct 3D writing, and from financial analysis to hallucination suppression, this round of capability leaps shares a common thread: models are no longer merely “answering questions” but “completing tasks.” The technical essence is systematic post-training on top of strong base models — treating tool use, long-horizon planning, self-correction, and factual calibration as explicit training objectives. Whether direct 3D script writing can truly break into creative industries depends on continued improvements in generation quality, professional pipeline compatibility, and cost. But for developers, a more definitive signal is clear: open-source models, with frontier-level performance, lower cost, and transparent weights, are becoming an option that must be taken seriously in agent application deployment. The rational approach is not chasing leaderboards, but validating against your own business evaluation sets — letting data make the decision for you.