Script-Direct 3D Creation: Evaluating LLM Capabilities and Deployment in Practice

1. From Benchmarks to Delivery: The Evaluation Perspective Is Shifting

If you follow the evolution of large language models closely, one trend stands out: the focus of evaluation is moving from “how human-like is the conversation” toward “can the task actually be delivered.” Early discussions centered on fluency and knowledge breadth. Today, what truly determines a model’s value is whether it can run code in a real engineering environment, produce usable 3D assets, and generate verifiable financial conclusions.

Three signals are worth noting:

1. Code evaluation is moving into real terminal environments. Earlier code benchmarks focused on function completion and isolated problems, which were disconnected from real development. Newer benchmarks place models in a full command-line environment and require them to handle environment setup, cross-file modifications, test execution, and final acceptance autonomously. The goal is to assess end-to-end engineering capability.
2. 3D creation is shifting from image generation to geometry generation. The route of generating images first and then reconstructing 3D models struggles to guarantee geometric accuracy and manufacturability. In contrast, parametric script generation lets models operate modeling software directly, producing editable and printable models.
3. Financial tasks now emphasize traceability. Real financial analysis involves retrieval, cross-document verification, and multi-step numerical reasoning. Any fabricated number invalidates the conclusion, making hallucination rate a hard metric.

The following sections break down each dimension.

2. Terminal-Based Coding Tasks: Real Engineering Capability and Cost Accounting

2.1 Why Terminal-Based Benchmarks Are More Convincing

Terminal-based coding tasks essentially treat the model as a “remote developer”: give it a clean environment and a task description, then require it to complete exploration, coding, debugging, and final acceptance on its own. These tasks naturally evaluate long-horizon planning, tool use, and error recovery, making them far closer to real work than single-point code completion.

Public results show that leading open-source models have entered the first tier on these tasks, narrowing the gap with the strongest closed-source models to a razor-thin margin. On some leaderboards, they have even pulled ahead. For enterprises that require private deployment, this means “open source is good enough” is becoming a reality rather than a wish.

2.2 Look Beyond Scores: Calculate the Cost

At comparable capability levels, inference costs can differ by several multiples across models. A rational selection process should focus on “effective capability per unit cost.” For the same terminal task, token consumption, API call counts, and retry rates should all be factored into the total cost. Industry observations confirm this: some models win on ultra-low unit pricing but have mediocre pass rates, while others strike a better balance between cost and pass rate. The latter often delivers better overall value in real business scenarios.

Practical recommendation: build a private evaluation set by sampling from your own real tickets, then rank candidate models using “pass rate × per-task value ÷ total cost.” This is more reliable than copying public leaderboards.

3. Script-Direct 3D Creation: Code as Creation

3.1 Two Technical Routes Diverge

There are two main approaches for involving models in 3D creation:

• Generative reconstruction route: generate an image from text first, then reconstruct the 3D model from the image. This approach delivers strong visual expressiveness but has uncontrollable geometric precision and messy topology, making it difficult to use directly for manufacturing.
• Script-direct route: the model generates executable scripts for modeling software directly, creating geometry through parametric methods, adjusting layouts, applying materials, and setting up lighting. Compared to indirectly operating software through plugins or tool-calling protocols, writing scripts directly bypasses the intermediate layer and offers greater flexibility. However, it places higher demands on the model’s combined spatial reasoning and code generation abilities.

In tool-assisted CAD-style modeling evaluations, advanced models have scored above 90 points, approaching the strongest closed-source models. The open-source camp has also reached parity with the first tier in this dimension for the first time. This indicates that while the script-direct route has a high barrier to entry, once crossed, the output quality and controllability significantly outperform pure generative approaches.

3.2 Illustrative Workflow and Pseudocode

A typical “natural language → parameters → geometry” workflow looks like this:

Requirements description → Model generates modeling script → Script executes to produce geometry
    ↑                                                          ↓
    └──── Modify parameters / provide feedback ←── Render preview and validation

Using “generate a 3D-printable desk lamp” as an example, the illustrative pseudocode is:

# Illustrative pseudocode: parametric generation of a printable desk lamp
lamp = Assembly("desk_lamp")
base = Cylinder(r=60, h=12)               # Base
arm  = ArcTube(radius=180, angle=120)     # Swing arm
head = Sphere(r=45, hollow=True)          # Lamp shade
lamp.add(base, arm, head)
lamp.union_at_joints()                    # Merge connection points
lamp.shell(thickness=2.4)                 # Uniform wall thickness
lamp.check(min_wall=2.0, max_overhang=45) # Printability check
lamp.export("desk_lamp.stl")              # Export to a common mesh format

Notice the “design intent” embedded in the script: wall thickness, overhang angle, and fillet radius are all explicit constraints. The model is not “drawing” a lamp; it is “defining” one. Changing a single parameter produces a variant, and the entire process is reproducible and versionable. That is the core value of parametric creation.

3.3 From Scripts to Usable Assets

Turning script-direct output into usable assets requires the final stretch:

• Online preview and editing: load the model directly in a web page, support parameter adjustments, and regenerate to form a “generate—preview—fine-tune” loop.
• Export pipeline: export common mesh formats such as STL for printing, or render videos and animated images for sharing.
• Automated validation: make wall thickness checks, overhang angle checks, and non-manifold detection standard pipeline steps. This can significantly reduce print failure rates.

This pipeline means product managers, teachers, and makers without modeling backgrounds can turn ideas into printable objects. Lowering the creation barrier is the real prerequisite for mainstream adoption.

4. Financial Scenarios: Retrieval, Verification, and Traceable Numerical Reasoning

Financial document analysis is a litmus test for model rigor. Its task characteristics differ fundamentally from general Q&A:

1. Multi-source retrieval: answers are scattered across annual reports, financial statements, and announcements, requiring precise localization.
2. Cross-document verification: the same metric may have inconsistent definitions across documents and needs cross-validation.
3. Multi-step numerical reasoning: reaching a conclusion from raw data often involves several calculation steps. An error at any step contaminates the result.

In evaluations covering a complete investment workflow (from screening targets and calculating metrics to profitability analysis), the newer generation of models shows significant improvement over the previous generation. Leading models can now handle most of the pipeline. To make model conclusions truly “citable,” three engineering practices are essential:

• Mandatory source attribution: every number must cite its source document and page.
• Numerical re-verification: use a code interpreter to recompute key calculations and compare results.
• Human review of critical conclusions: keep humans in the loop for high-risk decisions.

5. Hallucination Control: The Dividing Line Between “Can Talk” and “Can Be Trusted”

Hallucination rate is the dividing line that determines whether a model can enter production. Publicly disclosed evaluations show that the newer generation of models has reduced error rates in hallucination stress tests from over 20% in the previous generation to single-digit percentages. This order-of-magnitude improvement means model output is beginning to have a foundation of “trusted by default.” Human review can shift from sentence-by-sentence verification to spot-checking key conclusions.

Common approaches for managing hallucinations include:

• Retrieval augmentation: have the model answer based on provided materials rather than relying on parametric memory.
• Citation constraints: answers must link to sources. Content that cannot be traced must be explicitly marked as speculative.
• Self-verification: have the model perform consistency checks on its output and use tools to recompute numerical conclusions.
• Continuous stress testing: build regression test sets from confusable entities and error-prone calculations, then track them across version iterations.

6. Model Selection and Deployment Recommendations

Combining the dimensions above, here is an actionable evaluation framework:

Dimension	Key Question	Recommended Action
Capability	What is the pass rate on private tasks?	Build a private evaluation set and rank quantitatively
Cost	What is the total cost per task?	Track token consumption and retry rates
Risk	Does the hallucination rate meet business tolerance?	Stress testing + mandatory citations
Deployment	Is private deployment required?	Open-weight models + elastic cloud compute

For deployment, data-sensitive scenarios may prioritize private deployment of open-source models. If a cloud approach is chosen, general-purpose components such as model services, GPU compute, container platforms, and object storage from cloud providers can be used to assemble the inference and data pipeline (for example, platforms like Baidu AI Cloud offer such foundational components). Roll out gradually following the sequence of “offline evaluation → small-traffic canary → human review for critical business → full release” to avoid the uncontrolled risks of a big-bang switch.

7. Conclusion

Terminal-based coding, script-direct 3D creation, financial workflows, and hallucination control—these four dimensions together paint a picture of a model that “gets work done.” Among them, whether script-direct 3D creation can break into the mainstream creative community depends on whether the parametric workflow is smooth enough, whether printability works out of the box, and whether the creation barrier can be lowered to the point where users with no prior experience can get started. From a technology trend perspective, the “natural language → parameters → geometry” creation paradigm is already established. What remains is polishing the toolchain and building out the creative ecosystem. For developers, this is the optimal window to build evaluation systems and get the workflow running end to end.