The Leap in LLM Coding and 3D Generation: Technical Analysis and Practices

1. From “Chatting” to “Working”: The Shift in LLM Evaluation Paradigms

The iteration pace in the LLM space has noticeably accelerated recently. However, the real focus isn’t on parameter scales or benchmark scores, but rather the shift in evaluation dimensions: the industry is moving from multiple-choice knowledge tests to executable, task-based capability assessments. Taking recent model updates from various vendors as examples, three directions stand out:

• Terminal-level coding tasks: Models complete closed-loop operations such as environment configuration, bug fixing, and test passing in real command-line environments.
• 3D modeling and spatial reasoning: Models directly translate natural language descriptions into executable scripts for professional modeling software.
• Financial document analysis: Models perform retrieval, cross-document reconciliation, and multi-step numerical reasoning on real financial documents.

These three types of tasks share a common characteristic: verifiable results. Code either passes the tests or it doesn’t; generated 3D scenes either satisfy geometric constraints or they don’t; financial calculations are either correct or incorrect. Compared to subjective human scoring, verifiable evaluations better reflect a model’s true productivity. This is the fundamental reason why Verifiable Reward Reinforcement Learning (RLVR) has become the mainstream post-training technical pathway—reward signals no longer rely on human preference annotations but on objective environmental feedback.

For developers, understanding what these evaluations test and the underlying technical principles is far more valuable than memorizing a benchmark score. Let’s break them down one by one.

2. Coding Capability: Closed-Loop Execution of Terminal-Level Tasks

2.1 What Terminal-Level Evaluations Actually Test

Unlike traditional “function completion,” terminal-level coding tasks require the model to work like a junior engineer:

• Multi-turn interaction: The model doesn’t output code all at once. Instead, it executes commands, observes outputs, and adjusts strategies iteratively until the task is complete.
• Environment management: Handling real engineering issues like dependency conflicts, path permissions, and service startup failures.
• Implicit goals: Inferring acceptance criteria from an issue description rather than passively receiving explicit instructions.

These tasks assess a combination of the model’s planning, error recovery, and context management capabilities. Single-dimensional “code knowledge” is far from sufficient.

2.2 Core Technology: The Agent Execution Loop

The core architecture supporting this capability is the Agent loop—an iterative process of “Thought-Action-Observation.” A simplified schematic implementation is as follows:

# Agent execution loop for terminal-level coding tasks (schematic)
MAX_STEPS = 40
def solve_task(task_desc: str, sandbox: Sandbox, llm: LLM) -> Result:
    history = [system_prompt(task_desc)]
    for step in range(MAX_STEPS):
        action = llm.decide(history)           # Decide next step: execute command / modify file / submit
        if action.type == "finish":
            return verify(action.patch)        # Run test suite to verify if the task is truly completed
        observation = sandbox.execute(action)  # Execute command in an isolated sandbox
        history.append(compress(observation))  # Compress observation to control context length
    return Result(False, "Exceeded step limit")

There are three key engineering points:

1. Sandbox isolation: Commands generated by the model must be executed in a containerized environment to prevent damaging the host machine.
2. Context compression: Command outputs from long tasks can quickly exhaust the context window. Logs need to be truncated, summarized, and retain only key information.
3. Verification closed-loop: Task completion must be determined by a test suite, not model self-evaluation. This is crucial to prevent the model from “feeling good about itself.”

2.3 Cost Dimension: Look Beyond Scores to the Pareto Frontier

When new models are released, alongside performance improvements, cost control is becoming an equally important metric. For the same evaluation task, token consumption and retry rates can vary by multiples across different models. Enterprises should focus on the “performance-cost” Pareto curve during selection: only models with lower unit-task costs at equivalent pass rates have value for scaled deployment. Additionally, scenarios must be distinguished—offline batch processing can accept longer inference times in exchange for lower costs, while online interactive scenarios are latency-sensitive. The optimal choice for these two is often different.

3. 3D Creation: The Technical Logic of Direct Script Generation

3.1 Comparing Two Technical Pathways

There are currently two main industry pathways for involving LLMs in 3D creation:

• Pathway A: Tool Protocol Bridging. The model outputs structured calling instructions, which are forwarded through an intermediate protocol layer to the modeling software’s plugin for execution. The advantage is safety, controllability, and easy auditing; the downside is a long chain. The model’s capability is limited to a predefined set of interfaces, and the middleware itself becomes an integration and maintenance cost.
• Pathway B: Direct Script Generation. The model directly generates modeling software scripts (usually Python-style code executed by the software’s built-in interpreter) executed natively. The advantage is a high ceiling for expressiveness—theoretically accessing the software’s full API without an extra middleware layer. The downside is the strict requirement for code correctness; errors require the model to iterate and fix itself.

Recent model improvements in 3D tasks largely stem from the increased feasibility of Pathway B: the model’s code synthesis and spatial reasoning capabilities are now strong enough to directly write long, correct modeling scripts.

3.2 Why Spatial Reasoning is Difficult

3D modeling demands far more from a model than standard code generation:

• Coordinate systems and units: The position, rotation, and scale of all objects exist in a 3D coordinate system. A single symbolic error leads to model piercing or floating.
• Geometric constraints: Alignment, clearance, and symmetry between objects require the model to perform spatial calculations “in its head.”
• Parametric thinking: An efficient approach isn’t placing objects one by one, but using loops and functions to batch-generate regular structures.
• Material and lighting mapping: Mapping natural language descriptions like “brushed metal” or “diffuse white wall” into executable material parameters.

A schematic modeling script directly written by an LLM is shown below (the API is illustrative and executed by the modeling software’s built-in interpreter):

# Modeling script directly written by LLM (schematic API)
scene = Scene(unit="meters")
# 1. Large space shell
hall = scene.add_box(size=(120, 60, 12), center=(0, 0, 6))
hall.material = "concrete"
# 2. Parametric batch generation of seat array
for row in range(10):
    for col in range(30):
        seat = scene.add_box(size=(0.5, 0.5, 0.45),
                             center=(-40 + col * 1.2, -20 + row * 1.2, 0.225))
        seat.material = "fabric_gray"
# 3. Export for rendering, 3D printing, or online preview
scene.export("hall.fbx")

3.3 Engineering Practice Recommendations

When building a 3D generation workflow based on the direct script generation pathway, the following practices are recommended:

• Layered prompting: First, have the model output a scene structure list (objects, quantities, dimensions, materials). After manual confirmation, generate the script to avoid rework caused by trying to do it all in one step.
• Render feedback loop: Feed rendering screenshots back to the multimodal model for self-checking. Automatically correct the script upon detecting model piercing or proportion issues, forming an iterative loop of “generate-render-evaluate-correct.”
• Script as code: Generated scripts natively support version control management, enabling review, rollback, and reuse. This is a significant engineering advantage over manual modeling.
• Platformization support: Capabilities like online preview, editing, exporting print files, or sharing videos require cloud-side infrastructure support—object storage to manage model assets, container or function compute to carry format conversion tasks, and message queues to decouple asynchronous export processes.

4. Financial Analysis: Retrieval, Reconciliation, and Multi-Step Numerical Reasoning

4.1 Task Difficulties

Real-world financial document analysis is a recognized high-difficulty task. The challenges include:

• Complex document formats: Scanned copies, nested tables, footnotes, and cross-page data pose severe tests for document parsing.
• Cross-document reconciliation: Corresponding relationships between subjects across different statements require domain knowledge.
• Multi-step numerical reasoning: Locating data, calculating, and then comparing with previous years to determine trends. An error in any step propagates to the final conclusion.

4.2 Recommended System Architecture

Production-grade financial analysis systems typically use a layered architecture:

• Parsing layer: Layout analysis and table structure recognition convert unstructured documents into structured data.
• Retrieval layer: Hybrid keyword and vector retrieval, balancing exact matching with semantic recall.
• Reasoning layer: The model handles understanding and planning, while numerical calculations are handed over to code interpreters or calculation tools. Avoid letting the model “mental math” at all costs.
• Validation layer: Every number must carry a source citation. The system automatically traces back to the original text for verification.

4.3 General Techniques for Hallucination Suppression

Recent model improvements in hallucination control are backed by a combination of general techniques that enterprises can adopt as needed:

1. Verifiable reward training: Train the model with objective right/wrong signals, making “I don’t know” better than “fabricating.”
2. Refusal and uncertainty expression: Allow the model to output “insufficient evidence” rather than forcing an answer.
3. Self-consistency sampling: Sample the same question multiple times and take the majority-consistent result to eliminate occasional errors.
4. Mandatory citations: All numerical conclusions must be accompanied by original text positioning. If positioning fails, it is deemed untrustworthy.
5. Continuous stress testing: Construct error-prone question sets (similar subject names, unit conversions, cross-page tables) for regression testing to prevent degradation introduced by version iterations.

5. Enterprise Implementation: How to Evaluate and Select Models

Considering the above capability trends, here are four actionable recommendations for engineering teams:

First, build custom evaluation sets instead of blindly trusting public leaderboards. Extract real tasks from business logs (real issues, real statements, real modeling needs), define verifiable pass criteria, and cover edge cases. Public leaderboards reflect the upper limit of general capabilities, but your business performance depends on the overlap between your business distribution and the leaderboard’s distribution.

Second, use “unit task cost” as the budget model. Record the token consumption, retry rate, and manual intervention rate per task to calculate the true cost, rather than just comparing model unit prices.

Third, prioritize safety governance. Scripts and commands generated by the model must be audited before execution: sandbox isolation, least privilege, whitelisted commands, and manual confirmation for sensitive operations. This is especially true for the direct script generation pathway—the stronger the capability, the more gates are needed.

Fourth, leverage cloud infrastructure combinations. Implementing LLM applications is rarely solved by “one model.” It is an engineering combination of model services, container and function compute, object storage, logging services, and monitoring. Taking Baidu AI Cloud as an example, its LLM services and supporting compute scheduling, storage, and monitoring products can support the complete workflow setup from evaluation and fine-tuning to Agent deployment. Teams can map their architecture layers accordingly.

6. Future Trends

Overall, the evolution of LLM capabilities shows three clear main lines:

• From “writing code” to “operating software”: Models directly driving the script interfaces of professional software is becoming a universal interaction layer for professional production tools. 3D is just the first conquered domain.
• Multimodal feedback loops: Rendering images and execution screenshots flowing back to the model for self-checking. The automated iteration of “generate-verify-correct” will significantly reduce human intervention.
• Verifiability and traceability as adoption prerequisites: As models enter high-risk scenarios like finance and engineering, hallucination suppression and numerical traceability will determine enterprise adoption far more than raw performance.

For developers, the most worthwhile action right now isn’t chasing every model release, but building robust evaluation systems, feedback loops, and safety gates. Once the infrastructure is ready, every model iteration can be instantly transformed into business value.