Decoding the Leap in LLM Code Generation and 3D Modeling Capabilities

1. Background: The “Last Mile” from Base Capability to Usable Capability

Over the past two years, the focus of large language model competition has shifted from “pretraining parameter scale” to “usability gains brought by post-training.” The same base model, subjected to post-training of varying quality, can deliver dramatically different experiences to developers. The reason is straightforward: pretraining determines the upper bound of a model’s knowledge and reasoning, while post-training determines whether the model can reliably convert that upper bound into performance on real tasks—whether it can execute dozens of consecutive commands in a terminal and self-correct, or understand geometric constraints in 3D space and generate executable modeling scripts. These are not abilities that simply emerge from “having seen it during pretraining.”

From an engineering perspective, the value of post-training can be summarized in three points:

• Aligning to task format: Adjusting model behavior from single-turn Q&A to multi-turn agentic modes of “act—observe—correct.”
• Aligning to verifiable signals: Teaching the model to optimize its behavior toward verifiable outcomes (passing tests, successful compilation, satisfied constraints).
• Aligning to cost constraints: Reducing ineffective turns and token consumption while maintaining equivalent quality.

The following sections use two high-value directions—agentic coding and 3D modeling—to break down the technical logic behind these capability leaps and provide actionable evaluation and model selection methods.

2. How Post-Training Drives Capability Leaps

2.1 Three Mainstream Technical Approaches

Supervised Fine-Tuning (SFT): Directly teaches the model “how to do it” using high-quality task trajectories. For a coding agent, a trajectory typically includes: task description, initial environment state, multi-turn command execution, per-turn observed output, intermediate errors and corrections, and final verification results. Trajectory quality matters far more than quantity—trajectories containing real failures and recovery processes are especially effective at improving the model’s self-correction ability.

Verifiable Reward Reinforcement Learning: For tasks like code and 3D generation where objective judgment criteria exist, RL rewards need not rely on human preferences and can come directly from environment verification signals:

• Code tasks: unit test pass rates, compilation and static analysis results, whether runtime performance meets targets.
• 3D tasks: whether geometric constraints are satisfied (dimensions, spacing, fit), whether scripts execute successfully, and consistency between rendered output and intended goals.

This “verifiable outcome” characteristic makes reward signals difficult to game, and the resulting capabilities align more closely with real production scenarios.

Process Supervision and Iterative Correction: Beyond outcome rewards, scoring intermediate steps (e.g., “does the model correctly identify the root cause after an error”) significantly improves stability on long-horizon tasks and reduces the probability of the model going off track during multi-turn interactions.

2.2 Data Engineering Is the Key Determinant

The ceiling of post-training effectiveness is often determined not by algorithms but by data. In practice, the following require close attention:

• Trajectory collection: Record complete interactions in real or high-fidelity simulation environments, rather than isolated single-turn Q&A pairs.
• Difficulty stratification: Bucket tasks by difficulty to ensure hard samples receive sufficient training weight, avoiding a model that “repeatedly practices what it already knows and avoids what it doesn’t.”
• Decontamination: Remove samples that duplicate or closely resemble public evaluation sets to ensure credible eval results.
• Environment diversity: Cover different operating systems and toolchain versions to prevent the model from overfitting to a single environment.

2.3 An Example Training Signal

Input: task goal + environment observation
Output: model action (execute command / modify file / adjust script)
Reward computation (illustrative):
reward = 0
if all test cases pass:        reward += 1.0
elif partially pass:           reward += pass_ratio * 0.5
if interaction turns <= budget: reward += 0.2
if recovered after error:      reward += 0.1
if invalid repetitive action:  reward -= 0.1

This composite reward—outcome-primary, process-supplementary, negative-suppression—is a common approach in current post-training for coding agents.

3. Evaluating Coding Agents: Testing Models in Real Terminals

3.1 Why Evaluate in a Terminal

Traditional code evaluation mostly uses a single-turn pattern of “given a code snippet, run some tests,” which is far removed from real development. Real tasks are characterized by: stateful environments (the output of the previous command influences the next decision), diverse tooling (shell, editors, package managers), and indirect goals (“make this service run” rather than “write this function”).

As a result, a “terminal task evaluation” paradigm has emerged in the industry: provide the model with a real command-line environment in an isolated sandbox and a goal; the model must autonomously plan, execute, observe, and correct until the task is complete or the budget is exhausted. Scores from this type of evaluation correlate significantly more strongly with agent performance in real-world scenarios.

3.2 Key Components of an Evaluation Framework

A reproducible terminal evaluation framework consists of at least four components:

1. Sandbox environment: Containerized isolated execution environment with pre-installed task dependencies, ensuring identical initial states for every evaluation run.
2. Task definition: Initial state, natural language goal, and operational budget (maximum turns, maximum tokens).
3. Evaluation loop: Model outputs action → environment executes → observation returned → repeat.
4. Grader: Automated scoring based on objective criteria such as file assertions, service health checks, and test suites.

Illustrative code:

def run_episode(task, agent, max_steps=40):
    env = Sandbox(image=task.env_image)
    env.setup(task.init_script)                 # restore initial state
    history = [obs(env)]
    for _ in range(max_steps):
        action = agent.act(goal=task.goal, history=history)
        result = env.execute(action)            # execute in sandbox
        history.append(result)
        if result.contains_stop_signal:
            break
    score = grader.check(task.assertions, env)  # objective scoring
    metrics = {"score": score, "steps": len(history),
               "tokens": count_tokens(history)}
    env.destroy()
    return metrics

3.3 How to Read Leaderboards Correctly

When facing public evaluation scores, developers should cultivate four habits:

• Look at distributions, not just averages: Under the same total score, different models may vary dramatically on debugging, configuration, and refactoring tasks. Compare subsection scores against your own business task types.
• Look at variance and stability: Score fluctuations across multiple runs of the same task directly reflect the model’s reliability as an agent.
• Look at cost metrics: It is very common for models with similar scores to differ in cost by several multiples. Selection must be a two-dimensional comparison.
• Watch for data contamination: If an evaluation set has been widely publicized, cross-validate with a self-built evaluation set.

4. Getting the Cost Right: Performance–Cost Two-Dimensional Selection

4.1 Per-Task Cost Model

The cost of an agentic task is far more than just “how many tokens were output”:

per-task cost ≈ Σ(per turn: input tokens × input price + output tokens × output price)
             + fixed environment setup overhead
             + expected cost of failed retries

Three easily overlooked items deserve attention. First, in multi-turn interactions, each turn must carry the full history, causing input tokens to grow roughly quadratically with turn count. Second, tool execution results (logs, error stacks) are often long and expensive. Third, failure retry rates amplify total cost multiplicatively.

4.2 Engineering Approaches to Cost Reduction

• History compression: Summarize earlier turns, retaining only key states and conclusions.
• Tool output truncation: Keep only the tail segments of logs needed for analysis; deduplicate error stacks.
• Result caching: Reuse cached results for identical subtasks (such as environment setup).
• Tiered routing: Assign simple tasks to smaller models and escalate difficult tasks to flagship models, with confidence estimation for automatic routing.

4.3 Suggested Selection Process

Use a “three-dimensional matrix” for selection: task type × performance metric × unit cost. Start by building a small private evaluation set from your own business samples (even a few dozen tasks suffice). Then run complete episodes on candidate models, recording scores, turns, tokens, and end-to-end latency. Finally, decide based on the principle of “lowest cost above the passing threshold.” In practice, conclusions drawn from your own data often diverge from the intuitive impressions given by public leaderboards.

5. 3D Modeling and Spatial Reasoning: A Script-Driven Creation Paradigm

5.1 Why 3D Is a Hard Problem

• Spatial reasoning: Requires fine-grained reasoning over 3D coordinates, orientation, occlusion, and proportional relationships. “Rotate the chair 45 degrees and place it against the wall” involves continuous spatial transformation calculations.
• Geometric constraints: In industrial and architectural scenarios, dimensions, tolerances, and fit relationships must be precisely satisfied. “Close enough” is not acceptable.
• Long-horizon planning: A complete scene—from layout to materials to lighting—is a sequence of dozens of interdependent decisions.
• Complex interfaces: Professional modeling software exposes extensive scripting APIs that the model must accurately recall and combine.

5.2 Comparison of Two Interaction Paradigms

Dimension	Fine-grained tool calls	Whole-script generation
Interaction	One wrapped operation per step	Generate an executable script in one pass
Turns and cost	More turns, longer context, higher cost	Fewer turns, higher information density per output
Error recovery	Step-level rollback, fine granularity	Overall correction after error feedback
Reproducibility	Depends on session records	Script as an asset; versionable and reusable
Suitable scenarios	Exploratory creation with frequent trial and error	Structured, parameterized batch production

It is worth emphasizing the value of “script as an asset”: generating parameterized scripts means subsequent modifications only require changing parameters and rerunning—no need to restart the conversation from scratch. This allows 3D production pipelines to adopt software engineering practices such as version control, review, and batch variant generation.

5.3 A Script-Driven Workflow

# Modeling script generated by the model (illustrative; APIs are generic pseudocode)
scene = Scene()
# 1. Structure setup: parameterized generation for later adjustment
desk = scene.add_box(name="desk", size=(1.6, 0.8, 0.75))
desk.translate(z=0.375)                    # bottom face touches the ground
chair = scene.add_box(name="chair", size=(0.45, 0.45, 0.9))
chair.place_behind(desk, gap=0.05)         # constraint-based placement instead of hardcoded coordinates
# 2. Materials and detailing
desk.set_material(roughness=0.6)
chair.set_material(roughness=0.4)
# 3. Validation and export
scene.validate(min_gap=0.01)               # check for intersections and spacing constraints
scene.export("./assets/desk_scene")

The key to closing the engineering loop is the feedback path: when script execution fails, error messages are sent back to the model for correction. After rendering, screenshots can be fed back as multimodal input, allowing the model to iteratively adjust layout, materials, and lighting against the intended goal. The model handles “understanding intent—generating scripts—interpreting feedback,” while the modeling software handles “precise execution and rendering”—each doing what it does best.

5.4 How to Evaluate 3D Capabilities

A self-built 3D capability evaluation can design its grader across four dimensions: geometric accuracy (whether deviations from target dimensions, positions, or angles fall within tolerances), constraint satisfaction rate (whether fit, spacing, and alignment constraints all hold), editability (whether the output is parameterized and recalculates correctly after parameter changes), and visual consistency (semantic match between rendered results and text descriptions).

6. Enterprise Deployment: An Engineering Path from Evaluation to Production

1. Build a private evaluation set: Extract tasks from real business scenarios, covering high-frequency and edge cases, to serve as the yardstick for selection decisions.
2. Sandboxed execution: All model-generated code and scripts run in isolated environments with resource limits and allowlist mechanisms.
3. Trajectory retention and review: Record every interaction turn completely for regression testing and as data assets for future optimization.
4. Asset and state management: 3D artifacts, scripts, and evaluation results—unstructured assets—should be stored in object storage and placed under version control.
5. Observability: Track online task success rates, turn counts, and costs through logging services and monitoring alerts, with circuit breakers for anomalies.

At the infrastructure level, all of the above capabilities have mature cloud-native components: container platforms for sandboxes and elastic compute, object storage for artifact management, and logging and monitoring services for observability. Major cloud providers such as Baidu AI Cloud have productized these capabilities as standardized services, allowing teams to focus on business logic rather than building foundational infrastructure from scratch.

7. Conclusion

Improvements in evaluation scores are only the surface. Beneath them lies systematic progress across three dimensions: post-training methodology, evaluation systems, and engineering-driven cost control. For developers, a more valuable question than “which model is stronger” is: What is my task format, what evaluation should I use to measure it, and what engineering approaches should I employ to optimize cost and reliability? Once this methodology is established, teams can quickly validate and switch between models regardless of how they evolve, converting new capabilities into real productivity.