Open-Source LLM Post-Training Breakthroughs: Engineering Practices from Code Execution to 3D Script Generation

1. The Evolution of Post-Training Paradigms: Why “Small Iterations” Deliver Big Jumps

A notable trend has emerged in the open-source model community recently: a model that, after just one round of post-training targeting specific capabilities, achieved substantial improvements across multiple challenging benchmarks. The value of such point-release updates is often underestimated. These updates do not pursue parameter-scale expansion. Instead, they unlock more of the base model’s latent capabilities through targeted data composition, reward model optimization, and reasoning pipeline restructuring.

From an engineering perspective, post-training offers far greater controllability than pre-training. Teams can construct high-quality trajectory data for vertical capabilities such as terminal interaction, spatial reasoning, and numerical computation over long documents. Combined with a hybrid strategy of process rewards and outcome rewards, this approach significantly improves model stability on multi-step tasks. This is precisely why open-source models continue to close the gap with leading closed-source models on frontier benchmarks.

The sections below analyze the technical approaches and practical implementation paths across four capability dimensions.

2. Code and Terminal Tasks: From Completion to Autonomous Execution

2.1 What Terminal Benchmarks Actually Measure

Terminal benchmarks such as Terminal Bench differ fundamentally from traditional code completion evaluations. They require the model to operate in a real command-line environment: understanding natural language instructions, planning multi-step operations, executing commands, reading output, and adjusting its strategy based on errors until the task converges. This is essentially an agentic task with tool feedback rather than a single-turn generation task.

Evaluation metrics typically cover three levels:

• Task completion rate: whether the final state meets expectations
• Interaction steps: whether the task converges in fewer rounds, reflecting planning efficiency
• Cost per task: including token consumption and tool call frequency, which directly determines usability

2.2 How Post-Training Improves Terminal Capabilities

To make a model work reliably on terminal tasks, post-training typically requires the following types of data:

1. Successful trajectories: complete operation sequences labeled by strong models or human annotators, covering environment exploration, plan formulation, execution, and verification
2. Failure recovery trajectories: deliberately injected erroneous commands or abnormal outputs to train the model to recognize failure signals and roll back
3. Tool call format constraints: unified interface conventions for command execution, file I/O, and result parsing to reduce format drift

In reward design, outcome rewards handle “getting it right,” while process rewards handle “doing it cleanly.” The latter can be implemented by penalizing redundant commands, repeated exploration, and ineffective retries. In practice, introducing process rewards can reduce the average number of interaction steps by 20 to 30 percent at the same completion rate, which is especially critical for cost-sensitive production environments.

2.3 Cost Control Is the Dividing Line for Engineering Deployment

A frequently overlooked fact: capability rankings on terminal tasks often diverge from cost rankings. Some lightweight models achieve extremely low per-task costs through aggressive inference budget compression, but their insufficient completion rates limit real-world usability. Conversely, more capable models can incur per-task costs too high for scaling without proper constraints.

Evaluation should therefore focus on the composite metric of cost per successful task, rather than cost alone or score alone. Practical optimization approaches include:

• Adaptive inference budgets: dynamically allocating thinking length based on task complexity
• Caching and reuse: session-level caching of repeated environment exploration results
• Early termination: proactively ending interaction once verification signals meet expectations

3. Direct 3D Script Writing: Unlocking Capabilities by Bypassing Intermediate Layers

3.1 From Plugin Dependence to Direct Script Generation

Early approaches to involving large models in 3D modeling relied primarily on intermediate protocol layers, such as wrapping modeling software capabilities as tools through some form of model context protocol, which the model would then invoke. This approach involves long pipelines, difficult debugging, and limitations imposed by the granularity of tool encapsulation.

The new approach lets the model directly generate native scripts for modeling software, for example, driving geometry creation, material assignment, layout adjustment, and subdivision modification through a scripting language. The model no longer needs to understand an intermediate protocol; it directly outputs executable code for the modeling tool. This yields two immediate benefits:

• Higher expressive ceiling: scripts can accomplish combined operations not covered by tool interfaces
• Shorter debugging path: errors point directly to script lines, enabling faster iteration loops

3.2 Spatial Reasoning and CAD Benchmarks

CAD benchmarks assess a model’s understanding of geometric constraints, dimensional relationships, and assembly logic. Typical tasks include generating parameterizable parts from natural language descriptions, adjusting structures under given constraints, and refining existing models locally.

These tasks place three core demands on the model:

1. Numerical precision: dimensions, angles, and coordinates must be self-consistent
2. Structural validity: generated geometry must satisfy manufacturability constraints
3. Editability: output should be a parameterized script, not a one-off mesh

In post-training, teams typically construct “description—script—rendered result” triplets as training data and use rendering consistency as a weak supervision signal to help the model establish the mapping between textual descriptions and spatial structures.

3.3 Online Preview and Export Pipeline

From an engineering deployment perspective, script generation is only the first step. The complete pipeline also includes:

Natural language description
   ↓
Parameterized script generation
   ↓
Online rendering and preview
   ↓
Interactive editing (material, geometry, layout adjustments)
   ↓
Export to printable format / video / dynamic image

The key to this pipeline is the stability of the intermediate representation. If script generation results are not reproducible, subsequent preview and editing become impossible. It is therefore advisable to introduce deterministic constraints at the generation stage, such as fixed random seeds, standardized units, and explicitly declared coordinate systems.

4. Financial Document Reasoning: Retrieval, Verification, and Multi-Step Numerical Computation

4.1 Breaking Down the Task Challenges

Benchmarks based on real financial documents typically present the following challenges:

• Cross-document verification: differences in how the same metric is defined across different reports
• Multi-step numerical reasoning: derived indicators that require connecting multiple statements
• Retrieval and localization: accurately locating relevant paragraphs and tables in long documents

This differs significantly from traditional QA tasks. Answers are rarely found in a single location. The model must proactively plan retrieval paths and execute computations.

4.2 Improvement Paths

Strategies that can be applied during post-training include:

1. Structured table pre-training: familiarizing the model with table hierarchies and merged cell semantics
2. Computation trajectory supervision: requiring the model to explicitly output intermediate calculation steps for verification
3. Definition alignment data: constructing paired samples of the same metric under different definitions to train the model’s discrimination ability

For hallucination suppression, internal stress testing is irreplaceable. By constructing highly misleading distractor documents and near-identical numerical values, teams can quantify how error rates change under pressure. Practice shows that after targeted post-training, error rates on such stress tests can drop significantly, which is crucial for compliance requirements in financial scenarios.

5. The Relationship Between Safety Capabilities and Post-Training Scale

One phenomenon worth continuous observation: as post-training scale expands, model performance on security tasks such as vulnerability identification and code auditing typically improves in tandem. The reason is that security tasks are essentially combinations of multi-step reasoning and tool feedback, sharing substantial underlying capabilities with terminal tasks.

However, two boundaries deserve attention:

• Capability improvement does not mean zero risk: identifying vulnerabilities and fixing them are different capabilities. The latter requires a stricter verification loop.
• Evaluation must cover adversarial scenarios: high scores on standard benchmarks cannot substitute for adversarial testing.

It is recommended that internal evaluation systems break down security capabilities into four stages: discovery, localization, verification, and remediation, measuring each separately to avoid a single aggregate score masking weaknesses.

6. Engineering Deployment Recommendations

Based on the analysis above, here are several recommendations for teams evaluating or introducing such capabilities:

1. Select models by task type: terminal interaction, script generation, and document reasoning place very different demands on models. Do not expect a single model to handle everything.
2. Establish a cost-per-success metric: combine completion rate with overhead to avoid being misled by any single dimension.
3. Prioritize reproducibility validation: especially for script generation tasks, determinism is a prerequisite for scaling.
4. Build internal stress test sets: general benchmarks only reflect average performance. Real business distributions require custom evaluation.
5. Keep human verification in the loop: in high-risk scenarios such as finance and security, model outputs should serve as initial screening, not final judgment.

The trend of open-source models approaching frontier performance through post-training is already clear. For developers, the real opportunity lies not in chasing leaderboard scores but in understanding the engineering constraints behind these capabilities and transforming them into production pipelines that run reliably.