Open-Source LLM Post-Training: Engineering Practices from Agentic Coding to 3D Script Generation

Over the past year, competition among large language models has shifted from “pretraining parameter scale” toward “targeted enhancement during post-training.” Once a base model is released, targeted post-training is used to push vertical capabilities—coding, spatial reasoning, document analysis—to higher levels, and this has become a widely adopted technical route across the industry. This article does not discuss any specific vendor’s product. Instead, it uses the capability characteristics reflected in a recent class of open-source model updates as an entry point to review the evaluation logic and engineering practices behind post-training enhancements in several key technical directions, helping developers and enterprise technical decision-makers understand “what these scores mean” and “how to put them to work.”

1. Coding Tasks: Why Terminal-Level Benchmarks Deserve More Attention

Coding evaluation has long suffered from a mismatch: many benchmarks test “function-level completion” or “single-file algorithm problems,” while in real development scenarios the model faces multi-step terminal operations, dependency installation, error backtracking, and file-system state management. Such tasks are often called terminal-level or agentic coding tasks. The evaluation environment gives the model a container and requires it to accomplish a specified goal through commands; scoring is based on the final environment state rather than text matching.

These benchmarks are difficult for three reasons:

1. Long-horizon state dependency: the output of an earlier command affects later decisions, so the model must support a closed-loop “read the output, then act” workflow;
2. Error recovery: failed commands, missing dependencies, and permission issues are the norm; the model needs to diagnose problems rather than retry the same action;
3. Cost constraints: agentic tasks often require dozens of interaction turns, and token consumption and call counts directly determine usability.

So when we see an open-source model scoring close to top closed-source models on terminal-level coding benchmarks while driving per-task cost down to a low level, what really deserves attention is not a single number but whether the “performance–cost” curve has entered the range where engineering deployment becomes feasible. For enterprises, a simple two-dimensional evaluation can be set up during model selection:

• Horizontal axis: task pass rate;
• Vertical axis: average cost per task (including retries).

Only solutions falling within the “acceptable success rate + controllable cost” quadrant are suitable for CI pipelines or internal toolchains. Below is the schematic structure of an evaluation script for reproducing this kind of comparison in your own environment:

# Pseudocode: batch evaluation framework for terminal-level coding tasks
def evaluate_agent(model_client, tasks, max_turns=30):
    results = []
    for task in tasks:
        env = Sandbox(image=task.image)      # isolated container
        history = []
        for turn in range(max_turns):
            obs = env.observe()               # current terminal output and file state
            action = model_client.act(history, obs)
            env.exec(action.command)
            history.append((action, env.observe()))
            if env.task_solved(task.goal):
                break
        results.append({
            "task_id": task.id,
            "solved": env.task_solved(task.goal),
            "turns": turn + 1,
            "cost": estimate_cost(history),
        })
    return results

The value of this framework is that it breaks “scores” down into three observable dimensions: success rate, turn distribution, and cost. Many models perform similarly on easy tasks, but on long-tail tasks their turn counts balloon and costs spiral out of control—a problem that only self-built evaluation can expose.

2. 3D Modeling: Why Direct Script Writing Is More Flexible Than Tool Calling

3D content creation is one of the areas where LLM capabilities have spilled over most visibly in recent years. The traditional approach is to have the model operate modeling software through tool-calling protocols (such as function calling or MCP-style protocols), but this path has inherent bottlenecks:

• Tool interfaces usually expose only a limited set of operations, making complex modeling requirements hard to cover;
• Multiple rounds of tool calls introduce high latency and high failure rates;
• Adjusting layout, materials, and geometric details requires fine-grained parameter control, which the interface granularity does not provide.

The “direct script writing” route takes a different approach: the model directly generates the modeling software’s native scripts (for example, the Python scripting interfaces commonly supported by general-purpose 3D software, or parametric CAD modeling scripts), which the host software then executes. The benefits are:

1. Expressiveness equivalent to the software itself: the APIs a script can call represent the software’s full capability, unconstrained by a preset tool set;
2. Reusable and versionable: the generated script is itself an asset that can go into a code repository, be diffed, and serve as a parametric template;
3. Easy to modify locally: adjusting materials or layout only requires changing a few parameters in the script, rather than re-running the entire tool-call chain.

From an evaluation perspective, parametric modeling benchmarks (such as CAD-based modeling tests) examine the model’s understanding of geometric constraints, dimensional relationships, and operation ordering. A workable engineering pipeline looks roughly like this:

# Schematic: typical execution chain for script-based modeling
script = model.generate(
    prompt="Generate a script for an engine model with helical blades, including casing, rotor, and support structure",
    constraints={"units": "mm", "parametric": True},
)
validate_syntax(script)          # static check to catch obvious errors
result = run_in_host_app(script) # execute in the modeling software sandbox
if result.errors:
    script = model.repair(script, result.errors)  # self-repair based on error messages
export(result.scene, formats=["step", "stl"])     # export printable/interchangeable formats

Two engineering points deserve special attention: first, sandboxed execution—scripts generated by the model must run in an isolated environment to avoid arbitrary code execution risk; second, error feedback loops—structuring the host software’s error messages and passing them back to the model for repair has a significant impact on final success rates.

For teams that want to integrate 3D capabilities into their business processes, a pragmatic approach is to first build a “generate–validate–execute–repair–export” pipeline, then gradually accumulate a reusable script template library. The template library both reduces the uncertainty of each generation and serves as a source of fine-tuning data.

3. Document-Intensive Reasoning: The Real Challenges in Financial Scenarios

Financial document analysis is another capability significantly enhanced by post-training. The typical form of such tasks: given a batch of real disclosure documents (annual reports, quarterly reports, prospectus materials), the model must perform cross-document retrieval, numerical verification, and multi-step calculation. The difference from ordinary Q&A lies in:

• Answers are verifiable: numbers must be traceable to positions in the source text, and errors are costly;
• Multi-step reasoning chains are long: a conclusion often depends on three to five intermediate calculations;
• Distracting information abounds: documents contain many near-identical statements and differences in metric definitions.

When evaluating this capability, the industry typically uses two types of benchmarks: one focuses on “complex numerical reasoning over real documents,” and the other on “support for complete investment workflows,” examining each step from target screening to profitability analysis. Regardless of how a specific benchmark is designed, engineering deployment should track the following metrics:

Dimension	Observed Metric	Engineering Significance
Accuracy	Consistency between numerical answers and source text	Determines whether results can be trusted directly
Traceability	Whether citation locations are provided	Determines the cost of manual review
Hallucination rate	Share of unsupported statements	Determines whether risk control signs off
Stability	Variance across repeated runs of the same question	Determines whether the process can be automated

The hallucination rate is especially critical. Industry stress tests typically treat compressing the error rate from the low twenties (in percent) down to single digits as the usability threshold. For enterprises, a practical approach is to split model output into a “conclusion layer” and an “evidence layer”: the conclusion layer undergoes automatic validation (e.g., numerical ranges, units, and consistency of metric definitions), while the evidence layer is required to cite source text passages; only when both layers pass does the output enter downstream systems.

4. Engineering Lessons Behind the Capability Gains

Putting these three directions side by side reveals a common logic of post-training enhancement: it is not about making the model “know a little of everything,” but about giving it closed-loop execution capability for specific task formats. This yields three lessons for developers.

First, build your own evaluation. Public benchmark scores are only useful for initial screening; real business task distributions, tool versions, and document formats all differ. Following the evaluation framework approach in Section 1, migrate a subset of public benchmarks into your own environment and run them to obtain your own “success rate–cost” curve.

Second, treat cost as a first-class citizen. The cost of agentic tasks grows non-linearly with the number of turns; when selecting a model, be sure to include the costs of retries, repairs, and manual review. Cost-effective solutions are often not the cheapest per call, but the ones with the highest “right-the-first-time” rate.

Third, scripting is the lever that amplifies capability. Whether for coding tasks, 3D modeling, or data processing, having the model output executable scripts rather than just text turns model capability into reusable assets. The return on investment from supporting sandboxing, validation, and repair mechanisms is usually higher than simply switching models.

5. An Implementation Checklist

For teams preparing to integrate these capabilities, here is a concise checklist:

1. Clarify the task format: single-turn Q&A, multi-turn agent, or script generation? Different formats place very different demands on the model;
2. Build an isolated execution environment: container sandbox, resource limits, and timeout control are all indispensable;
3. Establish error feedback loops: structure execution errors and pass them back so the model can self-repair—this is the most cost-effective way to raise success rates;
4. Design the human-machine division of labor: keep manual review nodes for high-risk conclusions, and use citations and the evidence layer to reduce review costs;
5. Evaluate continuously: include success rate, turns, cost, and hallucination rate in routine monitoring, and compare across model versions.

The continued enhancement of open-source models during post-training is steadily pulling complex tasks—previously covered only by closed-source solutions—into the range of self-hostable, customizable deployment. For developers, the real opportunity lies not in chasing one-off leaderboard rankings, but in connecting these capabilities into their own engineering systems in a scripted, pipeline-based way—it is both technical work and an architectural decision.