The Evolution of LLM Post-Training: From Code Agents to 3D Modeling and Financial Reasoning

1. Background: Post-Training as the New Differentiator

Over the past year, the competitive landscape of Large Language Models (LLMs) has shifted focus from “pre-training scale” to “post-training quality.” While pre-training dictates a model’s breadth of knowledge and linguistic foundation, post-training—including supervised fine-tuning, preference alignment, reinforcement learning, and tool-use training—determines its usability in real-world tasks. A practical observation is that base models with similar capabilities can perform drastically differently in agentic and domain-specific tasks after undergoing varying post-training strategies.

A recent flagship model update followed this exact trajectory. Building upon its previously released base version, the model underwent systematic post-training enhancements, resulting in significant progress across three relatively independent capability tracks: code agents, 3D content creation, and financial document analysis, alongside a major reduction in hallucinations. This article does not serve as a product review but rather uses this update as a case study to deconstruct the technical focal points and engineering implications of the post-training phase, offering a reference for developers during model selection and system architecture.

2. Code Agents: Terminal Tasks as the New Benchmark

2.1 Why Terminal Tasks Are Harder

Traditional code generation evaluations largely remain at the “function-level completion” stage: given a function signature and comments, the model outputs the function body, which is then validated against unit tests. The limitation of such tasks is that they barely assess multi-step planning, environment interaction, and error recovery. Terminal-style benchmarks, on the other hand, require the model to operate within a real command-line environment to complete objectives like “locate and fix a build failure” or “implement a feature in a given repository and pass tests.” The model must:

1. Comprehend natural language requirements and decompose them into an executable sequence of steps;
2. Utilize command-line tools to inspect the environment, read files, and run tests;
3. Adjust strategies based on execution feedback, handling errors and edge cases;
4. Converge on a verifiable result within a limited number of steps.

These tasks are typically scored by “task pass rate” and demand exceptionally high model stability—a single erroneous command can corrupt the environment state.

2.2 Capability Structure and Cost Constraints

Based on public evaluation metrics, this new generation of post-trained models has entered the first tier in terminal-style task pass rates, narrowing the gap with top-tier closed-source models to single-digit percentages. Furthermore, at equivalent pass rates, the inference cost is significantly lower. This is highly meaningful for developers: the cost bottleneck for agentic tasks often lies not in the single-call price, but in the number of retries and the context length. A model with a 60% pass rate that consumes 3x tokens per task on average might actually be more expensive than a model with a 55% pass rate that succeeds on the first try.

Therefore, when evaluating code agents, it is recommended to monitor three metrics simultaneously:

• Pass Rate: The percentage of tasks successfully completed on the first attempt;
• Average Interaction Rounds: Reflects planning quality and error recovery efficiency;
• Token Consumption per Task: Determines the marginal cost of scaled deployment.

In engineering practice, a small-scale horizontal validation of candidate models can be conducted as follows:

# Pseudo-code: Batch evaluation framework for terminal tasks
tasks = load_benchmark("terminal_tasks.jsonl")
for model in candidate_models:
    results = []
    for task in tasks:
        env = Sandbox(image=task.image, timeout=300)
        trajectory = run_agent(model, task.instruction, env, max_steps=30)
        results.append({
            "passed": env.verify(task.tests),
            "steps": trajectory.steps,
            "tokens": trajectory.token_usage,
        })
    report(model, pass_rate(results), avg_steps(results), avg_tokens(results))

It is crucial to emphasize that such evaluations rely heavily on sandbox isolation, timeout controls, and reproducible image environments; otherwise, the results are not comparable.

3. 3D Content Generation: From “Calling Tools” to “Writing Scripts”

3.1 The Paradigm Shift in Capabilities

Early methods for LLMs participating in 3D creation primarily involved driving modeling software via tool-calling protocols (such as MCP-like solutions): the model outputs structured tool-calling instructions, which are executed by an external adaptation layer. While this approach offers clear interfaces and strong controllability, its capability is limited by the API granularity exposed by the tool—if the tool doesn’t provide a function, the model cannot use it.

The new paradigm directs the model to write modeling scripts directly, such as targeting the scripting interfaces of general 3D suites or CAD systems. The model generates complete parametric modeling code, which is then executed in the host environment. This shift brings several benefits:

• Higher Ceiling of Expressiveness: Scripts can combine basic geometric operations to achieve complex structures not directly exposed by tool APIs;
• Reproducibility and Versioning: As text, scripts naturally integrate with code repositories and diff workflows;
• Easier Parametric Adjustments: Modifying dimensions, materials, or layouts requires changing parameters rather than rebuilding from scratch.

The trade-off is a higher demand on the model’s spatial reasoning capabilities and familiarity with domain APIs. A single error in coordinate system conventions within the script can lead to a complete geometric misalignment.

3.2 Typical Tasks and Evaluation Dimensions

Based on public cases, these capabilities can already cover interior scene setup (e.g., vegetation and structural layout in large spaces), mechanical structure modeling (e.g., component assembly for rotating machinery), and printable model generation for additive manufacturing. Evaluations typically revolve around CAD-style benchmarks, assessing:

• Geometric correctness: Whether dimensions, topology, and assembly relationships match the description;
• Parametric degree: Whether adjustable parameters are output instead of hardcoded coordinates;
• Materials and layout: Whether material properties and spatial distribution are correctly set;
• Script robustness: Whether the script remains executable across different host software versions.

In practice, a typical script generation snippet looks roughly like this (illustrative):

# Illustrative: Parametric generation of a base plate with an array of holes
import cad_api as cad
base = cad.Box(width=120, depth=80, height=10)
holes = []
for i in range(4):
    for j in range(3):
        holes.append(cad.Cylinder(radius=3, height=12,
                                  center=(20 + i*25, 15 + j*25, 5)))
model = cad.subtract(base, cad.union(holes))
cad.export_step(model, "bracket.step")

The model needs to understand coordinate system conventions, boolean operation sequences, and export formats. An error in any of these will render the result unusable. Therefore, in actual deployment, it is recommended to build a closed loop of “script generation — sandbox execution — geometric validation,” replacing manual visual inspection with automated checks.

3.3 Online Preview and Distribution Pipeline

Supporting online capabilities typically include model preview, lightweight editing, exporting to printable formats, and exporting as videos or dynamic photos for sharing. The engineering value of this pipeline lies in directly integrating generation results into consumption scenarios: creators do not need to shuffle files between modeling software and sharing platforms. For enterprises, this means “text description → parametric model → printing/rendering” can be orchestrated into an automated pipeline.

4. Financial Document Reasoning: Retrieval, Verification, and Multi-step Numerical Calculation

4.1 Task Characteristics

Document question-answering in the financial domain differs fundamentally from general QA: answers are rarely found in a single document. They require cross-document verification (e.g., matching reports with footnotes), multi-step numerical reasoning (e.g., calculating ratios, year-over-year or quarter-over-quarter comparisons, adjusting accounting scopes), and accurate parsing of table structures. The challenges here include:

• Documents are lengthy and table-dense; retrieval recall quality directly determines the performance ceiling;
• Numerical calculations demand precision; any rounding error in intermediate steps propagates to the final answer;
• The model must distinguish between “numbers explicitly stated in the document” and “numbers requiring derivation.”

4.2 Evaluation Metrics and Capability Implications

Benchmarks targeting real-world financial documents typically cover retrieval, cross-document comparison, and multi-step numerical reasoning, scored based on numerical consistency between the answer and the ground truth. Other evaluations focus on complete investment workflows, covering the chain from target screening to profitability analysis, testing whether the model can maintain context consistency across multi-stage tasks.

Public results indicate that new-generation post-trained models have achieved leading levels on such benchmarks, with noticeable improvements in complete workflow tasks compared to predecessors. The takeaway for developers is that model selection for financial scenarios cannot rely solely on general QA scores. Focus should be placed on verifying table parsing accuracy, numerical calculation stability, and citation traceability. A practical approach is to build an internal “golden dataset”: extracting several real reports, manually annotating key values and derivation chains, and running batch regressions using fixed templates.

5. Hallucination Suppression: An Underestimated Foundational Capability

The biggest obstacle to deploying models in professional scenarios is often not a lack of intelligence, but rather “fabricating highly plausible falsehoods.” In internal stress-test benchmarks, the error rate of the new generation model has dropped significantly compared to its predecessor, entering a range close to top-tier models. Methods for hallucination suppression typically include:

• Training-side: Constructing “unanswerable” samples to train the model to explicitly express uncertainty when information is insufficient;
• Decoding-side: Introducing citation constraints, requiring answers to include verbatim excerpts from the source text;
• Engineering-side: Incorporating consistency checks and secondary verification into the Retrieval-Augmented Generation (RAG) pipeline.

It is important to note that the absolute value of hallucination metrics highly depends on how the test set is constructed; cross-vendor numbers should not be directly compared. Enterprises should build test sets based on their own business corpora and separately track “factual hallucinations” and “citation hallucinations.”

6. Implementation Recommendations and Risk Considerations

Synthesizing the four capability tracks above, here are several actionable recommendations:

1. Select Models by Task Type: Evaluate code agents by pass rate and interaction rounds; evaluate 3D generation by script executability and parametric degree; evaluate financial analysis by numerical consistency. Do not base decisions on a single aggregate score.
2. Factor Cost into Evaluation: The cost of agentic and long-document tasks is highly sensitive to retry counts. Always measure marginal costs under realistic workloads.
3. Build Closed-Loop Validation: Script execution must be sandboxed; numerical answers must be traceable; generation results must be reproducible.
4. Beware of Benchmark Overfitting: Public benchmark scores serve only as initial filtering signals. Final selection should be based on internal golden datasets.
5. Monitor Version Compatibility: Post-training updates may alter output styles and tool-calling habits. Conduct regression tests before upgrading.

Advancements in post-training are pushing LLMs from “being able to chat” to “being able to work.” However, the closer the capability gets to production, the stricter the engineering constraints become. Understanding the task structure and cost models behind each capability track is far more valuable than memorizing a few benchmark scores.