Open-Source Models Make New Gains in Terminal Tasks and 3D Generation: Engineering Takeaways

Why This Update Deserves Developer Attention

In the latest round of open-source model updates, the most notable change is not better conversational ability. Instead, models are starting to move into terminal task execution, 3D creation, financial analysis, and safety research — scenarios much closer to real production environments.

In the past, many models scored well on public benchmarks but showed clear degradation when faced with complex tool calling, multi-step reasoning, or cross-document verification. This time, the shift is different: models are no longer just producing more human-like text. They are showing measurable improvement in actually completing concrete tasks.

For developers, this means several key changes:

1. Terminal tasks are moving from demo-ready to engineering-ready. Models are no longer limited to running a few commands. They can plan around a real objective, try different approaches, correct mistakes, and adjust their path when something fails.
2. 3D and spatial reasoning capabilities are now explicitly part of the model’s skill set. Previously, “multimodal” mostly meant images and text. Now models are beginning to understand geometric structures and spatial relationships, and can directly generate scripts usable in professional software.
3. The balance between cost and performance has become a new competitive focus. The goal is no longer simply achieving the highest score. It is about delivering comparable task performance while significantly reducing inference cost and API overhead.
4. Hallucination control is shifting from post-hoc detection to training-stage optimization. Error rates under stress testing have dropped significantly, which is especially critical for enterprise applications.

These changes make it worth re-examining model selection and application architecture from a developer’s perspective.

Terminal Tasks: From Writing Commands to Completing Work

In traditional evaluations, many models perform well on single-turn code generation tests. But in a terminal environment, a task is rarely as simple as “write a correct piece of code.” It involves:

• Understanding the current directory structure and project context
• Determining which file to modify and which command to execute
• Handling errors, retrying, and switching strategies
• Maintaining goal consistency throughout long-running tasks without drifting off course

As a result, terminal benchmarks are becoming an important reference for measuring a model’s real-world working capability. In this update, the model’s score on relevant terminal benchmarks rose to approximately 65.7, placing it in the top tier. More notably, this result was achieved at a relatively lower cost.

There is an easily overlooked point here: cost in terminal tasks is not fixed. It is strongly correlated with the number of retries, context length, and tool calls. A model that appears cheap but makes frequent errors can end up costing far more than a model with a higher per-token price but a high first-attempt success rate. Therefore, when evaluating terminal capability, you cannot look only at the per-call price. You need to look at the total cost of completing a task.

An engineering evaluation flow can be designed like this:

Task definition: In an empty repository, generate a runnable project structure based on requirements
Success criteria:
  1. The project builds successfully
  2. Core functionality tests pass
  3. No obvious security risks are introduced
Metrics:
  - Completion time
  - Number of tool calls
  - Number of failed retries
  - Total token consumption

Only when these metrics are tracked together does model comparison become meaningful. Otherwise, conclusions drawn from single-point capability scores tend to fail in production environments.

A New Path for 3D Generation: Writing Scripts for Professional Software

The most easily overlooked part of this update for non-specialist users — but the most impactful for creators and developers — is the improvement in 3D modeling capability.

Previously, there were three common approaches to text-to-3D:

1. Generative 3D models: The model directly outputs meshes or neural radiance fields. Fast, but with poor controllability, insufficient precision, and difficulty in editing.
2. Parameterized templates: Reliant on preset libraries, with limited flexibility.
3. Driving professional modeling software through Python scripts: The highest flexibility and controllability, but extremely demanding on the model’s language generation and spatial reasoning abilities.

The third path is becoming more viable. The model no longer needs to output complex binary model files. Instead, it generates a script that can be directly executed by professional software — for example, using Blender’s Python API to create geometry, set materials, and adjust lighting, or using CAD scripts to generate parametric parts.

This requires several key capabilities simultaneously:

• Spatial reasoning: Understanding how spatial relationships such as “above,” “symmetric,” “rotation,” and “array” are expressed in code
• Precise knowledge of script syntax: API calls cannot be generated from vague impressions, or execution will fail immediately
• Iterative modification: When a user says “move this part to the left and increase the hole diameter,” the model must locate the corresponding code block and modify it correctly

On relevant CAD benchmarks, the model achieved 92.7 points under tool-assisted conditions, approaching leading closed-source models. This is currently the best result achieved by an open-source model. For scenarios requiring batch generation and parametric design, this capability can significantly reduce the time cost of manual modeling.

A typical engineering workflow might look like this:

User input:
  "Create a 3D-printable desk lamp model with a circular base and hexagonal lampshade"
Model output:
  1. Structure analysis: base, support column, and lampshade
  2. Generate parametric script:
     - Base: cylinder, diameter 80mm, height 10mm
     - Support column: cylinder, diameter 15mm, height 120mm
     - Lampshade: hexagonal prism, inscribed circle diameter 60mm, height 50mm
  3. Set materials and export parameters

Users can import the script into modeling software for preview, adjustment, or direct export of files for 3D printing. This workflow has more engineering value than generating a non-editable mesh.

Online preview and editing capabilities are also worth noting. Users can complete initial model generation and viewing in a web browser without installing a complex local modeling environment, then decide whether to move into professional software for deeper processing. This lowers the barrier to 3D creation and enables cloud-based collaboration.

Financial Reasoning: Cross-Document and Multi-Step Numerical Challenges

Financial scenarios demand very different capabilities from general question answering. A complete investment analysis typically requires:

• Extracting key financial metrics from multiple earnings reports
• Cross-checking information across different documents
• Handling inconsistencies in units, accounting standards, and time periods
• Performing multi-step numerical reasoning where each step can affect the final conclusion

On OfficeQA Pro v2, a benchmark based on real financial documents, the model achieved 64.4 points, placing it among the leading models currently available. On tests covering the full investment workflow, it also improved significantly over the previous generation, reaching 58.6 points.

The key to this improvement is not “knowing more financial knowledge.” It is more stable long-range reasoning and cross-document information integration. Financial documents can run to hundreds of pages, with critical information scattered across footnotes, appendices, and the main text. The model needs to maintain a kind of working memory, keeping consistency across multiple sources and avoiding the loss of critical constraints as context grows longer.

For enterprise users, this means the model can be considered for embedding into more scenarios such as report generation, due diligence assistance, and financial anomaly detection. However, it is important to note that hallucinations carry an extremely high cost in financial contexts. A single incorrectly cited number can lead to a completely wrong analytical conclusion. Therefore, improved hallucination control is an important prerequisite for financial applications to become viable.

Hallucination Control: From Detectable to Suppressible

In this update, the model’s hallucination error rate in internal stress testing dropped from 23.4% in the previous generation to 8.3%. While results from different test sets cannot be directly compared, the trend itself is meaningful.

Reducing hallucination error rates has multiple engineering implications:

1. Lower fact-checking costs. If model outputs are reliable in most cases, the burden of manual review drops significantly.
2. Higher user trust. In many enterprise scenarios, users do not require the model to “know everything.” They require it to “clearly say when it does not know.”
3. Reduced risk of cascading errors. In complex workflows, an error in one step affects all subsequent steps. Improved reliability at each individual step is amplified through the entire pipeline.

That said, no stress test can cover all real-world situations. In production environments, it is still advisable to maintain appropriate fact-checking and result confirmation mechanisms. Especially in high-risk domains such as finance, healthcare, and law, model outputs should be treated as decision support rather than final conclusions.

The Cost Perspective: What Really Matters Is End-to-End Cost

A commonly discussed phenomenon in public comparisons is that some models have extremely low unit prices but underperform on complex tasks, while others perform well but cost more. This update attempts to find a better balance between the two.

But cost evaluation should not stop at “how much per million tokens.” A more reasonable approach is to calculate the end-to-end task cost:

Total task cost = input cost + output cost + tool call cost + retry cost + human intervention cost

Human intervention cost is often overlooked, but it may be the highest of all. A cheap model that requires human intervention three times to complete a task can be more expensive than a slightly pricier model that completes it in one attempt.

Therefore, when selecting a model, developers should build their own task-level evaluation sets rather than relying solely on public leaderboards. A practical approach is:

• Select 20 to 50 tasks highly relevant to your actual business
• Define clear success criteria for each task
• Record completion time, number of calls, token consumption, and human intervention count for each model
• Calculate the total cost of completing all tasks

The data generated this way is far more valuable than any public score. It directly reflects how a model performs in your own business environment and provides a more reliable basis for model selection and optimization.