Open-Source Models Make Further Gains: Terminal-Level Code Generation and 3D Script Modeling in Practice

1. Core Changes in Model Capabilities

A recent class of open-source models has demonstrated notable improvements across several high-difficulty tasks. These models typically build on large-scale pretraining and then undergo post-training through reinforcement learning, process supervision, and tool-calling training. The result is more stable performance in areas such as code generation, spatial reasoning, and financial analysis.

Public evaluation results show that the model’s performance on terminal-level coding tasks is particularly noteworthy. Relevant benchmarks primarily assess whether the model can complete file operations, script writing, debugging, version management, and dependency installation in a command-line environment based on natural language requirements. Compared with earlier models, the new generation shows significantly reduced error propagation in long-horizon tasks, higher tool-calling success rates, and fewer instances of giving up midway or producing invalid commands when handling complex tasks.

In the 3D modeling domain, the model no longer merely generates static descriptions or simple geometric shapes. Instead, it can directly output executable modeling scripts. By learning the scripting languages of professional modeling tools, the model can generate model structures with spatial consistency, including composite geometry, boolean operations, material binding, and lighting layouts. These capabilities provide a new automation entry point for industrial design, digital content production, and simulation scene construction.

At the same time, the model shows stronger capability in financial document analysis. For tasks involving cross-document financial data extraction, multi-step numerical reasoning, and table structuring, the model can maintain a low hallucination rate within a retrieval-augmented framework and produce more traceable reasoning paths.

2. Terminal-Level Code Generation: From Completion to Task Execution

Terminal-level code generation is fundamentally different from common code completion. The latter primarily predicts the next segment of code based on the current file context, while the former requires the model to complete goal-oriented complex tasks in a real command-line environment. Typical tasks include:

• Locating key modules based on repository structure
• Writing scripts for batch file processing
• Running tests and modifying code based on failure logs
• Installing dependencies and resolving version conflicts
• Using command-line tools for data cleaning and format conversion

These tasks place high demands on the model’s planning ability, environment understanding, and error recovery. A common failure scenario is when the model generates the correct command but fails to check the current working directory, resulting in incorrect file paths. Another is when the model, after a test failure, does not read the logs but instead repeatedly generates similar code.

The new generation of models has significantly improved on these issues through post-training. On public terminal task benchmarks, the model’s overall score on coding tasks has entered the top tier. Compared with similar open-source models, it shows more stable trends in task completion rate, average completion steps, and invalid command rate.

For developers integrating such models into local development environments, the following configuration approaches are worth considering:

1. Restrict execution permissions: By default, the model is only allowed to read files and run read-only commands. Write operations and installation operations require manual confirmation.
2. Log feedback mechanism: After each command execution, standard output and standard error should be fed back to the model to help it plan the next step.
3. Task decomposition: Break complex goals into multiple sub-tasks, recording state after each sub-task to prevent the model from losing track of the overall goal when the context becomes too long.

A sample workflow looks like this:

User request: Find all code locations in the project that use the legacy config option,
and generate modification suggestions
Step 1: Model reads the project file tree
Step 2: Searches for the config option keyword
Step 3: Collects matching files and line numbers
Step 4: Reads the relevant code snippets
Step 5: Outputs modification suggestions and an impact analysis

In this process, the model does not need to directly modify files. Instead, it outputs structured analysis results that developers review before executing changes. This pattern is better suited to current engineering practices and makes auditing and traceability easier.

3. 3D Script Modeling: Letting the Model Generate Executable Models

Traditional 3D modeling relies on manual operation of graphical interfaces, which is time-consuming and difficult to replicate at scale. Generative models have changed this workflow: users can describe model requirements in natural language, and the model outputs scripts for professional modeling tools, which are then executed by the modeling software to generate geometry.

The current model’s main capabilities in 3D modeling include:

• Parametric geometry generation: Generating parametric models based on dimensions, proportions, and shape requirements
• Spatial relationship understanding: Handling relative positioning, rotation, arrays, and nesting between components
• Material and lighting configuration: Generating basic material nodes, lighting layouts, and rendering settings
• Model revision and refinement: Adjusting local model structures such as hole diameters, chamfers, and wall thickness based on follow-up instructions

Public evaluation results show that, with tool support, the model’s 3D modeling task scores are approaching those of leading closed-source models. This means the model does not generate scripts in isolation but can iteratively revise them based on tool feedback. For example, when script execution fails, the model can read error messages and adjust parameters rather than regenerating a completely different script.

In practice, model-generated modeling scripts can be divided into several levels:

Level 1: Generate a single geometric body with transformation parameters
Level 2: Generate composite geometry with boolean operations and spatial arrangement
Level 3: Generate a complete scene with lighting, materials, and camera
Level 4: Generate iterable models with parametric modification and version management

For 3D printing scenarios, the model can generate model files with support structures, wall thickness optimization, and assembly clearances. Users do not need to master professional modeling script languages to quickly produce printable models. Generated models can be exported to common formats and imported into slicing software for further processing.

It is worth noting that the current model still has issues in 3D modeling, such as geometry intersection errors, abnormal normal directions, and imprecise dimensions. Therefore, models for critical engineering still require manual verification. The model is better suited for proof of concept, rapid prototyping, educational demonstrations, and batch creative generation.

4. Complex Financial Document Analysis: Multi-Step Reasoning and Hallucination Control

Financial document analysis has long been a challenging scenario for natural language processing. Financial reports, prospectuses, research reports, and contracts have complex structures, with data scattered across multiple tables and paragraphs and numerous cross-document references. The model must perform retrieval, extraction, alignment, calculation, and verification across multiple steps.

On complex financial question-answering benchmarks, the model has achieved leading results. The key improvements come from two areas:

1. Clearer multi-step reasoning paths: The model explicitly lists data sources, calculation steps, and intermediate results in its responses, making manual review easier.
2. Notably lower hallucination rates: When sufficient information is not provided, the model is more inclined to state that it cannot confirm rather than generate plausible-looking but incorrect data.

Hallucination control is particularly important in financial scenarios. In real-world deployment, it is recommended to introduce retrieval augmentation and rule-based validation mechanisms to cross-check model output against authoritative data sources. For example, perform format validation and range checks on critical figures such as amounts, dates, and ratios, and verify the existence of cited sources.

A typical financial analysis task flow looks like this:

Input: Three financial report documents
Question: Calculate the change rate of net operating cash flow over the last two quarters,
and compare it with the industry average
Step 1: Retrieve sections related to cash flow from the three documents
Step 2: Locate the values for the last two quarters
Step 3: Calculate the quarter-over-quarter change rate
Step 4: Extract industry average data
Step 5: Compare and provide a conclusion, while preserving citation locations

In this workflow, every step of the model’s output can be tracked and verified, reducing the black-box risk.

5. Safety Capabilities and Deployment Recommendations

As model usage expands in code execution and file operation scenarios, safety capabilities become an essential consideration. The model needs to identify potentially dangerous commands, sensitive path access, privilege escalation attempts, and other risks.

In internal safety testing, the model’s error rate on basic safety tasks has declined significantly compared with previous generations. Common safety tasks include:

• Identifying malicious patterns in scripts
• Determining whether a command carries destructive risk
• Analyzing unauthorized access behavior in code
• Refusing high-risk operations in permission-restricted environments

For enterprise deployment, it is recommended to add a security proxy layer around the model service. The security proxy is responsible for intercepting dangerous commands, recording operation logs, enforcing allowlist and blocklist policies, and providing safety context to the model. This architecture effectively reduces the probability of model misoperation while meeting audit requirements.

6. Overall Assessment and Engineering Implementation Considerations

From the overall evaluation results, the model has achieved a certain level of production usability on terminal-level coding tasks, 3D modeling tasks, and financial document analysis tasks. In particular, on the cost side, its per-unit inference cost on multiple tasks is lower than that of comparable closed-source models, making it suitable for teams that need batch invocation or local deployment.

For engineering implementation, the following points are worth attention:

• Task boundary definition: Clearly define the scope of tasks the model can complete autonomously and the steps that require manual confirmation.
• Context management: Long tasks require appropriate state compression and memory mechanisms to avoid context overflow.
• Evaluation system development: Build an internal evaluation set to continuously monitor the model’s performance changes across different tasks.
• Multi-model collaboration: Combine general-purpose models with specialized smaller models to strike a balance between cost and effectiveness.
• Observability: Record every tool call, parameter, and result from the model to facilitate troubleshooting and model optimization.

Overall, open-source models are rapidly approaching leading closed-source models in high-difficulty coding and spatial reasoning tasks. For development teams, the current stage is better suited for adoption through assisted development, prototype validation, automated script generation, and batch modeling, gradually expanding model usage scenarios within manageable boundaries. As model capabilities continue to improve, intelligent agents built around development workflows will deliver greater practical value.