Deep Evaluation and Practical Analysis of Chinese Open-Source LLMs for Software Development in Late 2025
This article examines the engineering capabilities of Chinese open-source large language models in frontend development. Based on a hands-on test in which models were asked to build a 10-page e-commerce website using open-source coding tools, it evaluates project architecture, TypeScript migration, responsive UI implementation, toolchain integration, and autonomous agent behavior, followed by practical recommendations for developers.
As large language models continue to evolve, the performance of Chinese open-source models in coding assistance has become an important topic for developers. To assess their real-world engineering capabilities, we conducted a practical test from start to finish: each model was asked to build a medium-to-large e-commerce website with 10 pages using open-source intelligent coding tools and without external intervention.
The test focused not only on code generation speed, but also on deeper engineering capabilities, including project architecture, type-system migration, responsive UI implementation, and toolchain integration through mechanisms such as MCP operations and command-line execution. The following sections summarize the results and discuss their practical implications.
1. Project Architecture and Engineering Design
Frontend architecture directly affects the maintainability and extensibility of a project. During the test, models from different generations demonstrated significant differences in their ability to abstract and organize application structure.
Some earlier models produced overly minimal project structures, typically creating only three basic directories for pages, components, and styles. This flat organization is inadequate for an application with 10 pages. It creates tight coupling between business logic and the view layer while failing to separate state management, API requests, routing configuration, and utility functions. Large-scale refactoring would eventually become unavoidable.
By contrast, some newer or specially optimized models showed a basic understanding of architectural design. They proactively separated UI components from layout components, distinguishing shared structures such as global headers and sidebars from independent business elements such as product cards and modal dialogs. This layered approach follows modern frontend best practices and reduces coupling between components.
Overall, however, most models remained at the “works well enough” stage rather than achieving elegant architecture. Developers still need to establish explicit architectural constraints in the initial prompt, including directory structure, component boundaries, and state-management conventions.
2. Coding Standards and Type-System Migration
TypeScript has become standard in modern frontend development, making a model’s ability to handle types an important measure of its programming quality. The migration from JavaScript to TypeScript exposed clear weaknesses in several models.
In one test, a model generated the full set of page code using TypeScript syntax but failed to perform strict type checking and validation. More than 20 type errors remained in the codebase. This kind of “pseudo-TypeScript” provides neither reliable compile-time safety nor a better development experience. Instead, it increases the mental burden on developers who must locate and resolve the remaining problems.
Some models also encountered reasoning loops while fixing type issues. They repeatedly revisited the same styling or type-compatibility bug without gaining a broader understanding of the surrounding context.
Other models performed much better during type repair. When faced with dozens of problems introduced by the type system, they were able to trace type-inference chains, define appropriate generics, and resolve the issues in a relatively smooth process. This suggests that a model’s ability to correct code may be a more meaningful indicator of engineering value than its initial generation capability.
3. Visual Fidelity and Responsive UI Implementation
When building an e-commerce site from scratch, visual completeness and mobile compatibility were key evaluation criteria. The test revealed major weaknesses in the CSS strategies used by some models.
A typical negative example involved a model that first wrote all HTML and JavaScript logic and only added CSS at the end. This approach conflicts with modern component-based development practices and produced poor final results. Because structure and styling were developed separately, the project accumulated numerous style conflicts and layout problems. Even after extensive later fixes, basic mobile compatibility remained difficult to achieve.
The completeness of individual pages was another recurring issue. Models often implemented the global header reasonably well but omitted important footer elements such as site maps and copyright information. Other common problems included disorganized product-card layouts on the homepage, uncontrolled image aspect ratios, and overly simple product-detail pages.
These issues indicate that current models still struggle with long-context engineering tasks. They tend to prioritize core logic while paying insufficient attention to secondary details and page completeness.
4. Toolchain Integration and Agent Capabilities
The ability to integrate deeply with development tools is a key factor in the transition from a code-completion assistant to an autonomous development agent. The models showed substantial differences in MCP operations and command-line execution.
A major weakness of some models was limited MCP capability. They could not independently use scripts to generate and initialize a project. Instead, they adopted a fallback strategy: generating a package.json file and asking the user to install dependencies manually. Although these models often had relatively low error rates in pure logical reasoning, their lack of environmental awareness and execution capabilities significantly limited their level of automation.
Other models successfully called command-line tools and independently performed tasks such as installing dependencies and initializing project scaffolding. Models with this kind of agent-based execution capability provided a substantially better development experience.
However, the interaction also exposed boundary issues. In one case, a model unexpectedly switched all subsequent output to English without being corrected or instructed to do so. This loss of language consistency suggests that multilingual instruction following still requires improvement.
5. Conclusions and Practical Recommendations
Based on the test results from late 2025, Chinese open-source LLMs have acquired basic capabilities for generating multi-page, structured frontend applications. Nevertheless, they continue to show clear weaknesses in engineering standards, implementation details, and autonomous agent execution.
Earlier models tended to be more conservative and stable, while newer models were more aggressive in toolchain integration but also more prone to unexpected behavior.
Developers applying these models in real projects should consider the following practices:
Define architectural constraints in advance. Do not expect a model to independently design a complex architecture. Specify the directory structure, component hierarchy, and state-management approach in the initial prompt.
Validate types incrementally. Avoid asking a model to generate an entire TypeScript application in one step. Generate code by module and require type checking after each stage so that compilation errors can be eliminated progressively.
Make mobile requirements explicit. Clearly require a mobile-first or responsive breakpoint strategy. Avoid workflows in which logic is written first and styling is added later. Structure, styling, and interaction should be developed together.
Evaluate agent permissions carefully. For models capable of executing command-line operations, define an allowlist of permitted commands. This helps prevent destructive actions and endless loops during automated bug fixing.
The programming capabilities of large language models will continue to improve through iterative development. For now, the most effective approach is to understand each model’s boundaries and adopt a human-in-the-loop workflow rather than delegating the entire development process to the model.
