A Practical Evaluation of Domestic Open-Source LLMs for Front-End Code Generation

1. Introduction: LLM Coding Tools Enter a Phase of Practical Scrutiny

In recent years, large language models (LLMs) have seen growing adoption in code generation. In particular, some Chinese models marketed as “open source” and claiming to “match top international models” have drawn considerable attention and discussion within the developer community. Yet a significant gap often exists between marketing narratives and actual capability—can a model truly take a natural-language requirements document and deliver a complete, accurate, runnable front-end project? This article uses a full front-end project specification as a test sample to systematically evaluate the code generation capabilities of mainstream Chinese open-source LLMs, and to discuss their open-source attributes and AI development trajectories in depth.

It should be noted that all tests in this article are based on a general front-end technology stack (the React ecosystem). They are not intended to promote or disparage any specific product, but rather to provide an objective analysis from a technical capability perspective.

2. Testing Methodology: Building a Reproducible Evaluation Framework

2.1 Test Scenario Design

To ensure objectivity, the test used a front-end project covering three typical page types:

Page Type	Core Requirements	Evaluation Focus
Login page	Carousel, form validation, responsive layout	Basic UI reproduction and functional completeness
Product listing page	Multi-dimensional filtering, pagination, sorting interactions	Complex logic comprehension and interaction implementation
Detail form page	Multi-field linkage, validation rules, dynamic hints	Fine-grained requirement reproduction and state management

2.2 Evaluation Dimensions

The test was conducted across the following five dimensions:

• Requirement comprehension accuracy: Whether the model can correctly identify every functional point in the document;
• Code completeness: Whether the generated code runs directly and whether any key modules are missing;
• UI fidelity: Whether the visual style approaches the quality of a mature design system;
• Generation efficiency: The time cost from requirement input to a usable project output;
• Deployability: Whether the output provides an accessible online address or a complete project structure.

3. Test Results: Breaking Down Model Performance by Module

3.1 Login Page — Baseline Capability Met, Project Usable

The login page is one of the most basic front-end pages, with a relatively fixed structure. Test results show that Chinese LLMs performed fairly stably on this module: the carousel component was correctly implemented, form fields were complete, and basic responsive layout was taken into account. Overall, this page reached a level where it could be deployed with only minor adjustments.

This suggests that current Chinese models already possess a degree of engineering viability for highly standardized, pattern-driven page types. For startup teams rapidly building MVPs (minimum viable products) or internal tool prototypes, this capability has practical value.

3.2 Listing Page — Depth of Understanding Begins to Show Weaknesses

Moving to the product listing page, problems became apparent. The requirements document explicitly described multi-dimensional filter conditions (such as category, price range, and status) along with their corresponding interaction logic (filter linkage, pagination switching), yet the model’s output completely lacked the filter options. This issue was not isolated; it recurred across multiple listing page tests.

From a technical standpoint, this reflects the model’s shortcomings in the following areas:

1. Attention allocation for long-text requirements: When a requirements document contains multiple parallel functional points, the model tends to “miss” details described in the latter half;
2. Limited implicit logic inference: Filtering functionality typically requires understanding data structures and state transfer between components. The model’s modeling of such implicit dependencies is not yet mature;
3. Effective use of the context window: Even when the model supports longer input, there is still room for improvement in extracting key information and judging priorities.

3.3 Form Page — Insufficient Reproduction of Complex Interactions

Testing on the detail form page further confirmed the above assessment. Although the page could run overall, there were clear deviations from the requirements in the following respects:

• Incomplete field mapping: Some field types specified in the document (such as date pickers and cascading dropdowns) were simplified into basic input boxes;
• Missing or incorrect validation logic: For example, required-field hints, format validation, and cross-field linkage validation were not fully implemented;
• Lost interaction details: User experience details such as real-time error hints, disabled submit states, and success feedback animations were ignored.

The root cause of such problems is that complex forms involve multiple layers of technical logic, including data model design, state management, and event handling, which places higher demands on the model’s structured reasoning capability.

3.4 Overall UI Style and Generation Efficiency

On the visual level, the UI style generated by the model leans toward “engineering simplicity.” It is slightly higher in quality than the output of ordinary prototyping tools, but still falls short compared with directly using mature component libraries (such as templates based on a design system). The design feels conservative and lacks creative expression that stands out.

In terms of efficiency, the time from requirement input to project generation completion exceeded thirty minutes, which is related to factors such as the model’s inference speed, server-side load, and whether it is a free tier. By comparison, some leading international models can complete similar tasks within minutes, with the gap mainly reflected in inference optimization and service infrastructure.

4. Core Gap Analysis: Depth of Understanding, Not Surface Generation

Based on the above tests, the core gap between Chinese LLMs and leading international models is not “whether they can generate code,” but rather “whether they can deeply understand requirements.” This can be summarized in three levels:

Capability Level	Chinese Model Performance	Leading Model Performance
Surface structure generation	✅ Basically achieved	✅ Consistently achieved
Complete functional coverage	⚠️ Frequent omissions	✅ High coverage
Implicit logic and interaction details	❌ Clearly insufficient	✅ Relatively precise

The essence of this gap is a combined difference in training data quality, alignment strategies, and inference architecture. Due to constraints in training resources and data scale, open-source models currently still have a lower “understanding ceiling” on complex tasks than top-tier closed-source models.

5. The Deeper Significance of Open-Source Attributes for AI Development

5.1 Open Source Lowers the Technical Barrier

The most direct value of Chinese LLMs choosing the open-source route is that it enables small and medium-sized enterprises and independent developers to deploy locally and customize privately, without relying on external API calls, reducing data security risks and long-term usage costs. This is of great significance for promoting the democratization of AI technology.

5.2 The Open-Source Ecosystem Accelerates Iteration

Participation from the open-source community allows models to quickly receive feedback, fix defects, and adapt to new scenarios. Through developers’ actual use (such as the testing in this article), the model’s weaknesses are exposed, and the community can carry out targeted fine-tuning and optimization. This “usage-driven evolution” model is an advantage that closed-source models are hard-pressed to replicate.

5.3 Open Source Is Not the Endpoint, but the Starting Point

It is important to recognize clearly that open source does not equal “capability on par with top closed-source models.” At the current stage, Chinese open-source models are better positioned as assisted development tools rather than fully automated code generators. Developers should treat them as a “co-pilot” that improves efficiency, not as “autopilot” that replaces their own judgment.

6. Practical Recommendations for Developers

1. Define usage boundaries clearly: Use LLMs for standardized pages, boilerplate code, and component generation. Complex business logic still requires human oversight;
2. Write high-quality requirements documents: Structured, modular requirement descriptions with examples can significantly improve model output quality;
3. Use iterative strategies wisely: Do not expect a perfect result in one pass. A cycle of “generate—review—supplement—regenerate” is more efficient;
4. Pay attention to cloud platform capabilities: Model services and development toolchains offered by cloud platforms can provide more stable inference environments and security guarantees for enterprise-grade applications.

7. Conclusion

The progress of Chinese LLMs in code generation is evident, but there is still a way to go from “usable” to “genuinely good.” The core problem revealed by this test—insufficient depth of requirement understanding—is precisely the key direction for the next stage of technological breakthrough. Open-source attributes provide a unique ecosystem advantage for this process, and real feedback from developers is the most valuable resource for driving model evolution. Only when the technical community and industrial applications form a virtuous cycle can Chinese AI coding tools truly mature.