Evaluating LLM Frontend Code Generation: The Real Gap Between Open-Source and Closed-Source Models
This article evaluates the frontend code generation capabilities of open-source LLMs against closed-source flagships through a real-world project test. By analyzing the generation of login, list, and form pages based on a structured PRD, it reveals that while open-source models excel at standardizing tasks, they still lag in understanding complex business rules and deep requirement fidelity. The findings offer practical guidance for developers on choosing the right AI coding tools.
1. Background: Open-Source Models are Rapidly Approaching Closed-Source Flagships
Recent releases of new-generation open-source large language models (LLMs) have sparked industry discussions, with many claiming their capabilities now rival top-tier closed-source flagships. But do these claims hold water? For developers, benchmark scores offer some reference, but a model’s true value lies in its performance in real engineering tasks.
To draw a more practical conclusion, I designed a test: using a real-world frontend requirement document (PRD) covering three typical pages—login, product list, and form—I asked the models to generate an end-to-end runnable project. The test did not restrict the tech stack, and the models ultimately chose React. Here is my complete test record and analysis.
2. Test Environment and Prerequisites
To prevent misinterpretation, let’s clarify a few prerequisites:
- Input Method: A structured PRD was provided, explicitly detailing page elements, filter conditions, field definitions, validation logic, and interaction rules.
- Output Requirements: Beyond mere code generation, the models were required to deliver a complete, accessible, and runnable project.
- Baseline: The same PRD had previously been implemented using a closed-source flagship model, ensuring comparability.
It is worth noting that this test used the free tier of the open-source service, resulting in longer queue times and a total generation time of about half an hour. The speed factor may be related to resource quotas, so performance data is for reference only. The core focus is on generation quality and requirement fidelity.
3. Scenario-Based Test Results
3.1 Login Page: Usable Completion
The login page is a relatively standardized part of the requirements, including common components like carousels. The model’s output was fully functional with no obvious omissions, and the page structure was reasonable. It reached a “ready to go live with minor tweaks” level. This result indicates that for highly patterned pages, open-source models already possess production-grade code assembly capabilities.
3.2 List Page: Understanding Capability Starts to Lag
Issues were heavily exposed on the list page. The PRD explicitly specified filter conditions and interaction logic, but the generated result completely lacked the filter items. This issue recurred across multiple list pages, indicating a systematic understanding shortfall rather than an accidental omission.
This reflects a problem with the model’s “information extraction completeness” from structured documents: high-probability patterns like page skeletons are easily learned, but specific, personalized business rules in the document (such as filter fields, default values, and linkage relationships) are more likely to be ignored during generation.
3.3 Form Page: Insufficient Detail Fidelity
The form page ran smoothly overall, but there were noticeable deviations in field definitions, validation logic, and interaction details compared to the document description. For example, mandatory field rules, format validation, and linkage disabling specified in the document were mostly not accurately implemented. Forms are the most logic-dense part of frontend business systems, and such deviations directly lead to rework. This is an aspect that should not be masked by “good completion rates” when evaluating code models.
3.4 Overall Style: Engineering Usability over Product Polish
From a UI perspective, the generated result was better than typical prototype demos, but there was still a gap compared to a mature component library template system. The design felt more engineering-oriented, falling into the “usable and not ugly” category. To summarize the positioning of the open-source model: Engineering Usability > Product Polish.
4. Comparison with Closed-Source Flagships: The Gap is in “Understanding Depth”
Comparing the output with that of a closed-source flagship model for the same requirement yields a clearer layered conclusion:
| Scenario | Open-Source Model | Closed-Source Flagship |
|---|---|---|
| Login Page | Completion is close | Completion is close |
| List Page | Filter logic missing | More complete document understanding |
| Form Page | Obvious validation & interaction deviations | Accurate restoration of field relations & validation logic |
| Generation Speed | Slower (about 30 mins) | About 3 mins |
The conclusion is clear: for simple, patterned tasks, open-source models are on par; the gap is concentrated in the depth of understanding complex requirements. This reminds us that evaluating code models shouldn’t just focus on “can it generate a page,” but “can it restore business rules.”
5. Implications for the Open-Source Ecosystem and AI Development
Setting aside specific products, the industry signals behind this test are worth discussing:
- Open-source models are iterating faster. From “completely unusable” to “matching flagships on simple tasks,” the catch-up cycle for the open-source camp has significantly shortened. This is good for the industry, as competition will continue to drive down model usage costs.
- The verifiability advantage of open-source is prominent. Open model weights mean the community can independently reproduce evaluations rather than relying on benchmark scores published by vendors. Real-world project tests like this are exactly the transparency dividend brought by the open-source ecosystem.
- Developer Selection Insights:
- For prototype building and standardized page generation, open-source models are ready to serve as productivity tools.
- For scenarios involving complex business rules and multi-constraint validations, closed-source flagships or deep human intervention are still required.
- Regardless of the choice, the degree of PRD structuring directly determines generation quality. Writing filter conditions and field rules in a machine-readable format is much more effective than verbalizing requirements.
- View “explosive” marketing rationally. There is a gap between benchmark scores and real engineering performance. As developers, building your own test set (even if it’s just a few real business pages) is far more valuable than chasing marketing rhetoric.
6. Conclusion
The capability leap of open-source models is real, but the claim of “rivaling flagships” needs to be broken down: in the breadth of code generation, the gap is no longer large; in the precision of requirement understanding, there is still a visible gap. For enterprises and developers, this is actually the healthiest situation—open-source guarantees the cost floor and ecosystem autonomy, while closed-source defines the capability ceiling. Together, they push AI coding tools toward true production-grade status. The next step worth watching is whether the open-source camp can shore up its shortcomings in the critical dimension of “complex instruction following.”
