Benchmarking a Domestic Large Model for Frontend Code Generation: Capability Analysis and Engineering Insights

1. Background: Beyond the “Parity” Claims

Over the past year, domestic large language models have accelerated their iteration speed in code generation, with claims of “capability parity with leading international models” frequently appearing in product announcements. While encouraging, such statements can mislead developers—there is often a significant gap between benchmark scores and real-world project outcomes.

This article does not focus on any vendor’s marketing language. Instead, it returns to engineering practice: using a complete frontend project requirement as input, we observe how a model transforms a requirement document into runnable code, and extract a reusable framework for evaluating code generation capabilities. The evaluation target is described neutrally as “a domestic large model,” with an “international mainstream model” used as a reference. The emphasis is on methodology rather than ranking.

It should be noted upfront that a single evaluation cannot represent a model’s full capability. The value of this article lies in providing a reusable observation framework, not a definitive conclusion.

2. Evaluation Design: Making Capability Observable

2.1 Task Selection Principles

To ensure comparability, the task design follows three principles:

• Authenticity: Requirements come from real project pages, not toy examples.
• Completeness: Covers three high-frequency page types—login, list, and form—to assess display-oriented, data-oriented, and interaction-oriented capabilities respectively.
• Judgability: The requirement document explicitly states functional points and interaction logic, making “implemented or not” an objectively verifiable fact rather than a subjective impression.

2.2 Input Format

The input is a structured requirement document containing page structure descriptions, field definitions, filter conditions, validation rules, and interaction descriptions. No specific tech stack was mandated; the model was free to choose its framework—this is critical because the reasonableness of the tech stack choice is itself part of the capability. The generated output used React, a mainstream industry choice that requires no criticism.

2.3 Observation Dimensions

The evaluation spans five dimensions, which also form the analytical backbone of this article:

Dimension	Observation Point
Usability	Whether the output runs directly and is accessible
Completeness	Proportion of functional points from the requirement document that are implemented
Understanding Depth	Fidelity of complex interactions, field relationships, and validation logic
Design Quality	UI style, component reuse, and engineering structure quality
Efficiency	End-to-end generation time and stability

3. Test Results: Capability Boundaries by Layer

3.1 Usability: Runnable Output Is a Significant Plus

One easily overlooked but important point is output form. Some tools provide code snippets requiring manual assembly of the project skeleton; in this evaluation, the model directly provided an accessible project URL, with the engineering scaffold, dependency configuration, and routing structure generated together. This significantly shortens the chain from “getting code” to “seeing the page.”

However, service availability showed fluctuation. The first invocation displayed a queue notice, requiring multiple retries to enter the generation flow. For an individual, this is merely an inconvenience; but for teams planning to integrate the model into CI/CD or development pipelines, service stability and quota mechanisms are engineering constraints that must be assessed in advance, not assumed to be always available.

3.2 Login Page: Display-Oriented, High Completeness

The login page is the most “standardized” of the three: fixed layout, simple interactions, and clear visual elements. The carousel component specified in the requirement document was correctly implemented, and the page was functionally complete with no obvious omissions.

This page’s performance can be summarized as “close to shippable”—meaning the developer’s primary work shifts from rewriting from scratch to adapting the output to team conventions. For display-oriented pages, current large model capability already has practical value.

3.3 List Page: Correct Structure, Declining Functional Understanding

Problems begin to appear on the list page. The page skeleton, table structure, and pagination components were generated correctly, but the filter conditions and filter interactions explicitly stated in the requirement document were not implemented.

This phenomenon deserves deeper analysis. It is not an isolated omission but was repeated across multiple list pages, indicating a systemic capability gap: the model excels at reproducing “static structure” but is weak at reproducing “state-driven interaction logic.” Filtering functionality inherently requires the model to understand the complete chain of “user action → state change → data filtering → view update,” not just rendering filter controls.

From an engineering perspective, fixing such omissions is not cheap: the developer must re-architect the data flow, add state management, and integrate filtering logic—workload comparable to writing it from scratch. The list page is therefore a key area requiring manual review at this stage.

3.4 Form Page: Insufficient Fidelity in Fields and Validation

The form page exposes another facet of the same problem. The page runs, but the fields, validation rules, and interaction details defined in the requirement document were not fully implemented.

Forms are one of the highest “requirement density” scenarios in frontend development: a single field may simultaneously involve type constraints, required rules, conditional visibility, and error message timing. The model can generate the visual structure of a form, but its handling of constraints between fields and validation trigger timing—semantic information—is clearly insufficient. This is the same root cause as the list page issue: sensitive to structure, insensitive to rules.

3.5 Design Quality: Engineering Usability Prioritized, Aesthetic Polish Lacking

The UI style is generally moderate: better than most low-fidelity wireframes, but short of the quality of directly applying mature component library templates. The style leans simple and engineering-focused, lacking distinctive design elements.

This positioning is not inherently a defect. For internal tools and admin dashboards, “engineering usability over visual appeal” is a perfectly reasonable trade-off. But for customer-facing product pages, dedicated design review and visual refinement are necessary after generation. Treating the model as a “scaffold generator” rather than a “designer” is the more pragmatic expectation management at this stage.

3.6 Efficiency Comparison: Speed vs. Accuracy Trade-off

Under identical requirements, the international mainstream model’s end-to-end time was on the order of minutes, while the domestic model took significantly longer (roughly half an hour under the free tier). It must be noted objectively that the time difference may partially stem from queueing and rate limiting in the free tier, not solely from model inference capability.

More important is the accuracy difference. On the login page, both models achieved similar completeness; on the list and form pages, the international model reconstructed filtering logic, field relationships, and validation details more completely. The core gap is not “can it generate,” but “how deeply does it understand”—this returns to the central judgment of this article.

4. Gap Attribution: From “Generating Code” to “Understanding Requirements”

Abstracting the observations above yields a capability layering model:

L1 Structure Generation: page skeleton, component layout, style fidelity     — fairly mature
L2 Logic Generation: state management, event handling, data flow            — partially present
L3 Semantic Understanding: requirement document → constraint rule mapping   — clear weakness
L4 Engineering Judgment: tech selection, directory structure, maintainability — initial capability

The domestic model performs consistently at L1, inconsistently at L2, and L3 is the primary bottleneck. Yet L3 is precisely the key to reducing manual rework—what developers really want to outsource is not typing code, but translating documents into constraints.

Two angles help explain the L3 shortfall. First, rule information in requirement documents is often implicit and requires contextual reasoning; high-quality “document + full implementation” pairs are relatively scarce in training data. Second, verifying interactive logic requires runtime feedback, which purely static generation cannot self-check. This also explains the gap between “it runs” and “it runs correctly.”

5. Implications for Open-Source Attributes and AI Development

Returning to the question posed in the title, and setting aside marketing language, this generation of domestic models brings at least three layers of practical significance:

First, open and semi-open ecosystems lower the barrier to capability access. When model weights or services with mid-to-upper-tier code generation ability are provided in an open manner, small and medium teams can fine-tune and adapt them in private environments (e.g., injecting team component library conventions, internal API contracts)—a flexibility closed APIs rarely offer. For developers, customizability often matters more than absolute performance.

Second, competition drives improvement in “documentation quality” as a hidden infrastructure. The most valuable finding in this evaluation is that model performance heavily depends on the clarity of requirement documents. This creates reverse pressure on teams to write requirements in a more structured, more verifiable way—which is itself a long-term dividend for R&D efficiency.

Third, capability layering provides a clear path for engineering integration. Since L1 is strong and L3 is weak, the sensible integration approach is: use the model to generate scaffolds and boilerplate code, use human review for business rules and interaction details, and set up a “post-generation checklist” in the workflow. The pseudocode below illustrates a typical integration constraint:

Generated output → Static checks (lint / type) → Structure comparison (against requirement document feature list)
                  → Interaction review (filter, validation, linkage item-by-item) → Manual merge

Placing the model in the “drafting” stage of the pipeline rather than the “delivery” stage is the lowest-risk, most certain-value usage at this point.

6. Practical Recommendations: How to Use the Current Generation of Models Effectively

Based on the evaluation, here are several actionable suggestions:

1. Structure the requirement document. Write functional points as checkable lists, and express validation rules as “field + condition + message” triplets—this measurably improves fidelity.
2. Submit page by page, module by module. Submitting an entire site requirement in one batch dilutes the model’s attention. Split by page, generate incrementally, and accept each page individually to reduce rework cost.
3. Prioritize template-style pages. Login, registration, and detail display pages yield the highest return; complex forms and multi-condition lists should be human-led with model assistance.
4. Establish a post-generation checklist. At minimum cover: whether filtering and sorting work, whether required and format validation triggers, whether field linkage behaves correctly, and whether empty states and error states are handled.
5. Assess service stability. If integration into team workflows is planned, confirm quota, rate limit, and availability policies in advance to avoid introducing uncertainty into delivery cadence.

7. Conclusion

The progress of domestic large models in frontend code generation is real: the output is runnable, the structure is sound, and template-style pages are close to usable. But “parity” claims should not be read as full equivalence—the gap in requirement understanding depth remains clearly visible, concentrated in semantic-layer capabilities such as interaction logic and constraint rules.

For developers, the most rational stance is to treat the model as a collaborator with clearly defined capability boundaries: use it where it excels, and retain human judgment where it is weak. For the industry, the long-term value of customizability and ecosystem vitality brought by open and open-access models may deserve more attention than any single benchmark result. What ultimately determines the success of AI-assisted development is never a model’s standalone score, but whether a team can design a sound engineering workflow around its capability boundaries.