Large Language Model Frontend Code Generation, Tested: Reproducible Benchmarks and the Case for Open-Source Models

    This article presents a reproducible test of large language models (LLMs) for frontend code generation, comparing an open-source model against a closed-source API model across three typical page types: a login page, a list page, and a form page. It introduces a five-dimensional scoring table that turns subjective impressions into verifiable acceptance criteria, and reports where models succeed (page structure and layout) and where they consistently fall short (business semantics such as combined filtering, field linkage, validation, and state synchronization). It also covers engineering usability, generation speed, accessibility, and security integration, then discusses the value, boundaries, and suitable use cases of open-source models. Readers will learn how to set up repeatable benchmarks and make more rational decisions about LLM-based frontend development and model selection.

    1. Why Frontend Code Generation Needs Verifiable Testing

    In a real development workflow, the value of LLM-based frontend code generation is not simply whether it can produce a page. It is whether the model can understand requirements, organize components, reproduce interactions, and reduce manual rework.

    For teams, the point of testing frontend code generation is to turn marketing claims into verifiable conclusions. Model selection should not rely on leaderboards alone. Instead, it should draw on typical pages such as login pages, list pages, and form pages, and examine functional completeness, semantic fidelity, and engineering usability. This article also introduces an open-source model comparison to help readers understand the relationship between private deployment, data security, and ecosystem collaboration.

    2. Test Setup: Making Results Reproducible

    To avoid a loose “if it runs, it counts” judgment, this test uses a fixed baseline:

    • Test subjects: Model A (an open-source model, version A-OpenModel-v1.5, privately deployed) and Model B (a closed-source API model, version B-ClosedModel-v3.8, accessed through an API). To avoid brand bias, this article uses codenames, but the versions, prompts, and acceptance criteria stay consistent.
    • Requirement samples: An e-commerce frontend requirements document covering a login page, a product list page, and a product application form page. The document specifies fields, filter conditions, validation rules, and interaction descriptions.
    • Generation environment: Unified prompts, a unified dependency template, and unified container resources, so that environment differences do not affect results.
    • Repetitions: Each page was generated three times per model. The test records the functional pass rate, the number of blocking errors, and the time spent on manual fixes.
    • Acceptance criteria: All key features must be implemented with no blocking errors; core flows must be clickable, refreshable, and submittable. A functional pass rate of at least 80% is recommended before entering engineering integration; below 70% is not recommended for direct use.

    3. A Five-Dimensional Scoring Table: From Feeling to Verifiable Acceptance

    During evaluation, the following scoring table can turn a vague “seems fine” into evidence-based conclusions:

    Dimension Weight Evidence Deduction Rules Final Conclusion
    Functional completeness 30% Requirement feature checklist, click-by-click screenshots, flow results Deduct 3 points per missing key feature, 1 point per missing secondary feature, and 5 points for a blocking flow A missing key feature means immediate failure; it does not proceed to later scoring
    Semantic fidelity 25% Field meaning, field relationships, state transitions Deduct 2 points for an incorrect field explanation, 3 points for an incorrect state switch, and 4 points for misreading a business rule Semantic errors directly undermine functional trust
    Interaction logic 25% Filtering, linkage, validation, submit feedback Deduct 2 points for a single broken interaction, 3 points for a broken combination logic, and 5 points for failures repeated across multiple pages A combination-interaction failure is treated as a capability gap
    Engineering quality 10% Project structure, component reuse, maintainability Deduct 2 points for a disorganized directory, 2 points for duplicated/redundant components, and 1 point for missing basic conventions Engineering usability takes priority over visual polish
    Generation efficiency 10% End-to-end time, accessibility, debugging cost Deduct 2 points for severe timeouts, 5 points if it cannot run, and 3 points for excessive fix time Low efficiency reduces the value of a pilot

    The final score is a weighted calculation: 85 or above is recommended for a pilot; 70 to 85 is suitable after manual enhancement; below 70 warrants cautious evaluation.

    4. Page-by-Page Testing: Three Levels of Capability Boundaries

    4.1 Login Page: Basic Structure Was Fairly Complete

    In this test, the login page requirements included basic elements such as a carousel, input fields, and buttons. Both Model A and Model B completed the page structure, and basic components like the carousel, inputs, and buttons rendered correctly. It should be noted that this conclusion applies only to the login-page scenario in this test and cannot be generalized into “mainstream models work out of the box.” Production environments still need API integration, captcha, security checks, analytics/tracking, and exception handling.

    4.2 List Page: Combined Filtering Exposed Semantic Weaknesses

    Problems began on the list page. The requirements document asked the product list to support “city + price range” combined filtering. For example, selecting Beijing as the city and setting the price range to 100–300 should display only matching products.

    In testing, the generated pages usually included a city selector and a price input, but the filter conditions did not work correctly. Some models implemented only single-city selection and ignored the price condition; some performed only front-end filtering without passing query parameters; and some did not refresh the list results after the city was switched.

    This shows that models are better at generating structures like tables, pagination, and cards, but weaker at business semantics such as “how a filter is triggered, how conditions are combined, and how results are refreshed.” The list page therefore became the first dividing line between model capabilities.

    4.3 Form Page: Field Linkage and Validation Rules Are the Hardest

    The form page had the highest complexity. The product application form in the test included fields such as product name, price, stock, invoice type, and shipping address. The requirements specified that selecting an invoice should reveal the invoice title field; stock quantity could not be less than zero; the phone number had to pass format validation; and submit had to prevent duplicate clicks.

    In practice, the page ran overall, but key logic was largely missing. For example, after selecting an invoice type, the invoice title field did not appear as expected; negative stock was not blocked; a phone number with an invalid format could still be submitted; and the button lacked loading or duplicate-click protection. The difficulty of the form page lies in the fact that field visibility, dependencies, validation rules, and error feedback often overlap, and the model must stay consistent across multiple constraints.

    5. A Clear Gap Between Page Structure and Semantic Understanding

    Placing the three page types together yields a clear conclusion: when generating frontend code, a model’s page-structure ability is not the same as its functional-semantic ability.

    At the structure and presentation level, models can usually generate pages with reasonable layouts and complete components. But at the semantic and logic level, fidelity drops noticeably once business rules such as filtering, linkage, validation, and state synchronization come into play.

    The reason is not complicated: structural information maps fairly directly from the requirement text to components, whereas business logic requires cross-context reasoning and constraint solving. As a result, different models may perform similarly on login pages but tend to diverge on list pages and form pages.

    6. UI Style and Engineering Usability: Componentization First, Aesthetics Second

    Based on the results, generated pages were usually more complete than rough prototypes, but still fell short of mature product visuals. For engineering teams, this gap is acceptable, because style can be unified quickly through a component library and a theme system, while missing interaction logic has to be filled in manually, point by point.

    The following engineering-usability checks are recommended as priorities:

    • Whether the page is built on a mature component library rather than a large amount of repeated custom styling;
    • Whether it supports theme variables, a unified color palette, and spacing standards;
    • Whether it has responsive adaptation for both desktop and mobile layouts;
    • Whether key controls have accessibility, such as button names, input labels, keyboard operation, and contrast;
    • Whether it keeps a clear directory structure to make it easier to add routing, state management, and an API layer later.

    7. Generation Speed, Accessibility, and Security Integration

    Generation speed is also an important dimension. Generating a complete project often takes a long time, and complex projects are further affected by compute resources and queuing policies. When evaluating, three points are worth attention: first, whether the output is an accessible project URL rather than scattered code fragments; second, whether the end-to-end time is manageable; and third, whether the generated result can be integrated into a build, deployment, and monitoring pipeline.

    In real engineering, general cloud capabilities can be used for hosting and governance, for example using a container platform to deploy applications, a log service to collect runtime information, monitoring and alerting to ensure stability, object storage to manage static resources, and a key management service to protect sensitive configuration.

    At the same time, generated code cannot be assumed to be secure. Before going live, at minimum check for dependency vulnerabilities, log masking, least-privilege permissions, and no hardcoded keys. For pages that involve user information, also add input validation, exception handling, and access control.

    8. Open-Source Model Comparison: Value, Boundaries, and Use Cases

    The significance of open-source models goes beyond the outcome of a single benchmark. Compared with closed-source API models, the value of open-source models comes mainly in three areas. First, developers can deploy and try them locally, lowering the cost of learning and innovation. Second, enterprises can deploy models in a private environment, reducing the risk of sensitive data leaks. Third, teams can fine-tune models on their own business data so that general capabilities fit vertical scenarios.

    The boundaries also need to be viewed objectively. Currently, open-source models are narrowing the capability gap with closed-source models in some areas, such as structured page generation, code completion, and private deployment tasks; but in highly complex interactions, long-chain reasoning, and frontier capabilities, continued observation is still needed. When selecting a model, it is not appropriate to simply conclude that “open-source is always better” or “closed-source is always stronger.” A combined assessment should account for compliance requirements, cost budget, maintenance capability, and actual task performance.

    9. Implementation Recommendations and FAQs

    How Should You Choose?

    Start by clarifying the requirement type. If the work is mainly component pages, list pages, and admin forms, and if data compliance and private deployment matter, open-source models are worth testing first. If the goal is exploring complex interactions and fast API access, closed-source models are worth comparing. Whichever route you take, test across multiple rounds using real requirement documents.

    What Are the Common Mistakes?

    Common mistakes include: looking only at generated screenshots without checking interaction results; drawing conclusions from a single prompt; misjudging occasional errors as a lack of model capability; having no feature checklist or acceptance criteria; and ignoring the security, performance, and maintainability of generated code.

    What Should You Note for Private Deployment?

    Before private deployment, confirm the model license, hardware resources, concurrency capacity, upgrade mechanism, and data-processing boundaries. After going live, establish log auditing, permission management, and model version records, so that problems can be traced back to their source.

    Overall, the focus of evaluating an LLM’s frontend code generation is not whether it can “draw” a single page, but whether it can reliably understand and reproduce business logic. Building a reproducible test baseline, a five-dimensional scoring table, and a checklist-based acceptance method helps teams make more rational decisions about model selection, open-source model comparison, and engineering adoption.