Evaluating Large Language Models on End-to-End Frontend Project Generation

    A practical methodology for evaluating how well large language models generate complete, shippable frontend projects—not just isolated code snippets. The article proposes a three-layer rubric (runnability, feature coverage, maintainability) with a machine-readable scoring template, explains why login, list, and form pages sit at sharp points along a difficulty gradient, and attributes capability gaps to long-context constraint preservation, implicit-constraint reasoning, cross-file consistency, and self-correction. It also covers writing structured, numbered requirements, wiring generated output into automated acceptance gates, and the real value of open-weight models.

    1. From “Can Generate” to “Can Deliver”

    Over the past couple of years, large models’ capability curve in code generation has clearly trended upward, and the gaps between the top entries on public leaderboards have kept shrinking. Yet the experience of teams deploying these models in production often diverges from the leaderboards. The reason is simple: leaderboards measure the correctness of isolated functions and single-file fragments, whereas real project delivery measures cross-file consistency, constraint preservation, and rework cost.

    A real frontend requirement typically contains dozens of feature points spread across routing, state management, component communication, API adaptation, and styling conventions. There is a sizable engineering gap between a model that can produce a page that “looks right” and one that can produce a project that is “ready to ship.” The first principle of evaluation should therefore be: replace toy examples with real requirements, and replace subjective impressions with executable acceptance.

    2. Evaluation Design: Three Layers and One Acceptance Sheet

    Split the evaluation into three layers and converge layer by layer, avoiding unreproducible conclusions like “it feels good overall.”

    Layer 1: Runnability. Can the project install dependencies, start locally, pass the build, and report zero console errors? This layer is a gate; if it fails, later scoring is meaningless.

    Layer 2: Feature conformance. Convert the requirement document item by item into a checklist of feature points, and count how many are actually implemented. This is the dimension that most reliably exposes a model’s weaknesses, especially for metric-driven requirements (filtering, sorting, pagination, linkage, validation).

    Layer 3: Engineering maintainability. Is the directory structure reasonable, are components over-coupled, are type definitions complete, is there a lot of leftover placeholder content, and are styles consistent with the design conventions?

    A scoring sheet that can be used directly might be organized like this:

    1. # eval/rubric.yaml
    2. project: front_end_generation
    3. levels:
    4. - name: runnability
    5. weight: 0.2
    6. checks: [install_ok, build_ok, dev_server_ok, console_error_count]
    7. - name: feature_coverage
    8. weight: 0.5
    9. items:
    10. login_carousel: true
    11. login_form_validation: true
    12. list_filters: true # key observation item
    13. list_pagination: true
    14. form_field_dependencies: true
    15. form_error_messages: true
    16. - name: maintainability
    17. weight: 0.3
    18. checks:
    19. type_coverage_ratio: ">0.8"
    20. placeholder_left_count: 0
    21. hardcoded_style_ratio: "<0.2"
    22. circular_dependency_count: 0

    The key practice is: count feature coverage separately and do not blend it with aesthetics. The most common distortion in practice is that a polished interface inflates the overall impression while large-scale gaps in core interactions go unnoticed.

    3. The Capability Gradient: Why Three Kinds of Pages Differ So Much

    Under the same requirement and the same prompting conditions, different types of pages show a clear difficulty gradient. This is not accidental; it is determined by the structure of the task.

    3.1 Login Page: The Most Patternized

    Login page code is extremely dense in open-source corpora, and its structure is highly converged: a form, validation, submit state, and a carousel or background decoration. The model can almost complete the task by pattern memory, so its completion rate is usually the best—essentially “ready to ship after minor tweaks.”

    Even at this level, I recommend checking two details: whether error messages distinguish network failures from validation failures; and whether form submission handles repeated clicks and request cancellation. These two points are the dividing line between a “demo-grade” and a “production-grade” result.

    3.2 List Page: From “Writing a Page” to “Designing a State Machine”

    The difficulty of a list page is that it is essentially a multi-state combination problem: data loading, empty data, load failure, filters applied, filter with no results, pagination boundaries, and selections preserved across pages. Models can often build a correct skeleton but miss linkage logic such as filters, cross-page selection, and URL synchronization.

    There are two reasons. First, filter rules in requirement documents often appear as scattered points, and models tend to suffer constraint decay in long contexts—earlier constraints get overwritten by later descriptions. Second, linkage relationships require explicit reasoning, and the fact that “the filters aren’t implemented” shows that the model treated filtering as an optional decoration rather than part of the main flow.

    This is not an occasional problem but a systematic tendency that spans pages, and it should be a key observation item in evaluation.

    3.3 Form Page: From “State Machine” to “Rule Engine”

    The form page demands the most, because it involves inter-field dependencies (the value of field A determines whether field B is required), layered validation (format validation, business validation, and populating errors returned by the backend), dirty-value handling (clearing invalid values after switching an option), and differences between draft and submit data structures.

    Common problems for models at this level: fields are complete but relationships are missing; validation exists but is in the wrong order; the page can render but cannot correctly handle “back/rollback” operations. This makes the form page the key dividing line for distinguishing a model’s “depth of understanding.”

    3.4 Visual Style: Usability over Wow Factor

    The UI of generated projects is usually better than early prototypes, but compared with output that applies mature component-library templates, there is still a gap in design consistency: an inconsistent spacing system, arbitrary use of state colors, and coarse handling of responsive breakpoints. For internal systems in the engineering-validation stage, this style is acceptable; if targeting external users, it is still advisable to do a second pass of governance based on a component library and design specification.

    4. Attributing the Differences: Where Exactly Does Depth of Understanding Differ

    Consolidating the observations across the three page types, the capability gap can be attributed to four verifiable dimensions rather than a vague “smarter”:

    • Long-context constraint preservation: the longer the requirement, the higher the loss rate of earlier constraints.
    • Implicit-constraint reasoning: deriving “needs an empty state, needs URL synchronization” from “filtering” requires domain priors.
    • Cross-file consistency: whether routing, state, types, and API definitions are self-consistent.
    • Self-correction ability: after an error, whether the model can locate the root cause or repeatedly patches locally.

    At the same time, one must distinguish capability problems from service-quality problems. Generation time degrading from minutes to tens of minutes is usually affected by queue wait, concurrency quota, model size, and inference configuration, and does not necessarily mean a drop in generation quality. When selecting models, enterprises should measure “output quality” and “service performance” separately: the former via offline batch evaluation, the latter via load testing and quota planning.

    5. Improving Generation Quality: Write Requirements as Machine-Parseable Constraints

    The most easily overlooked variable in evaluation is input quality. With the same model, differences in how structured the requirement document is can produce output differences larger than the gap between models. I recommend a “constraints first + numbered traceability” style:

    1. ## Module 2: Product List Page
    2. - R2.1 A fixed filter bar at the top with three items: category, price range, listing status
    3. - R2.2 Filter changes take effect immediately and sync to the URL query
    4. - R2.3 The list supports pagination, 20 items per page; empty results show an empty state and a "clear filters" entry
    5. - R2.4 Cross-page multi-select is supported, and the selected count shows in real time in the action bar
    6. ## Module 3: Product Edit Form
    7. - R3.1 The "unit" field is shown and required only when category = weighed goods
    8. - R3.2 Switching the category must clear the entered value of "unit"
    9. - R3.3 Validation order: required -> format -> range -> populate errors returned by the backend

    The value of numbering is that the acceptance stage can directly produce a “requirement-implementation” mapping matrix, so missing items are obvious at a glance and the communication cost of rework drops significantly.

    6. Engineering Closed Loop: Make Acceptance Automatic

    Manually clicking through pages is not sustainable. I recommend wiring the generated output into the existing R&D pipeline and replacing subjective judgment with deterministic tools:

    1. # Example: acceptance gate
    2. npm ci && npm run lint && npm run typecheck && npm run test -- --coverage
    3. npm run build && npm run test:e2e -- --grep "@critical"

    Supporting recommendations:

    1. Contracts first: freeze interface type definitions first, then have the model generate the UI, to reduce cross-file inconsistency.
    2. Slice generation: generate in batches by module, keeping each slice within a few hundred lines for easier review and rollback.
    3. Test-driven acceptance: turn the feature points above directly into end-to-end test cases so that problems like “missing filters” surface at commit time.
    4. Observability: record the model version, elapsed time, and number of rework rounds for each generation, forming baseline data for sustainable comparison.

    On deployment form, container platforms handle running the build artifacts and elastic scaling; logging services and monitoring/alerting collect build failures and production anomalies; object storage holds static assets. These general capabilities can be combined on cloud platforms such as Baidu AI Cloud, independently of the model choice, with no need to bind them together.

    7. The Real Value of Open-Source Attributes

    When the public evaluation scores of open models approach those of closed-source flagships, the significance of the discussion is not “they’ve caught up,” but that it changes three things.

    First, private deployment and auditability. In scenarios involving internal code, financial risk control, and government systems, controllable model weights mean data does not leave the domain, and behavior is reproducible and regression-testable.

    Second, say over evaluation standards. Open weights let researchers build benchmarks closer to real engineering—cross-file consistency, constraint preservation rate, defect-fix rate—which reflect delivery capability better than single-point correctness.

    Third, cost and replacement elasticity. Once models are pluggable, teams can adopt “use the expensive model for architecture and hard problems, use cheap open models for batch generation” as a routine strategy, rather than being locked to a single vendor.

    One point to be clear-eyed about: open source does not equal free performance. Inference resources, tuning investment, and evaluation-system building are all hidden costs; meanwhile, the real location of the capability gap is in requirement understanding and engineering consistency, not simply code volume.

    8. Conclusion

    For the proposition “can it match flagship models,” a more rigorous way to answer is to translate it into engineering language: under structured requirement constraints, can it deliver in one shot a project that is buildable, testable, and low-rework? Based on this standard, four practical conclusions:

    • Patternized modules like login pages already reach a directly usable level;
    • The filter linkage, field dependencies, and validation order in list and form pages are the current main weaknesses and should be evaluation priorities;
    • Generation speed is affected by service quota and needs to be measured separately from quality metrics;
    • Structured numbered requirements plus automated acceptance gates improve output quality more reliably than switching models.

    Progress in model capability is certain, but what it replaces is the “writing boilerplate code” part. Architecture decisions, constraint definition, and quality gatekeeping still belong to engineers. Only by building up the evaluation system can a team truly turn the uncertainty of model capability into the certainty of engineering delivery.