LLM Frontend Code Generation Tested: From Requirements Docs to Runnable Projects — and How to Close the Gaps

    An end-to-end test of a large model's frontend code generation, driven by a real requirements document and benchmarked against a peer frontier model. It breaks down where the model succeeds (complete, runnable, multi-page projects) and where it falls short (filter logic, field linkage, and complex validation on constraint-dense pages), while also covering usability issues like queuing, visual quality, and generation speed. The article then lays out six engineering practices — structuring requirements, turning rules into tests first, a generate–validate–fix loop, page-by-page generation, quantified evaluation, and clear human–machine boundaries — and closes with an objective look at capability benchmarking and open weights. Readers get a practical framework for judging and integrating LLM code generation into reliable production workflows.

    LLM Frontend Code Generation Tested: From Requirement Doc to Runnable Project — What’s Missing and How to Fix It

    Note: This article is based on a developer community member’s public hands-on test of a newly released large model. To keep the focus on transferable technical methods, specific product and framework names have been omitted. The tested system is referred to throughout as “the model under test,” and the comparison system is referred to as “the reference object (a peer frontier model).”

    After a new generation of large models was released, developer communities were quick to praise them, with claims like “matching the top frontier models” and “automatically generating complete frontend projects” spreading widely. But most of these conclusions stay at the level of demo clips or isolated features. For engineering teams, the real questions are: given a real requirements document, can the model generate a runnable, deliverable, and maintainable project? And compared to the top tier, exactly where do the gaps appear? This article uses one end-to-end test driven by a real requirements document as its main thread, breaks the problem down layer by layer, and offers engineering recommendations that can be put into practice directly.

    I. Test Background and Method Design

    1.1 Why Evaluate “End-to-End Delivery”

    Generating a component or fixing an error is a completely different scale of capability from delivering a full frontend project.

    • Single-point generation: the input is clear, the expected output is unique, and you only need to glance at the result to judge correctness.
    • Complete project: the input is a requirements document containing implicit assumptions, relationships between pages, and constraints scattered throughout. The deliverable must pass functional, interaction, visual, and maintainability checks at the same time.

    When stakeholders say “can the model be used,” they usually mean the latter. So judging by demo effects or a single benchmark metric alone is easy to mislead; end-to-end validation with real requirements is necessary.

    1.2 Test Materials: A Requirements Document Close to Real Development

    This test used a real frontend project’s requirements document as input, covering three typical page types:

    • Login page: carousel, login form, and similar needs; relatively simple structure.
    • List page: multiple lists, each with clearly specified filter conditions and interaction logic.
    • Form page: multiple fields, plus the relationships between fields, validation rules, and interaction details.

    The value of this material lies in its typical real-world characteristics: the requirements are complete, but the information is distributed in “narrative” form rather than as a structured field list.

    1.3 Variable Control and Observation Dimensions

    To stay close to how developers actually work, the test followed two principles:

    1. No special optimization: don’t rewrite prompts to suit the model’s preferences, and don’t repeatedly tune toward a desired output.
    2. Describe only the goal: don’t constrain the tech stack, don’t specify a component library, don’t provide a design mockup — only describe what is wanted.

    The observation dimensions covered five aspects:

    Dimension Key question Basis for judgment
    Functional completeness Are all features in the doc implemented Line-by-line comparison against the requirements doc
    Requirement understanding Do fields, interactions, and validation match the doc Check filter options, validation logic, and other key details
    Visual completeness Is the layout and style at a usable standard Compare with prototypes and mature component solutions
    Delivery form Is the output a code snippet or a runnable project Can it be opened, accessed, and used directly
    Efficiency & stability Time taken, whether queuing occurs, whether retries are needed Full timing, logging of anomalies

    II. Hands-On Experience: Delivery Form Is Improving, but Usability and Efficiency Still Need Work

    2.1 Usability: Queuing and Rate Limiting Surface First

    • Phenomenon: during testing, “too many users, please queue or pay” messages appeared repeatedly, and access was granted only after repeated attempts.
    • Insight: generative services are essentially compute-intensive inference tasks; during peak times they are constrained by compute and scheduling, so queuing is normal. Enterprises should assess concurrency capacity in advance and prepare backup options for critical scenarios.
    • Countermeasure: when integrating LLM generation into business systems, you must design timeout, retry, degradation, and async task mechanisms. Time-consuming tasks like this shouldn’t sit in a synchronous request–response chain:
    1. User submits a generation task
    2. └─ Server writes the task to a queue and immediately returns a task ID
    3. ├─ Frontend polls / receives a callback and shows progress
    4. └─ Generation completes → return artifact URL and preview link
    5. └─ Timeout / queue failure → show queue position, or suggest retrying later

    2.2 Generation Form: From “Code Snippets” to “Runnable Artifacts”

    Without a specified language, the model under test defaulted to a component-based frontend framework, taking about half an hour from start to completion. Compared to similar models that usually finish within minutes, this is clearly slower, possibly related to the free tier used during testing.

    Its output form is worth noting: it returned a directly accessible project URL rather than scattered code snippets. This difference matters because:

    • It lowers verification cost: developers don’t need to manually assemble dependencies or configure the build environment; they just open the link and experience the app page by page.
    • It broadens the user base: non-developer roles like product and operations can also take part in acceptance.
    • It aligns with the direction of “intelligent coding”: the focus shifts from “writing code” to “delivering runnable software,” consistent with the philosophy of conversational prototyping and one-click deployment tools.

    2.3 Efficiency: Total Time and Rework Rate Should Be Counted Together

    “Half an hour” alone can’t support a conclusion. Evaluating generation efficiency requires at least three things:

    1. Whether the output includes a complete project structure;
    2. Whether significant rework is needed to reach a deliverable state;
    3. Whether results are stable across multiple runs of the same requirement.

    When evaluating tool effectiveness, “time to first generation” and “total effort to fix until acceptance passes” should be measured in the same formula.

    III. Page-by-Page Breakdown: Fidelity Gaps Come Down to “Constraint Density”

    3.1 Login Page: Clear Requirements, Fixed Structure, Near Reference Level

    The carousel required in the doc was implemented in the output, and the login page’s functionality was generally complete with no obvious omissions. The author judged it “ready to ship with minor tweaks.”

    Pages like this are characterized by concentrated requirements and highly standardized structure; they appear repeatedly in public code data, so the model can generate them correctly through pattern matching. For this reason, good performance on the login page is a “passing line” and shouldn’t be the main basis for judging overall capability.

    3.2 List Page: Filtering Missing, and Not a One-Off

    Problems first appeared on the list page. Filter conditions and interaction logic clearly written in the doc were all missing from the output. More importantly, this wasn’t a one-off omission on a single page but a common issue across multiple list pages.

    The page skeleton was fine; what was missing was the functional layer. Filtering looks simple but involves several states:

    • Whether filter conditions need to sync with query parameters in the URL, to support refresh persistence, link sharing, and browser back/forward;
    • Whether the current page number resets after switching a filter, and whether the searched keyword is retained;
    • What to show when there are no results;
    • How to clear all filters with one click.

    These states are often scattered, implicit, or even unwritten in requirements docs. This shows that even with a correct page skeleton, the model underestimates the implicit interaction chains.

    3.3 Form Page: It Runs, but Fields and Validation Are Hard to Reproduce Faithfully

    The form page ran overall, but a fair number of fields, validation logic, and interaction details weren’t implemented per the doc. Forms are a typical “constraint-dense” scenario:

    • Field linkage: for example, the selected type determines the subsequent options;
    • Validation timing and rules: real-time vs. on-blur, and how required, length, and format are each handled;
    • The form and placement of error messages;
    • State handling during submission and on submit failure.

    Even when the doc is written clearly, the model under test still falls short on reproducing complex forms. This shows the key isn’t “can it write it out” but “can it faithfully execute constraints line by line.”

    3.4 Horizontal Comparison: The Shortfall Concentrates in “Depth of Understanding”

    Coincidentally, the author had previously completed the same requirements with the reference object, and the two showed clear tiering:

    Dimension Model under test Reference object Nature of the gap
    Login page Complete functionality Complete functionality Basically on par
    List page Filtering missing Filtering and field structure understood more completely Gap in understanding depth
    Form page Some fields and validation not implemented Field relationships, validation, and interactions reproduced accurately Gap in understanding depth
    Generation speed Clearly slower (possibly related to the free tier) About a few minutes Gap in engineering experience

    Conclusion: the current gap isn’t mainly about “can it generate a page that looks decent” but “can it accurately understand and fully implement the requirements.” This gap falls in the parts where business value is most concentrated: the more complex the page, the more interactions and validation decide success or failure, and the more pronounced the gap.

    IV. Visuals: Engineering Usability Isn’t the Same as a Good-Looking Product

    The overall interface generated by the model under test was “a bit better than most prototypes,” but the author admits it still falls short of the polish and harmony of mature component solutions.

    A common industry breakdown helps understand this difference:

    • Usability: it runs, doesn’t crash, and features are visible;
    • Consistency: color, spacing, corner radius, and similar visual norms are unified, and components behave consistently across pages;
    • Aesthetics: hierarchy, whitespace, and font weight have design intent and match the product’s tone.

    The model under test reached “usability,” still short of the “consistency + aesthetics” represented by mature component solutions. The reason is that its style leans simple and engineering-oriented, drawn more from the form-style page patterns common in training data than from a systematic design language.

    For scenarios like admin and enterprise backends that emphasize stability and consistency, a low-cost approach is to specify design constraints in the prompt so the output fits existing norms:

    1. Design spec:
    2. - Primary color: brand blue #2B6CF0
    3. - Corner radius: 8px
    4. - Base spacing: multiples of 8px
    5. - Body text 14px, headings 16px, weight 500
    6. - Tables use zebra striping, row height 48px

    V. Building a Reliable “AI-Generated Frontend” Workflow

    The problems this test surfaced show that treating LLMs as a “requirements-doc reader + fully automatic developer” is still unreliable today. A more realistic approach: let the model generate, and let the engineering system act as the gatekeeper. Six practices follow.

    5.1 Structure the Requirements Document: Reduce the Loss of Implicit Constraints

    Narrative docs naturally drop information. Convert them into a “feature list + page state table + validation rules table”:

    1. ## Feature list
    2. - F01 Login page carousel
    3. - F02 List page keyword search
    4. - F03 List page multi-condition filtering
    5. ## Filter state table
    6. | Condition | Control type | Default | Behavior on change |
    7. | --- | --- | --- | --- |
    8. | Keyword | Input | empty | Enter or click search triggers a query; page resets to 1 |
    9. | Category | Dropdown | All | Selecting triggers a query immediately; combined with the keyword condition |
    10. | Price range | Dual-handle slider | Any | Triggers a query on release |
    11. | In stock | Toggle | Off | Toggling triggers a query |
    12. | All conditions | — | — | Supports one-click clear; clearing restores the default list |
    13. ## Form validation rules
    14. | Field | Type | Required | Rule | When to show the error |
    15. | --- | --- | --- | --- | --- |
    16. | Phone | Text | Yes | 11 digits, starts with 1 | On blur |
    17. | Email | Text | No | Standard email format | On blur |
    18. | Notes | Multiline | No | ≤ 200 characters | Validated on submit; remaining characters shown in real time |

    This has three benefits:

    • The model implements per the tables, and the omission probability drops noticeably;
    • The same tables can be turned directly into test cases;
    • When rules change, you can update them locally without rewriting the whole prompt.

    5.2 Turn Rules into Tests First, Then Let the Model Write the Implementation

    Each filter or validation rule maps to an automatically assertable test case:

    1. function test(name: string, fn: () => void) {
    2. try { fn(); console.log('PASS:', name); }
    3. catch (err) { console.error('FAIL:', name, err); process.exitCode = 1; }
    4. }
    5. function assert(cond: boolean, msg: string) {
    6. if (!cond) throw new Error(msg);
    7. }
    8. // Test case corresponding to filtering
    9. test('After switching category, query params sync and page resets to 1', () => {
    10. applyFilter({ category: 'electronics' });
    11. assert(getPageParam() === 1, 'Page not reset');
    12. assert(getQueryParam('category') === 'electronics', 'Category not synced to query param');
    13. });
    14. test('After one-click clear, the list returns to default results', () => {
    15. resetFilters();
    16. assert(isDefaultList(), 'Default list not restored after clearing filters');
    17. });
    18. // Test case corresponding to form validation
    19. test('Phone format error shows a message; valid input does not', () => {
    20. assert(validatePhone('12345') !== '', 'Should show a phone format error');
    21. assert(validatePhone('13800000000') === '', 'A valid phone should not error');
    22. });

    Giving these cases to the model along with the requirements is like handing it an executable acceptance spec; after the model generates, the team only needs to run the tests to quickly locate omissions.

    5.3 Establish a “Generate–Validate–Fix” Loop

    A single generation round rarely passes in one shot; a reliable approach embeds validation into the flow:

    • Static checks: type checking and linting to catch obvious syntax and type errors;
    • Unit tests: verify pure logic like filtering and validation;
    • End-to-end tests: drive a browser through page operations, covering key paths like carousels, filtering, and form submission;
    • Visual regression: screenshot key pages and compare against a baseline to avoid visual drift;
    • Automatic fixing: feed failure info back to the model along with the relevant code, constrain the scope of changes, and iterate 2–3 rounds.

    In mainstream agent development methodologies, the “continuous integration and continuous delivery” philosophy applies equally to AI-generated code: each generation result should be treated as a commit and must pass the same quality gates as human-written code.

    5.4 Generate Page by Page, Module by Module

    Asking for the whole project at once makes it easier to “lose sight of one thing while attending to another” the longer the context becomes. A more robust order:

    1. First establish the project structure, routing, and data model;
    2. Generate page by page, bringing global conventions (route table, design constraints, data model) as fixed context with every request;
    3. Extract shared components into shared modules for pages to reuse.

    5.5 Quantify Evaluation So Model Selection Is Evidence-Based

    Following this test’s observation dimensions, you can build a lightweight evaluation set: each real requirement gets a structured spec and test cases covering key interactions. Example metrics:

    1. Requirement coverage = tests passed / total requirement items # "Did we build it all?"
    2. First-pass rate = share of tasks passing on the first run # "Chance of getting it right the first time"
    3. Avg. fix rounds = total fix attempts / generation tasks # "Delivery efficiency"
    4. Visual consistency = manual or automated score of key pages # "Interface quality"
    5. Cost per requirement = total tokens × unit price / requirement items # "Economics"

    When switching models, changing prompts, or upgrading versions, use the same evaluation set for horizontal comparison rather than drawing conclusions from a single experience.

    5.6 Define the Human–Machine Collaboration Boundary

    The test has shown the model is well suited to “clearly structured” work like scaffolding, basic page generation, and first-pass styles; “constraint-dense” parts like filter logic, field linkage, and complex validation still need human oversight. A pragmatic split: the model produces, the developer reviews and fills gaps. This is also the mainstream collaboration paradigm of today’s “intelligent coding” tools.

    VI. An Objective View: Capability Benchmarking and Open Weights

    The community buzz about “matching the top frontier models” is essentially a discussion of the model’s capability tier. Combining the test results with the evaluation method yields a few relatively objective observations.

    6.1 “Benchmarking” Should Look at the Full Capability Curve, Not a Single Point

    The login page being basically on par with the reference object shows that leading models of this generation have reached the same tier on basic generation. But on reproducing complex pages’ requirements, there’s still a perceptible gap, and generation speed is affected by version and compute. A more meaningful comparison examines the full capability curve under the same set of real tasks and the same evaluation standard, rather than a single demo scenario.

    6.2 Open Weights: The Spread and Divergence of Code Generation Capability

    If the relevant model is offered with open weights, its impact on the industry shows mainly at four levels:

    • Data and code stay controlled: enterprises can deploy the model in their own environment, and source code and business data need not leave the internal network. This is especially critical for data-sensitive industries like finance, government, and healthcare.
    • More room for domain customization: enterprises can fine-tune with their own codebases, component libraries, and design specs, making the output fit their internal tech stack better and easing the “doesn’t understand our business” problem of general-purpose models.
    • Capability can be independently verified: once weights are open, third parties can reproduce tests in a unified environment. The standardized process used in this evaluation applies equally to open-source model assessment.
    • Pushes capability gradients downward: frontier capability gradually spreads from a costly, restricted state to smaller teams and regional markets, delivering two direct kinds of value:
      • Individual learning: understanding the code generation process in a local environment;
      • Prototype building for small teams: validating product ideas at a lower barrier.

    At the same time, stay clear-headed:

    • The current gap shows that open models still need continued iteration to faithfully reproduce complex requirements;
    • Private deployment requires compute investment and engineering capability; “open” doesn’t mean “zero cost”;
    • Ecosystem maturity, context length, and inference efficiency also need to be factored into evaluation.

    So open and closed source are more likely long-term complements than replacements: open source handles control and customization, closed source handles the capability ceiling, and enterprises combine them by scenario.

    6.3 Implications for the Direction of AI

    This test reflects three trends:

    1. From “completion” to “delivery”: the capability focus is shifting from generating code snippets to delivering runnable, deployable projects; the emergence of conversational prototyping and one-click deployment tools reflects exactly this trend.
    2. From “single model” to “system”: intelligent coding capability is increasingly made up of “model + tools + engineering constraints”; existing practices like requirement structuring, automated testing, and CI are becoming infrastructure for AI workflows.
    3. From “benchmark scores” to “real-task validation”: the gap between community buzz and actual experience shows that the industry needs more real-world evaluation standards rather than just watching leaderboard rankings.

    VII. Summary: From “Usable” to “Reliably Usable”

    The end-to-end test’s conclusions boil down to four points:

    • It can deliver: it can generate a complete project with multiple pages that is directly accessible;
    • There are gaps: logic reproduction like filtering and validation on complex pages falls short, and generation speed is slow;
    • The difference is in depth: the gap with peer frontier models is mainly in depth of understanding, not in whether it can generate a page;
    • The key is engineering: requirement structuring, automated testing, and the generate–validate loop are what let these tools truly deliver value.

    For developers, rather than chasing the wow factor of a single generation, it’s better to invest in a reusable evaluation and validation system. For enterprises and platform providers, the point to see clearly is this: LLM capabilities are rapidly approaching the frontier, offering more choices in openness, private deployment, and customization; the focus of competition is shifting from “model leaderboards” to “engineering delivery capability.” Whether you can plug model capability into a reliable production process is what ultimately decides the value of AI-assisted development.