LLM Frontend Code Generation Tested: Capability Boundaries and How to Evaluate Them

    When a model launches, "comparable to industry leaders" is almost boilerplate. For developers who actually have to ship generated code to production, the outcome depends on three plainer questions: does the model read requirements accurately, does it reproduce business logic completely, and does it deliver maintainable code reliably? This article reports a hands-on test of frontend code generation with two differently positioned models — a domestic open-source model (Model A) and an overseas closed-source flagship (Model B) — against the same structured requirements document. It breaks the results down page by page, compares the two models, analyzes why comprehension depth rather than coding ability becomes the dividing line, and lays out practical engineering methods (structured requirements, decomposition, mandatory human review, deployment and observability) for getting usable output from LLMs.

    1. Why a Test Was Necessary

    At launch time, “capabilities on par with the industry’s best” is almost standard phrasing. But for developers who have to put generated code into production, success rarely depends on a single polished demo or a benchmark score. It depends on three much plainer questions: can the model read the requirements accurately, can it reproduce the business logic completely, and can it deliver maintainable code consistently?

    Code generation is fundamentally a compound task — requirements comprehension plus engineering implementation. A model that looks impressive across a few single-page demos may not hold up on a real project with multiple pages, multiple interactions, and multiple validation states. So instead of being swayed by scattered marketing claims, it is more useful to build a reproducible evaluation framework and test engineering readiness against real requirements. That is the question this article sets out to answer.

    2. Evaluation Dimensions: Decide What You Are Actually Testing

    Before writing anything, you need a coordinate system. I evaluate a model’s frontend code generation across four dimensions:

    • Depth of requirements comprehension: does it genuinely understand both the explicit requirements and the implicit conventions in the document, rather than settling for “close enough”?
    • Functional completeness: is every feature stated in the requirements implemented, or are there systematic omissions?
    • Visual and interaction fidelity: do the UI style, component organization, and interaction details reach a level that is engineering-usable, or even genuinely good-looking?
    • Generation efficiency: how long from task submission to usable output, and how much do free versus paid tiers affect usability?

    To keep the comparison fair, I chose a real frontend requirement I had previously implemented with another mainstream model as the baseline sample — same task, with a reference answer available. The two subjects were positioned quite differently: one domestic open-source model (Model A) and one overseas closed-source flagship (Model B).

    3. Method: Making the Comparison as Fair as Possible

    I deliberately did not constrain the tech stack, providing only a structured requirements document, so that the test also covers how well each model selects and lands its own implementation. Both models defaulted to a mainstream component-based frontend framework, which is a reasonable choice on its own — but as the results show, a reasonable choice of stack does not guarantee a well-executed implementation.

    To keep the comparison valid, a few things were fixed in advance:

    • The same requirements document was fed to both models, so the input was identical;
    • No leading follow-up prompts were added, so the models’ first-pass comprehension could be observed;
    • Output had to be a directly accessible instance, not just code snippets, so each page could be checked in a browser.

    One caveat worth noting: during testing I frequently hit messages saying usage was currently high and that I needed to queue or upgrade to a paid tier, so first attempts often failed and only repeated retries succeeded. Usability of that kind belongs in the evaluation too.

    4. Page-by-Page: Where the Boundaries Actually Are

    1. Login page: stable on basic scenarios

    The requirements document explicitly called for a carousel, and Model A implemented it completely. The page had all its functions with no obvious omissions — the kind of output you could ship directly with minor adjustments. On simple pages with clear structure and linear logic, mainstream models already have fairly strong delivery capability.

    2. List page: comprehension starts to slip

    Problems surfaced on the product list page. The document spelled out the filter conditions and their interaction logic, yet Model A implemented no filters at all. More importantly, this was not an isolated miss but a shared issue across multiple list pages — evidence of a systematic weakness at the comprehension layer: the page skeleton was right, but fidelity to the functional semantics was clearly discounted.

    3. Form page: complex logic is not faithfully reproduced

    The form page ran end to end, but a large number of fields, validation rules, and interaction details were not implemented as documented. I believe the document itself was quite clear, yet the model’s reconstruction of a complex form was still insufficient. Forms are precisely the scenario that stresses comprehension depth — field dependencies, conditional validation, and submission state management are intertwined, and any single misunderstanding produces something that looks right but behaves wrong.

    4. Overall style: engineering-usable rather than product-grade

    The UI style sits somewhere “better than a rough prototype, but short of directly reusing a mature component library or design system.” The design sense is plain and engineering-oriented, with nothing striking. If I had to place it: engineering-usable outranks good-looking.

    5. Side-by-Side Comparison: The Gap Is Mostly About Comprehension Depth

    Because the same requirements had also been implemented with Model B, the comparison is fairly direct:

    Page Model A (domestic open-source) Model B (overseas closed-source flagship)
    Login page Comparable completeness, directly usable Comparable completeness
    List page Filters and other requirements missing entirely Fuller grasp of filtering and field structure
    Form page Fields and validation details poorly reproduced More accurate reproduction of field relationships, validation logic, and interactions
    Generation speed Noticeably slower (roughly half an hour from start to finish) Much faster (a matter of minutes)

    The conclusion is not complicated: the two are close on simple pages, and the gap concentrates in comprehension accuracy on complex requirements. As for speed, Model A’s slowness may be related to the free tier and cannot be attributed to model capability alone — but taken together, comprehension depth is the dividing line.

    6. Root Cause: Why Comprehension Depth Becomes the Dividing Line

    Abstracting one level up, the difference in model performance does not come from “whether it can write code” but from semantic parsing of the requirements:

    • Explicit requirements versus implicit conventions: simple pages have linear, explicit requirements that models hit easily. Complex pages contain many conventions that are “written down but scattered across paragraphs,” and the model has to consolidate them across sections and turn them into implementation constraints. That is the step most often lost.
    • Complexity of the state space: list filtering and form dependencies are essentially a combinatorial explosion of “input → state → output.” Without a complete state model, a model implements only the visible happy path and omits branch logic.
    • Global consistency: interaction patterns shared across pages require project-level consistency. Most models do reasonably well within a single page but struggle to stay consistent across pages.

    Insufficient comprehension depth is the root cause of degraded code generation quality in complex frontend scenarios.

    7. Engineering Practice: How to Actually Get Value from LLMs

    Capability limits do not mean a model has no value. The right method can raise output quality by a clear margin:

    1. Structure the requirements document. Rewrite prose descriptions into itemized contracts, listing fields, validation, and interactions one by one, which reduces the burden of cross-paragraph consolidation. For example:
    1. ## List page · Filter area (functional contract)
    2. - Filters: category (dropdown, single-select), price range (numeric, min <= max), on-shelf status (toggle)
    3. - Dependency: category change -> reset pagination to page 1
    4. - Validation: when price min > max, show "Invalid range" and disable the search button
    5. - Empty state: when no results, show a placeholder graphic plus a "Clear filters" button
    1. Divide and conquer. Split a large requirement into independent subtasks — login, list, form — and generate them one at a time, which reduces the comprehension decay caused by feeding everything in at once.
    2. Human review is not optional. For critical logic such as filtering, validation, and dependencies, check every item against the requirements. Treat the model as a highly productive implementer that still needs strict review.
    3. Combine with general-purpose cloud capabilities. A generated frontend artifact usually needs more before going live: a container platform to run it, object storage to host static assets, and logging plus monitoring and alerting for observability. Such capabilities are general-purpose categories offered by cloud providers such as Baidu AI Cloud, and can be paired with code generation tools to form a complete “generate — deploy — observe” pipeline.

    8. Broader Thoughts: Open-Source Models and the Direction of AI

    Back to the original question: what does progress in domestic models mean for open source and for AI development? Three points.

    First, open source narrows the capability gap. As the usability of open-source models gradually approaches closed-source flagships, developers are no longer locked to a single vendor, and requirements like private deployment, data compliance, and cost control become easier to satisfy.

    Second, competition accelerates iteration. When open-source and closed-source models chase each other on the same set of real tasks, the whole development ecosystem benefits. Taking evaluation methodology seriously is a sign of that competition maturing — a shift from “who is loudest” to “who is most verifiable.”

    Third, general engineering capability still has to accumulate. Current models are close to usable on simple pages, but complex logic reconstruction clearly has room to improve. Treating comprehension depth as a long-term target is more useful for getting AI into real engineering work than chasing single-point demos.

    9. Conclusion

    The biggest takeaway from testing on a real project is that rational evaluation is more valuable than chasing marketing claims. The frontend code generation capability of LLMs is already enough to speed development up significantly, but a model behaves more like a high-output implementer that needs clear instructions and strict review than a black box that can carry all complex logic on its own. Building your own evaluation framework, writing requirements clearly, and keeping a close eye on critical logic — that is the path from “usable” to “good.”