Evaluating LLM Frontend Generation: End-to-End Testing, Capability Limits, and Why Open Source Matters

    Model release announcements and leaderboard scores routinely claim frontier-level coding ability, but shipping a real frontend is a long chain, not a single-turn Q&A. This article evaluates how current models perform when asked to generate a runnable, accessible frontend project end to end from a requirements document with built-in acceptance criteria. Across login, list, and form pages, fidelity drops as constraint density rises: pattern-heavy pages score high, while cross-page state and interaction logic stay weak. The analysis argues that understanding depth, not raw benchmark points, is the real dividing line, and explains why open-weight models approaching frontier capability matter for data residency, reproducibility, customization, and ecosystem growth. Practical guidance covers structuring requirements into machine-readable specs, front-loading design tokens and component contracts, replacing manual review with acceptance scripts, staged delivery, and a self-built internal evaluation set.

    1. Leaderboard Buzz Is Not the Same as Ship-Ready

    Every time a new model generation ships, social platforms fill with verdicts like “matches the international frontier” or “a drop-in replacement for task X.” These conclusions usually come from single-turn, Q&A-style tests: one clear instruction, one verifiable output, and a score becomes the conclusion.

    But real frontend delivery is not a single-turn Q&A. It is a long chain:

    Requirement parsing → information architecture and page breakdown → component and state design → interaction and linkage implementation → validation and error-state handling → visual consistency → runnability and maintainability.

    If a model’s understanding “slows down” at any link in the chain, the final output ends up looking finished but being unusable. So the engineering value of a model has little to do with a few extra points on a general leaderboard. What matters is whether it can hold a stable level of fidelity on tasks that are constraint-dense, shared across pages, and detail-heavy.

    This article uses a frontend requirements document with built-in acceptance criteria as input and asks a model to produce, end to end, a complete, runnable, and accessible project (without prescribing a tech stack). We then check each page against its functional points and interaction details, and summarize the technical causes behind the gaps and the engineering responses that address them.

    2. Turning “Feel” into a Measurable Acceptance Framework

    To make evaluation results reproducible and comparable, you first have to fix the acceptance criteria. We suggest six dimensions, each mapped to a checkable artifact:

    Dimension What to check How to judge
    Runnability Can dependencies be installed, does the build pass, is a reachable URL provided Can it be opened and used directly
    Requirement coverage Whether every functional point in the document has a corresponding implementation Line-by-line checklist
    Interaction fidelity Linkage, reset, pagination, empty states, error states, URL state sync Walk the operation paths
    Code quality Component split granularity, state ownership, type annotations, naming consistency Structural review
    Visual consistency Unified design tokens, responsive breakpoints, loading and empty states Screenshot comparison across viewports
    Delivery efficiency Queue and concurrency limits, single-generation time, multi-round fix cost Timing and round-count logging

    The value of this framework is that it translates “it just feels a bit off” into concrete statements such as “the filter linkage on page type 3 is missing, and the validation rules on page type 4 deviate in 3 places.” Only what is locatable can be improved.

    3. Layered Test Results: A Capability Gradient Across Three Page Types

    3.1 Login pages: fewest closed-loop nodes, highest completion

    The login page is usually the best-generated page. Carousels or background images, username and password fields, CAPTCHA, “remember me,” agreement checkboxes, error prompts, and submission feedback—these elements appear with very high frequency in public corpora and engineering templates. They sit in the model’s “high-frequency pattern memory zone,” so generation needs almost no reasoning, only pattern completion. The result is a complete, feature-rich page with no obvious gaps, essentially “tweak a few details and ship.”

    The conclusion is not that “the model is good at login pages.” It is that the task itself has low constraint density and high pattern reuse, which inflates the score. If you only pick these pages when evaluating, you will systematically overestimate the model’s ability.

    3.2 List pages: correct skeleton, decaying functional understanding

    On product or content list pages, the problems become concentrated. The page structure, grid layout, and card styling are basically correct, but entire filter groups spelled out in the document are missing, or only the UI was built without wiring it up:

    • The linkage rules among filtering, sorting, and pagination are not implemented (the page index does not reset after a filter changes).
    • Filters are not two-way synced with the URL query parameters (the state cannot be shared and is lost on refresh).
    • Empty-result states, loading skeletons, and retry-on-failure are missing.
    • The “clear all” behavior for combined filters and the default sort order are undefined.

    This is not an occasional omission but a systematic weakness on this class of page. The root cause: the difficulty of a list page is not rendering but the state source and state machine. A rule chain in the document like “filter change → reset pagination → merge parameters → re-request” spans multiple sections, and the model tends to pick up only the explicit subtask of “render the list” while ignoring the state constraints that are prerequisites.

    3.3 Form pages: field relationships and validation details distort most easily

    Form pages generally run, but the field-level deviations are the most numerous:

    • Conditional display / dependent disabling is not implemented (the unified social credit code field appears only when “enterprise” is selected).
    • Validation rules are oversimplified: digit count, ranges, and cross-field consistency checks are missing.
    • Asynchronous validation (for example, whether a phone number is already registered) is not wired up, or lacks debouncing and race handling.
    • The submit state has no protection against duplicate clicks, and a failure does not preserve the already-entered content.
    • Error messages are not tied to specific fields and only give a global prompt.

    Form pages have the highest constraint density: every field carries a type, required flag, validation rules, linkage relationships, and copy requirements. In the document, this information is often scattered across different sections as lists or tables. When compressing a long document, the model tends to keep structural information (which fields exist) at the expense of relational information (how the fields affect one another).

    3.4 Visual style: engineering usability beats design dazzle

    The generated UI is at a level that is “more polished than a hand-drawn prototype but clearly engineered”: spacing, font sizes, and corner radii are basically consistent, but design hierarchy is missing—weak primary/secondary contrast, little brand feel, and insufficient detail polish (transitions, micro-interactions, loading states). It still lags behind the complete design language that ships with mature component libraries.

    This dimension is usually not the model’s capability bottleneck but the result of missing prompts and constraints: the requirements document specifies only functionality, with no design tokens, color semantics, font scale, or component-mapping rules. So the model falls back to the “safest” generic aesthetic from its training.

    3.5 Delivery efficiency: queuing and latency are real costs

    Queue waiting during high-concurrency periods, single-generation times on the order of half an hour, and free-tier limits all directly affect usability. Compared with closed-source baselines of similar capability (on the order of minutes for the same tasks), open-source or free entry points still lag in throughput and engineering orchestration. This is a reminder that model capability evaluation must include the service form, not just the weights themselves.

    4. Why “Understanding Depth” Is the Real Dividing Line

    Attributing the phenomena above converges on four points, which are also the focus of all subsequent engineering optimization:

    1. Keeping long-document global constraints. Rules shared across pages (a unified pagination protocol, a unified date format, unified error handling) must be distilled into a global configuration from the start, rather than re-inferred on every page.
    2. Inferring implicit requirements. The document will not spell out “pagination changes should scroll to the top” or “a delete confirmation should show how many items are affected.” Whether these default interactions are filled in determines whether the output looks like it was made by someone who has done this before.
    3. Constraint adherence and post-generation self-check. A strong model self-checks against the requirements line by line before outputting; a weak model only guarantees syntactic and runtime correctness.
    4. Repairability. Whether the generated code can be modified locally without ripple effects—whether components are decoupled, state is centralized, and styles are tokenized—directly determines the cost of multi-round iteration.

    Add two often-overlooked variables. First, evaluation set saturation: public problems have been trained on repeatedly, so scores cannot reflect real-world performance. Second, the leverage of prompts and scaffolding: a structured requirements document plus explicit constraints often improves final quality more than switching models does.

    5. How to Rationally Read “Performance Matches the Frontier” Claims

    Capability approaching the frontier is worth attention, but it holds only under three qualifications:

    • Same-condition comparison: the same requirements document, the same prompts, and the same acceptance script, sampling multiple times to get a success-rate distribution—not taking the best result from a single demo.
    • A capability matrix: score by page type (login / list / form / detail / dashboard), by complexity (single page / multi-page linkage / cross-platform), and by stage (generation / debugging / refactoring), rather than issuing a single overall score.
    • Threshold thinking: the question that matters in engineering is not “who is a few points higher” but “whether it has crossed the threshold for unattended delivery.” The current shared position of mainstream models is: simple pages have crossed the line, complex interactions are still below the threshold, and human-involved review and repair are still required.

    6. What Approaching the Frontier Means for the Open-Source Model

    When open-weight models close in on closed-source frontier capability on real tasks, the significance is not the word “drop-in replacement” but four shifts at the industry-structure level.

    First, keeping data in-domain becomes an option. When user data, internal codebases, or regulated business are involved, private deployment is the only reliable path. Downloadable weights mean inference can happen inside your own network, so data does not have to leave the boundary.

    Second, results are reproducible and processes are auditable. With a fixed weight version, fixed quantization precision, and fixed inference parameters, the same input yields a predictable output. This is decisive for engineering scenarios that require auditing and traceability, such as finance, healthcare, and government.

    Third, customization moves from prompt engineering to weight engineering. With the weights in hand, you can do continued pre-training on a private codebase, instruction tuning on domain samples, and alignment on preference data, then distill the capability into a smaller-parameter model that runs on-device and at the edge. Cloud providers such as Baidu AI Cloud offer model services, model development, and coding-assistant capabilities that productize this “data preparation → training → evaluation → deployment” pipeline and lower the barrier to self-building.

    Fourth, the ecosystem flywheel starts turning. Inference-framework adaptation, quantization and compression, heterogeneous-compute support, and co-built toolchains and evaluation sets are all positive feedback loops unique to open source: every adaptation lets more hardware run it, and every public evaluation lets more teams see it clearly.

    At the same time, two premises deserve a rational view. Open source and closed source are a division of labor, not a replacement: closed-source solutions explore the capability ceiling while open-source solutions determine the pace of adoption. And open weights do not equal unrestricted commercial use: license terms, compliance of third-party licenses in generated code, and security review of output content must all be built into the process.

    7. Actionable Engineering Practices

    7.1 Structure the requirements document: turn implicit rules into machine-readable constraints

    Do not feed the model a long natural-language document directly. First convert it into a structured spec:

    1. {
    2. "page": "product_list",
    3. "route": "/products",
    4. "filters": [
    5. { "key": "category", "type": "single-select", "options_from": "category_dict" },
    6. { "key": "priceRange","type": "range" },
    7. { "key": "stockOnly", "type": "boolean", "default": false }
    8. ],
    9. "interactions": {
    10. "on_filter_change": ["reset page_index to 0", "merge query params into the URL", "keep current sort"],
    11. "on_reset": ["clear all filters", "restore default sort"],
    12. "empty_state": "empty-state illustration + clear-filters button",
    13. "url_sync": "two-way sync of filters and pagination; state survives refresh"
    14. },
    15. "acceptance": [
    16. "rapid category changes produce no duplicate requests",
    17. "page index resets after a filter change",
    18. "URL is shareable and refresh restores identical state"
    19. ]
    20. }

    The key is to extract “rules” from prose into fields. The more structured the spec, the less the model has to infer, and the higher the fidelity.

    7.2 Front-load constraints: provide design tokens and component boundaries first

    Inject global conventions before generation so styles are not reinvented on every page:

    1. // Global design tokens and component contracts, generated before pages
    2. const tokens = {
    3. color: { primary: "#1a73e8", danger: "#d93025", bg: "#f7f8fa" },
    4. space: [4, 8, 12, 16, 24, 32],
    5. radius: { sm: 4, md: 8, lg: 12 },
    6. font: { base: 14, title: 20, caption: 12 }
    7. };
    8. // Validation rules as data, not scattered across page templates
    9. const rules = {
    10. phone: { required: true, pattern: /^1[3-9]\d{9}$/, msg: "Invalid phone number format" },
    11. smsCode: { required: true, dependsOn: "phone", length: 6, msg: "Verification code is 6 digits" },
    12. submit: { debounce: 800, preventDuplicate: true }
    13. };

    7.3 Replace manual visual inspection with acceptance scripts

    Turn the linkages most likely to go missing (see 3.2) into automated cases and run them directly against the model’s output:

    1. // Framework-agnostic acceptance example
    2. test("page index resets to 0 after a filter change, with no duplicate requests", async () => {
    3. const p = await open("/products");
    4. await p.select("category", "book");
    5. expect(await p.pageIndex()).toBe(0);
    6. expect(await p.countRequests("/api/products")).toBe(1);
    7. });
    8. test("filter state is restored after refresh", async () => {
    9. const p = await open("/products?category=book&page=2");
    10. expect(await p.selected("category")).toBe("book");
    11. expect(await p.pageIndex()).toBe(1);
    12. });

    With a script, “looks about right” becomes “12 of 17 passed,” improving both review efficiency and objectivity.

    7.4 Deliver in stages, not all at once

    We recommend four stages: information architecture and routing → static skeleton and component contracts → state and interactions (including filtering, pagination, and forms) → styles, responsiveness, and error states. Each stage’s output is smaller and easier to verify, errors surface earlier, and repair cost drops by an order of magnitude.

    7.5 Build a small internal evaluation set

    Pick 20–30 cases from historical, real requirements, each paired with a structured spec and acceptance scripts, to form your team’s own baseline. Compared with public leaderboards, it more closely matches your business distribution and gives a comparable regression curve as models iterate.

    7.6 Three reviews that must pass before launch

    Security review (input validation, authorization boundaries, sensitive-data leakage), license compliance review (authorization terms for dependencies and generated code), and maintainability review (component decoupling, state ownership, duplicate code). These three cannot be self-certified by the model and must stay in the process.

    8. Conclusion

    The conclusion from one end-to-end test is actually quite plain: current models are strong at pattern-based pages and weak at constraint-dense and cross-page linkage. The meaningful reading of “matches the frontier” is not to swap out the toolchain but to take three actions—structure the requirements, automate acceptance, and keep the right to review complex interactions with humans.

    For open-source models, capability approaching the frontier points to something more long-term: LLM capability no longer has to be an exclusive resource of a handful of closed-source services, but can become deployable, modifiable, and auditable infrastructure inside an enterprise. Only when more teams can tune a model into their own shape using their own data does the barrier to AI application innovation truly begin to fall.