Evaluating LLM Front-End App Generation: Why Structurally Correct Apps Miss Functional Requirements
A hands-on evaluation of how well today's general-purpose LLMs generate complete front-end applications from real requirements documents. Based on an end-to-end test across login, list, and form pages, it explains why models frequently deliver apps that are structurally correct yet functionally incomplete, and lays out engineering practices—requirements structuring, contract-style prompting, task decomposition, automated verification loops, and human-in-the-loop collaboration—that turn one-off generation into reliable delivery. It also compares a domestic general LLM against an international flagship model and discusses what this means for evaluation standards and the shift toward engineering agents.
Whenever a new general-purpose large language model is released, the community quickly fills with threads claiming the model “matches some international flagship,” and benchmark screenshots and evaluation posts flood in within hours. Behind the hype, though, what developers and enterprises actually want to know is simple: can it help me get the work done? Does it only produce scattered code snippets, or can it start from a requirements document and deliver a well-structured, correctly interactive, ready-to-run front-end application?
I previously ran an end-to-end application-generation test using a real project’s front-end requirements document. In one sentence, the result was this: it looks convincing, but the details don’t hold up to scrutiny. The page skeleton, visual style, and basic interactions all came out cleanly, yet features the document spelled out—filter conditions, field validation, state linkage—were partially omitted or oversimplified.
This isn’t an isolated problem with any single product; it’s a common weakness that surfaces when current LLMs generate complete applications. Rather than getting stuck in a “who’s stronger” argument, it’s more useful to answer three questions with real engineering value:
- How do you scientifically evaluate a model’s application-generation ability?
- Why do models produce output that is “structurally correct but functionally incomplete”?
- How can developers and enterprises turn generation ability into stable, reliable delivery?
1. Evaluation Prerequisites: From “Single-Point Capability” to “End-to-End Tasks”
Traditional code-capability benchmarks are mostly built around function-level problems: given a problem description and a function signature, the model completes the function body, and test cases decide whether it passes, with metrics like first-attempt pass rate. This kind of evaluation is cheap and reproducible, and it reflects a model’s grasp of syntax and algorithms reasonably well—but it’s clearly distant from everyday enterprise development.
Delivering a front-end application at a company involves at least four stages: understanding the business requirements, designing the page and component structure, implementing interaction logic and boundary states, and passing tests to go live. A mistake at any stage gets amplified in the next. That’s why evaluating a model should use “end-to-end tasks”: feed in a complete requirements document, get out a runnable project, and verify it layer by layer against a unified standard. A few principles matter here:
- Authentic tasks: Use real requirements documents accumulated on actual projects, not artificially constructed toy problems.
- Quantifiable acceptance: Turn “is it good?” into check items that can be executed automatically, reducing subjective judgment.
- Observable process: Record generation time, number of failed retries, and manual rework time—not just the final result.
- Reproducible results: Fix the runtime environment, dependency versions, and prompt template; run the same task at least three times and take the median rather than the single best run.
One thing worth stressing: free tiers and paid services differ substantially in concurrency quotas, rate limits, and queuing behavior. Time-to-completion and completeness measured under different conditions aren’t directly comparable, so be especially careful about this in any side-by-side comparison.
2. Evaluation Design: A Three-Tier Acceptance Standard
Given how front-end applications are delivered, I split the acceptance standard into three progressively deepening tiers:
| Tier | What to check | How to verify |
|---|---|---|
| Structure (can it run?) | Whether the project installs dependencies and builds successfully, whether routes and pages are complete, whether the component breakdown is reasonable | Automated build, manual browsing |
| Interaction (is the logic correct?) | Whether filtering, pagination, sorting, form validation, state linkage, and exception/empty-state handling match the document | Manual verification item by item, plus automated cases when feasible |
| Restoration (does it look right?) | How closely the visual style, responsive layout, and detail polish follow the design spec | Manual walkthrough, comparison against page screenshots |
These tiers are progressive. If the structure tier fails, the interaction tier is moot; if the interaction tier fails, even the most polished visuals are just a “nice-looking prototype.” In testing, many results that “look usable” broke down at the second tier.
3. Test Breakdown: Login, List, and Form Pages
The test used one real requirements document covering three typical page types: login, list, and form. The document was written in natural language without specifying a front-end framework, so the model chose a mainstream component-based framework on its own.
3.1 Usage Experience: A Double Test of Usability and Efficiency
On the first attempt, the server repeatedly returned a rate-limit message like “too many users right now—please queue or upgrade to a paid plan,” and it took several refreshes to get through. This has little impact on exploratory use, but for an enterprise team, peak-hour throttling directly affects development scheduling, so capacity planning has to factor it in.
Once generation began, it took about half an hour from start to finish—clearly on the slow side. One point in its favor: the output was a project you could open online directly, not just scattered code snippets. That means evaluation can happen in a real runtime environment, cutting the cost of manually setting one up.
3.2 Login Page: Close to Production-Ready
The login page had the highest completion. The carousel mentioned in the document was implemented, and form inputs, basic validation, and submit feedback were all present with no obvious gaps. Routes navigated correctly, making it essentially “ready to launch with minor tweaks.”
The reason is that the login page is a high-frequency, pattern-driven screen: form controls, validation prompts, and carousel switching all follow mature patterns, and public code samples are abundant. The model has seen countless similar implementations, so generating it is close to “reproducing a pattern,” with very low cognitive load.
3.3 List Page: Where Understanding Starts to “Slow Down”
Problems first appeared on the list page. The document clearly specified filter conditions and interaction logic—filtering by a status dropdown, keyword search, pagination—but the model didn’t fully implement the filters. More telling, this wasn’t a one-off: several list pages showed the same issue.
The page skeleton was correct—the table, pagination component, and route structure were all in place, and there was nothing visually wrong. But checking against the document item by item revealed clear functional gaps. In one sentence: the structure is right, the features are missing. Worse still, the model gave no hint; the omission happened “silently,” so a reviewer who isn’t paying close attention will easily miss it.
3.4 Form Page: Weak Restoration of Complex Requirements
The form page ran, but its field composition, validation rules, and interaction details weren’t fully implemented as the document described. For example:
- The document required certain fields to appear only under specific conditions, but the model laid out all fields at once;
- Validation rules were simplified to “required + length”;
- Field-to-field linkage (such as auto-filling a related field when a certain option is selected) wasn’t implemented.
The form page is the “deep end” of business logic: many fields, many rules, many condition combinations, and much of it scattered across the document in natural language, requiring the model to connect ideas across paragraphs. That cross-paragraph association is exactly where generative models are relatively weak.
3.5 Visual Style: Competent, Not Impressive
On the visual side, the output was slightly above a typical wireframe prototype, but still behind pages built directly on a mature design system (such as a mainstream enterprise component library or an atomic styling approach).
The overall design leaned “engineering-minded”: tidy layout, uniform spacing, but little brand identity or visual memorability, and unrefined interaction feedback like loading skeletons, transition animations, and empty-state illustrations. If I had to position it, it’s roughly “engineering-usable rather than product-beautiful.”
3.6 Side-by-Side Comparison: The Gap Is in “Depth of Understanding,” Not “Code Quality”
Coincidentally, I had previously implemented the same feature set with an international flagship model, which allowed a direct comparison (results for reference only):
| Aspect | Model A (a domestic general-purpose LLM) | Model B (an international flagship model) |
|---|---|---|
| Login page | Comparable completion, full functionality | Comparable completion, full functionality |
| List page | Correct page structure, filters missing | Filter conditions and field structure restored more completely |
| Form page | Runs, but validation and linkage simplified | Field relationships, validation logic, and interaction details restored more accurately |
| Generation time | Noticeably longer (affected by service load and quota; not a strict comparison) | Roughly a few minutes |
| Output | Directly accessible online | Directly accessible online |
The two were close on pattern-driven pages; the real dividing line showed up on pages that demand deep understanding of business rules. This means judging “parity” can’t rest on overall impression alone—you have to break it down to specific task types.
4. Root-Cause Analysis: Why “Structure Right, Features Missing”
At least five factors behind the phenomena above are worth understanding.
4.1 Instruction-Following Decay Under Long Context
Requirements documents often run into the thousands of words, with key constraints scattered across sections. It’s widely observed that models use information at different positions in a long context unevenly, and instructions in the “middle zone” are more easily overlooked. The more complex the document, the higher the chance of omission. Constraints like filters and field linkage are exactly the high-risk areas.
4.2 Implicit Dependencies Are Hard to State Explicitly
Human developers fill in implicit constraints from common sense: reset the page number when a filter changes; have keyword search stack with filter conditions; show an empty state rather than a blank table when there’s no data. Models may lack the same “default consensus,” so once the document doesn’t spell these out item by item, they tend to omit or simplify them.
4.3 One-Shot Generation Lacks a Feedback Loop
In conversational generation, the model outputs a large amount of code in one pass, with no way to run, test, or self-check against the acceptance criteria during generation. What it optimizes is “producing code that looks complete and reasonable,” not “passing requirements acceptance.” As a result, errors go straight into the final output.
4.4 A “Path of Least Resistance” Tendency
From a probabilistic standpoint, generating basic structure and conventional styles carries the least “resistance,” while implementing fragmented validation branches and boundary states costs more “cognitive effort.” Models tend to take the path of least resistance, which shows up as “a complete skeleton with diluted details.”
4.5 Ambiguity in the Requirements Input Itself
Natural-language requirements are inherently ambiguous: what dimensions does “supports filtering” refer to? Which rules exactly count as “reasonable validation”? The granularity of the requirements document itself often sets the ceiling on how well the model can understand.
5. Engineering Practices: Turning “Occasionally Usable” into “Stably Usable”
Once you understand the causes, the solution becomes clear: don’t expect the model to get everything right in one shot; use engineering means to keep uncertainty contained.
5.1 Structuring Requirements: Making Implicit Constraints Explicit
Don’t hand the model a long natural-language document directly. Instead, organize it into a structured “task card.” For a list page, something like this format works:
page: order-listfilters:- key: statuscontrol: selectoptions: [All, Unpaid, Paid, Shipped]behavior: on_change(reset_page(1), refetch)- key: keywordcontrol: searchbehavior: on_enter(reset_page(1), refetch)table:columns: [Order ID, Order Time, Amount, Status, Actions]pagination:page_size: 20keep_filters_on_page_change: truestates:loading: table skeletonempty: empty-state illustration + "Clear filters" buttonerror: error message + Retry buttonacceptance: # acceptance criteria, each item verifiable- "When a filter changes, reset the page to 1"- "After paging, keep the current filter conditions"- "When there is no data, show an empty state rather than a blank table"
The value of this step is that it turns the “implicit dependencies” the model most often misses into explicit, written-down constraints—and provides a basis for automated acceptance checks later.
5.2 Contract-Style Prompting: Specifying Input, Output, and Acceptance Criteria
The task instruction should read like a development ticket, with at least four parts: input (design mockups, API definitions), output (page structure and file organization), constraints (tech stack, coding standards, prohibitions), and acceptance criteria (functional check items listed one by one).
[Task] Implement the order list page[Input] Design mockup (see attached image), order query API definition (see document)[Output] A standalone page component + filter component + pagination config, with necessary comments[Constraints]- Do not omit any requirement item; if something can't be implemented, explain why first, then propose an alternative- All API calls must handle loading, empty, and error states- Form validation rules must strictly match the field table below; do not simplify them on your own[Acceptance criteria]1. After switching the status dropdown, the table refreshes and the page resets to 12. After entering a keyword and pressing Enter, it stacks with the status filter3. Current filter conditions are preserved when paging4. When there is no data, show an empty state and a "Clear filters" button
The rule “do not omit any requirement item” may look simple, but it significantly reduces “silent omissions,” prompting the model to surface problems when its ability falls short instead of quietly skipping over them.
5.3 Task Decomposition: Break Long Tasks Down, Generate Step by Step
Instead of asking the model to generate the entire project at once, advance layer by layer along “page → component → interaction”:
- Generate the page skeleton and routes first, and confirm the structure is correct;
- Implement the components and data requests one by one;
- Finally, fill in validation rules, boundary states, and style details.
After each step, confirm manually or verify automatically, and only move on once it’s confirmed correct. This keeps the context size of any single task under control and catches deviations early.
5.4 A Build-Verification Loop: Let the Model “Run Through” It
When conditions allow, put the generation process inside an “execution agent” framework to form this loop:
Requirement task card → generate code → automated build / type check → run acceptance cases↑ │└──────── feedback on failures to fix ←─────────┘
When a build fails or an acceptance case doesn’t pass, feed the error message back to the model to correct it, repeating until it passes. Compared with one-shot generation, this “generate → verify → fix” cycle sharply reduces low-level errors and gives developers a unified yardstick for objective comparison across multiple models.
5.5 Human-Machine Collaboration: Defining the Division of Labor
Generative tools aren’t omnipotent; a sensible split balances efficiency and quality:
| Stage | Suited to the model | Needs human oversight |
|---|---|---|
| Page building | Routes, layout, basic components, boilerplate code | Information architecture, interaction-flow design |
| Business logic | Routine CRUD, form validation, list interactions | Complex business rules, permission and compliance logic |
| Visual design | Basic styles, layout adaptation | Brand standards, design details, motion and interaction polish |
| Quality assurance | Generating test cases, fixing compile errors | Architecture decisions, security review, final acceptance |
5.6 The Engineering Foundation: Design Systems and Component Assets
The “unimpressive visuals” in the test largely stem from the model lacking a unified design-language constraint. Rather than letting the model freely improvise each time, give it your team’s own component library, design tokens, and coding standards as context so it can “create within constraints.” This ensures consistency and lets the team’s engineering experience accumulate into reusable generation assets.
In addition, accessibility requirements—semantic markup, keyboard operability, form-label association, alternative text for images—should also be written into the acceptance criteria as routine deliverables rather than after-the-fact patches.
5.7 From Generation to Launch: An Engineering Chain in the Cloud
Generated code only creates real business value once it enters a standard development pipeline. A typical chain looks like this:
Requirements doc → task card → model generation → automated acceptance → code hosting → continuous integration → object storage/CDN or container-platform deployment → logging service / monitoring and alerting
Using Baidu AI Cloud as an example, developers can combine an LLM inference service with cloud products such as object storage, a content delivery network, a container platform, code hosting, continuous integration, a logging service, and monitoring and alerting, embedding “model generation” into the full “build → test → release → observe” chain. This upgrades application delivery from a one-off code output into a sustainable engineering capability. (For specific product availability and configuration, refer to the official documentation.)
6. Implications for the Open Ecosystem and AI Development
Back to the original question: as domestic general-purpose LLMs compete with international flagships on general capability, what does that mean for the open ecosystem and AI development? At least three things are worth noting.
6.1 From “Follower” to “Diverse Supply”
Domestic general-purpose LLMs competing through open weights or open services give developers, beyond the closed-source flagship, a domestic option that is cost-controllable and lets them keep data under their own control. For enterprises with data-compliance requirements, or those wanting private deployment or deep customization, this diverse supply is itself a sign of a healthy ecosystem. But we should stay clear-eyed: local approximation ≠ full parity. Whether a model can deliver reliably in complex business scenarios is the true touchstone for how much “parity” is worth.
6.2 Evaluation Standards Should Move from “Benchmark Scores” to “Real Tasks”
Today’s LLM leaderboards are mostly built on static benchmarks like Q&A, reasoning, and single-function code completion, which can’t reflect end-to-end delivery ability. My test confirmed this: looking at any single dimension’s score, you might conclude the two are “on par,” but once it comes down to a real project, the gap shows up in the details. So the industry needs more evaluation systems aimed at real engineering tasks, for example:
- Application-generation evaluations based on real requirements documents;
- Executable, verifiable engineering-task benchmarks;
- Comprehensive assessment covering security, maintainability, and long-term evolution.
Continued investment by domestic vendors such as Baidu in LLM evaluation and industry-standard building will help shape evaluation criteria closer to real industry needs, turning “capability parity” from a slogan into a measurable fact.
6.3 From “Code Generation” to “Engineering Agents”
The problems this test exposed all point in the same direction: models don’t lack the ability to “write code”—they lack the closed-loop ability to “deliver according to requirements.” The future path of evolution is to give models a complete workflow of planning, execution, verification, and correction, turning them into “engineering agents” that can understand requirements, write code, run tests, and iterate based on feedback. This is precisely the significance of continued investment in areas like intelligent coding and agent platforms.
For developers, rather than chasing “who just matched whom,” it’s better to invest energy in refining how requirements are expressed, improving engineering processes, and accumulating domain knowledge. Tools will keep being updated, but the ability to turn vague requirements into clear tasks and unreliable output into reliable delivery will always remain an engineer’s core competitiveness.
7. Conclusion
A few conclusions from this test:
- Domestic general-purpose LLMs have made substantial progress in end-to-end application generation, with pattern-driven pages like the login page approaching usable quality;
- On complex list and form pages, models still share the common weakness of “structure correct, features missing,” and their understanding ability needs improvement;
- The gap between models lies mainly in depth of understanding, not in code quality itself;
- Structuring requirements, contract-style prompting, task decomposition, automated verification, and human-machine collaboration can significantly reduce generation risk and genuinely integrate the model into the development workflow.
The value of technology doesn’t lie in leaderboard rankings but in whether it solves real problems. For domestic LLMs, going from “usable” to “excellent,” and from “matching” to “surpassing,” will take sustained refinement in real business scenarios. For developers, staying rational and mastering engineering methods is what keeps you competitive through this wave of AI.
