Testing LLM-Generated Frontend Projects: Capability Assessment and Engineering Implementation
This article reports a hands-on test of using an LLM to generate a complete, runnable frontend project from a real requirement document. It documents the test methodology and a reproducible information block, quantifies the usage experience (access stability, generation time, build acceptance, and delivery format), and evaluates per-page capabilities across login, list, and complex form pages. It analyzes why capability decays as complexity rises, discusses UI quality and the significance of advancing open-source LLMs, and offers practical guidance for bringing AI coding tools into engineering workflows—including complexity-based usage tiers, a structured requirement-document template, a manual review checklist, automated validation commands, and security and compliance boundaries.
Introduction: Choosing LLMs for Frontend Projects—What to Watch and Which Errors Are Common?
As LLMs continue to improve, “input a requirement and directly generate a runnable frontend project” is no longer just a demo concept—it is starting to enter real production scenarios. But when facing the various reviews that accompany each new model release, what teams actually care about is not community sentiment but three questions: How do you choose an LLM to generate frontend projects? What should you watch during evaluation? Which errors are most common in actual delivery?
This article reports a complete hands-on delivery test based on a real frontend requirement document. Without constraining implementation details, it lets the model generate a runnable project in a single pass after being given the requirement, then systematically breaks the results down—from test data, usage experience, and per-page capabilities to root-cause analysis, UI style, the significance for open source, and engineering implementation recommendations. Rather than simply discussing “whether the model can write code,” this article focuses more on “whether the generated result can enter the engineering workflow.” It is intended for developers and decision-makers who are evaluating whether AI coding tools can join team collaboration.
1. Test Methodology and a Reproducible Information Block
Many model capability evaluations produce misleading conclusions—not because the model’s performance itself lacks value, but because the evaluation design is not rigorous enough: the requirements are too toy-like, the input too fragmented, and the acceptance criteria unclear. This test follows these principles:
- Use real requirements, not toy demos. Requirements should come from actual projects and include multiple pages, multiple roles, multiple states, and complete business flows.
- Provide complete documents, not scattered prompts. Requirements are organized into a structured document that includes a feature list, field definitions, interaction logic, validation rules, and page relationships.
- Do not constrain implementation details; observe default choices. Without pre-specifying the language, framework, or component library, observe the model’s default technical-selection tendencies under open constraints.
- Focus on runnable deliverables, not isolated code snippets. The final acceptance target is an accessible project, not scattered functions.
To make the evaluation more reproducible, this test adds the following information block:
- Model name/version: ERNIE-5.5-flash-preview (example version; when teams retest other models, they should replace it and record the specific version number).
- Requirement document size: about 4,200 words, including feature descriptions, field definitions, interaction descriptions, validation rules, and page relationships.
- Page list: login page, product list page, order/customer profile form page.
- Runtime environment: Node.js 18, npm 10; default frontend stack React 18 + TypeScript 5 + Vite 5; browser environment Chromium.
- Access method: local development address http://localhost:5173; the generated result also provides a project archive and a preview entry.
- Build commands: npm install, npm run typecheck, npm run lint, npm run build.
- Time measurement: from submitting the complete requirement document to the project being first accessible. If queuing or retries occur, record the count separately.
With this handling, readers see not “it feels strong” or “it feels weak,” but a verifiable model version, input size, environment configuration, and acceptance result.
2. Usage Experience: Quantitative Records Instead of Emotional Judgments
Overall usage experience determines whether a tool can enter daily development. This test focuses on quantifiable metrics rather than simply describing “fast” or “slow.”
Access Stability
During peak hours, the page showed a “too many users—please queue or pay” prompt 3 times; after 5 re-entries, the task successfully started. For developers who pursue efficiency, such queuing interrupts the workflow, so evaluation cannot look only at generation quality—it must also consider stability.
Generation Time
From submitting the requirement document to the project being first accessible took about 31 minutes. This is clearly longer than the second-level response of code completion, because the generation process is not a single code completion but closer to a long-chain task execution: understanding the requirement, planning pages, generating files, assembling the project, and attempting to run it. The longer the chain, the higher the time cost and uncertainty.
Build Acceptance Results
The first check after generation produced these results:
- lint: passed.
- typecheck: 2 type errors initially.
- build: could complete the build, but had incomplete route navigation.
- after manual fixes: build passed, pages accessible.
Therefore, a more reasonable conclusion is not “the generated result is definitely usable” or “the generated result is unusable,” but rather: the model can produce a runnable project skeleton, but whether it can enter production still depends on subsequent testing, fixes, and business acceptance.
Delivery Format
A highlight of this test is that the model did not only provide several code snippets but produced an accessible project address and a project archive. This means the output is closer to a “runnable artifact” than “parts that developers must assemble themselves.” For AI coding tools moving from assistive completion toward autonomous delivery, this is a positive signal.
But teams should also note the risk: do not directly upload real user data, keys, or internally sensitive business information to generative tools. For real projects, desensitize the data and set access controls.
3. Per-Page Capability Assessment: Conclusions First
Beyond the overall experience, what truly tests the model is how faithfully each page’s functionality is reproduced. The test results show that capability decays as page complexity rises. Here is the conclusion first:
| Page Type | Generation Completeness | Manual Completion Ratio | Applicable Scenarios |
|---|---|---|---|
| Login page | High; basic skeleton usable | ~20% | Internal system login, prototype validation |
| List page | Medium; complete skeleton but weak filtering | ~50% | Admin dashboards, operations tools |
| Complex form page | Low; clear gaps in fields and validation | ~70% | Orders, approvals, customer profiles, and other core forms |
Login Page: A Usable Baseline Skeleton That Still Needs Testing and Business Acceptance
The login page includes common interactions such as forms, validation, navigation, and a carousel. The model can complete the basic page structure and main interactions, and its overall usability is relatively high. The reason is that login page logic is relatively standard; the industry already has many mature patterns, and the model has seen enough diverse samples, so it can reproduce them at high quality more easily.
But one point needs correcting: the login page is not something you “just tweak and ship.” A more accurate judgment is: the basic skeleton is usable but still needs testing and business acceptance. Before launch, you should at least check login-failure messages, CAPTCHA logic, permission-based navigation, secure transmission, error copy, analytics/tracking interfaces, and so on.
List Page: Filter Logic Is the Dividing Line of Comprehension
Once you reach the product list page, problems begin to surface. The requirement document explicitly requires filtering by category, price range, and listing status, with linked pagination, but the filter options in the model’s result were not fully implemented. This is not an isolated single-page issue but a systemic shortcoming that appears across multiple list pages.
For the list page, here is a recommended acceptance checklist:
| Check Item | Result This Time | Fix Recommendation |
|---|---|---|
| Filter fields | Not fully implemented | Add category, price range, status, and other fields per requirements |
| Filter-linked pagination | Does not reset pagination | Force page to reset to 1 when filters change |
| Empty/loading states | Missing | Add loading, empty-data, and request-failure states |
| Query parameter retention | Not retained | Write filters and pagination into the URL query for easier refresh and sharing |
If the requirement specifies React 18 + TypeScript 5, the filter logic should at least include state definitions, filter updates, and pagination resets. For example:
type Filters = {category: string;maxPrice: number;status: string;page: number;};const DEFAULT_FILTERS: Filters = {category: 'all',maxPrice: 9999,status: 'all',page: 1,};const [filters, setFilters] = useState<Filters>(DEFAULT_FILTERS);const applyFilter = (patch: Partial<Filters>) => {setFilters(prev => ({ ...prev, ...patch, page: 1 }));};
Note that in the example, maxPrice should not use Infinity, because an infinite price can cause anomalies in serialization, API parameters, and UI display. In a real project, replace it with a specific price ceiling such as 9999, or extract it into a configurable constant.
Models can usually generate attractive list rendering and pagination but tend to miss filter state management, filter-triggered pagination resets, and filter-query linkage. This kind of cross-component, cross-state logic is precisely the key dividing line for list-page comprehension.
Form Page: Insufficient Fidelity for Complex Fields and Validation
The complex form page runs overall, but its fields, validation, and interaction details were not fully implemented per the document. Complex forms are difficult because they contain many implicit constraints: field linkage, conditional required fields, cross-field validation, submit states, error states, and API-failure echoes. These logics do not exist in isolation but form an interdependent network of constraints.
The complex form acceptance table is as follows:
| Check Item | Acceptance Requirement | Result This Time |
|---|---|---|
| Field linkage | Show the unified social credit code field when an enterprise customer is selected | Partially implemented |
| Conditional required | The unified social credit code is required for enterprise customers | Missing or unstable |
| Cross-field validation | End time must be later than start time | Partially implemented |
| Submit state | Disable the button while submitting to prevent duplicate submissions | Missing |
| Error state | Complete field-level error messages and submission-failure messages | Incomplete |
| Echo/retention | Retain already-entered content after a failure | Incomplete |
The following test cases illustrate the failure points:
- Customer type is “enterprise” but the unified social credit code length is not 18 characters — should show “Unified social credit code format is incorrect.”
- End time earlier than or equal to start time — should show “End time must be later than start time.”
- After clicking submit, if the API responds slowly, the submit button should be disabled to avoid duplicate submissions.
- When the API returns a failure, the user’s input should be retained and the failure reason displayed.
Example logic can be written as follows:
const validate = (values: FormValues) => {const errors: Partial<Record<keyof FormValues, string>> = {};if (values.customerType === 'enterprise') {if (!/^[\w]{18}$/.test(values.creditCode || '')) {errors.creditCode = 'Unified social credit code format is incorrect';}}if (values.endTime && values.startTime && values.endTime <= values.startTime) {errors.endTime = 'End time must be later than start time';}return errors;};
Models tend to turn forms into a “stack of fields”—rendering all fields but incompletely reproducing field dependencies and conditional rules. This is also an important reason why form pages lag behind the login page.
4. Root Cause Analysis of Capability Shortcomings
Taken together, the capability decay is not random but has several explainable root causes:
- Instruction-following decay in long contexts. The longer the requirement document, the more the model’s attention to detail instructions is diluted. Earlier, structured instructions—such as page skeletons and field lists—are completed at a high level; later instructions requiring cross-paragraph association—such as filter linkage and cross-field validation—are easily overlooked.
- Insufficient ability to model complex dependencies. The core of a real frontend project is not static UI but state and dependency relationships. Models are good at generating interfaces that “look right” but struggle to stably maintain the state machine that “runs right.”
- Missing self-verification. Human engineers test as they write, whereas models lack effective self-verification and regression during long-chain automatic generation, so missed logic is hard for them to detect themselves.
In this task sample, compared with top closed-source models, the gap is small for standardized pages like the login page, but the gap in comprehension accuracy is more pronounced for list pages and form pages. The core of the gap is not “whether it can write code” but “the depth of understanding of the requirement document.” This is precisely what determines whether AI coding tools can take on complex business.
5. UI Style: Engineering Usability Over Product Aesthetics
At the visual level, the model-generated UI style is unremarkable: better than most wireframe prototypes but still short of mature component libraries or atomic CSS solutions—overall leaning toward simple engineering and lacking a strong sense of design.
For generative frontend tools, a more pragmatic positioning is: engineering usability over product aesthetics. When evaluating UI, it is not advisable to look only at “whether it looks good”; consider adding these dimensions:
- Whether grid spacing is stable;
- Whether responsive breakpoints are clear;
- Whether component styles are consistent;
- Whether form controls are accessible;
- Whether buttons, tables, and dialogs have unified interactions;
- Whether there is basic mobile adaptation.
If the goal is to validate an idea, build an admin dashboard, or generate a runnable prototype, such output is already efficient enough; if the goal is a polished product for end users, designers still need to be involved for refinement.
6. The Deeper Significance of Improving Open-Source LLM Capabilities
Setting aside the success or failure of a single evaluation, a more valuable question is: as open-source model capabilities gradually improve, what does this mean for the AI ecosystem? Based on this task sample, several positive implications emerge:
- Lowering the barrier and cost of accessing capability. More teams can obtain near-first-tier capability at lower cost and in a more controllable way, which is especially suitable for data-sensitive scenarios that require private deployment.
- Driving the ecosystem from single-point leadership toward multi-point competition. When the gap between open-source and closed-source narrows, the focus of competition shifts from “whose parameters are larger” to “who understands the scenario better, who engineers better, and whose inference is more cost-effective.”
- Accelerating capability equalization and innovation diffusion. Open source means reproducible, fine-tunable, and auditable; researchers and enterprises can perform deep optimization for vertical domains on top of an open-source base.
- Exposing shortcomings and pointing to the direction of evolution. The insufficient depth of understanding exposed by this test highlights the next optimization priorities: long-context instruction following, complex state modeling, and a self-verification loop.
Of course, it also needs to be viewed objectively: in this task sample, open-source models still have room for improvement in comprehension accuracy for complex business scenarios. Capability equalization is a trend, but closing the gaps still takes time.
7. Practical Recommendations and a Reusable Template for Engineering Implementation
If a team is preparing to bring AI coding tools into a real project, the following approach is recommended.
1. Use by Page Complexity Tier
- Login, registration, basic CRUD: can be handed to the model for generation first, with manual testing added.
- Complex lists, filtering, sorting, batch operations: manually focus on checking filter linkage, pagination resets, empty states, and query parameters.
- Complex forms, approval flows, financial fields: manual intervention is required to complete field linkage, conditional required fields, cross-field validation, and error echoes.
2. Structured Requirement Document Field Template
The requirement document should explicitly list the following fields:
| Field Name | Type | Required | Validation Rule | Linkage Rule | Error Message | Page Location | Acceptance Criterion |
|---|---|---|---|---|---|---|---|
| customerType | enum | Yes | Individual/Enterprise | Controls display of the unified social credit code field | Customer type cannot be empty | Basic info section | Field visibility correct after switching |
| creditCode | string | Conditionally required | 18 characters | Required for enterprise customers | Invalid format | Enterprise info section | Not shown for individual customers |
3. Manual Completion Checklist
After generation, check against the following list:
- Whether filter fields are complete;
- Whether filter changes reset pagination;
- Whether empty, loading, and error states are complete;
- Whether query parameters are retained in the URL;
- Whether form field linkage is correct;
- Whether conditional required fields take effect;
- Whether cross-field validation is bound to the correct error location;
- Whether the submit state prevents duplicate submissions;
- Whether input is retained after an API failure;
- Whether access control is implemented;
- Whether sensitive data is desensitized;
- Whether dependency security and licensing are reviewed.
4. Recommended Automated Validation Commands
Do not rely only on manual visual inspection; incorporate the generated project into an automated validation pipeline:
npm run typechecknpm run lintnpm testnpm run buildnpm audit --production
If the team uses a license compliance tool, dependency license checks can also be added to avoid introducing component libraries or third-party packages that do not meet project requirements.
5. Security and Compliance Boundaries
In real projects, it is not advisable to directly provide raw business data, user information, or keys to generative tools. Before launch, also check whether the generated code leaks sensitive information, has privilege-escalation risks, or exposes internal logic through error messages.
Summary
This test shows that current LLMs are already capable of delivering runnable frontend projects. Standardized pages have high completeness, but for tasks requiring deep comprehension—such as list filtering and complex form validation—capability decays noticeably as complexity rises. The core gap is not “whether it can write code” but “the depth of understanding of the requirement document.”
For open-source LLMs, continuously approaching capability is significant: it lowers the barrier, drives competition, promotes capability diffusion, and points to optimization directions by exposing shortcomings. For developers, the most pragmatic path is to treat the model as a high-quality scaffold, while keeping complex logic, security boundaries, and final business acceptance in the hands of engineers.
