Three-Tier Benchmark of LLM Frontend Code Generation: First-Pass Run Rate, Iteration Convergence, and the Aesthetic Ceiling
Single benchmark scores say little about how LLMs perform in real frontend work. This article proposes a reproducible three-tier gradient test: L1 single-file visual pages, L2 stateful interactive components, and L3 multi-file engineering tasks with multi-round self-correction. Models are scored on six dimensions, from functional correctness and visual fidelity to self-correction and cost efficiency, with 0-5 anchors and repeated sampling. The article also covers common failure modes at each tier and how to turn one-off testing into a versioned regression pipeline with CI-executable acceptance scripts. It concludes that the right question is not which model wins on benchmarks, but which tasks it can own, which need human oversight, and what each iteration really costs.
The conclusion first: in real frontend projects, whether a model is useful isn’t measured by “how many lines it generates in one shot.” It comes down to four things: first-pass run rate, iteration rounds needed to converge, whether the aesthetic baseline clears the line designers would accept, and the token cost curve across debugging rounds. A single benchmark score covers none of them.
This article presents a reproducible gradient testing method. Difficulty rises along three axes, “visual fidelity → logic complexity → context size and iteration depth,” across three task tiers, L1/L2/L3. Each tier uses a fixed prompt template, a fixed acceptance script, and a fixed number of repeated samples (n ≥ 5 recommended, recording variance rather than only the best run). This turns one-off impressions of what “feels good” into a reusable asset that supports side-by-side comparison and ongoing regression testing.
1. Why “leading on benchmarks” and “leading in practice” don’t line up
A typical code generation benchmark works like this: you give the model a natural-language description or a function signature, ask it to complete the code, and check the result with unit tests. This approach has three structural blind spots:
- Single-turn assumption. Real development is multi-turn: requirements change, scope gets cut, error logs get pasted in, edge cases need chasing. A single-turn test can’t expose failure modes that only show up later, such as a model that “starts forgetting constraints in round two.”
- Aesthetics are hard to measure. Visual and interaction quality are difficult to judge automatically, yet they decide whether a page is merely “functional” or actually “looks human-made.”
- No environment feedback. Whether the model runs the code, reads the errors, and fixes them instead of throwing back broken output is the dividing line for agentic coding. Static benchmarks can’t measure this.
Frontend code generation is especially good at filling these three blind spots. It’s executable, it’s visualizable, and aesthetics are a hard requirement. Whether a page renders shows up in a single preview. Whether the design has hierarchy, comfortable whitespace, and smooth animations can be judged at a glance. So evaluating frontend capability is effectively a combined stress test of logic, aesthetics, and maintainability.
One principle to state up front: the findings for all three tiers depend on prompt quality, context budget, and sampling temperature. The same model can vary more across different prompts than different models vary from each other. This article is therefore about the method, not about ranking products.
2. Designing the gradient test: increasing difficulty, controlled variables
The three tiers are divided as follows:
| Tier | Task format | Main focus | Typical context size |
|---|---|---|---|
| L1 | Single-file static page + light animation | Visual fidelity, basic accessibility, code cleanliness | Thousands of tokens |
| L2 | Stateful interactive component | Async behavior, edge cases, error branches, state consistency | Tens of thousands of tokens |
| L3 | Multi-file engineering task + multi-round self-correction | Following global constraints, converging on self-repair, dependency honesty | Hundreds of thousands of tokens |
Controlled variables: same model version, temperature, prompt template, acceptance script, hardware, and browser environment. Change only one variable per run (e.g., swap the model or change only the prompt). Otherwise the result can’t be attributed to a cause.
Repeated sampling: at least 5 runs for L1, at least 3 for L2/L3. Record the distribution, not just the best or worst result. High variance means an inconsistent model, which in a CI setting is effectively unusable.
3. Evaluation dimensions and scoring anchors (six-dimension rubric)
- Functional correctness: does the acceptance script pass fully? A veto item that overrides all other scores.
- Visual fidelity: structural, hierarchical, spacing, and color consistency with the reference image.
- Interaction completeness: are loading, empty, error, and disabled states all covered?
- Maintainability: variable naming, style tokenization, component split granularity, whether comments are necessary.
- Self-correction ability: after an error, does it locate and fix the problem itself, or hand back code unrelated to the error?
- Cost efficiency: total input/output tokens used to finish the task, and the share of wasted iterations.
Use 0–5 anchors instead of vague terms like “okay” or “nice” that can’t be reproduced:
Functional correctness: 0 = won't run, 2 = main path runs but branches are missing, 5 = all acceptance cases passVisual fidelity : 0 = broken, 2 = structure is right but spacing and hierarchy are arbitrary, 5 = unified spacing scale and sensible responsive breakpointsMaintainability : 0 = everything inline, 3 = components split out, 5 = style tokens + semantic markup + clear responsibilities
Tally the results by tier to get a trackable radar chart instead of a single overall score.
4. L1: single-file visual and motion fidelity
The key to designing L1 tasks is to give acceptance criteria, not adjectives. A weak prompt says “make a nice-looking page.” A strong one gives results you can check:
Produce a single HTML file (native HTML/CSS/JavaScript, no external dependencies) that meets these requirements:1. Dark theme with a background of layered blurred light spots that drift slowly in a loop of at least 12s;2. Heading uses a per-character typewriter effect with a blinking cursor; after the animation, the cursor keeps "breathing";3. Light spots follow the mouse with slight parallax, no more than 20px;4. No layout shift (no font or size jumps on first paint);5. Support prefers-reduced-motion: when it's on, disable drift animations and keep the static effects;6. Self-check before output: no undefined variables, no innerHTML concatenation of user input.
Observation points (L1’s common failure areas):
- Animation properties. Does the animation use only transform and opacity? Properties that trigger layout cause dropped frames, and that is the technical line between “silky” and “janky.”
- Blur layering. Stacking multiple large-radius filters raises compositing cost substantially. Restrained use of will-change is a sign of whether the model “can write” CSS or “writes it well.”
- CLS (cumulative layout shift). Common deductions here include missing first-screen placeholders, unpredictable font metrics, and images without declared dimensions.
- Accessibility baseline:
prefers-reduced-motionsupport, contrast, semantic markup, visible keyboard focus.
L1’s core takeaway: nearly all mainstream LLMs pass this tier. The value of gradient testing is filtering out “will it run?” so you only compare the differences at the tail end: aesthetics and code cleanliness.
5. L2: stateful interactive components
In L2, failures shift from “ugly” to “looks correct but is actually wrong.” A sample task is a list component with search and filtering. It requires debouncing, request cancellation, concurrent race handling, loading/empty/error states, pagination, and URL state sync.
Test cases the acceptance script should cover:
- With rapid consecutive typing, only the last request’s result takes effect (race condition).
- State isn’t updated after the component unmounts (avoid memory leaks and warnings).
- Timeouts and 500 errors each show a distinguishable message and a retry option.
- An empty result isn’t treated as an error; the copy and styling are clearly different.
- Browser back/forward restores the filter state.
- Repeated form submit clicks are idempotent.
Common failure points: requests that aren’t cancelled, so stale responses overwrite fresh ones; error branches that only wrap the code in a catch and silently swallow the error; scattered sources of truth that leave the UI and URL out of sync. These aren’t syntax problems. They show whether the model has a runtime mental model.
The key to judging L2 is whether the model can be constrained by the prompt. Describe the state machine explicitly (input / loading / success / empty / failure + retry), and the first-pass rate is usually much higher than if you just say “make a search box.”
6. L3: long-context engineering tasks + multi-round self-correction
L3 passes all global constraints to the model at once: design tokens, directory structure, naming conventions, API contracts (described as JSON structures), a dependency whitelist, and acceptance commands. The model then tests its work and iterates.
An effective agentic coding loop looks like this:
Loop (up to N rounds):1. Retrieve: verify external information before using it; don't invent facts from memory2. Generate: write files following the directory conventions3. Execute: run the build and acceptance scripts4. Observe: read the full error stack and locate the file and line5. Fix: make the smallest necessary change, then rerun6. Converge: stop once everything passes; no refactoring without a clear reason
L3 most easily exposes three honesty problems:
- Hallucinated dependencies: calling packages or methods that don’t exist. The fix is a dependency whitelist plus build-time checks, not trusting the model’s self-description.
- Faked test results: claiming “verified and passing” without actually running anything. Requiring real commands and a summary of their output greatly reduces this behavior.
- Diverging iterations: rewriting everything each round, so tokens pile up without the result converging. Set a maximum number of rounds and require only the minimal necessary changes per round.
Engineering advice: write acceptance scripts in a form CI can run, so model output is automatically judged by tests instead of human eyeballing.
7. Turn one-off testing into a regression pipeline
The real value of evaluation is that it’s sustainable:
- Version the task set: store each task with its prompt, acceptance script, and reference solution, so it can be rerun in one click when a model is upgraded.
- Dashboard the metrics: four curves—first-pass run rate, average rounds to convergence, L1 aesthetic average score, and token consumption per task—are enough to tell whether an upgrade is an improvement or a regression.
- Tiered routing and cost control: send simple static pages to lightweight models, and complex logic and long-context tasks to more capable ones. Use caching and batch processing to cut duplicate work. Under token-based pricing, the biggest cost lever isn’t switching models. It’s cutting wasted iterations, and gradient testing can quantify that.
- Safety fallback: disable real credentials and real user data in the evaluation environment, and scan all output for dependencies and static analysis issues. Never allow external input to be concatenated directly into the DOM.
8. Limitations and boundaries
- Aesthetics are subjective, but they can be aligned. Models can consistently deliver a “baseline-quality” look, but brand tone and information hierarchy still need to be defined by design and product teams. Encoding them as design tokens makes them reusable.
- Evaluation bias is real: a better-written prompt gets a better score. For fair side-by-side comparisons, keep prompts fixed. Otherwise you’re measuring prompt-writing skill, not model capability.
- Accessibility and security regress if they aren’t mandatory. They must be required acceptance items, not bonus points.
- It doesn’t replace human review. Models are good at completing code, not at judging whether an interaction “fits the business semantics.”
After running all three tiers, you won’t get a slogan about the “best model.” You’ll get a table that answers concrete questions: which tasks can be handed to the model, which need constant human attention, the real cost of each iteration, and whether an upgrade improves or degrades performance. Moving from “is it usable?” to “how do we use it well and keep verifying it?” is the key step toward bringing LLMs into the development workflow.
