Evaluating LLM Frontend Coding Capabilities: Methods, Scoring, and Test Cases
Evaluating how well large language models (LLMs) write frontend code is too often reduced to a gut-feel comparison, which is neither reproducible nor explainable. This article presents a practical methodology for building a reproducible, regression-friendly evaluation: how to lock down variables such as capability tier, delivery mode, and runtime environment; how to design test cases that expose the widest capability surface (using a web-based Pomodoro timer as the running example); and a five-dimension weighted scoring system in which core usability acts as a veto. It also details common frontend correctness traps—timer drift in background tabs, autoplay-policy audio issues, and poor input validation—alongside a separate bug-fixing assessment. Finally, it shows how to turn a one-off review into a sustainable engineering workflow with baselines, automated execution, visual regression, and monitoring, so readers can build a capability profile they can trust for technology selection.
1. Why “It Feels Good to Use” Is Not a Valid Conclusion
The most common way to evaluate an LLM’s coding ability during tool selection or technical pre-research is to hand the same requirement to several tools and see which one produces the most convincing code on the first try. The problem with this approach isn’t the observation itself—it’s that the conclusion can’t be transferred. It’s neither reproducible nor able to explain where the gap comes from.
Concretely, informal evaluations typically suffer from three types of systematic bias:
- Prompt drift. The wording used to describe a requirement varies slightly each time, and the granularity of the requirement, implicit constraints, and whether examples are included all significantly affect the output. In the end, you’re comparing prompts rather than models.
- Incomparable scoring dimensions. Some people value “does it run,” others value “is the code elegant,” and still others value “does it pass on the first try.” Naturally, the conclusions differ from person to person.
- Insufficient sampling. Generative output is stochastic; a single sample gives you one data point, not a distribution. One brilliant performance or one failure is not enough to represent the true capability level.
Therefore, the goal of an evaluation shouldn’t be to rank tools, but to produce a reproducible, explainable, regression-friendly capability profile: for a given task type, how usable is each tool’s lower bound, where are its typical defects concentrated, and to what extent does it need human backup.
2. Evaluation Design: Lock the Variables First
2.1 Define Capability Tiers and Runtime Modes
The same model family often offers multiple specifications at once (a lightweight tier, a flagship tier, a reasoning-enhanced tier), and the delivery method splits into interactive web use versus API calls. The behavioral differences between the two are real: the web version usually ships with peripheral capabilities like retrieval, file reading, and conversation-context management, while API calls rely more on the caller to assemble context and toolchains themselves.
For this reason, an evaluation report must explicitly record three pieces of metadata: capability tier, delivery method, and whether a deep-reasoning toggle is enabled. Without this information, cross-tool comparison loses its premise. Pay particular attention to comparisons between paid and free tiers—at their core these are “price–capability” trade-offs rather than pure capability differences, and the conclusions should be stated separately.
2.2 Fix the Runtime and Acceptance Environment
The environment in which generated code runs directly determines the pass/fail judgment. It is recommended to fix it as: a single HTML file, native browser capabilities, with no external dependencies and no network resource requests allowed. This constraint may look simple, but it rules out a large number of interference factors at once—no need to deal with package management, build tools, cross-origin issues, or CDN availability—so the evaluation focus can stay on “the model’s understanding of the requirement and its code organization ability.”
2.3 Record Complete Experiment Logs
For each sample, retain at least: the raw prompt, the generation time, the full output, screenshots or run logs of the manual judgment process, and the point-deduction details. The value of these records is that when a conclusion is later disputed, you can trace back to the specific artifact rather than relying on memory.
3. Building the Test Set: How Much Capability One Problem Covers
The design principle for evaluation problems is to trigger the widest capability surface with the smallest problem statement. This article uses a task like a “web-based Pomodoro timer” as an example, because it simultaneously hits several key frontend development capabilities:
- State management: The timer has to handle the state transitions “not started → running → paused → finished”—the part most easily simplified away and most prone to boundary defects.
- Real-time rendering and performance awareness: Updating the UI every second or even every frame tests whether you understand render throttling and compositing overhead.
- Native browser API calls: Audio cues require correctly handling the browser’s autoplay policy, and time progression requires handling timer throttling in background tabs.
- Visual presentation and requirement understanding: A background that gradients smoothly over time requires understanding “gradient” as a function that continuously interpolates with progress, rather than hardcoding a static gradient image.
A qualified evaluation prompt should write the requirements as a decidable checklist, not as vague adjectives. Reference template:
Build a web-based Pomodoro timer in a single HTML file, with these requirements:1. Minimalist UI, single file, no external dependencies;2. Configurable focus duration (minutes) and break duration;3. Support start / pause / resume / reset;4. Play an alert sound when the countdown ends, with a mute toggle;5. The page background color gradients smoothly with task progress;6. Validate inputs (range, type, empty values); prompts must not block the main flow;7. When the page becomes visible again after being in a background tab, the timer result stays correct.
Item 7 is the key discriminator between “runs” and “reliable”—it pushes the test subject from feature implementation to correctness reasoning.
4. Scoring System: Five-Dimension Weighting and Judgment Criteria
Scoring must be built on observable behavior. The following structure is recommended, with a total of 100:
| Dimension | Observation Point | Judgment Method | Weight |
|---|---|---|---|
| Usability | Whether the core interactions actually work | Binary; failing gives an overall 0 | Veto |
| Feature completeness | Coverage of the requirement checklist item by item | Deduct per missing item; double the penalty for key items | 30 |
| Robustness | Input validation, boundary states, background recovery | Construct abnormal inputs and boundary scenarios and test them | 20 |
| Visual & requirement understanding | Whether the layout is clean, whether the gradient is truly continuous | Subjective scoring backed by objective sampling | 25 |
| Code quality | Structure, readability, division of responsibilities | Manual code review | 15 |
| Prompt extrapolation | Industry-standard practices not explicitly stated | e.g., accessibility, responsiveness, error messages | 10 |
On usability as a veto item: The starting point for accepting this kind of task is “clicking it produces a response.” If the generated code simply can’t run, discussion of the remaining dimensions is meaningless, and the report should mark it directly as an unusable sample, to avoid an inflated capability average caused by spreading the points around.
On the visual dimension not relying on subjectivity alone: Subjective impressions need to be backed by objective evidence, or they’re easily swayed by personal preference. An actionable approach is to sample the same task multiple times at a fixed progress point and compare screenshots, observing whether the color scheme is harmonious, the whitespace is reasonable, and the font hierarchy is clear, and to record decidable defects such as “whether there is overlap, overflow, or misalignment.”
On the tiered judgment of “gradient understanding”: This is the observation point that best reflects the depth of requirement understanding in this problem, and it can be split into three tiers:
- 0 points: No gradient, or a static gradient background that doesn’t change with progress.
- Partial points: A gradient exists, but color switching is abrupt (for example, switching palettes directly by stage).
- Full points: Color continuously interpolates with remaining time; at any moment, taking two adjacent progress points yields a color difference too small for the naked eye to detect.
Expressing the difference between these three tiers in code makes it clearer. A jump-style implementation usually maps progress segments to fixed color values:
// Jump: color changes abruptly when progress crosses a thresholdfunction updateBg(progress) {const stage = Math.floor(progress * 4);document.body.style.background = ['#0f172a', '#1e3a5f', '#7c2d12', '#b91c1c'][stage];}
A continuous-interpolation implementation does linear mixing at the channel level:
const START = [15, 23, 42]; // starting RGBconst END = [185, 28, 28]; // ending RGBconst lerp = (a, b, t) => a + (b - a) * t;const mix = (c1, c2, t) =>`rgb(${c1.map((v, i) => Math.round(lerp(v, c2[i], t))).join(',')})`;function paint(progress) {// progress ∈ [0, 1], derived from the remaining-time ratiodocument.body.style.background = mix(START, END, progress);}
For the driving mechanism, redrawing per progress via a frame callback is closer to the literal requirement of “smooth” than switching per second via a timer:
let running = false;function frame() {if (running) {paint(1 - remainingMs / totalMs);requestAnimationFrame(frame);}}
5. Robustness: Making Easily-Overlooked Correctness Traps Explicit
Frontend timers have three classic defects that almost inevitably appear in implementations that only care about “does it run,” which makes them ideal constructive test cases.
First, using accumulation instead of a time difference, causing the background tab to lose time. The browser throttles timers on invisible pages. If each trigger only decrements the remaining seconds by one, then after the page sits in the background for a few minutes and returns to the foreground, the displayed time will be noticeably too large. The correct approach is to model “remaining time” as “the difference between the end timestamp and the current moment,” replacing the accumulation logic with a difference calculation:
let totalMs, remainingMs, endAt, timerId = null;function tick() {remainingMs = Math.max(0, endAt - Date.now());paint(1 - remainingMs / totalMs);renderTime(remainingMs);if (remainingMs === 0) { stop(); playAlert(); }}function start() {if (timerId !== null) return;endAt = Date.now() + remainingMs;timerId = setInterval(tick, 200); // poll faster than the display precision}function pause() {if (timerId === null) return;clearInterval(timerId);timerId = null; // remainingMs already synced in the last tick}
The value of this single change is that it solves three problems at once—pause/resume, background recovery, and display error—a typical example of “using the right model rather than patching symptoms.”
Second, the alert sound is silent in some environments. Mainstream browsers require the audio context to be created or resumed by a user gesture. If it’s initialized directly at page load, the countdown-end sound very likely won’t be heard. A decidable acceptance method is to complete audio-context initialization and resume on the first “start” click.
let audioCtx;function unlockAudio() {const Ctx = window.AudioContext || window.webkitAudioContext;audioCtx = audioCtx || new Ctx();if (audioCtx.state === 'suspended') audioCtx.resume();}function playAlert() {if (!audioCtx) return;const osc = audioCtx.createOscillator();const gain = audioCtx.createGain();osc.frequency.value = 880;gain.gain.value = 0.2;osc.connect(gain).connect(audioCtx.destination);osc.start();osc.stop(audioCtx.currentTime + 0.3);}
Third, the way input validation is implemented. Legality checks are a bonus, but implementation quality varies greatly: interrupting the main flow with a blocking popup, reporting errors in real time during input, and silently correcting illegal values to defaults—all three differ in both experience and implementation cost. A more reasonable approach is inline hints plus blur validation, keeping the duration within a valid range. During evaluation, “whether there is validation” and “whether the validation method is reasonable” should be split into two independent observation points, otherwise crude implementations get rewarded.
6. Bonus Question: Independent Evaluation of Bug-Fixing Ability
Building on the implementation above, deliberately inject several defects (for example, changing endAt to accumulation, removing the event binding for the mute toggle, making the gradient depend on a fixed color table), then ask the test subject to fix them without rewriting. This can independently assess code-comprehension ability. The recommended tiered judgment is:
- Accurate localization but no fix: Identifies where the defect is and its cause—gets base points.
- Minimal fix that follows the original design: A controlled change scope that doesn’t break the existing structure—gets a higher score.
- A fresh approach with a full refactor: Functionally correct, but you need to assess the regression risk it introduces and the manual merge cost—scores slightly lower than the minimal-change solution.
The third option isn’t given the top score because, in real engineering collaboration, “making the minimal necessary change within the existing architecture” is usually preferred over “starting over from scratch”: it’s more reviewable, has a smaller conflict surface, and has a lower rollback cost. This trade-off is itself a judgment standard that the evaluation should convey to readers.
7. Common Pitfalls and Improvement Suggestions
- Drawing conclusions from a single sample. Generation is stochastic. Repeat the same problem 3–5 times and report both the mean and the dispersion. A high mean with high variance means you must prepare a retry mechanism before shipping.
- Prompts that get vaguer over time. “Make it look nicer” is not decidable; “ample whitespace, no more than three font levels, no large solid-color blocks” is.
- Only testing generation, not fixing. Generation ability determines onboarding speed; fixing ability determines long-term collaboration cost. The two should be scored separately.
- Ignoring accessibility and responsiveness. Keyboard reachability, visible focus, adequate contrast, and no overflow on narrow screens are industry-standard requirements that should count as bonus observation points even if the prompt doesn’t mention them.
- Mistaking peripheral capabilities for coding ability. Bundled capabilities like retrieval and file reading change the usable scenarios but shouldn’t be counted in a comparison of pure code-generation ability, or the conclusion gets diluted by peripheral features.
8. Turning Evaluation into a Sustainable Engineering Process
A one-off evaluation only solves tool selection; regression evaluation solves evolution. When putting it into practice, the process can be split into four parts:
- Case and baseline management: Put prompts, expected behavior, and scoring rules under version control. Scoring-rule changes must leave a trace, or historical scores become incomparable.
- Automated artifact execution: Load the generated HTML in a unified runtime, complete clicks, inputs, and waits programmatically, and collect console errors, unhandled exceptions, and key DOM states.
- Visual and behavioral regression: Take screenshots at fixed progress points and compare pixel-level differences against the baseline, turning “whether the UI changed noticeably” into an alertable metric; also assert observable behaviors such as the countdown-end event, the mute toggle, and state transitions.
- Result archiving and monitoring: Archive artifacts, logs, and screenshots to object storage, write scores and execution results to a logging service, and use monitoring alerts to track key-metric trends, forming a longitudinal view of “version—score—defect distribution.”
This way, a one-off subjective experience becomes auditable data: when a tool’s capabilities upgrade, you can clearly answer “which dimensions improved, which scenarios regressed, and whether the human-backup strategy needs adjusting.”
9. Conclusion
The core of evaluating an LLM’s coding ability isn’t finding a “strongest” answer, but establishing a set of rules that make conclusions hold up: first lock the variables, then use decidable prompts to expose capability differences, replace impression-level judgment with behavior-level observation, and finally automate the process for long-term tracking. By practicing with this article’s five-dimension scoring system and correctness-trap checklist, readers can produce, within a few hours, a capability profile far more reliable than a “gut-feel comparison,” and reuse it directly in subsequent technology selection and R&D process improvement.
