LLM Code Generation in Practice: The Three-Tier Evaluation Method

    This article introduces the Three-Tier Evaluation Method for assessing large language models on real-world front-end code generation. Instead of relying on public benchmark rankings, the method evaluates models across three progressive tiers—static visual fidelity, interaction and state management, and engineering quality with self-correction—and quantifies results with a unified scorecard for side-by-side comparison and regression testing. It also covers the four stages of agentic coding, three key metrics (context window, reasoning ability, and cost per call), and practical guidance on design tokens, phased generation, feedback loops, version regression, and full-chain cost accounting. Teams can use it to turn subjective impressions into observable, quantifiable, and repeatable metrics, moving models from impressive demos to reliable everyday tools.

    1. The Evaluation Gap: Why a High Benchmark Score Doesn’t Mean It’s Good to Use

    When choosing an LLM, many teams start by looking at public leaderboard rankings, then comparing context window length and API pricing. But once you plug a model into an IDE or coding assistant and let it handle real front-end development tasks, a different class of problems surfaces:

    • It runs, but it looks bad: generated pages have harsh color schemes and disordered spacing, stuck in the visual style of generic UI frameworks from several years ago.
    • Static is fine, interaction is weak: simple display pages may pass on the first try, but once a task involves form interdependencies, filtering and sorting, or asynchronous data loading, the error rate rises noticeably.
    • It can’t fix errors: when a runtime error occurs or the page goes blank, the model just pastes the error message back verbatim, unable to locate the root cause and fix it.
    • Context is wasted: even when you provide the design spec, API docs, and component guidelines, the output only reflects a small fraction of that information.

    The root cause is that public leaderboards mostly focus on tasks that are easy to score automatically—math reasoning, single-function completion, unit test pass rates—whereas “turning a natural-language requirement into a deliverable page” is a multi-constraint, multi-turn task that also requires aesthetic judgment. So evaluating code generation capability needs a method that closely mirrors the real development workflow.

    This article proposes a Three-Tier Evaluation Method: design tasks around three progressive levels—static visual fidelity, interaction/state management, and engineering quality with self-correction—and then quantify the results with a unified scorecard. It can be used both to compare different models side by side and as a regression baseline after a model version upgrade.

    2. Understanding the Foundation: Agentic Coding and Three Key Capabilities

    In recent years, Agentic Coding has become an important direction for LLM applications. Instead of a one-off “ask—generate” exchange, the model autonomously plans around a goal, calls tools, runs verification, and iterates on fixes to close the loop. In front-end development, this capability typically shows up across four stages:

    1. Understanding and planning: break down the requirement and decide on page structure, component breakdown, and implementation path;
    2. Tool use: search for information, consult documentation, run commands, read and write files;
    3. Execution and verification: start the local environment, render the page, inspect console errors and network request results;
    4. Self-correction: locate the cause based on errors and page behavior and fix it, rather than throwing the problem back to the human.

    When describing and validating this capability, three foundational metrics matter most:

    • Context window: determines whether you can fit the full design spec, API docs, and project guidelines in one pass, reducing the information loss from multiple rounds of back-and-forth;
    • Reasoning ability (e.g., chain-of-thought enhancement): determines how robustly the model handles complex interactions, edge cases, and asynchronous logic;
    • Cost per call: under a mature billing model, this usually includes both a component proportional to input/output length and a significantly discounted component for repeated context after a cache hit, directly affecting whether the model can be used at scale and high frequency within a team.

    Note that terms like “first-pass success rate” and “aesthetic score” below are definitions at the evaluation-method level and don’t refer to any specific product. Each model’s actual performance on different tasks should be based on the real results from your own internal evaluation set.

    3. Core Method: The Design Logic of the Three-Tier Evaluation Method

    The three tiers correspond to three progressive levels of front-end capability:

    • Tier 1 (Static): can it “get the look right”?
    • Tier 2 (Interaction): can it “get the logic right”?
    • Tier 3 (Engineering): can it “build a good product”?

    To make results reproducible and comparable, every task should use a fixed input format: requirement description + constraints + acceptance checklist. Also record the model version, prompt version, and key generation parameters. Beyond the model’s own output, consider including the full trajectory of the agentic coding tool—which tool calls it made, how many correction rounds it went through—rather than looking only at the final code.

    Prepare 3–5 tasks per tier, each corresponding to a common business form such as a marketing landing page, an admin list page, a data dashboard, or a form workflow page, and keep the tasks consistent across every evaluation run.

    Tier 1: Static Visual Fidelity—Aesthetics and First-Pass Success

    Focus: understanding design intent, controlling visual hierarchy and color, and the soundness of layout structure.

    Task design: give explicit content and style requirements. For example: build a tech product landing page with a dark theme, a background with a soft gradient glow, a hero headline with a typewriter effect, three equal-width feature cards below, and an overall emphasis on whitespace and visual hierarchy.

    Scoring points:

    • Overall style: does it accurately deliver the theme and color palette, or is it gaudy or cluttered from piling on “all the trendy elements”;
    • Typography details: is alignment clean, does spacing follow a consistent rhythm, and do font sizes create a clear hierarchy;
    • Color quality: are the colors restrained and harmonious, and does contrast meet readability requirements (body text to background contrast should be at least 4.5:1);
    • Responsive: do cards stack naturally on narrow screens, and do navigation and images adapt;
    • Code cleanliness: does it use CSS variables to manage colors and spacing consistently, or does it contain meaningless nesting and repeated styles;
    • First-pass success rate: the proportion of tasks that pass acceptance without manual modification.

    Common failure modes: elements nested layer upon layer with chaotic positioning; overuse of shadows, gradients, and glow effects; missing responsive breakpoints.

    Tier 2: Interaction and State Management—Logic and User Experience

    Focus: state design, event handling, edge-case coverage, and accessibility basics.

    Task design: on top of the Tier 1 page, add an “article filter list” module that includes:

    • category tab switching;
    • keyword search with debounce (~300 ms pause before triggering);
    • sorting by publish time or popularity;
    • data from a local mock API, with three states: loading, empty result, and request failure.

    Scoring points:

    • State design: does it avoid maintaining the same data in multiple places, and are derived values computed in real time;
    • Interaction completeness: are click, input, and keyboard interactions all present, and does the result update immediately and correctly;
    • Edge-case handling: does the empty state show a friendly message, is the loading state guarded against duplicate submissions, and can a failure be retried with one click;
    • Usability details: do buttons show clear disabled/loading feedback, do controls have accessible labels, and is color not the sole carrier of information;
    • Performance awareness: does it consider virtualization or pagination when the list grows large.

    Illustrative implementation snippet:

    1. // Search debounce + AbortController cancels stale requests so old results don't overwrite new ones
    2. function debounce(fn, wait = 300) {
    3. let timer;
    4. return (...args) => {
    5. clearTimeout(timer);
    6. timer = setTimeout(() => fn(...args), wait);
    7. };
    8. }
    9. const searchArticles = debounce(async (keyword) => {
    10. controller?.abort();
    11. controller = new AbortController();
    12. try {
    13. const res = await fetch(`/api/articles?keyword=${encodeURIComponent(keyword)}`, {
    14. signal: controller.signal,
    15. });
    16. if (!res.ok) throw new Error(`Request failed: ${res.status}`);
    17. articles = await res.json();
    18. render();
    19. } catch (err) {
    20. if (err.name !== 'AbortError') renderError('Failed to load, please retry');
    21. }
    22. }, 300);
    23. input.addEventListener('input', (e) => searchArticles(e.target.value));

    Tier 3: Engineering and Self-Correction—Closed-Loop Capability

    Focus: code maintainability, build runnability, and the ability to iterate and fix based on feedback.

    Task design: require a complete, runnable small app that includes routing, component breakdown, type definitions, and basic unit tests, and ensure the build command passes. Then enter the “self-correction test”:

    1. Code review: an independent review model or script checks the code;
    2. Generate test cases: produce test cases for edge cases and run them;
    3. Feedback loop: organize the failing cases and build/runtime logs and hand them back to the generating model, asking it to fix only the problems without introducing extra changes;
    4. Convergence: repeat the process, record how many rounds are needed for defects to converge, and whether a “fix one thing, break another” loop appears.

    Illustrative feedback template:

    1. ## Run Results
    2. - Build: Failed
    3. - Unit tests: 3 cases passed, 1 failed
    4. ## Error Message
    5. TypeError: Cannot read properties of undefined (reading 'map')
    6. at renderList (src/components/List.jsx:42:18)
    7. ## Requirements
    8. Please locate the root cause and make a minimal change. After fixing, resubmit the complete file. Do not modify unrelated logic.

    Three-Tier Scorecard

    For easy side-by-side comparison, use a unified 1–5 scoring card (higher is better), with weights adjustable per business priorities:

    Dimension Suggested Weight Key Metrics
    Correctness 30% First-pass success rate, unit test pass rate, type and build checks
    Visual aesthetics 25% Style fit, layout hierarchy, color, responsiveness, accessibility
    Interaction and engineering 25% State design, edge-case handling, code structure, maintainability
    Self-correction 20% Localization accuracy, fix convergence rounds, regression rate

    In practice, losing too many points on lower-tier tasks indicates weak foundational capability, while high-tier performance is what determines whether a model can enter the production workflow. Sample each task at least 3 times to reduce the interference of randomness from a single generation.

    4. Advanced Practice: Making Prompts and Engineering Systems Work Together

    Evaluation is just the starting point; consistently producing high-quality code also depends on prompt design and supporting engineering systems.

    1. Provide a “design system” first, then ask for “pages”

    Rather than describing the visual style every time, have the model produce or confirm a set of Design Tokens up front, and have all subsequent pages reference them uniformly. This significantly improves visual consistency:

    1. :root {
    2. --color-bg: #0b1020;
    3. --color-surface: rgba(255, 255, 255, 0.06);
    4. --color-text: #e6e9f0;
    5. --color-accent: #6ea8fe;
    6. --space-1: 4px;
    7. --space-2: 8px;
    8. --space-3: 16px;
    9. --space-4: 32px;
    10. --radius-md: 12px;
    11. --font-heading: 700 clamp(2rem, 4vw, 3.5rem) / 1.2 system-ui, sans-serif;
    12. }

    2. Generate in phases, not in one shot

    For complex pages, break the work into phases: requirement and constraint confirmation → page skeleton → component implementation → interaction and data → self-check and optimization, with each phase producing a human-reviewable intermediate result. Decomposing large tasks aligns with the iterative delivery approach used in mature software development.

    3. Build a verifiable feedback loop

    Visual deviations can be caught with screenshot comparison—for example, capture key pages with a headless browser and compare them against the target design at the pixel or structural level. Functional deviations are left to automated tests, then tracked continuously via logging services and monitoring alerts for post-launch error rates and page performance. The model’s value lies in shortening the “find—locate—fix” chain, not in replacing the testing system.

    4. Establish a version regression mechanism

    Fix the Three-Tier tasks as an internal evaluation set under version control, and re-run scoring after a model or prompt upgrade to avoid unknowingly regressing specific scenarios while the overall picture looks better. Enterprise teams can also use general-purpose capabilities from cloud platforms such as Baidu AI Cloud—model services, object storage, logging services—to build a unified, traceable evaluation and invocation pipeline.

    5. Account for cost across the full chain

    The price of a single generation is only part of the cost; you also need to count:

    • the time spent on manual fixes;
    • the extra token consumption from multiple retries and self-correction;
    • maintenance costs, such as code readability and extensibility.

    If the invocation supports context caching, put the system prompt and project guidelines that change less frequently at the front, so they can reuse the cache and reduce input billing costs—refer to the official documentation of the platform you use for the exact billing rules.

    5. Conclusion: Replace Intuition with Quantifiable Methods

    The competition in front-end code generation is shifting from “can it write code that runs” to “can it write code that looks good, works well, and is easy to maintain.” The core value of the Three-Tier Evaluation Method is that it breaks “how it feels” into observable, quantifiable, and regression-friendly metrics:

    • the static tier measures aesthetics and consistency;
    • the interaction tier measures logic and edge cases;
    • the engineering tier measures maintainability and the self-correction loop.

    For a team, the more pragmatic path is human-AI collaboration: let the model handle repetitive work like page skeletons, boilerplate code, and common components, while developers focus on interaction design, performance optimization, and business rules. Set the tone with the Three-Tier Evaluation, safeguard it with a design system and testing system, and control scale with cost accounting—only then can you turn an LLM from a “dazzling demo” into a reliable, everyday productivity tool.