Three Evaluation Tiers for Measuring the Capability Boundaries of Agentic Frontend Coding

    Large language models can already generate runnable frontend code. The more important question is where they can work independently and where they still fail. This article presents a reproducible three-tier evaluation workflow covering algorithmic logic, visual and interaction fidelity, and self-correction under long-form documentation. Each tier defines its environment, task, prompt, acceptance criteria, scoring weights, cost metrics, and security boundaries, so teams can assess whether an AI coding agent is ready to participate in real development work.

    Large language models can already generate runnable frontend code. The more important question is where they can work independently and where they still fail.

    Agentic coding allows a model to inspect a project, run builds and tests, interpret errors, and modify the implementation without requiring the developer to manually move context between tools. This workflow is closer to real software development than a simple question-and-answer interaction.

    This article presents a three-tier evaluation process covering algorithmic logic, visual and interaction fidelity, and self-correction under long-form documentation. Each tier specifies the test environment, task, prompt, and acceptance criteria. It also defines scoring weights, cost-reporting methods, and security boundaries. Teams can apply the framework to their own projects to determine whether a model is suitable for entering the development workflow.

    1. What Is Agentic Coding?

    Traditional AI-assisted coding is usually conversational. A developer describes a requirement, the model returns code, and the developer pastes any resulting error messages back into the chat. This approach can be efficient for an individual function, but moving context back and forth becomes expensive in a multi-file project.

    Agentic coding allows the model to operate in a loop:

    1. Read project files.
    2. Modify the code.
    3. Run build or test commands.
    4. Read the output.
    5. Decide what to do next.

    For frontend development, three capabilities are especially important:

    1. Context retrieval: The model can read type definitions, API documentation, and component specifications to fill in information that is missing from the initial request.
    2. Execution and observation: The model can run tests, builds, or a development server in an isolated environment and inspect the console output.
    3. Self-correction: When an error occurs, the model can use stack traces to locate the cause and modify the code. This loop can repeat for multiple rounds, but each round needs an explicit stopping condition, such as “all tests pass” or “the maximum number of rounds has been reached.”

    Execution capability also introduces risk. Code run by the model may read or write files, access the network, or call external services. The execution environment must therefore be isolated from the developer’s host system and production credentials. Specific security boundaries are defined in Section 5.

    2. Two Types of Frontend Generation Challenges

    Frontend code requires both logical correctness and visual quality. If either is missing, the page is not ready for delivery.

    Logic

    Modern framework features such as state management, effect cleanup, and list rendering are easy to implement incorrectly. Typical problems include:

    • A missing variable in a useEffect dependency array that creates a stale-closure bug.
    • An asynchronous response arriving out of order and leaving the UI in an inconsistent state.
    • Incorrect offset calculations in a virtualized list with dynamically sized items.

    Visual Quality

    Visual quality can be evaluated through measurable criteria. This framework uses the following indicators:

    • Spacing should use multiples of 4px or 8px, and a page should use no more than five spacing levels.
    • The contrast ratio between body text and its background should be at least 4.5:1. Large headings should reach at least 3:1, based on the WCAG 2.1 AA standard. Contrast should be measured with axe or browser developer tools on a display at normal brightness.
    • Interaction animations should last between 150ms and 300ms. Animations should preferably use transform and opacity, and support the prefers-reduced-motion media query.

    Because these criteria can be checked with tools, they are suitable for acceptance testing.

    3. The Three-Tier Evaluation

    The three tiers increase in task complexity. Each tier records the environment, task description, and acceptance criteria so that results can be reproduced.

    Tier 1: Algorithmic and State Logic

    Test environment: React 18, TypeScript 5, Node.js 20, and Vitest.

    Task: Implement a virtualized list component that supports dynamic item heights. The list contains 10,000 items, and each item’s height is determined by its content.

    Example prompt:

    1. You are a senior frontend engineer. Implement a virtual scrolling list component using React 18 and TypeScript 5.
    2. Requirements:
    3. 1. The dataset contains 10,000 items. Render only the nodes in the visible area and its buffer.
    4. 2. Item heights are determined by their content and should be measured and cached after the initial render.
    5. 3. First explain the height calculation strategy and buffer design, then provide complete code with comments.
    6. 4. Perform no more than five self-correction rounds. Stop when all tests pass.

    Acceptance criteria:

    • Items do not jump or leave blank areas when scrolling to the top or bottom.
    • When the dataset changes from 10,000 items to 100 items, the scroll position is reset correctly.
    • After mounting and unmounting the component 100 times, the number of event listeners and timers returns to its initial value.
    • Unit tests cover empty data, a single item, and changing item heights. All tests must pass.
    • Stopping condition: Stop when all tests pass. If test cases still fail after five rounds, mark the task as failed and retain the logs from the final round.

    Tier 2: Visual and Interaction Fidelity

    Test environment: Plain HTML, CSS, and JavaScript without a UI component library. Test in the latest stable versions of Chrome and Safari at viewport widths of 1440px and 390px. Use the latest versions of axe DevTools and Lighthouse for inspection.

    Task: Generate a dark-mode AI product landing page with a blurred light-spot background, a top navigation bar that hides while scrolling, a typewriter-style heading, and an asymmetric grid layout.

    Example prompt:

    1. Create a dark-mode AI product landing page using only native HTML, CSS, and JavaScript.
    2. Requirements:
    3. 1. Use multiple colored blocks with filter: blur to create a blurred light-spot background.
    4. 2. Hide the top navigation when the page scrolls down and show it when the page scrolls up.
    5. 3. Display the main heading character by character, with a 60ms interval between characters.
    6. 4. Use CSS Grid for an asymmetric layout. Keep the body text width at no more than 680px.
    7. 5. Use neon purple and fluorescent blue as accent colors. Body text must have a contrast ratio of at least 4.5:1.
    8. 6. When prefers-reduced-motion is enabled, disable the character-by-character animation.

    Acceptance criteria:

    • A full-page axe DevTools scan shows no serious contrast-related issues.
    • When prefers-reduced-motion is enabled, the character animation is disabled and the complete heading is displayed immediately.
    • Neither viewport has a horizontal scrollbar, and the navigation does not cover the heading.
    • The Lighthouse accessibility score is at least 90.

    Tier 3: Long-Form Documentation and Self-Correction

    Test environment: An approximately 8,000-character product requirements document containing five API specifications and one interaction flowchart. The backend uses a local mock service, with its address and port fixed in the document. The model cannot access the external network.

    Task: Implement an order management page based on the requirements document, call all five APIs, and handle every branch in the flowchart, including approval, rejection, and timeout scenarios.

    Chunking strategy: Split the document by level-two headings, with no more than 1,500 characters per chunk. Include the API identifier during retrieval. First convert the flowchart into a textual step table, then map each step to the relevant API specification.

    Example prompt:

    1. You are a senior frontend engineer. The following content contains chunks of a product requirements document, and each chunk includes an API identifier.
    2. First list the request method and field mapping for all five APIs, then implement the order management page.
    3. Requirements:
    4. 1. Every branch in the flowchart must have corresponding handling code and test cases.
    5. 2. Run tests after each modification. Use the errors to locate and fix problems.
    6. 3. Perform no more than 10 correction rounds. Stop when all tests pass.

    Acceptance criteria:

    • All five APIs are called, and request parameters match the fields specified in the document.
    • Every branch in the flowchart has corresponding handling code and test cases.
    • For 20 randomly selected details from the document, the model must retrieve and answer correctly at least 90% of the time.
    • Stopping condition: Stop when all tests pass. If test cases still fail after 10 rounds, mark the task as failed and retain the logs from the final round.

    4. Scoring Weights and Passing Threshold

    Score the three tiers using the following table. Each dimension is first evaluated against its passing threshold. A dimension that meets the threshold receives full credit; otherwise, it receives zero. The weighted scores are then added together.

    Dimension Weight Passing threshold
    Logical correctness 35% All Tier 1 unit tests pass, with no closure or state-management errors
    Visual fidelity 25% No serious contrast issues in axe, a Lighthouse accessibility score of at least 90, and spacing and animation meet the criteria in Section 2
    Self-correction 25% The task converges within the maximum number of rounds, with the human limited to acceptance testing and making no manual changes to business code
    Security and compliance 15% No unauthorized access, credential leakage, or destructive operations without confirmation

    The decision rule is as follows: a total score of at least 80 is required, and the security and compliance dimension must receive full credit before the model is considered ready to enter the development workflow. If the security and compliance dimension fails, the result is an automatic failure rather than a weighted-score adjustment.

    Run each tier at least five times and use the median result.

    5. Cost Comparison Methodology

    Cost comparisons should use measured data rather than estimates. Track the following five metrics:

    • Number of API calls: The number of model requests made from the start of the task to its completion.
    • Input and output tokens: Convert token usage into cost using the price table fixed at the beginning of the test. The report should state the date of the price table.
    • Correction rounds: The average number of actual modification-and-test cycles.
    • Human intervention time: The time, in minutes, from task submission to acceptance, including the time spent reading, modifying, and resubmitting work.
    • Failure rate: The proportion of tasks that do not pass when the stopping condition is reached.

    The report should include the median and failure rate rather than only the best result. When comparing approaches, convert human intervention time using the team’s actual hourly labor cost and compare it with the cost of completing the same task manually.