Agentic AI Coding Models: A Three-Tier Evaluation from Code Generation to UI Quality
This article presents a three-tier framework for evaluating AI coding models in agentic programming workflows. The tiers cover basic code generation and logical correctness, multi-file engineering with autonomous repair, and interface aesthetics with interaction quality. It also offers a scoring approach and practical guidelines for developers who work with these models.
In traditional AI-assisted programming, a developer asks a question, the model returns code, and the exchange usually ends there. This works for simple functions, explaining error messages, or providing partial implementations. But when a task involves multiple files, complex dependencies, ongoing debugging, and interface details, one-off text generation often shows clear problems. The code may lack consistency across files, the model cannot actively verify its results, and when errors occur it can only wait for the developer’s next feedback. The final output often stays at “it runs, but it is not really usable.”
The core change in agentic programming is that the model moves from a pure code generator to a collaborative system. It observes its environment, makes plans, takes actions, gets feedback, and keeps correcting itself. This does not mean the model can fully replace developers. It means the model starts to follow workflows closer to real engineering practice. Developers no longer judge only whether a single piece of code is correct. They also need to assess whether the model can keep moving a task toward its goal.
1. Understanding the Agentic Programming Loop
A complete agentic programming process usually includes the following steps:
- Understand the goal: Identify business requirements, inputs and outputs, constraints, and acceptance criteria.
- Break down the task: Split the overall requirement into subtasks such as pages, APIs, data structures, tests, and deployment.
- Check the context: Read the project directory, existing code, configuration files, and runtime environment to avoid generating code blindly, detached from the real project.
- Make changes: Create or update files, add implementation logic, and adjust styles and configuration.
- Run verification: Execute builds, tests, static checks, or local run workflows.
- Analyze feedback: Locate problems using logs, exception stack traces, and test results.
- Iterate and fix: Modify the implementation and verify again until the preset criteria are met.
- Summarize and deliver: Explain the changes, verification results, known limitations, and suggested next steps.
The most important factor is not whether the model can write code, but whether it can complete multiple rounds of this loop driven by feedback. For enterprise development, this usually matters more than the speed of a single generation.
2. A Three-Tier Testing Framework: From Correct Functions to Product Completeness
To avoid judging models based only on subjective impressions, testing can be divided into three progressive tiers. They cover logical ability, engineering ability, and aesthetic ability, with increasing difficulty. The results are closer to real development scenarios.
Tier 1: Basic Code Generation and Logical Correctness
Tier 1 checks whether the model has stable basic coding ability. The task should not be overly complex, but it should include clear inputs, outputs, boundary conditions, and error-handling requirements.
For example, ask the model to implement a data processing module that:
- Receives a set of structured records;
- Filters and sorts them by specified fields;
- Handles empty values, duplicate values, and invalid input;
- Returns results in a consistent format;
- Writes the necessary test cases.
At this stage, focus on four areas.
First, whether the requirements are understood accurately. Did the model miss key constraints? Did it misread “sort by time, newest first” as oldest first? Does it distinguish an empty result from an error? Each of these directly affects the final behavior.
Second, whether the code structure is clear. Even if code passes simple tests, muddled function responsibilities, vague variable names, or deeply nested branches make later maintenance costly.
Third, whether boundary handling is complete. High-quality implementations should not cover only normal inputs. They should also consider empty arrays, missing fields, type errors, extreme values, and repeated calls.
Fourth, whether the model verifies its work actively. Ask the model to run tests after writing code and report which checks passed and which failed. If it submits code without verifying it, its working style is still close to one-off question answering.
A task description for a basic test can follow this structure:
Please implement a data transformation module.Requirements:1. The input is a collection of structured records;2. Support filtering, sorting, and pagination;3. Handle missing fields explicitly;4. Do not modify the original input;5. Add tests for both normal and error scenarios;6. After completing the work, run the tests and report the results.
During evaluation, do not look only at the final output. Also record whether the model asks clarifying questions about ambiguity, states its assumptions, and corrects its code based on test results. This gives a more reliable conclusion than simply comparing code length.
Tier 2: Multi-File Engineering Collaboration and Autonomous Repair
Tier 2 expands the scope from a single function to a complete project. Tasks may include building a small management page, implementing an API module with data validation, or adding a feature to an existing project.
At this level, the model must handle directory structure, module dependencies, configuration files, data flow, and run commands at the same time. Key test points include:
- Whether it inspects the existing project first instead of overwriting files directly;
- Whether it can identify call relationships between modules;
- Whether it keeps existing interfaces and data structures compatible;
- Whether it can handle build failures, missing dependencies, and type errors;
- Whether it reruns the verification process after making changes;
- Whether it can distinguish environment problems from business logic problems.
This tier is especially useful for observing the model’s reflective ability. For example, after the first run produces an error, does the model immediately try random edits? Or does it first read the full log, locate the source of the error, and then propose a minimal fix? In engineering practice, the second approach is usually safer.
You can use the following process for testing:
1. Provide Incomplete but Realistic Project Context
Do not paste all the code to the model at once. Let it explore the directories and files within the project environment. This tests how well it gathers information and organizes context.
2. Set Verifiable Acceptance Criteria
Acceptance criteria should be as objective as possible, for example:
- The page loads normally;
- Form validation covers required fields and format errors;
- Clear messages appear when data submission fails;
- Core logic has automated tests;
- The build completes without blocking errors;
- Existing features are not affected.
3. Record Each Action and Its Result
The evaluation record should include what the model did, the reason for each action, what problems appeared, how they were fixed, and whether verification was finally completed. The action trace shows whether the model truly understood the project or relied on extensive trial and error to get lucky.
4. Check Side Effects of Changes
The model may make broad changes to solve one problem, so check whether it introduced new risks. These include interface changes, broken styles, configuration pollution, swallowed exceptions, and missing security validation.
The core of Tier 2 is not finishing in one attempt. It is whether the model can keep going after failures. If the model can extract useful information from build logs, gradually narrow the problem, and verify again after each change, it has strong value as an engineering collaborator.
Tier 3: Interface Aesthetics, Interaction Quality, and Product Completeness
Tier 3 targets front-end pages and application prototypes. It focuses on whether the model can turn abstract requirements into an interface with visual hierarchy and a good user experience.
Many generated results have no obvious functional errors
