A Three-Stage Evaluation Framework for Code Generation Models: From Functions to Frontend Aesthetics
This article presents a practical three-stage framework for evaluating code generation models in real engineering contexts. It begins with single functions and components, moves to interactive pages and stateful workflows, and ends with repository-level agentic coding. For each stage, the article explains what to test, how to design prompts, which dimensions to review, and how to combine automated checks with human review. It also covers controlled agent loops, safety boundaries, automation and governance, common pitfalls, and the shift from impressive one-off demos to stable, verifiable delivery.
A Three-Stage Evaluation Framework for Code Generation Models: From Functions to Frontend Aesthetics
Using large models for programming is no longer mainly a question of whether they can generate code. The real question is whether they can deliver verifiable results under explicit constraints. Many demos look smooth when there is no data, no error handling, and no maintenance requirement. Once they enter real engineering, problems appear: missing state, boundary errors, unmaintainable styles, and uncontrolled dependencies. For this reason, code generation models should not be judged only by the first impression of a single output. They need to be tested in a graded, repeatable, and automatable way.
This article introduces a three-stage evaluation framework for engineering practice. The first stage examines single functions and components. The second stage examines interactive pages and closed state loops. The third stage examines repository-level agentic programming. Each stage considers functional correctness, maintainability, frontend aesthetics, interaction experience, and security boundaries. The goal is not to produce a simple ranking of models, but to turn “can it be used?” into an engineering judgment: “can it deliver stably?”
1. Why Graded Testing Is Necessary
The complexity of code generation tasks varies widely. Asking a model to write a pure function is fundamentally different from asking it to modify an existing repository, understand interface contracts, run tests, and fix failures. If a model is evaluated with only one page screenshot or one code snippet, the conclusion is likely to be distorted.
The core idea of graded testing is to move from local generation to system delivery and to define acceptance criteria at each level:
- Micro level: single functions, single components, and local styles. Focus on logical correctness, boundary handling, semantic structure, and basic accessibility.
- Meso level: complete pages or small interactive applications. Focus on state management, asynchronous flows, error recovery, responsive layout, and visual consistency.
- Macro level: repository-level changes and agentic closed loops. Focus on planning ability, context understanding, tool-call boundaries, self-correction, and change control.
The three levels are not isolated. A micro-level failure often means that basic code quality is insufficient. A meso-level failure often means that state modeling is weak. A macro-level failure may come from poor context management, ineffective use of test feedback, or an unsuitable tool environment. Only by locating failures by level can teams determine whether the problem lies in the model, the prompt, or the engineering process.
2. Evaluation Dimensions: From “Looks Right” to “Verifiable”
“Strong coding ability” and “good aesthetics” are too vague. A reusable evaluation system needs observable dimensions.
| Dimension | Core question | How to check |
|---|---|---|
| Functional correctness | Does it satisfy the main path and boundary conditions? | Unit tests, interaction tests, exception-path checks |
| Maintainability | Are naming, modularity, duplication, and dependencies reasonable? | Code review, complexity checks, dependency change review |
| Visual system | Are spacing, font size, color, and hierarchy consistent? | Design token checks, screenshot comparison, human review |
| Interaction experience | Are loading, empty, error, disabled, focus, and motion states complete? | Browser automation, keyboard operation, slow-network simulation |
| Accessibility | Are semantics, contrast, keyboard reachability, and screen-reader hints correct? | Semantic tag checks, contrast checks, focus-order checks |
| Agent capability | Can it plan, execute, read feedback, and repair? | Multi-step tasks, failure injection, step limits |
| Engineering security | Does it introduce risky dependencies, leak sensitive information, or perform unauthorized actions? | Sandbox execution, dependency allowlists, secret scanning, permission isolation |
| Cost efficiency | How many rounds and how much context are needed to reach a usable result? | Record repair rounds, input/output size, and manual rework time |
Aesthetics is one of the most underestimated dimensions. Aesthetics is not simply “looking good.” It means whether the visual system is coherent, the information hierarchy is clear, interaction feedback is timely, and accessibility has not been broken. A practical approach is to break aesthetics into layout grids, spacing scales, font-size hierarchy, color semantics, border radius and shadows, motion duration, focus styles, error-message styles, and copywriting conventions. Only then can models understand the requirement and reviewers reproduce the evaluation.
3. Stage One: Micro-Level Testing of Functions and Components
3.1 Task Design
Micro-level tests should not be too simple. Asking a model to write an addition function does not reveal exception handling or engineering habits. A better task is a local implementation with explicit boundary conditions, for example:
- Implement an input parsing function that handles null values, duplicate keys, invalid encoding, and length limits.
- Implement a notification component that supports multiple states, auto-dismiss, pause on hover, keyboard dismissal, and reduced motion.
- Implement a form-field component with error messages, help text, disabled state, and focus styles.
The task must constrain the technology and require the model to provide a test plan. Otherwise, the model may introduce unnecessary dependencies or sacrifice maintainability for demo effects.
3.2 Reusable Prompt Template
You are a senior frontend engineer. Complete the following local implementation task.Goal: Implement a notification component using native web technologies without third-party dependencies.Constraints:1. Support four semantic states: info, success, warning, and error.2. Support manual dismissal and auto-dismiss; pause the timer on hover.3. Manage focus properly after opening; support keyboard dismissal.4. Preserve enough reading time before auto-dismiss.5. Respect the system's reduced-motion setting.6. Do not rely on color alone for semantics; also provide text or icon cues.Visual requirements:1. Use a 4px base spacing scale.2. Use no more than three font-size levels.3. Use shared variables for border radius, shadows, and border widths.4. Meet common accessibility contrast requirements for body text.5. Use motion only to express state changes; do not add meaningless animation.Output order:1. Implementation approach and risks.2. Complete runnable code.3. Acceptance checklist.4. Self-check results, identifying parts that still require human confirmation.
3.3 Example Visual Variables
Instead of writing “premium, spacious, modern” in a prompt, provide executable design variables:
:root {--space-1: 4px;--space-2: 8px;--space-3: 12px;--space-4: 16px;--radius-sm: 6px;--radius-md: 10px;--font-sm: 12px;--font-md: 14px;--font-lg: 18px;--motion-fast: 120ms;--motion-base: 200ms;--color-surface: #ffffff;--color-text: #1f2937;--color-muted: #6b7280;--color-accent: #2563eb;--color-danger: #b91c1c;}@media (prefers-reduced-motion: reduce) {* {animation-duration: 1ms !important;transition-duration: 1ms !important;}}
These variables are not meant to be applied mechanically. They help the model understand that aesthetic requirements can be expressed as engineering constraints. Reviewers should also check whether the variables are actually used and whether there are many hard-coded colors and spacing values.
3.4 What to Score at the Micro Level
At the micro level, do not only check whether the code runs. Focus on:
- Whether null values, very long input, repeated calls, and rapid consecutive triggers are safe.
- Whether component semantics are correct: is a button really a button, and can assistive technology recognize state messages?
- Whether styles are centrally managed and whether repeated magic numbers exist.
- Whether motion respects reduced-motion settings.
- Whether error messages are user-facing rather than exposing internal exceptions.
- Whether the model proactively lists uncertainties and test gaps.
If the micro level already shows many meaningless dependencies, ignored boundary conditions, and main-path-only implementations, do not move directly to more complex tasks. First improve the prompt and acceptance checklist.
4. Stage Two: Meso-Level Testing of Interactive Pages
4.1 Task Design
For the meso level, design a page with filtering, loading, empty states, errors, and detail interactions. It does not need a complex backend, but it must cover typical states in real frontend engineering:
- Initial loading, refresh, and retry.
- Rapid requests after filter changes and the problem of stale results overwriting newer ones.
- No data, partially missing data, and request failure.
- Keyboard navigation, focus management, and closing drawers or overlays.
- Narrow-screen layout, content overflow, and long-text wrapping.
- Timing consistency between motion and state feedback.
A common failure mode is a static page that looks beautiful but lacks loading and error states. Rapid clicks cause states to overwrite each other, and focus is lost after an overlay closes. These problems are invisible in screenshots but directly affect usability.
4.2 Prompt Strategy for the Meso Level
Meso-level tasks should translate “page requirements” into “testable behavior.” The prompt should include at least four types of information:
- Information architecture: page regions, primary actions, and data relationships.
- State machine: initial, loading, success, empty, error, and retrying.
- Visual requirements: spacing, color semantics, font sizes, and responsive breakpoints.
- Acceptance criteria: each item should be verifiable through interaction or automation.
Example fragment:
Implement a data-filtering page using mock data. Do not connect to a real service.Required states:- Initial loading.- Request succeeds but returns no results.- Request fails and can be retried.- When filter conditions change rapidly, only the last valid result is applied.- The detail panel can be opened, closed, and closed with Esc.- On narrow screens, the filter area can collapse and the detail panel does not block key actions.Acceptance criteria:1. All interactive elements are keyboard accessible, with visible focus styles.2. After closing the detail view, focus returns to the trigger button.3. Long text does not cause horizontal scrolling.4. Loading state must not use an infinite spinner as the only feedback.5. Error messages must explain the next action the user can take.6. Color must not be the only way to distinguish states.
4.3 Human Review Method for the Meso Level
Automation can determine whether something exists, but visual and interaction quality still need human review. A fixed review order helps:
- First inspect information hierarchy: can users immediately tell what the page does?
- Then inspect state completeness: are loading, empty, error, disabled, selected, hover, and focus states present?
- Then inspect responsiveness: are narrow screens, long text, very long lists, and soft-keyboard scenarios stable?
- Finally inspect motion: does it serve state changes, or does it cause persistent distraction or excessive animation?
During review, do not only say “this does not look good.” Point to the violated rule, such as “primary and secondary buttons lack contrast,” “spacing does not follow the same scale,” “error state changes only color,” or “focus order does not match visual order.” The more specific the issue, the easier it is to correct in the next prompt.
5. Stage Three: Repository-Level Agentic Programming
5.1 What the Macro Level Examines
At the macro level, the task is no longer “generate a page.” The model must complete a small engineering task in a controlled environment: read requirements and a repository summary, propose a change plan, generate a limited diff, run checks, read failure information, repair issues, and summarize risks.
The key is not to let the model act with unlimited autonomy. The key is to give it a closed loop that can stop, be audited, and be rolled back. A mature agentic programming workflow usually includes:
- Read requirements, interface contracts, directory summaries, and relevant files.
- Output a plan, impact scope, and risks.
- Modify only necessary files and avoid unrelated refactoring.
- Run static checks, unit tests, builds, and basic accessibility checks.
- Read failure logs, locate causes, and make minimal fixes.
- Stop when passing conditions or the step limit is reached, then output a change description and remaining risks.
5.2 Controlled-Loop Pseudocode
context = repository summary + requirements + interface contracts + current test resultsfor step in 1..MAX_STEPS:plan = model.plan(context)patch = model.generatePatch(context, plan)result = sandbox.run({staticCheck,typeCheck,unitTest,build,accessibilityCheck}, patch)if result.pass:breakif result.errorIsUnrelatedToPatch:stopAndReport()context = context + trim(result.failures, BUDGET)context = summarize(context)
This pseudocode emphasizes engineering constraints rather than a specific product capability. It must include a maximum step count, a context budget, failure classification, and stop conditions. Otherwise, the model may repeat the same failed attempt or make overly broad changes just to pass tests.
Security boundaries are equally important. The execution environment should have no network access by default or allow access only through an allowlist. Dependency installation should be controlled. Secrets must not enter the context. Generated code should run in an isolated environment. Final changes must go through human review. For enterprise projects, teams should also record prompts, diffs, test results, and approval information so that issues can be traced.
5.3 Macro-Level Acceptance Checklist
- Do tests pass, and were any failing test cases introduced?
- Do static checks, type checks, and builds pass?
- Were unapproved dependencies introduced?
- Were files unrelated to the task modified?
- Are there hard-coded secrets, debug output, or dangerous execution logic?
- Is compatibility with existing interfaces preserved?
- Is the change description accurate, and are risks reported honestly?
- Under failure injection, can the model use logs to narrow down the problem?
Only when these conditions are met can agentic programming move from “demo” to “engineering-ready.”
6. Advanced Prompting: Turning Adjectives into Constraints
The key to high-quality code generation is not stacking role descriptions such as “you are an expert.” It is turning requirements into constraints that the model can execute and reviewers can verify. The following techniques are useful.
6.1 Replace Aesthetic Adjectives with Design Variables
“Modern” cannot be accepted or rejected consistently. “4px spacing scale, three font-size levels, low-saturation background, a single accent color, visible focus ring” can be verified. For UI tasks, provide:
- Spacing scale.
- Font-size hierarchy.
- Color roles rather than specific brand colors.
- Border-radius and shadow rules.
- Motion duration and trigger conditions.
- Responsive breakpoints.
- Accessibility requirements.
- Copywriting tone and punctuation conventions.
6.2 Ask for a Plan and Acceptance Criteria Before Code
Ask the model to output in a fixed order:
1. Requirement understanding and uncertainties.2. Implementation approach and state model.3. Impact scope and risks.4. Code diff.5. Test checklist and self-check results.
This has two benefits. First, the plan reveals whether the model truly understands the task. Second, the acceptance checklist can be converted directly into test cases, reducing the chance of “looks complete but is actually unusable.”
6.3 Specify Exception Paths and Stop Conditions
Prompts should explicitly require handling of exception paths such as “request failure,” “duplicate submission,” “rapid switching,” “insufficient permission,” “no data,” and “content too long.” Also define stop conditions, for example: “If the same test still fails after two consecutive repair rounds, output the failure reason and a minimal reproduction, and do not expand the scope of changes.”
6.4 Control Context Instead of Flooding It
Long-context capability does not mean that every file should be inserted at once. A more reliable approach is to provide a repository summary, directory structure, interface contracts, relevant file fragments, and test results first, then add more context as needed. The longer the context, the more noise it contains, and the more likely the model is to ignore key constraints. For large projects, use a “summarize, locate, expand, verify” process instead of treating context length as understanding ability.
6.5 Require Self-Checks but Do Not Trust Them Blindly
A self-check prompt can be written as: “Check each acceptance criterion and identify items that are satisfied, unsatisfied, or cannot be confirmed.” This improves transparency, but it cannot replace real tests. Any conclusion about runnable behavior should be confirmed by tests, builds, static checks, or manual operation.
7. Automation and Governance: Making Evaluation a Pipeline
A one-time test only shows the result for one task. Continuous delivery requires a repeatable evaluation set. Teams should build a set of “golden tasks” covering functions, components, pages, exception paths, and repository changes, then run them regularly with the same prompts and acceptance criteria. When recording metrics, do not only record whether a single run passes. Track:
- Pass stability across multiple runs.
- Average repair rounds.
- Manual rework time.
- Defect escape situations.
- Number of new dependencies.
- Number of accessibility and security issues.
- Visual regression differences.
- Context and invocation cost.
Governance should include at least sandbox execution, dependency allowlists, sensitive-information scanning, license checks, change review, and audit logs. Aesthetic review should also use a fixed checklist rather than personal preference. Only when tests, design specifications, and engineering constraints are moved earlier can models operate effectively within clear boundaries.
8. Common Pitfalls
- Using a single demo as a system evaluation: A beautiful page does not represent stable capability. Observe performance across different tasks, different randomness, and failure injection.
- Equating long context with global understanding: Context is capacity, not effective retrieval and correct attention. More irrelevant files mean more noise.
- Looking only at the main path: Loading, empty, error, disabled, overlong-text, and keyboard states are the real test of engineering quality.
- Over-correcting: Self-repair without step and budget limits can easily loop repeatedly or expand the scope of changes.
- Separating aesthetics from accessibility: Low contrast, hidden focus, and color-only state communication are all cases of “looks good but is unusable.”
- Leaving security until the end: Generated code may introduce risky dependencies, leak secrets, or perform unauthorized actions. It must run inside a sandbox and review process.
- Treating prompts as magic: Prompts can improve expression, but they cannot replace clear requirements, interface contracts, test cases, and human review.
9. Conclusion
The evaluation of code generation models should shift from “is a single generation impressive?” to “is it stable and verifiable under constraints?” The micro level verifies basic code quality. The meso level verifies state and interaction closed loops. The macro level verifies planning, execution, feedback, and repair. Only by combining all three can teams clearly see logical correctness, frontend aesthetics, and the real limits of agentic programming.
For teams, the most important task is not chasing a particular demo. It is building a reusable engineering method: turn vague requirements into acceptance criteria, turn aesthetic specifications into design variables, connect test feedback to the generation loop, and place security and dependency governance inside the execution environment. When requirements are verifiable, feedback is automatable, and boundaries are isolated, model-generated code can move from “looks usable” to “maintainable and shippable.”
