Evaluating Chinese LLMs in Practice: A Three-Stage Test for Code and Frontend Aesthetics

    This article presents a practical evaluation framework for Chinese LLMs that goes beyond single-shot code generation. It defines six assessment dimensions and three test stages: basic code generation, engineering collaboration, and agentic closed-loop development. It also introduces verifiable criteria for frontend aesthetics, offers prompt-design guidance, and highlights common evaluation pitfalls so teams can judge whether a model can support real development workflows.

    1. From “Can Generate” to “Can Deliver”

    In the past, evaluations of large language models’ programming ability often focused on surface-level indicators: whether the model could output code in a specified language, complete a function, generate a simple page, or answer quickly. In real development, however, code generation is only the starting point.

    A genuinely useful intelligent coding model should address at least the following questions:

    1. Can it accurately understand natural-language requirements instead of mechanically assembling code?
    2. Can it work within an existing project structure without breaking existing modules?
    3. Can it proactively identify runtime errors, type errors, and interaction defects?
    4. Can it revise its work based on test results instead of repeatedly outputting the same solution?
    5. Can it produce frontend pages with visual hierarchy, interaction feedback, and responsive adaptation?
    6. When requirements are incomplete, can it identify risks and ask necessary questions?

    Therefore, evaluating Chinese LLMs should not rely only on parameter scale or single-response quality. It should also examine stability across continuous tasks. This article divides the testing process into three levels: basic code generation, engineering-level collaboration, and agentic closed-loop development. It also adds a dedicated verification step for frontend aesthetics and interaction experience.


    2. Establishing a Unified Testing Standard

    To avoid impression-based evaluation, define assessment dimensions before testing. The following six indicators are recommended.

    2.1 Requirement Understanding Accuracy

    Does the model capture the key constraints in the task? For example:

    • Which technology stack must be used?
    • Does the page need mobile compatibility?
    • Can the data be empty?
    • Are error prompts required?
    • Must existing interfaces be preserved?
    • Are there permission or security restrictions?

    A high-quality model does not simply restate the requirements. It actively clarifies functional boundaries and points out possible ambiguities.

    2.2 Code Correctness

    Code correctness includes syntax, types, dependencies, execution flow, and boundary conditions. Common problems include:

    • Referencing a variable that does not exist;
    • Failing to await asynchronous logic correctly;
    • Not handling error states;
    • Crashing when a list is empty;
    • Submitting repeatedly when the user clicks multiple times;
    • Inconsistent component parameter naming;
    • Mismatched frontend and backend data structures.

    This dimension cannot be judged by reading code alone. The code must be placed in an actual runtime environment for verification.

    2.3 Engineering Fit

    Generating a standalone example is not difficult. The real challenge is modifying an existing project. For testing, provide a medium-sized project with multiple directory levels, configuration files, shared components, and business modules, then ask the model to complete a specified feature.

    At this point, observe whether the model:

    • Checks the project structure first;
    • Understands existing coding conventions;
    • Reuses shared components;
    • Modifies only necessary files;
    • Keeps existing interfaces and routes stable;
    • Explains the scope of changes after modification;
    • Avoids large-scale rewrites just to complete the task.

    2.4 Debugging and Self-Correction

    It is not a serious problem if the model’s first output has issues. The key question is whether it can quickly locate the root cause based on error information.

    An effective debugging process usually includes:

    1. Reading the complete error message;
    2. Determining the stage at which the error occurs;
    3. Narrowing the problem scope;
    4. Modifying the minimum necessary code;
    5. Re-running the project or suggesting verification steps;
    6. Explaining the reason for the fix and its potential impact.

    If the model only makes surface-level replacements each time, or ignores error logs, it is still stuck at the text-generation stage.

    2.5 Frontend Visual and Interaction Quality

    Frontend quality should not be judged only by whether buttons and tables exist. It also requires attention to:

    • Whether the page hierarchy is clear;
    • Whether primary and secondary information are properly emphasized;
    • Whether colors are consistent;
    • Whether spacing feels natural;
    • Whether font sizes are reasonable;
    • Whether loading, success, and failure states are clear;
    • Whether actions provide immediate feedback;
    • Whether horizontal overflow occurs on mobile;
    • Whether empty states are designed;
    • Whether animations serve interaction rather than decoration alone.

    Aesthetics are not entirely subjective. By using a unified test task, visual effects can be broken down into observable indicators.

    2.6 Output Maintainability

    High-quality code must not only run; it must also be easy for others to maintain. Pay attention to:

    • Whether functions have a single responsibility;
    • Whether names express real meaning;
    • Whether duplicate logic exists;
    • Whether state management is clear;
    • Whether styles are easy to extend;
    • Whether comments explain key reasons instead of repeating the code;
    • Whether external input is validated.

    3. Stage One: Basic Code Generation Testing

    The goal of the first stage is to verify whether the model has basic code understanding and implementation ability. The test task should not be overly complex, but it must include clear inputs, outputs, and exception conditions.

    Test Case: Implement a Data Filtering Module

    The requirement can be stated as follows:

    Create a data list page that supports keyword search, status filtering, pagination, loading indicators, empty-data prompts, and request-failure prompts. When the user changes filter conditions, the page should automatically return to the first page.

    This task appears ordinary, but it covers multiple foundational capabilities.

    The core logic the model needs to handle includes:

    • Re-requesting data when the search term changes;
    • Resetting the page number when filter conditions change;
    • Showing a loading state during requests;
    • Keeping a retry entry when a request fails;
    • Showing an empty state when there is no data;
    • Avoiding stale requests overwriting newer results during rapid consecutive actions;
    • Correctly disabling pagination buttons under boundary conditions.

    What to Observe

    First, does the model omit states? Many low-quality implementations define only list data and loading state, without considering error state and empty state.

    Second, does the model handle asynchronous requests correctly? A typical problem is this: the user first searches for keyword A, then quickly searches for keyword B, but A’s response returns later than B’s, and the page ends up displaying A’s results. A more robust approach is to add a request sequence marker, a cancellation mechanism, or validate the current query conditions when the response arrives.

    Third, does the model consider input debouncing? For a search box, a request should not be sent immediately on every keystroke. The model should provide a debouncing strategy for the scenario and explain that the delay setting needs to be adjusted based on API response speed and user experience.

    Fourth, can the model explain its design? Outputting code without explaining state transitions usually leads to higher maintenance costs.


    4. Stage Two: Engineering-Level Collaboration Testing

    The second stage no longer asks the model to create files from scratch. Instead, it asks the model to enter an existing codebase and complete a refactor or feature change. This is closer to real development.

    Test Task Design

    Prepare a sample project with the following characteristics:

    • Basic routing and layout;
    • Existing shared button, modal, and table components;
    • Pages using unified theme variables;
    • Data fetched through a wrapped request module;
    • Some existing test cases;
    • A boundary-case issue that needs to be fixed.

    Then provide a sequence of tasks:

    1. Add a statistics card area;
    2. Add batch operations to the list;
    3. Fix the issue where pagination does not reset after filter conditions change;
    4. Add a retry mechanism for API failures;
    5. Add tests for key logic;
    6. Keep the existing page style unchanged.

    Ideal Workflow

    The model should first scan the directory and key files rather than immediately creating new files. It then needs to identify:

    • Where the page entry is;
    • Which module handles data requests;
    • How component parameters are named;
    • How style variables are organized;
    • What structure the tests use;
    • Whether the project has an established error-handling approach.

    Before making changes, the model should ideally provide a brief implementation plan, for example:

    • Reuse the existing statistics card component first;
    • Add batch-operation fields at the request-parameter layer;
    • Move pagination-reset logic into the filter-change handler;
    • Add a retry method for failure states;
    • Finally, run existing tests and check the mobile layout.

    This plan is not formalism. It reduces disordered modifications when the model handles complex tasks.

    Scoring Method for Engineering Tests

    A five-level scoring system can be used:

    • 1: Cannot run or breaks existing functionality;
    • 2: Basic functionality is completed, but obvious errors remain;
    • 3: Main flow works, but boundary handling is incomplete;
    • 4: Feature is complete, structure is clear, and maintainability is good;
    • 5: Completes the requirement and proactively adds tests, exception handling, and compatibility notes.

    Pay special attention to the “scope of change.” If a small feature causes many unrelated files to be rewritten, the model lacks engineering discipline even if the final page runs.


    5. Stage Three: Agentic Closed-Loop Development Testing

    The third stage assesses whether the model has stronger autonomous collaboration ability. The task can be described as follows:

    Based on a product requirement, complete a showcase page with a gradient background, content cards, a search entry, operation feedback, and a responsive layout. Run it locally and fix any issues found.

    For this type of task, do not look only at the final code. Observe the entire process:

    1. Does it break down the requirements?
    2. Does it identify missing assets?
    3. Does it plan the page structure?
    4. Does it generate an initial implementation?
    5. Does it proactively run the project?
    6. Does it check the console and page behavior?
    7. Does it fix issues based on the results?
    8. Does it provide a final usability summary?

    Why “Run—Observe—Fix” Matters

    Single-shot generation cannot catch many problems, such as:

    • A style file not being imported correctly;
    • Path casing causing resource loading failures;
    • Page overflow on narrow screens;
    • Click events bound to the wrong element;
    • Animations causing content flicker;
    • A component accessing a null object before asynchronous data returns;
    • Focus not returning to the original trigger after a modal closes.

    Only by placing the code in a runtime environment does the model have a chance to discover these issues. Therefore, the core of agentic capability is not “writing more code,” but continuously taking actions toward the goal.


    6. Frontend Aesthetics Testing: From Subjective Impressions to Verifiable Indicators

    Frontend aesthetics are often reduced to “whether it looks good,” but they can actually be broken down into multiple dimensions.

    6.1 Visual Hierarchy

    The page should have a clear primary visual focus. Titles, core data, primary actions, and supporting information need a clear priority order. If all text, buttons, and cards use the same color and size, users will struggle to understand the page structure quickly.

    6.2 Whitespace and Density

    Insufficient whitespace makes a page crowded, while excessive whitespace reduces information density. The model should decide layout based on content type instead of mechanically applying fixed spacing.

    6.3 Color System

    A limited set of primary, secondary, and status colors is recommended. Success, warning, and error colors should remain semantically consistent. Attention should also be paid to contrast between text and background, avoiding sacrifices in readability for visual effect alone.

    6.4 Interaction Feedback

    Buttons should provide clear feedback after being clicked. Submission processes should show an in-progress state. Successful actions should show a result prompt, and failures should explain what to do next. A page without feedback, even if visually polished, makes users wonder whether their click worked.

    6.5 Restrained Motion

    Effects such as fade-in, translation, and fade transitions can improve perceived quality, but they should not interfere with reading or operational efficiency. Motion should have a clear purpose, such as indicating a state change, establishing spatial relationships, or reducing the abruptness of page transitions.

    6.6 Responsive Adaptation

    At minimum, check three scenarios: desktop widescreen, standard laptop width, and mobile narrow screen. Common problems include:

    • Fixed-width cards causing horizontal scrolling;
    • Navigation content that cannot collapse;
    • Tables squeezing text;
    • Modals extending beyond the screen;
    • Touch targets that are too small;
    • Fonts remaining at desktop size on small screens.

    7. How to Design High-Quality Prompts

    Model performance is closely related to prompt quality. A requirement should be divided into six parts.

    7.1 Role and Goal

    Clarify whether the model needs to produce a page prototype, production code, a bug fix, or a technical proposal.

    7.2 Technical Boundaries

    Specify allowed languages, frameworks, component approaches, file scope, and runtime conditions.

    7.3 Functional Requirements

    Describe requirements by user flow rather than listing only visual elements. For example, “the user enters a keyword and views filtered results” is more precise than “add a search box.”

    7.4 Exceptions and Boundaries

    Specify cases such as empty data, failure, repeated clicks, overly long text, narrow screens, and insufficient permissions.

    7.5 Visual Direction

    Describe layout hierarchy, color tone, information density, and motion principles, but avoid vague terms such as “premium” or “good-looking.”

    7.6 Acceptance Criteria

    Ask the model to check after completion:

    • Whether the project can start;
    • Whether console errors exist;
    • Whether the core flow is operable;
    • Whether mobile behavior is normal;
    • Whether modified files and verification results are explained.

    8. Common Pitfalls and Improvement Suggestions

    Pitfall 1: Looking Only at the First Output

    Fast first generation does not mean strong engineering ability. Add at least one round of error feedback and one requirement change, then observe whether the model maintains context consistency.

    Pitfall 2: Testing Only Simple Pages

    Static pages easily hide model weaknesses. More valuable tests should include asynchronous requests, state transitions, error handling, and responsive layouts.

    Pitfall 3: Equating Aesthetics with Gradients and Animations

    Visual effects are not a pile of decorative elements. Genuine aesthetics include information hierarchy, readability, operational efficiency, and overall consistency.

    Pitfall 4: Ignoring Human Acceptance

    Models can assist development, but they cannot replace final verification. Code involving permissions, data security, payments, user privacy, and production deployment still requires human review and automated testing.

    Pitfall 5: Ignoring Context Management

    The larger the project, the more easily the model is distracted by unrelated files. Use directory notes, module summaries, change scope, and staged summaries to control input context and avoid inserting large amounts of irrelevant content at once.


    9. Conclusion: The Real Ceiling Depends on Closed-Loop Ability

    Competition among Chinese LLMs in code generation is no longer just about “who can write more code.” It is about “who can more reliably complete the full loop from understanding requirements to delivering results.”

    A mature evaluation method should cover at least three layers:

    • The basic layer: syntax, logic, and boundary handling;
    • The engineering layer: project fit, collaboration conventions, and maintainability;
    • The agentic layer: running, observing, debugging, and continuous correction.

    At the same time, frontend pages need separate evaluation of visual hierarchy, interaction feedback, responsive layout, and restrained motion.

    If a model can accurately understand requirements, proactively break down tasks, generate runnable code, fix issues based on errors, and balance visual and engineering quality, then it is genuinely valuable for daily development workflows. For developers, the most effective approach is not to pursue one-shot generation blindly. Instead, integrate the model into a standard development process of “requirement analysis—implementation—verification—fix—review,” making it a collaborative engineering assistant rather than a mere code-snippet generator.