A Practical Guide to Evaluating LLMs for Frontend Code Generation and Agent Capabilities

1. Why You Need Your Own Evaluation Framework

Whenever a new LLM version is released, the tech community’s immediate reaction is often to “test it out”: throwing a few flashy prompts at the model, taking screenshots, recording screens, and posting the results. While this type of content helps build an intuitive impression, it falls short as a basis for engineering decisions due to several flaws:

1. Survivorship Bias: Demonstrators typically only showcase successful cases. Failures and missteps are rarely fully presented.
2. Irreproducibility: Model services continuously iterate. The same prompt might yield different results at different times, making demonstrations without version locking difficult to verify.
3. Context Mismatch: Others’ demonstration scenarios might not match your business. A team building marketing landing pages and a team building internal data dashboards have completely different definitions of “good.”

Therefore, a more reliable approach is to upgrade from “watching others’ tests” to “building your own evaluation system.” The value of an evaluation system isn’t about assigning an absolute score to a model, but rather: providing a comparable baseline for model selection; enabling rapid regression validation during model upgrades; and transforming “feels nice” into discussable, traceable engineering metrics.

2. Frontend Code Generation: Five Core Evaluation Dimensions

1. Visual Aesthetics and Design Fidelity

The baseline for modern page generation is no longer “can it render,” but “does it look decent when rendered.” During evaluation, focus on:

• Whether color schemes and contrast ratios meet basic design standards, and if there are typical “machine-like” issues such as highly saturated color clashes.
• Whether font hierarchy, spacing rhythm, and whitespace form a clear visual flow.
• Given a specific design description (e.g., “minimalist style, dark theme, card-based layout”), whether the output faithfully restores the constraints.

It is recommended to prepare 3 to 5 fixed description texts for each dimension, feed them horizontally to different models, and generate comparison images using a unified screenshot workflow.

2. Spatial Understanding and Layout Capabilities

Layout is a frequent failure point in frontend generation. Evaluation cases should intentionally include:

• Descriptions requiring precise spatial relationships, such as “fixed top navigation, 240px left sidebar, bottom-attached pagination in the right content area.”
• Responsive breakpoint requirements to verify if the model actually writes media queries or simply overflows on narrow viewports.
• Composite layouts: mixing grid and flexbox, nested scrollable areas, etc.

The validation method is straightforward: use a headless browser to take screenshots at multiple viewport widths and manually inspect if they meet expectations.

3. 3D and Visualization Performance

3D scenes and data visualizations are where models differentiate the most. Evaluation points include:

• Whether common 3D rendering solutions can be correctly imported and basic scenes set up (camera, light sources, materials).
• Light, shadow, and atmosphere expression: whether shadows are reasonable, or if materials look “muddy.”
• Chart requirements: whether data mapping is correct, axes and legends are complete, and interactive tooltips are usable.

Make sure to simultaneously record rendering performance (frame rate) and the number of browser console errors, as these two metrics are often ignored in demonstration-type content.

4. Animations and Interaction Details

• Whether transitions and keyframe animations are smooth, and whether duration and easing are restrained.
• Whether intermediate states like hover, focus, and loading are handled.
• Bonus points: whether accessibility is proactively considered, such as respecting the system’s “reduce motion” preference.

5. Code Engineering Quality

Rendering effects are just the surface; whether the code can go to production is key:

• Whether component splitting is logical, or if hundreds of lines are crammed into a single file.
• Whether state management is clear, and if there are obvious redundant rendering pitfalls.
• Whether runnable instructions and a dependency list are included, and what the handover cost is for the team.

A practical approach is to feed the model’s output directly into static analysis and build pipelines, letting the toolchain guard the quality for you.

3. Agent Capabilities: From “Calling Tools” to “Knowing What It’s Doing”

Agent evaluation is much more complex than single-turn code generation. The core observation point is the model’s “process quality” rather than just the final result.

1. Task Decomposition and Planning

Give a vaguely defined task with few constraints (e.g., “add an export feature to this project”) and observe whether the model:

• Clarifies requirements first or makes reasonable assumptions, rather than immediately writing code.
• Outputs a plan with the correct dependency sequence, rather than a simple parallel list.

2. State Awareness and Self-Positioning

This is a key characteristic distinguishing Agent maturity. An excellent Agent can clearly articulate during a long workflow: what stage it is currently at, what has been completed, and why it is doing the next step. During evaluation, you can intentionally introduce a failure in an intermediate step and observe whether the model loses its goal and loops in place, or if it can recover the context, adjust its plan, and continue.

3. Accuracy of Tool Calling

• Whether parameters are constructed correctly and whether the actual return values of the tool are read.
• Whether “hallucinatory calls” occur, where execution results are fabricated.
• When facing tool errors, whether it blindly retries, or analyzes the root cause and corrects parameters or switches paths.

4. Convergence and Termination Judgment

A good Agent knows when to stop. You can measure the step distribution “from task start to self-determined completion,” as well as the match between the deliverable and acceptance criteria, to avoid the runaway loop of “forever tweaking the last line.”

4. An Actionable Evaluation Workflow

1. Build an Evaluation Dataset

Stratify by scenario: prepare several cases for landing pages, data dashboards, 3D showcases, and multi-step Agent tasks. Each case should fix three elements: input prompt, expected output highlights, and scoring criteria. The evaluation dataset should be version-controlled and continuously expanded as the business evolves.

2. Design a Scorecard

Use a structured format to solidify scoring criteria and reduce subjective fluctuations:

{
  "case_id": "landing_dark_01",
  "dimensions": {
    "visual_aesthetic": {"weight": 0.3, "score": null},
    "layout_accuracy":  {"weight": 0.25, "score": null},
    "interaction":      {"weight": 0.2, "score": null},
    "code_quality":     {"weight": 0.15, "score": null},
    "stability":        {"weight": 0.1, "score": null}
  },
  "notes": ""
}

3. Automated Initial Screening + Manual In-depth Review

Automation acts as a safety net, while humans handle aesthetics and experience judgments:

# Evaluation pipeline illustration (pseudo-code)
for case in eval_cases:
    result = llm.generate(case.prompt)      # Call the model under test
    build_ok = try_build(result.code)       # Build validation
    shots = capture(result.code, viewports=[375, 768, 1440])
    errors = collect_console_errors(result.code)
    save_artifact(case.id, result, shots, errors)
# Manual review only processes objects that pass automation, scoring via the scorecard

It is recommended to run the same case multiple times against the same model to calculate the success rate and variance—stability often holds more engineering value than a single peak performance.

4. Record, Compare, and Regress

Archive the model version, parameter configuration, and result artifacts from each evaluation to form a baseline that evolves over time. When a new model is released, simply rerun the evaluation dataset to answer with data whether “it is worth migrating.” The infrastructure requirements for this workflow are not high; it can be built using general capabilities provided by cloud platforms (such as object storage, logging services, and monitoring/alerting). Mainstream cloud service providers, including Baidu AI Cloud, offer corresponding product categories for this purpose.

5. Common Evaluation Pitfalls

1. Only Looking at Best Outputs: A single impressive result doesn’t mean it’s production-ready. The distribution is more important than the peak.
2. Prompt Misalignment: Different models have varying sensitivities to prompt styles. A fair approach is to make minimal prompt adaptations for each model and note them in the report.
3. Ignoring Cost and Latency: At equivalent quality, lower latency and cost mean a higher upper limit of engineering feasibility.
4. Replacing Measurement with Demos: Demos are for “generating interest,” while evaluations are for “decision-making.” The two cannot replace each other.

6. Conclusion

LLM capabilities iterate rapidly, but evaluation methodology is a slow-changing variable. Instead of chasing every wave of “hands-on tests,” it is better to cultivate an evaluation dataset and scoring system tailored to your own business: for the frontend, focus on the five dimensions of aesthetics, layout, 3D, animations, and engineering quality; for Agents, monitor the four observation points of planning, state awareness, tool calling, and convergence capabilities. Then, use an automated pipeline to turn subjective impressions into reproducible data. When the next generation of models is released, you shouldn’t wait for others’ conclusions—you should run your own evaluation and let the data make the judgment for you.