Evaluating LLM Coding Skills in Practice: Frontend Design, 3D Rendering, and Cost Efficiency

1. Background: Coding Has Become the Main Arena for LLM Competition

Over the past two years, the boundaries of large language model code generation have been continuously pushed outward. Across the industry, this leap in capability has been driven primarily by three forces.

First, reinforcement learning post-training has gone deeper. Mainstream research teams now commonly introduce a large-scale reinforcement learning phase after pretraining, using verifiable rewards as the optimization signal—for example, whether generated code passes unit tests, whether a program runs successfully, or whether rendering output meets specified constraints. Compared with relying entirely on subjective human feedback, this approach of optimizing toward measurable outcomes has enabled steady progress in hard domains such as mathematics and programming, with noticeably improved output stability and first-attempt pass rates.

Second, the open-source camp is rapidly approaching flagship-level performance. New-generation open-source models keep climbing third-party benchmark leaderboards, with some models now competitive against top closed-source models on code generation and reasoning. For enterprises, this means options such as private deployment, keeping data within their own infrastructure, and on-demand fine-tuning have become genuinely viable.

Third, API prices continue to fall. With inference architecture optimization and intensifying competition, pricing for new-generation models has hit repeated new lows. Some open-source model APIs now cost a fraction of traditional closed-source flagship APIs. “Getting usable coding capability at lower cost” is shifting from a marketing slogan to an engineering reality.

That said, there is often a gap between benchmark scores and the real development experience. Third-party composite benchmark “intelligence indices” can reflect a model’s general reasoning level, but they cannot answer the questions developers care about most: Does the page it writes look good? Can a 3D scene run correctly on the first attempt? Will multi-turn modifications break existing logic? Answering these questions requires a systematic evaluation methodology.

2. Why Real-World Testing Matters Beyond Benchmark Scores

Before designing an evaluation, it’s worth understanding why public leaderboards alone are insufficient.

1. Benchmark contamination and ceiling effects. Public test questions have a non-trivial probability of appearing in training data, so models may have effectively “memorized the answers,” inflating scores. At the same time, top models now score close to perfect on mainstream benchmarks, making them less useful for differentiation. For an evaluation to be informative, it needs fresher, more business-relevant tasks.

2. Visual aesthetics are hard to quantify automatically. Traditional code evaluation focuses on whether code compiles and passes tests. But frontend development output also involves substantial aesthetic judgment: whether the color scheme is harmonious, whether whitespace has rhythm, whether animations are restrained. These dimensions cannot be judged by assertion scripts alone; they require a combination of human review and model-based review.

3. Engineering context differs. Benchmark tasks are mostly isolated problems, while real development happens within specific tech stacks, existing codebases, and team conventions. The same model can perform dramatically differently when writing a single-file page from a blank canvas versus making incremental changes in a large repository. Evaluations should reproduce the target scenario as closely as possible.

3. Evaluation Design: Five Core Dimensions

Drawing on common industry practice, LLM coding capability evaluation can be broken down into five dimensions, each with its own dedicated tasks.

3.1 Frontend Page Generation: Visual Aesthetics and Fidelity

This is the area where models are improving fastest and where differences are most immediately felt. Typical tasks involve giving a model a natural language requirement and asking it to produce a runnable single-file HTML page—for example, a product landing page, a data dashboard, or an interactive form. Evaluation focuses on:

• Layout quality: Is the grid structure sound? Is the information hierarchy clear? Are navigation, body, and footer complete?
• Visual details: Color coordination, typographic hierarchy, whitespace rhythm, and restraint in shadows and border radii.
• Responsive behavior: Is the page usable across desktop, tablet, and mobile widths?
• Interaction completeness: Are hover states, form validation, and scroll animations implemented as specified?

Practical experience shows that models differ little on the baseline of “code correctness,” but diverge significantly on “visual impact.” This is precisely where sampling effort should be concentrated.

3.2 3D and Data Visualization: Spatial Reasoning and Mathematical Depth

3D tasks are a litmus test for a model’s spatial reasoning and mathematical ability. Typical tasks include: rendering a draggable, rotatable geometric object with lighting using native browser WebGL; drawing an animated particle system with Canvas; and implementing 3D camera controls and projection transforms. Evaluation focuses on:

• Mathematical correctness: Are transformation matrices, normal calculations, and lighting models correct?
• First-attempt pass rate: Can rendering succeed without human intervention?
• Interaction implementation: Are mouse drag and scroll zoom handlers smooth?
• Performance awareness: Are buffers reused sensibly, avoiding per-frame object reconstruction?

A key advantage of these tasks is relatively objective judgment—rendering failures and console errors are exposed immediately, making them suitable as a foundation for automated regression testing.

3.3 Code Correctness and Engineering Structure

Beyond “does it run,” the question is “is it maintainable”:

• Structure: Are functions and modules split sensibly, or is all logic crammed into one giant function?
• Naming and comments: Are names meaningful, and are comments present where the logic genuinely needs explanation?
• Defensive programming: Are boundary conditions and exceptional inputs handled adequately?
• Dependency restraint: Are native capabilities preferred over unnecessary dependencies?

3.4 Instruction Following and Multi-Turn Iteration

Real development rarely happens in a single step. Multi-turn tasks can be designed as follows: generate an initial version in round one, request modifications in round two (e.g., “switch the color scheme to cool tones” or “add a column to the pricing table”), and request a partial refactor in round three. The evaluation checks whether the model can precisely modify the target area without breaking existing functionality or expanding the scope of changes beyond what was asked. This is a critical capability that many leaderboards fail to capture—yet it defines the day-to-day experience.

3.5 Cost and Efficiency

Record input/output token consumption and response latency for each task to provide data for later cost-efficiency analysis.

4. Task Set Design and Automated Scoring

4.1 Three Principles for Task Set Design

• Executable: Output must be runnable code that can be loaded and verified by a headless browser.
• Quantifiable: Every task maps to explicit scoring items, avoiding vague “feels good” conclusions.
• Reproducible: Prompts, sampling parameters, and runtime environments are fixed so results across models and over time can be compared fairly.

4.2 Scorecard Design

Dimension	Weight	Key Scoring Points
Visual aesthetics	30%	Layout, color scheme, typography, whitespace, motion
Feature completeness	25%	Requirement coverage, interaction usability
Runtime stability	20%	Zero console errors, cross-device compatibility
Code quality	15%	Structure, naming conventions, maintainability
Cost efficiency	10%	Quality score normalized by unit cost

For subjective dimensions such as visual aesthetics, a double-blind approach combining human review and model-based review can be used: a reviewer model first outputs structured scores against a unified rubric, then human reviewers sample and calibrate the results. This balances efficiency and credibility.

4.3 Automated Evaluation Script

import json
import pathlib
# 1) Task set: covers frontend, 3D, multi-turn iteration, and other scenarios
TASKS = [
    {
        "id": "fe-landing-01",
        "type": "frontend",
        "prompt": "Generate a single-file HTML landing page with a top navigation bar, "
                  "hero section, three-column feature introduction, pricing table, and "
                  "footer. The layout must be responsive, with restrained colors and a "
                  "clear visual hierarchy.",
    },
    {
        "id": "3d-cube-01",
        "type": "3d",
        "prompt": "Using only native browser WebGL, render a cube that supports mouse "
                  "drag rotation. Include ambient light and directional light, with the "
                  "light direction changing slowly over time.",
    },
]
# 2) Call a generic chat completion API (pseudocode; adapt to your model service)
def call_model(model: str, prompt: str) -> dict:
    ...  # Returns {"content": str, "usage": {"input": int, "output": int}}
# 3) Headless browser check: load the artifact, collect console errors and render state
def headless_check(html_path: pathlib.Path) -> dict:
    ...  # Returns {"rendered": bool, "errors": list}
def evaluate(model: str) -> list[dict]:
    reports = []
    for task in TASKS:
        resp = call_model(model, task["prompt"])
        out_dir = pathlib.Path("artifacts") / model
        out_dir.mkdir(parents=True, exist_ok=True)
        out_file = out_dir / f"{task['id']}.html"
        out_file.write_text(resp["content"], encoding="utf-8")
        check = headless_check(out_file)
        reports.append({
            "task": task["id"],
            "rendered": check["rendered"],
            "console_errors": len(check["errors"]),
            "input_tokens": resp["usage"]["input"],
            "output_tokens": resp["usage"]["output"],
        })
    return reports
if __name__ == "__main__":
    print(json.dumps(evaluate("model-a"), ensure_ascii=False, indent=2))

With a scoring prompt for the reviewer model, subjective dimensions can also be incorporated into the pipeline:

You are a senior frontend reviewer. Score the generated page on the following five
dimensions (0-10): visual aesthetics (layout/color/typography/whitespace/motion),
feature completeness, code quality, responsiveness and interaction, and stability.
Output JSON only, without explanation.

5. Bringing Price into Evaluation: Cost-Efficiency Analysis

Model selection is never about picking “the strongest”—it’s about picking “the most appropriate.” When new-generation open-source model APIs cost a fraction of traditional closed-source flagship APIs, cost-efficiency analysis becomes especially important.

Step 1: Normalize costs. Most model services price input and output tokens separately, with output tokens typically more expensive. Record token consumption for each evaluation task and convert it into an “average cost per task” as a unified basis for comparison.

Step 2: Calculate a cost-efficiency score. A simple illustrative formula:

Cost-efficiency score = Composite quality score ÷ Average cost per task

As a demonstrative example: suppose a task averages 2,000 input tokens and 8,000 output tokens. Model A scores 85 in quality with a per-task cost at baseline 1.0, yielding a cost-efficiency score of 85. Model B scores 78 in quality but at one-quarter of the baseline cost, yielding a cost-efficiency score of 312. If the quality gap between the two is acceptable for the business, the lower-priced model may deliver significantly higher overall value.

Step 3: Route by scenario. A more engineering-oriented approach is not “either/or” but task layering: assign core paths and complex refactoring to the most capable model; assign boilerplate code, simple pages, and batch generation to cost-efficient models; then further reduce costs through prompt caching, result reuse, and related techniques. Many teams have found that a combination of “strong model as safety net + cheap model for volume” outperforms using a single model for everything.

6. From Evaluation to Selection: Practical Guidance for Teams

1. Build an internal evaluation set from real work. Extract 30–50 anonymized typical tasks from historical requirements, including frontend pages, visualizations, bug fixes, and incremental refactoring. This represents your actual needs far better than any public leaderboard.
2. Treat evaluation as regression testing. Model versions iterate frequently. Run the internal evaluation set before every upgrade and let score changes inform the decision, avoiding the “upgrade immediately breaks things” scenario.
3. Set up multi-model routing. At the gateway layer, automatically select models by task type, complexity, and budget, shielding downstream consumers from differences. Developers can also use cloud platform model services with unified access to quickly compare models.
4. Continuously monitor production metrics. After rollout, track code adoption rate, manual modification magnitude (diff size), runtime error rates, and similar indicators. This turns evaluation from a one-off experiment into an ongoing mechanism.

7. Conclusion

LLM coding capability is evolving at a pace visible to the naked eye: reinforcement learning makes output more stable, open source expands the range of choices, and falling prices accelerate adoption. But for engineering teams, leaderboard scores remain only a reference point. What ultimately determines decision quality is a quantifiable, reproducible evaluation system tailored to your own business. By building task sets, automated scoring, and cost-efficiency analysis into team infrastructure, you can make rational choices in a timely manner no matter how the model landscape shifts.