A Practical Evaluation Framework for LLM Coding Ability: Frontend and 3D Generation

1. Background: Why Hands-On Testing Is Necessary

Large model iterations have accelerated noticeably in recent months. New generations of models from multiple vendors are commonly trained with long-horizon reinforcement learning (RL) post-training, yielding substantial gains in core capabilities such as reasoning and coding. Some models have entered the global top tier on public intelligence benchmarks, while API prices continue to fall, creating a pricing trend of “near-flagship capability at mid-range prices.” This benefits developers but introduces a new problem: there is often a clear gap between leaderboard scores and real engineering experience.

The gap stems from three main sources:

1. Data contamination risk: Public evaluation sets may already be in training corpora, inflating scores.
2. Task distribution mismatch: Leaderboards emphasize algorithm problems and standardized tasks, whereas real development is dominated by weakly constrained, highly subjective tasks like “frontend pages with aesthetic requirements” and “interactive 3D scenes.”
3. Uneven capabilities: The same model can vary widely across sub-capabilities, and an aggregate score masks weaknesses.

So rather than waiting for leaderboards, it is better to build your own hands-on evaluation process. This article focuses on coding capability and presents a quantifiable, reproducible evaluation method covering frontend visuals and 3D generation.

2. Evaluation Design: Five Core Metrics

The first step in hands-on testing is defining “what good means.” We recommend building a scorecard from the following five dimensions:

1. Runnability: Whether generated code runs directly in the target environment, and what the error rate is. This is a hard metric and should carry the highest weight.
2. Feature completeness: Whether each requirement is implemented. For example, if “search box, sorting, pagination” is required, each missing item loses a point.
3. Visual and aesthetic quality: For frontend tasks, evaluate layout rationality, color harmony, spacing rhythm, and component details (rounded corners, shadows, hover feedback, etc.). Aesthetics are hard to fully quantify; averaging scores from multiple raters reduces subjective bias.
4. Interaction and 3D performance: Whether animations are smooth, whether 3D scenes support drag-rotate and scroll-zoom, and whether lighting and materials look natural.
5. Stability: Result variance across multiple samples of the same prompt. Poor stability means uncontrollable production behavior.

A reference weight allocation is shown below:

Dimension	Weight	Notes
Runnability	30%	Compile or runtime errors score zero
Feature completeness	25%	Check requirement points one by one
Visual aesthetics	20%	Average of multiple blind ratings
Interaction/3D performance	15%	Smoothness and operability
Stability	10%	Variance across multiple sampling rounds

3. Methodology: Controlled Variables and Batch Sampling

1. Unified test set. Prepare 20–50 test cases stratified by difficulty:

• L1 single-file small tasks: e.g., “generate a single-file HTML countdown page”;
• L2 composite tasks: e.g., “a dark-themed data dashboard with a line chart, a donut chart, and KPI cards”;
• L3 complex tasks: e.g., “a browser-side 3D solar system supporting drag and zoom.”

2. Control variables. Fix the sampling temperature (e.g., 0.3), use the same prompt template, and make requirement descriptions as specific and checkable as possible. Sample each case at least 3 times and use averages rather than the single best result.

3. Automated recording. A schematic script can batch-call models and record results:

import time
CASES = [
    {"id": "fe-001", "type": "frontend",
     "prompt": "Generate a single-file HTML dark dashboard: line chart, donut chart, and 4 KPI cards, with harmonious colors"},
    {"id": "3d-001", "type": "3d",
     "prompt": "Use WebGL to implement a 3D cube with drag rotation, scroll zoom, and basic lighting"},
]
def run_eval(api_client, model, rounds=3, temperature=0.3):
    records = []
    for case in CASES:
        for r in range(rounds):
            t0 = time.time()
            resp = api_client.chat(
                model=model,
                messages=[{"role": "user", "content": case["prompt"]}],
                temperature=temperature,
            )
            records.append({
                "case": case["id"], "round": r,
                "latency_s": round(time.time() - t0, 2),
                "total_tokens": resp["usage"]["total_tokens"],
                "output": resp["text"],
            })
    return records

4. Frontend Visual Generation: Key Testing Points

Frontend generation is the lowest-barrier perceptual test scenario—you only know if the code is right by running it, but you can judge whether the page looks good at a glance. Three key points apply in practice:

1. Use single-file HTML to reduce environment noise. Require the model to output a “single-file, no local dependencies” page that runs by opening a browser, avoiding noise from build tools and dependency versions. If charts are needed, uniformly allow all tested models to reference the same class of general-purpose chart library to keep comparisons fair.

2. Break aesthetics into a checkable list:

• Layout: grid alignment and even whitespace;
• Color: restrained primary/secondary/accent colors and adequate contrast;
• Details: consistent card radii and shadows, natural hover and transition animations;
• Responsiveness: no misalignment when the window is resized.

3. Record both an “aesthetic score” and “modification cost.” A visually stunning result with chaotic structure requiring major rework has lower engineering value than a result scoring 80 on visuals with clear structure. You can additionally count “how many changes are needed from generated output to shippable” as a reference.

A schematic single-file page skeleton:

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <title>Data Dashboard</title>
  <style>
    :root { --bg:#0f172a; --card:#1e293b; --accent:#38bdf8; }
    body { background:var(--bg); color:#e2e8f0; font-family:system-ui; }
    .grid { display:grid; grid-template-columns:repeat(4,1fr); gap:16px; }
    .card { background:var(--card); border-radius:12px; padding:16px; }
  </style>
</head>
<body>
  <div class="grid"><!-- KPI cards --></div>
  <!-- Chart containers -->
</body>
</html>

5. 3D Scene Generation: Key Testing Points

3D generation is a highly discriminative task that tests a model’s “spatial imagination + API proficiency.” We recommend targeting browser-native WebGL standards or mainstream browser-side 3D rendering approaches, with cases progressing from simple to complex:

• L1: Rotating cube, validating the basic rendering pipeline;
• L2: Scene with lighting and shadows, validating understanding of materials and light sources;
• L3: Particle system or solar system, validating animation loops, camera controls, and math capability.

Evaluation focuses on three points: first, interactivity—whether drag-rotate and scroll-zoom event bindings are correct; second, rendering correctness—whether geometry is visible, lighting direction is reasonable, and typical errors such as black screens or depth conflicts occur; third, performance awareness—whether update logic is placed in the animation loop and whether geometry is rebuilt every frame.

A schematic 3D code skeleton:

<canvas id="scene"></canvas>
<script>
  const gl = document.getElementById('scene').getContext('webgl');
  // 1. Compile vertex/fragment shaders and link the program
  // 2. Write cube vertices and index buffer
  // 3. Each frame: update rotation matrix from mouse state -> clear -> draw
  function frame(t) {
    // update rotation from drag; draw arrays
    requestAnimationFrame(frame);
  }
  requestAnimationFrame(frame);
</script>

A common phenomenon in hands-on testing: a model writes 3D code that “looks correct” but produces an invisible object or black screen at runtime. Causes often include unhandled shader compilation errors, uninitialized matrices, and improper camera position. These problems are precisely the key samples that distinguish engineering competence and should be prioritized in the test set.

6. Cost and Performance Trade-offs: Calculating Unit Cost

The other half of evaluation is price. Current model APIs generally bill by token, with different input and output prices, so you should measure “the real cost of completing a unit task” rather than just looking at list prices:

def task_cost(tokens_in, tokens_out, price_in, price_out):
    """price unit: currency per million tokens"""
    return tokens_in / 1e6 * price_in + tokens_out / 1e6 * price_out

Example (illustrative numbers): a task averages 2,000 input tokens and 4,000 output tokens. Model A charges 8 and 24 currency units per million tokens for input/output respectively, so the single-run cost is approximately 0.002×8 + 0.004×24 = 0.112 currency units. At 10,000 calls per day, the monthly cost is about 33,000 currency units. Switching to a model priced 60% lower with a similar measured score could save about 20,000 currency units per month—this is what “flagship-level capability at a low price” really means in engineering terms.

Based on cost measurement, four general selection strategies emerge:

1. Layer by scenario: Use the strongest models for low-frequency, high-value tasks such as architecture design and difficult defect localization; use cost-effective models for high-frequency tasks such as code completion, comment generation, and batch refactoring.
2. Consider both open-source and hosted forms: Open-weight models can be privately deployed, suiting data-sensitive scenarios; hosted APIs eliminate operations overhead. Evaluate both against compliance requirements.
3. Build a cost-performance index: Use “measured composite score / cost per task” as the basis for horizontal ranking, avoiding comparisons based on price alone or score alone.
4. Verify the platform’s engineering support: Whichever provider you choose (for example, Baidu AI Cloud and other large-model services generally offer model access, monitoring/alerting, and logging capabilities), pay attention to rate-limiting policies, context length, and concurrency stability—they also affect the real cost per unit task.

7. Summary and Practical Recommendations

There is no definitive leaderboard for large-model coding evaluation—only hands-on testing tailored to your own business. Recommendations for implementation:

1. Build 20–50 layered test cases covering correctness, aesthetics, and 3D tasks;
2. Fix temperature and prompt templates, and average multiple samples per case;
3. Score with a scorecard while recording latency and token consumption;
4. Compute a “score/cost” cost-performance index and select models by task frequency tiers;
5. Put the test set under version control and rerun it after new model releases to continuously update selection conclusions.

When “capable enough” and “cheaper” can hold simultaneously, a scientific hands-on process is the only path to turning that possibility into a certainty. Rather than chasing hype and switching models, it is better to make the evaluation system part of the team’s infrastructure—that is the optimal answer to rapid model iteration.