Evaluating LLM-Generated 3D Frontends: A Three-Dimensional Framework and Common Pitfalls

Why Creative Tasks Are a Better Test of Model Capability

General benchmark scores only reflect a model’s statistical performance on standardized questions. Real-world development, by contrast, resembles a compound task of “open-ended creation under strict constraints.” Take 3D frontend creation as an example: the model must simultaneously accomplish three things—understand spatial composition, choose the correct rendering approach, and strictly honor numerical and behavioral constraints. These correspond to the three qualities of a human engineer: aesthetic judgment, technical depth, and engineering rigor. A deficiency in any one of them will show up directly in the final output.

Therefore, rather than chasing leaderboard scores, it is more revealing to construct a task that cannot be “fudged” and observe the model’s actual performance. This article uses a representative task: asking the model to generate a single offline HTML file containing a massive ring-shaped water gate, a terraced coastal city, and a tidal system that can rise and fall.

Test Design: Using Constraints to Eliminate “Lucky Guesses”

An effective creative evaluation task must first close off the paths that allow a model to cut corners.

1. Technical Stack Constraints

The task requires a single offline HTML file using only native WebGL2, with no external images, model files, or third-party libraries. This means all geometry, textures, and lighting must be generated procedurally. The model cannot pass by “importing assets.” The quality of the sky, water surface, and building materials depends entirely on the code itself.

2. Layered Prompt Constraints

A high-quality prompt should first establish the “static foundation” and then lock in the “dynamic behavior.” For example:

Please generate an offline-runnable HTML file:
[Composition & Materials] A massive ring-shaped water gate as the main subject,
  with a terraced coastal city distributed along the bay. The first screen should
  clearly present the layering of the building complex, with copper oxide and
  stone materials clearly distinguishable.
[Dynamic Behavior] During tidal rises and falls, ship buoyancy, waterline position,
  and the submerged range of plaza stone steps must change in a coordinated manner.
  Simply raising and lowering a blue plane to simulate water is not allowed.

The clause “simply raising and lowering a blue plane is not allowed” is critical—it preemptively blocks the laziest water-surface implementation and forces the model to invest in physical coordination rather than delivering a visually “fake tide.”

Dimension 1: First-Screen Composition—First Impressions Depend on Scene Hierarchy

The most visible difference in generated results often lies not in modeling precision but in composition: how the default camera organizes the scene’s primary and secondary elements.

A typical failure mode is “a massive subject upstaged by terrain.” Although the ring-shaped water gate is enormous, the surrounding mountains occupy more of the frame. The first screen is dominated by nearby slopes and distant peaks, while the coastal city shrinks into the background with low visual distinction—users struggle to identify where the main subject is.

A better approach organizes the ring gate, bridge openings, beacon towers, and tiered residential buildings into a coherent landscape: buildings spread along both banks, the central plaza displays clear concentric step levels, and the warm-cool contrast between copper oxide and stone makes the materials immediately recognizable. This comes closer to the “complete world” required by the prompt, rather than the crude collage of “placing a ring-shaped object on terrain.”

From this, three observable evaluation points emerge: first-screen information density (the proportion of the frame occupied by meaningful content), visual focus path (whether the viewer’s first fixation lands on the primary structure), and occlusion relationships (whether the main subject is crowded out by secondary terrain).

Dimension 2: Rendering Mechanism—Similar Appearance, Orders of Magnitude Apart in Implementation Depth

The water surface is the best litmus test for the technical depth of generated code. Two common implementation approaches differ significantly in quality:

Approach A: Analytical approximation. Using sky gradients, sun highlights, and light streaks to “suggest” reflections. Simple to implement with low performance overhead, but the reflections have no actual correspondence to objects in the scene and do not hold up under scrutiny.

Approach B: Mirrored-camera planar reflection. This introduces an additional virtual camera mirrored across the water plane at the code level. The city and ships are redrawn in their entirety and rendered to an offscreen framebuffer, which the water shader then samples as a texture. The core workflow:

// Planar reflection (mirrored camera) core steps
// 1. Mirror the main camera's position and orientation across the water plane
// 2. Clip the view frustum below the water surface, keeping only objects to reflect
// 3. Render to an offscreen framebuffer to obtain the reflection texture
// 4. In the water fragment shader, sample the reflection texture by projected
//    coordinates, then combine with ripple distortion and Fresnel blending

With Approach B, the reflections of buildings on the shore have a real spatial source. This is a generational difference in mechanism, not a matter of parameter tuning. The lesson: evaluating generation quality requires reading the implementation path—screenshot comparisons only cover the surface.

Dimension 3: Parameter Fidelity—Good Visuals Still Need Source-Level Verification

The most common hidden problem in generated code is “behavior that is declared but parameters that don’t add up.” A representative case: the prompt requires “most stone steps to be submerged at high tide.” In one generated version, a plaza with nine concentric step levels was submitted. According to the step height and maximum water level in the source code, only about the lowest four levels would be below the average water level at full tide—the requirement was not met. The verification cost for this type of problem is quite low:

const STEP_H = 0.6;   // Height increase per step level
const STEPS  = 9;     // Total number of step levels
const MAX_TIDE = 2.0; // Maximum tide level
const submerged = Math.floor(MAX_TIDE / STEP_H); // ≈ 3~4 levels
// Requirement: "most stone steps should be submerged"
// Actual result: only ~4 levels → parameter requirement not met

Conversely, another generated version with a weaker first-screen composition correctly covered the outermost step level with the maximum water level range. This shows that visual metrics and parameter correctness are two independent scoring dimensions and must be verified separately. It is advisable to codify such checks as automated assertions and include them as a fixed inspection item after every generation.

Version Comparison Methodology: Compare Against Same-Generation Baselines, Not Cross-Generation Leaderboards

Two additional lessons from evaluation practice are worth documenting.

First, same-generation comparisons carry more information. Cross-generation comparisons tend to be obscured by overall upgrades, while “improvements” and “regressions” between adjacent versions reveal more about iteration direction. When the training environment changes (for example, when distillation alters marginal gains), a model often strengthens one capability while sacrificing another—these trade-offs are invisible in aggregate leaderboard scores but immediately apparent in pairwise comparisons on specific tasks.

Second, expect capability divergence. Models typically improve faster on long-form programming and tool-assisted tasks than on pure visual creation. Significant divergence in scores across different task categories within the same version is normal. Evaluators should establish separate baselines per task family rather than summarizing everything with a single total score.

Practical Recommendations for Developers

1. Use a four-layer prompt structure: composition and materials → dynamic behavior → numerical constraints → acceptance criteria, progressing layer by layer, with explicit “prohibited items” written out.
2. Use a three-part acceptance checklist: first-screen appearance (composition and material recognizability), implementation mechanism (read the code to confirm the technical approach), and numerical correctness (independently verify key parameters). All three are essential.
3. Run fixed-variable regressions: use the same prompt and the same random seed when comparing across versions to distinguish “capability improvement” from “random fluctuation.”
4. Ask the model to self-check: request that generated code include a key parameter table (water levels, step heights, camera parameters, etc.) and perform a line-by-line self-review against the prompt. This significantly reduces parameter mismatch probability.
5. Engineer the evaluation process: in team workflows, use cloud platform capabilities (such as model services, container platforms, GPU compute, and monitoring and alerting) to run evaluation cases in batch, preserve generated artifacts for version regression, and turn “intuition-based evaluation” into measurable engineering assets.

Conclusion

LLM frontend creation capability is evolving from “generating a scene that runs” to “generating a self-consistent world.” Evaluating this evolution cannot stop at first impressions from a screenshot—it must penetrate three layers: scene organization ability, rendering implementation depth, and parameter fidelity. Building an evaluation framework of “appearance + mechanism + numerical correctness” not only identifies a model’s true capability more accurately, but also guides us toward writing prompts with tighter constraints and higher fulfillment rates. For developers facing rapidly iterating models, this is the meta-skill most worth mastering.