Evaluating LLMs on Complex 3D Frontend Generation: Where First Impressions End and Real Testing Begins

1. Why Use Complex 3D Frontend Generation to Test Models

Most public coding benchmarks focus on algorithm problems and function completion. In real-world frontend delivery, however, the more challenging and long-tailed problem is “long-horizon programming”: managing geometry construction, material lighting, animation timing, and interaction state within a single code file, where dozens of variables and functions must remain semantically consistent. Any break in that consistency shows up directly on screen. A complex 3D scene is a typical example of this kind of task—it is both a visual challenge and a logic challenge.

A new generation of large models has increasingly shifted its training focus toward long-horizon programming and tool-assisted tasks. For developers, rather than memorizing leaderboard scores, it is more useful to design a high-discrimination “exam” and directly observe how a model performs under real constraints.

2. Exam Design: Making Requirements “Hold Up”

Consider a typical task: using only a single offline HTML file and the browser’s native WebGL2 interface, render a giant ring-shaped water gate, a terraced sea city, and a rising and falling tide—without relying on external images, models, or third-party libraries.

The prompt is written in two layers:

• The first layer establishes the “static world”: first lock down the composition, architecture, and materials—the sense of scale of the ring structure, the way buildings spread across both banks, the layering of plaza steps, and the distinction between metal and stone textures.
• The second layer establishes “dynamic constraints”: explicitly state that “the tide must not be represented by simply raising and lowering a blue plane.” Ship waterlines, dock waterline marks, and plaza flooding states must all respond to the water level.

The second layer is the key to the entire exam. “It cannot just be a blue plane that goes up and down” is a typical anti-cheating clause: it forces the model to build linkage logic between water level and scene state, rather than faking it with a visual approximation.

Controlling variables is equally important:

• Comparative evaluations should be conducted within the same time window, on the same client, and under the same mode tier. Online model service behavior may change with deployment iterations, and comparisons separated by time introduce additional noise.
• For enterprise users, if model capabilities are accessed via a cloud platform (such as Baidu AI Cloud) through an API, inference parameters, context length, and sampling configuration should also be fixed, and the raw response of each request should be retained to ensure traceability and reproducibility.

3. Evaluation Dimension One: First-Frame Composition—The Value of a “First Impression”

Differences in perceived quality are often already visible in the first frame of the default camera. Given the same requirements, outputs of different quality levels tend to fall into two typical composition patterns:

• The subject is squeezed by its environment: although the giant structure is placed in the center of the frame, mountains on both sides consume a large portion of the image, nearby slopes and distant mountains fill the first screen, and the city shrinks deep in the bay with weak recognizability. This is the stiffness of “placing a single object on terrain.”
• The scene is organized as a coherent landscape: building clusters spread along both banks, the central plaza has clear concentric step layers, warm and cool materials (such as oxidized copper and stone) are distinguishable under default lighting, and the image has clear hierarchy and depth.

Specific judgment can be reduced to three questions:

1. Can the subject be found at a glance?
2. Is there a narrative relationship between the subject and the environment (do bridge openings, beacon towers, and layered housing echo one another)?
3. Are material differences recognizable without interaction?

Composition reflects the model’s overall understanding of spatial relationships in the prompt, not its ability to model local objects. Progress at this layer is easy to perceive, but it is only the entry threshold.

4. Evaluation Dimension Two: Rendering Mechanism—From “Looks Like” to “Actually Corresponds”

Water reflections are the best sample for distinguishing “simulation” from “implementation.” Common approaches fall into two tiers:

• Low-cost approximation: using analytic sky, sun highlights, and light trails to “paint” the feeling of water. It looks acceptable, but the reflections have no real spatial correspondence with the buildings on shore.
• Mechanism-based implementation: introducing a mirror camera for planar reflection—flipping the camera about the water plane, redrawing the city and ships from the mirrored viewpoint into an offscreen texture, and then sampling it into the water shader so that every reflection pixel has a real source.

Example code (three core steps):

function renderPlanarReflection(scene, waterPlane) {
  // 1. Mirror the current camera symmetrically about the water plane
  const reflCam = mirrorCamera(activeCam, waterPlane);
  // 2. Set a clipping plane to render only geometry below the water surface, avoiding artifacts
  reflCam.clipBelow(waterPlane);
  // 3. Render to an offscreen texture from the mirrored viewpoint for the water shader to sample
  const tex = renderToTexture(scene, reflCam);
  waterMaterial.setReflectionMap(tex);
}

The significance of this layer is that a correct mechanism means the model has mastered graphics structures, not just “seen good-looking images.” For enterprise frontends such as digital twins and industrial visualization, mechanism correctness matters more than single-frame appearance. It determines whether the effect still holds when the scene or time changes, and it also determines future maintenance costs.

5. Evaluation Dimension Three: Quantitative Constraints—The Real Divide

What most clearly separates outputs is often the least conspicuous part: whether numerical parameters fulfill the textual requirements.

A typical case: the requirement explicitly states that “after high tide, most of the stone steps are submerged.” One output looked visually quite complete, but when calculating from the number of step layers, single-layer height, and maximum water level in the source code, only the lowest four layers were below average water level at full tide—“most submerged” was not fulfilled. By contrast, another version that was less impressive in first-frame composition extended the maximum water level to cover the outermost steps.

This reveals a common phenomenon: models are more sensitive to “visual self-consistency” than to “quantitative semantics.” The image may look plausible, but the numbers can be far off. Once such deviations enter real business contexts—instrument thresholds, statistical definitions in financial research, engineering dimensions—the risk is very real.

An actionable countermeasure is “source constant verification”: use a small script to extract key parameters from the generated code and perform consistency checks.

// Extract key constants from the generated source to verify quantitative constraints
const STEP_COUNT = 9;      // Number of plaza step layers
const STEP_HEIGHT = 0.45;  // Height of a single step layer
const MAX_TIDE = 1.8;      // Maximum tide level (relative to datum)
// Number of step layers submerged at full tide
const submerged = Math.floor(MAX_TIDE / STEP_HEIGHT); // = 4
// Requirement is "most steps submerged" (at least 5 layers); current configuration does not meet it

Going further, acceptance criteria can be quantified at the prompt stage: “The plaza has nine step layers, each about 0.45 meters high, and the maximum tide level should submerge no fewer than six layers.” This converts vague semantics into checkable inequalities and greatly reduces ambiguity.

6. From Perception to Delivery: A Three-Tier Acceptance Checklist

1. Perception layer: composition hierarchy, material distinction, and dynamic linkage completeness.
2. Mechanism layer: whether effects such as reflections, shadows, and water level are “painted” or “computed”—this directly affects future maintenance and extensibility.
3. Parameter layer: every quantitative requirement in the prompt is converted into a checkable item and verified manually or by script.

Supporting practices include: staged generation (skeleton first, details later), adding automated assertions for scenes (running headless rendering in continuous integration for screenshot comparison), and retaining version snapshots of prompts and responses for regression. While long context brings a higher first-pass rate, it also makes “where the problem is” harder to locate by eye, which increases the value of automated verification.

7. Conclusion

The “first impression” of frontend 3D generation has improved visibly: composition is more complete, materials are more credible, and mechanisms are more solid. But the real test comes later—fulfillment of quantitative constraints, consistency of parameters across long contexts, and verifiability of output results. This is both the next stage of model evolution and a gate that developers must guard themselves in human-machine collaboration. Only by upgrading acceptance standards from “does it look good?” to “does it compute correctly?” can large models truly enter engineering delivery.