A Frontend 3D Generation Test for LLMs: Visual Quality, Rendering Mechanisms, and Consistency

1. Why Frontend 3D Is a High-Quality Test Bed for LLMs

Among code generation tasks, frontend 3D scene generation stands out as a rare “high signal-to-noise” evaluation target, for three reasons:

• Comprehensive: A decent 3D scene simultaneously tests spatial layout, graphics knowledge (rendering pipeline, material systems, camera models), numerical computation (coupling between tide levels and geometry), and long-range constraint tracking. Weakness in any one link shows up directly in the final product.
• Hard to fake results: Business code can be gradually improved through patch-style edits, but the look, layering, and physical plausibility of a 3D scene are presented as a whole—either the model truly “understands” the scene, or the awkwardness of “placing individual objects on a terrain” is immediately visible.
• Easy to isolate variables: Requiring an offline single-file HTML output, using the browser’s native 3D rendering interface, and depending on no external images, model files, or third-party libraries eliminates framework fallbacks and asset rescue, making the model’s own capability the only variable.

For this reason, 3D generation tasks are often used in recent years as a controlled experiment for examining the quality of a model’s post-training.

2. Evaluation Task and Prompt Design

The task in this case: implement a “tidal sea city” in a single offline HTML file—including a giant ring-shaped sluice gate, a terraced sea city, and a tide system that can rise and fall. Throughout the test, the same terminal and the same interaction mode were fixed, and the old and new generations of models were compared within a similar time window to avoid version drift interfering with conclusions.

The prompt design has three reusable points:

1. Establish composition and material tone first. Require the model to first determine the scene’s primary/secondary relationships, building distribution, and warm/cool material relationships before writing code. This is equivalent to front-loading “aesthetic decisions” as explicit instructions, preventing the model from devoting all compute to geometric detail while neglecting overall composition.

2. Write dynamic behavior as hard constraints. Explicitly specify: the rising and falling tide cannot rely solely on raising and lowering a blue plane—boats must rise and fall with the water level, the waterline must move across building surfaces, and the submersion state of the plaza steps must be coupled with the tide level. Only such constraints can distinguish whether the model “made an animation” or “built a water-level system.”

3. Embed verifiable quantitative indicators. For example, “at high tide, most of the stone steps should be submerged.” Such statements can later be verified by checking the step elevations and maximum water level in the source code, converting subjective impressions into objective checks.

An illustrative prompt skeleton is as follows:

Please implement in a single offline HTML file (native 3D rendering, no external dependencies):
1. Scene: giant ring-shaped sluice gate + terraced sea city
2. Composition: clear primary/secondary hierarchy under the default camera,
   building clusters spread along both banks,
   central plaza with clear concentric step layers
3. Materials: distinguish the warm/cool textures of oxidized copper and stone
4. Dynamics: tide level can rise and fall; boats, waterline, and plaza submersion
   state must be coupled with the tide level;
   raising and lowering a single blue plane alone is prohibited
5. Quantitative: at high tide, most stone steps should be below the water level

3. Dimension One: Scene Organization and “First Impression”

The default camera is the model’s “first impression” and the first watershed in terms of visual quality.

The problem with the older-generation model is an imbalance of primary and secondary elements: the giant ring itself is large enough, but the mountains on both sides occupy more of the frame. The first screen is filled with nearby slopes and distant mountains, the sea city is tucked deep in the bay, and the recognizability of the building clusters and plaza is weak—viewers have to work to identify where the protagonist is.

The newer-generation model demonstrates stronger scene organization: the giant ring, bridge arches, beacon towers, and layered residences are organized into a coherent landscape; building clusters spread along both banks; the central plaza has clear concentric step layers; and the warm/cool textures of oxidized copper and stone are easier to distinguish. It is closer to the “complete world” required by the prompt.

From this, a visual-quality evaluation checklist can be distilled:

• Primary/secondary relationship: Can the visual subject be identified at a glance on the first screen?
• Visual guidance: Do foreground, midground, and background form a progression rather than blocking each other?
• Layer density: Are there distinguishable substructures (steps, towers, bridge arches) rather than homogeneous geometric stacking?
• Material distinction: Do the warm/cool and light/dark relationships of different materials hold up?

4. Dimension Two: Depth of Rendering Mechanism Implementation

The mechanism differences beneath the visual quality are the more essential generational signal.

In the implementation of water surface reflection, the older model mainly uses analytical approximation: simulating reflection with an analytical sky, sun highlights, and light trails. Such schemes are computationally cheap, but the reflections have no real spatial source—they are essentially weighted approximations of environmental effects.

The newer-generation model introduces a mirror camera scheme (i.e., planar reflection in graphics): render the scene again after mirroring it about the water plane to obtain a reflection texture, which the water surface samples directly during shading. The core idea is illustrated below:

// Core idea of planar reflection (illustrative code)
const mirrorCam = mainCamera.mirroredAbout(waterPlane);
const reflTex  = renderSceneToTexture(scene, mirrorCam);
waterMaterial.reflectionSampler = reflTex; // water fragments sample by projected coordinates

This means the reflections of onshore buildings have a real corresponding spatial source—whatever is on the shore is reflected in the water. This step up in implementation mechanism shows that post-training has indeed enabled the model to master more complex graphics engineering schemes rather than remaining at the level of effect stacking.

During evaluation, it is advisable to develop the habit of “reading the source code”: two water surfaces with similar visual quality may differ by a generation in implementation; conversely, a mechanism-advanced scheme may also bury pitfalls elsewhere—which is exactly what the next dimension discusses.

5. Dimension Three: Parameter Consistency and Prompt Fulfillment

This is the most valuable finding of this evaluation: the newer-generation model is actually less thorough than the older model in handling the maximum tide level.

Checking against the step elevations and maximum water level in the source code: of the newer model’s nine plaza step levels, only the lowest four are below the average water level at high tide, failing to fulfill the prompt’s requirement that “most stone steps should be submerged”; while the older model, whose first-screen composition was less favorable, actually extended the maximum water level range to the outermost steps.

The verification logic is not complicated:

// Consistency check illustration
const stepHeights = [0.4, 0.8, 1.2, /* ... */ 3.6]; // nine step elevations
const maxTide = 1.6;                                 // maximum water level in source
const submerged = stepHeights.filter(h => h < maxTide).length;
// submerged = 4, less than half → fails to fulfill "most stone steps submerged"

This case yields two insights:

• Capabilities are differentiated: while visual quality and mechanism improve, numerical consistency may regress. “Looking good in a demo” and “fulfilling constraints” are two different things.
• Evaluation must land on source code and numbers: looking only at runtime effects will miss such problems; only by checking each quantitative requirement in the prompt against the source code can credible conclusions be obtained.

6. A Reusable Multi-Layer Evaluation Framework

Combining the above practice, LLM 3D generation capability evaluation can be organized into four layers:

1. Visual layer: fix the prompt and resolution for screenshots, and compare first-screen composition and scene organization;
2. Mechanism layer: read through the generated source code and identify the technical choices for reflection, shadows, water-level systems, etc.;
3. Consistency layer: extract the quantitative requirements from the prompt and verify fulfillment against the source code one by one;
4. Interaction layer: adjust runtime parameters such as tide level to verify whether the coupling of boats, waterline, and submersion state is complete.

This framework also applies to other generative tasks—breaking down “subjective visual quality” into objectively checkable items is a general methodology for all model evaluation.

7. Reading Model Evolution Trends from Generational Differentiation

Another observation dimension worth expanding is the time window. In recent months, the industry environment has undergone substantive changes: as distillation gains on the pre-training side converge, the room for improvement brought by pure scale expansion and inherited training is narrowing, and the direction of post-training investment has become the main variable in capability differentiation across model generations.

This explains why newer-generation models have made significant progress on long-range programming and tool-assisted tasks while showing “regressions” in certain specific details—model iteration is no longer a comprehensive linear improvement but a directional trade-off under a limited training budget.

The implication for users: do not make technical decisions on the assumption that “a new version is necessarily stronger across the board,” but instead build small evaluation sets for your core scenarios and let the data speak. Mainstream cloud platforms generally provide LLM APIs along with supporting logging, monitoring, and evaluation toolchains; for example, model services on platforms such as Baidu AI Cloud can be used to quickly set up similar comparative tests. Making evaluation engineered and routine is more valuable in the long run than chasing single leaderboard scores.

8. Conclusion

What makes 3D scene generation a “mirror” of LLM capability is that it simultaneously exposes three levels: the model’s aesthetic organization, graphics fundamentals, and constraint fulfillment. This case shows that the newer-generation model completed a generational upgrade in “first impression” and implementation mechanism, yet regressed on numerical consistency—which reminds us that model evaluation has no once-and-for-all conclusion. For developers, rather than waiting for an authoritative leaderboard, it is better to follow the four-layer framework in this article, accumulate a reproducible set of evaluation cases around your own business scenarios, and let every model upgrade be measured by the same ruler.