Hands-On Evaluation of LLM 3D Scene Generation: From Composition to Reflection

1. Why a “Tidal Sea City” Makes the Right Test

In front-end capability evaluations of large language models, tasks such as static landing pages and marketing pages no longer separate models meaningfully. These tasks are highly templated, tolerate errors generously, and models produce “looks acceptable” output with high probability. A genuine high-pressure test is 3D scene generation, because a single generation simultaneously stress-tests four capabilities:

1. Spatial composition: organizing the entity list from the prompt into camera language with clear hierarchy and depth layers;
2. Rendering mechanism selection: effects such as reflection, lighting, and water surfaces have multiple viable implementation paths, and the choice directly determines the ceiling;
3. Numerical consistency: parameters such as tide level, step heights, and waterlines must be mutually consistent, without contradictions;
4. Granularity of instruction following: whether a requirement with quantitative semantics—such as “most of the stone steps are submerged”—is actually honored, rather than approximated with a general mood.

This article therefore uses a unified test task: require the model to output a single, offline-runnable HTML file that uses native WebGL2 to implement a “giant circular water gate + stepped sea city + rising and falling tides,” with no external images, model files, or third-party libraries. The value of these constraints is that they eliminate external-resource interference: textures, models, and the engine are all constructed from scratch by the model in code, exposing purely the model’s own ability to organize code and space.

The task also includes a critical “anti-lazy” clause: the tide must not be implemented by simply raising and lowering a flat blue plane. Ship waterlines, shoreline water marks, and the plaza submersion state must all change in coordination. This clause separates “looks like water” from “behaves like water,” and it is the crux of the entire evaluation.

2. Evaluation Design: Controlled Variables and Reproducibility

For evaluation conclusions to hold, variables must be controlled. In practice, at least three things matter:

• Unify the runtime environment and inference mode: complete all comparison tests on the same client and the same inference tier;
• Compress the time window: model services may fluctuate across time windows. Comparisons made weeks apart can inject environmental noise larger than the generational difference itself, so comparison tests should be completed within closely spaced time windows;
• Preserve raw artifacts: save the generated HTML source rather than only screenshots, so that source-level auditing remains possible later—the value of this shows up in Section 5.

The prompt should be written in layers, escalating from composition to mechanism to quantitative acceptance:

Task: Output a single, offline-runnable HTML file using native WebGL2,
with no external images, models, or third-party libraries.
Scene requirements:
1. Composition first: in the default camera view, the circular water gate
   is the visual centerpiece and the sea city spreads along both shores,
   with a clear three-layer relationship—foreground (plaza steps),
   middle ground (building clusters), background (mountains);
2. Distinguishable materials: oxidized copper and stone must be clearly
   separated in warm versus cool tones;
3. Tidal mechanism: expose a water-level parameter. When the tide rises or
   falls, ship waterlines, shoreline water marks, and the submerged range
   of the plaza must all change in coordination; simply moving a
   solid-color water plane up and down is forbidden;
4. Quantitative acceptance: define step heights and the maximum water level
   as constants in the source code. At high tide, most steps should sit
   below the waterline.

3. Dimension One: First-Screen Composition Sets the First Impression

Both outputs run and render, but the default camera view immediately reveals the difference. One version places the giant ring at the center of the frame, yet the mountains on both sides claim much of the frame: the first screen is dominated by a near hillside and distant peaks, the city shrinks into the depths of the bay, and the building cluster is barely recognizable. The other version organizes the ring, bridge arches, beacon towers, and terraced housing into a coherent landscape: the plaza steps read clearly, and the warm and cool materials are distinguishable at a glance.

Abstractly, this tests the model’s scene-graph organization ability: how entities are distributed across foreground, middle ground, and background; how negative space is reserved for the visual subject; and how secondary elements are made to serve the subject rather than compete with it. This mirrors the composition intuition of a human art director—one of the implicit abilities that post-training can least easily patch in 3D generation.

The default camera matters most because most users never touch camera parameters—the first screen is everything. When judging a 3D generation, first check whether the default view is self-consistent; only then discuss detail precision.

4. Dimension Two: Reflection Mechanism—The Implementation Divide Beneath the Surface

Water reflection has two common implementation paths:

• Analytic approximation: sky color, sun highlights, and light streaks are layered analytically to “simulate” the feel of reflection. It is cheap and runs stably, but the reflections have no real spatial origin, and dynamic objects cannot enter them correctly;
• Planar reflection: construct a camera mirrored about the water plane, redraw the shore-side scene against the water into an offscreen texture, then sample that texture in the water shader, adding normal perturbation and a Fresnel term.

In this round of testing, the two model versions happened to land on these two different paths. The version that used a mirrored camera produced building and ship reflections with true spatial correspondence, and the reflections updated as the tide level changed—a substantive upgrade at the mechanism level. The core flow can be simplified as:

// Core steps of planar reflection (illustrative)
function renderReflection(frame) {
  // 1. Mirror the main camera about the water plane to get the reflection camera
  const reflectCam = mirrorCamera(mainCam, waterPlane);
  // 2. Redraw shore buildings and ships with the reflection camera
  //    into an offscreen texture
  bindRenderTarget(reflectionRT);
  drawScene(reflectCam, { clipPlane: waterPlane }); // clip away the underwater part
  // 3. In the main pass, sample the reflection texture by screen coordinates,
  //    adding ripple perturbation and a Fresnel term
  bind(null);
  drawScene(mainCam, {
    waterMaterial: {
      reflectionMap: reflectionRT.texture,
      normalPerturb: rippleNormal(frame.time),
      fresnel: computeFresnel(viewDir, waterNormal)
    }
  });
}

This yields an important evaluation principle: when the visuals look similar, the mechanism determines the ceiling. Analytic approximation can look no worse in a static screenshot, but as soon as objects move or the water level changes, the flaws surface. An evaluation must not stop at screenshot comparison—you have to read the code to confirm the rendering path.

5. Dimension Three: Parameter Auditing—The Last Mile

The most interesting divergence appeared on the quantitative clause. The prompt required “most steps submerged at high tide.” One version defined nine step heights and a maximum water level in its source code, but by the constants’ own arithmetic, only the lowest four steps sit below water at the average water level—“most” was never delivered. Conversely, the version whose first-screen composition was weaker set the maximum water level to cover the outermost steps. The audit itself is trivial:

// Step submersion audit (illustrative)
const steps = [0.4, 0.8, 1.2, 1.6, 2.0, 2.4, 2.8, 3.2, 3.6]; // nine step heights
const avgWater = 1.8;   // average water level
const maxWater = 3.4;   // maximum water level setting
const submerged = steps.filter(h => h < avgWater).length;
console.log(submerged); // 4 — fails the "most steps" semantic requirement
// To honor the prompt, raise maxWater to cover the outer steps,
// or reduce the step spacing

Two conclusions emerge from this case:

1. Visual improvement and numerical rigor do not necessarily advance in sync. A model’s progress in “atmosphere” does not automatically translate into progress in “honoring quantitative commitments”; thoroughness under extreme parameters is often an overlooked weak spot;
2. Readable, auditable generated code is an evaluation advantage unique to text-to-code generation. A hand-built scene can only be accepted by eye, whereas model-generated source allows a three-way mapping checklist—“prompt requirement → source constant → runtime behavior”—that converts fuzzy aesthetic judgments into verifiable numeric ones.

Therefore, write quantitative acceptance clauses explicitly into the prompt and require the model to define key parameters as constants, leaving an entry point for post-hoc auditing.

6. Lessons from Generation Comparisons: Capability Gains Are Non-Uniform

Within generational updates of the same technical route, different capability dimensions often improve unevenly, and some may even regress locally. At the macro level, shifts in data mix across training cycles, changes in post-training priorities, and varying distillation gains can make two generations with “similar total scores” behave very differently on specific dimensions.

This dictates the evaluation methodology: score by dimension, not by total alone. Score composition, materials, mechanism, and parameters independently, then combine them into a capability profile. This round’s results illustrate the point precisely: the newer generation clearly advanced in composition organization and reflection mechanism, yet regressed on thoroughness under extreme parameters. An overall impression score would completely mask this structural change—and it is precisely this structural change that indicates where the model’s capabilities are heading next.

7. A Reusable Evaluation Framework and Engineering Suggestions

The practice above can be distilled into a four-layer acceptance model that you can apply directly to your own evaluation set:

1. It runs: single file, zero external dependencies, no console errors on load;
2. It looks right: default-camera composition, material distinction, and lighting atmosphere all pass;
3. The mechanism is correct: the reflection path and dynamic coordination follow physical intuition rather than visual approximation;
4. The parameters are precise: quantitative clauses verified at the source-code level.

On the engineering side, batch evaluation can be assembled from generic cloud capabilities: object storage hosts each generation’s artifacts and screenshots, logging services retain prompts and full responses for traceability, and monitoring and alerting track batch task status—cloud platforms such as Baidu AI Cloud provide these generic services. To compare multiple models horizontally, route all calls uniformly through a large-model service API and fix sampling parameters such as temperature, so that differences come from the models themselves rather than sampling randomness.

8. Conclusion

First impressions can deceive; mechanisms and constants cannot. Composition determines whether a 3D generation looks like a “complete world,” the rendering mechanism determines its ceiling, and the numeric constants in the source determine whether it actually honored its promises. For developers, the better investment is not chasing one-off stunning screenshots but building a dimension-scored, auditable, reproducible evaluation framework—one that not only tells you whether a single generation is good, but reveals the true direction in which model capabilities are evolving.