How to Scientifically Evaluate a Large Model's 3D Scene Generation Capability

1. Why 3D Scene Generation Is a Litmus Test for Model Quality

In front-end development, 3D generation tasks test a model on many fronts at once: spatial imagination (how to organize a city, terrain, and water bodies), graphics fundamentals (cameras, lighting, render targets), mathematics (projection matrices, wave functions, tide-level interpolation), and engineering organization (hundreds or thousands of lines of code that must remain clearly structured with consistent variables). Compared with localized tasks like “write a sorting algorithm,” it is much closer to long-horizon programming in real business scenarios.

The choice of test vehicle matters just as much. An “offline single-file HTML + native WebGL2” format is recommended as the standard exam paper, for three reasons:

• Zero dependencies: External images, model files, or third-party libraries are not allowed. This removes ecosystem boosts and tests only the model’s own knowledge density and code organization ability.
• Easy to reproduce: The output runs with a double-click. Any evaluator who receives the same file sees the same result.
• Auditable: All source code is visible. You can check both whether the result “looks right” and whether the implementation is “actually correct” line by line.

2. Designing a Reproducible Controlled Experiment

A single question and a single answer prove nothing. Reliable evaluation must be built on controlled comparative experiments.

1. Use Case and Prompt Design

Consider a typical use case: generate a coastal city that includes a giant ring-shaped tidal gate, terraced plazas, and a tide that can rise and fall. Prompt design matters more than one might expect, and there are three key points.

First, constrain the “worldview” before discussing details. Explicitly require the model to establish the primary/secondary composition, building massing, and warm/cool material contrast first. Otherwise, the model tends to pour all its effort into a single subject while glossing over everything else.

Second, explicitly block “lazy paths.” The most perfunctory way to implement tide animation is to raise and lower a blue plane. This must be explicitly forbidden, and the prompt should require that boats, waterlines, and plaza flooding states change in coordination with the tide level.

Reference prompt:

Please implement a coastal city in a single offline HTML file
(native WebGL2, without referencing any external images,
models, or third-party libraries):
1. The scene includes a giant ring-shaped tidal gate, terraced plazas,
   and coastal building clusters. First establish the primary/secondary
   composition, building massing, and warm/cool material contrast.
2. The tide can rise and fall; during high and low tide, boats, waterlines,
   and plaza flooding states must change in coordination.
   Do not fake it by "raising and lowering a blue plane."
3. At high tide, most of the stone steps should be below the average water level.

Third, convert soft descriptions into hard metrics. Statements like “most of the stone steps are submerged” must later be reducible to checkable numeric conditions (see Section 5).

2. Comparison and Sampling Protocol

• Submit the same prompt to old and new generations of models separately, keeping all other conditions fixed: same inference mode, same parameter settings.
• If the API supports it, fix sampling parameters such as temperature. Run multiple sampling rounds for each item so that a single lucky run is not mistaken for capability.
• Complete tests within the same time window as much as possible. Model services undergo dynamic adjustments, and the same model may drift over time; cross-window comparisons can easily introduce noise.

3. First Dimension: First-Screen Composition and Scene Organization

The first glance at a generated file tests the model’s understanding of “visual hierarchy”—the dimension most easily overlooked and yet the one that most separates good from mediocre results.

Two typical failure modes:

• Imbalanced hierarchy: The main subject has large massing, but the terrain on both sides occupies more of the frame. The first screen is filled with nearby hillsides and distant mountains, while the city is tucked deep into the bay with weak recognizability. The subject is centered, but the viewer has to work hard to identify it.
• Isolated elements: The ring structure is merely “placed” on the terrain with no narrative connection to its surroundings. The whole thing looks like a stack of objects rather than a coherent world.

The corresponding excellent performance looks like this: the main subject (the giant ring), transitional elements (bridge arches, beacon towers), and the background (layered residences) are organized along the waterfront into a coherent landscape; the central plaza has clear concentric step layers; and the materials show a warm/cool contrast between copper patina and stone. Such output indicates that the model has internalized design knowledge like “composition,” rather than mechanically piling up geometry.

Evaluation tip: When comparing screenshots, don’t look at rendering precision first. Ask three questions—where does your gaze land in the first second? Is the subject integrated with its environment? If you remove the subject, does the image still hold up?

4. Second Dimension: Implementation Depth of the Rendering Mechanism

Two versions that look similar may differ radically in implementation. Water reflection is a classic probe for this dimension.

Approach 1: Analytical approximation. In the fragment shader, use sky color, sun highlights, and light smears to “simulate” reflections:

vec3 R = reflect(-viewDir, vec3(0.0, 1.0, 0.0));
float sun = pow(max(dot(R, sunDir), 0.0), 128.0);
vec3 refl = mix(skyHorizon, skyZenith, clamp(R.y, 0.0, 1.0))
          + sunColor * sun;   // reflection color is unrelated to the actual scene

This approach is cheap, but the reflection is fabricated—what appears in the water has no real correspondence to the scene on shore.

Approach 2: Mirror camera. Flip the camera across the water surface, redraw the entire scene into a texture, and then sample it when shading the water:

const mirrorCam = camera.clone();
mirrorCam.position.set(
  camera.position.x,
  2 * waterLevel - camera.position.y,   // flip across the water surface
  camera.position.z
);
renderer.setRenderTarget(mirrorRT);
renderer.render(scene, mirrorCam);      // draw the mirrored scene first
renderer.setRenderTarget(null);         // then sample onto the water

The essential difference with the mirror camera is that reflections of onshore buildings and boats now have a real spatial source and correspond one-to-one. This is an upgrade at the implementation-mechanism level, not a cosmetic tweak at the parameter-tuning level.

Evaluation insight: Screenshots can only answer “does it look right”; reading the source code answers “is it correct.” Mechanism depth is a better predictor of a model’s capability ceiling on more complex tasks.

5. Third Dimension: Constraint Fulfillment and Numerical Correctness

This is the part most easily masked by a beautiful first screen. A good-looking scene does not mean the prompt was fully fulfilled.

A real counterexample emerged during testing: the prompt required that “at high tide, most of the stone steps are submerged.” In one version with an excellent first screen, extracting the number of step layers, the per-step height, and the maximum water level from the source code and calculating revealed that at high tide, only the lowest four of nine steps were below the average water level. Meanwhile, another version with a less impressive first screen extended the maximum water level to cover the outermost steps.

The calculation is not complicated:

steps = 9        # number of plaza step layers
step_h = 1.2     # height of a single step
base_y = 0.0     # elevation of the lowest step
max_tide = 4.0   # high-tide water level
flooded = sum(1 for i in range(steps) if base_y + i * step_h < max_tide)
print(f"Steps submerged at high tide: {flooded}/{steps}")
# Output 4/9: does not meet the "most are submerged" requirement

This reveals a general pattern: models are not equally capable at “generating a scene that looks plausible” and “making scene parameters strictly satisfy constraints.” The former relies on pattern generalization; the latter requires global numerical consistency and is harder.

An engineering approach is to build a mapping table from “prompt requirement → verification method” and check each item off:

Prompt requirement	Verification method
Most stone steps submerged at high tide	Extract step and water-level constants, verify with a script
Boats rise and fall with the tide	Check whether boat height is bound to the tide-level variable
Waterline changes along building walls	Check whether the waterline references the same tide-level source
Single file, zero dependencies	Scan for imports and external resources

6. From a Single Test to a Sustainable Evaluation System

One comparison only produces talking points; a systematic evaluation is what supports technology selection. Consider building it up along these lines:

1. Version regression mindset: Fix a set of use cases and rerun them after each model iteration. Focus not only on “progress” but also on “regressions”—it is common for a new version to regress on some sub-capability.
2. Dimension-based scoring: Score visual composition, rendering mechanism, physical consistency, constraint fulfillment, and code quality separately. It is normal for a model’s performance to diverge across sub-tasks; a single total score only obscures the details.
3. Assetize verification scripts: Preserve the numeric verification scripts and dependency-scanning scripts from Section 5, and combine them with prompts to form a repeatable evaluation suite.
4. Make environment control routine: Fix the time window, fix parameters, run multiple sampling rounds, and stay sensitive to dynamic adjustments on the service side.

7. Conclusion

For developers, rather than chasing scores on general leaderboards, it is better to construct “verifiable hard tasks” around your own real business: write requirements as prompts with hard constraints, turn constraints into calculable check items, and solidify checks into regression-ready scripts. Whether you use open-source models or model services accessed through platforms such as Baidu AI Cloud, this evaluation framework of “constraint decomposition + source-level inspection + numerical verification” can be reused directly. What truly determines technology selection is never a single impressive first screen, but the capability boundaries that only surface through line-by-line inspection and item-by-item calculation.