Testing 3D Scene Generation in Large Models: The Gap Between First Impressions and Correctness

1. The Benchmark: A Test That Simultaneously Challenges “Perception” and “Constraints”

To evaluate a large model’s ability to generate 3D content, the benchmark itself must be sufficiently demanding. The task used in this test: generate a single-file HTML page that can be opened offline, using the browser’s native 3D graphics API (WebGL2), to implement a harbor city—featuring a giant ring-shaped water gate, terraced seaside buildings, and a tide system with adjustable water levels. Three hard constraints were imposed:

• No external images, model files, or third-party libraries are allowed. All geometry, materials, and lighting must be built from scratch in code.
• The composition, architecture, and materials must be visually credible—the first screen must deliver a complete landscape narrative.
• The tide cannot be a simple blue plane moving up and down—ship buoyancy, shoreline watermark positions, and the submersion state of plaza steps must all be linked to the water level.

The value of this task lies in its simultaneous coverage of two qualitatively different capabilities: the former tests “perceptual quality”—composition, color, layering—which determines first impressions; the latter tests “constraint satisfaction”—numerical relationships, state linkage—which determines whether the result holds up under scrutiny. Extensive testing shows that model performance across these two capabilities is not synchronized, and this is precisely the gap that demo effects tend to mask when selecting and deploying models.

Two methodological points deserve attention. First, old and new model generations must be compared within the same time window, using identical prompts and identical reasoning modes—because server-side policies change over time, and comparing across different windows introduces systematic bias. Second, beyond inspecting the rendered output, one must also open the generated source code and perform numerical verification—this step is missing from many evaluation pipelines, and its value will be demonstrated below.

2. First Impressions: Camera Composition Determines 80% of Visual Appeal

The “first impression” of a 3D scene is largely determined by the default camera framing—three factors matter most: subject proportion, visual guidance, and depth density.

In the previous generation’s output, the ring-shaped water gate was large enough and centered, but the surrounding mountains occupied more of the frame: the initial screen was filled with nearby slopes and distant peaks, with the city shrinking into the depths of the bay. The building cluster and central plaza were difficult to identify. The viewer’s gaze was drawn away by the terrain, and the main subject required effort to locate—a classic case of “subject present, hierarchy misplaced.”

The new generation, by contrast, organized the giant ring, bridge archways, navigation towers, and terraced housing into a coherent landscape: buildings spread along both banks, the central plaza featured clear concentric step levels, and the contrast between oxidized copper’s cool tones and stone’s warm tones was unambiguous—one could immediately distinguish “protagonist” from “supporting cast.” Both outputs contain the same ring structure, but the former merely “places a single object on terrain,” while the latter “builds a complete world.”

It is worth emphasizing that this kind of compositional ability is not mystical—it corresponds to trainable visual patterns: subject area ratio, foreground–midground–background depth layering, and the distributional balance of warm and cool color blocks. The new generation received stronger alignment on such signals during post-training, which is the direct reason for its superior ability to shape first impressions.

3. Water Rendering: From “Fake Reflection” to “Mirror Camera”

The difference in water handling between the two outputs reveals more about “implementation mechanism” than composition does.

The previous generation used an analytical approximation: sky gradients, a sun specular spot, and stretched light streaks to “suggest” the presence of reflections. This approach costs almost nothing to render and can pass visual inspection, but it has a fatal flaw—buildings on the shore are not “reflected” in the water. The reflections and the actual geometry have no spatial correspondence, and close inspection exposes the illusion.

The new generation introduces a proper implementation of planar reflection: the mirror camera technique. Its workflow is as follows: clone the main camera, apply a mirror transformation along the water plane; use a clipping plane to cull geometry below the water surface; render the mirrored view into an off-screen texture; finally, when shading water fragments, project screen coordinates onto that texture for sampling. A simplified code sketch:

// Core planar reflection workflow (pseudo-code)
const mirrorCam = mainCamera.clone();
mirrorCam.applyReflection(waterPlane);              // mirror camera across water plane
const reflectTex = renderToTexture(scene, mirrorCam, {
  clipPlane: waterPlane                             // cull geometry below water
});
waterMaterial.uniforms.reflectionMap = reflectTex;  // projective sampling during water shading

With this approach, every building and every boat has a true mirrored counterpart in the water—the reflection now has a spatial origin. The cost is that the scene must be rendered one additional time per frame, nearly doubling draw overhead. In practice, this is typically balanced by reducing the reflection texture resolution and narrowing the reflection frustum. This is an upgrade at the implementation-path level, not a parameter-tuning improvement—for assessing whether a model truly “understands” rendering principles, such mechanistic evidence is far more convincing than a final screenshot.

4. Correctness Regression: Numerical Oversight at Maximum Tide Level

However, on the “constraint satisfaction” side, the new generation shows a clear regression.

The task explicitly requires that “most stone steps are submerged at high tide.” Performing source-level verification of the generated output: read the step count, individual step height, and maximum tide constant from the code, then do a simple division. The result shows that, at maximum water level, only the lowest four of nine plaza steps fall below average water level—“most steps submerged” is not satisfied. The previous generation, whose first-screen appearance was less impressive, actually covered the outermost steps within its maximum tide range.

This case reveals a general pattern: models prioritize “looking correct,” while numerical consistency is the most likely element to be sacrificed. Three reasons stand out:

• Training signals are visually biased. Human feedback primarily comes from “a single look”—preference signals for composition, color, and lighting are dense, while “whether the step submersion ratio meets the requirement” almost never enters explicit feedback.
• Quantitative conditions are easily semanticized. Constraints like “most stone steps submerged” tend to be translated into a “feels about right” water level parameter, rather than establishing a precise numerical relationship between step height and tide level.
• Linked constraints form a long-tail path. Water level changes must simultaneously drive boat buoyancy, watermark position, and submersion state—any constant offset in this call chain is easily masked by the rich visual effects of the water surface.

5. Why Progress and Regression Appear Simultaneously

Zooming out, this paired “one up, one down” reflects a structural shift in current model iteration.

On one hand, as distillation’s marginal returns diminish, generation-over-generation differences increasingly depend on the design orientation of post-training objectives, rather than “imitating a stronger model at a larger scale.” When teams channel post-training resources toward directions like code and 3D creation—areas with strong demo impact and external evaluation signals—perceptual quality improvement is a natural outcome.

On the other hand, physical consistency and parameter linkage—“low-frequency, long-tail” constraints—lack dense feedback sources and cannot be easily inherited through distillation, making them prone to trade-off regressions. In an industry increasingly favoring lightweight, high-frequency, low-cost small models, teams that persist in pushing the frontier become more exposed to such structural trade-offs. This also reminds users: demos cannot reveal true differences between models. You must build your own evaluation set around your specific tasks, and the evaluation dimensions must cover both perceptual and constraint-related aspects.

6. Engineering Practice: Turning “First Impressions” into Verifiable Deliverables

For developers, the following practices can turn the above insights into concrete workflow:

1. Prompt layering. Separate visual requirements (composition, materials, lighting atmosphere) from constraint requirements (numerical relationships, state linkage). Where possible, express constraints as computable conditions—e.g., “at least 60% of steps below water at high tide”—rather than “mostly submerged.”
2. Four-category acceptance checklist. Score each category independently: composition (subject ratio, depth hierarchy), materials (warm/cool, semantic distinction), mechanism (reflection implementation, animation drive chain), and numerical values (key constant verification). This prevents overall visual appeal from hijacking judgment.
3. Static validation as a safety net. Write assertion scripts against key constants in the generated code, for example:

# Static verification of tide parameters in generated code (illustrative)
import re
def extract_const(code: str, name: str) -> float:
    m = re.search(rf"{name}\s*=\s*([\d.]+)", code)
    return float(m.group(1)) if m else None
step_h   = extract_const(gen_code, "STEP_HEIGHT")  # individual step height
steps    = extract_const(gen_code, "STEP_COUNT")   # number of steps
max_tide = extract_const(gen_code, "MAX_TIDE")     # maximum tide level
submerged = int(max_tide // step_h)
assert submerged / steps >= 0.6, "Most steps not submerged—acceptance failed"

1. Staged generation and regression. First have the model settle the implementation mechanism (reflection approach, animation drive structure), then iterate on parameters and art details. For each iteration, fix the prompt and randomness, and run the same validation script—preventing “increasingly good-looking, increasingly incorrect” outcomes.
2. Cloud-based evaluation pipeline. The generated single-file page can be hosted on object storage, distributed via CDN to reviewers, and headless browsers can be launched in container services or function compute for screenshot comparison and assertion checks. Such pipelines can be assembled using common cloud platform capabilities—no dedicated infrastructure required.
3. Same-window comparison with logs. Version comparisons must be performed within the same time window. Record prompts, reasoning modes, and generation timestamps to avoid interference from server-side policy changes.

7. Conclusion: After the First Impression, the Real Test Is in the Backend

The conclusion of this evaluation can be condensed into one sentence: the new generation is indeed better at “delivering first impressions”—composition is more disciplined, and reflection mechanisms have upgraded from imitation to proper implementation. But the numerical oversight on the constraint side reminds us that progress in perceptual ability does not automatically translate into progress in correctness. For users, evaluating whether a model is suitable for production requires looking beyond “impressive screenshots” and asking whether it can pass your own defined validation scripts. For model teams, the weight assigned to physical consistency and long-tail constraints in post-training signals will determine whether the next iteration is comprehensive progress or yet another trade-off. The bar for first impressions is rising quickly, but the battlefield where real differentiation happens is shifting to the “backend”—the places invisible in screenshots yet decisive for delivery quality.