Testing LLM 3D Scene Generation: Strong First Impressions Meet Engineering Reality

1. Why Use a “Single-File 3D Scene” to Test a Large Model

LLM-generated front-end code has evolved from “it runs” to “it looks good.” One of the best tests for this evolution is a strictly constrained offline 3D scene: it requires the model to handle spatial composition, architectural modeling, materials and lighting, animation driving, and state coupling — all within a single file. Any weak link shows up immediately on screen.

Current model iteration follows two tracks: one is the “high-volume, low-cost” fast lightweight model, focused on low cost and low latency; the other is the frontier flagship route, with parameter scales heading toward trillions, context windows expanding to millions of tokens, and post-training investment concentrated on complex tasks like coding and 3D creation. The value of evaluating the latter is not in benchmark scores but in comparing outputs from two generations of the same model (referred to below as “old version” and “new version”) on the same task. That comparison tells us where real progress happened: in surface-level appearance, or in underlying engineering.

A note on evaluation hygiene: this test was conducted in a unified environment — same prompt, same “advanced” mode, fixed sampling parameters, and evaluation completed as early as possible after version release with version numbers recorded. By industry convention, online models may drift after release due to server-side adjustments. Testing early and fixing variables is the prerequisite for trustworthy conclusions.

2. Test Design: Writing Requirements as “Acceptance Specs”

The prompt used here contains four key points:

1. Form constraint: An offline single HTML file based on the native WebGL 2.0 standard, with no external images, models, or third-party libraries;
2. Scene elements: A giant ring-shaped water gate, a terraced sea city, and a risable tide;
3. Priority declaration: Establish composition, architecture, and materials first, then discuss animation;
4. Hard coupling constraint: The tide must not be just a blue plane moving up and down — ships, waterlines, and plaza flooding states must all change accordingly.

The difficulty lies in the “compound” nature of the task: it is not a single-point capability test but a triple examination of spatial narrative, rendering technique, and parameter self-consistency. Point 4 is a deliberately planted watershed — it elevates water level from a “texture height” to a global state driving the entire scene. If the model merely translates a blue plane, it reveals itself on this point.

The test design methodology can be summarized in three principles:

• Offline verifiable: The output is self-contained, runs without network access, and review depends on no environment;
• Explicit coupling: State exactly which derived changes the state should drive, leaving no ambiguity;
• Quantified acceptance: Key requirements carry numeric constraints (e.g., “most stone steps should be submerged at high tide”), turning review from “I think” into “I calculated.”

In the actual test, both model versions delivered a valid native WebGL2 single file with zero external dependencies. This means the single-file baseline is no longer a filter — the real gap lies beneath the visuals.

3. The First-Impression Battle: First-Screen Composition and Visual Hierarchy

The first screen is a static poster. The old version’s problem: the giant ring is large enough, but the mountains on both sides swallow most of the frame. Near hills and distant mountains fill the first screen layer upon layer, the city huddles deep in the bay, and both the building cluster and central plaza have weak legibility. Visually, it reads as “a ring-shaped object placed on terrain,” not “a world created.”

The new version’s progress shows in three areas:

• Primary-secondary organization: The giant ring, bridge arches, beacon tower, and terraced housing are arranged into a coherent visual flow — the ring is the protagonist, the rest are supporting elements;
• Layout logic: Buildings spread along both banks, and the central plaza is presented with clear concentric step layers, establishing a spatial narrative;
• Material layering: The coolness of oxidized copper contrasts legibly with the warmth of stone — at a glance you can read “this is a gate, this is a city.”

This layer can be scored structurally:

1. Is the main subject recognized at first glance;
2. Do secondary elements serve the subject rather than compete with it;
3. Do first-screen colors and materials establish clear layering;
4. Does the image convey the scene’s functional narrative (water gate + sea city + tide).

Conclusion: The new-generation model’s first-impression progress is real. Its ability to understand and orchestrate the prompt’s composition and material requirements has stepped up. But this is only the first page of the exam.

4. The Mechanism Watershed: Implementation Tiers for Water Reflections

Water is where the technical substance of this task lies, and reflection implementation roughly falls into three tiers:

• Tier one, “pasted on”: Analytic sky gradient, sun specular highlight, plus a few light streaks. Essentially screen-space procedural fakery — reflections have no spatial correspondence to onshore entities and fall apart when the viewpoint changes;
• Tier two, “computed”: Mirror camera approach — the city and ships are mirrored across the water plane and redrawn, then sampled into the water shading. Reflections have a real spatial source;
• Tier three, full physical wave solving: Mostly used in offline rendering; rarely adopted proactively by models in real-time scenes.

In testing, the old version stayed at tier one, while the new version reached tier two. This is a substantive upgrade at the mechanism level, not parameter tweaking for beautification. Conceptual pseudocode for building a mirror camera:

// Conceptual sketch: reflection camera based on planar mirroring
function buildMirrorCamera(cam, waterY) {
  const m = cam.clone();
  m.position.y = 2 * waterY - cam.position.y; // flip viewpoint across water plane
  m.forward.y *= -1;                         // mirror the forward direction
  m.up.y *= -1;                              // preserve coordinate system handedness
  return clipBelow(m, waterY);               // oblique clip to keep underwater geometry out of reflections
}

The cost of this mechanism upgrade must also be recognized: geometry draw calls nearly double, oblique frustum clipping is required, and reflection buffer resolution plus water normal perturbation sampling together determine wave-light quality. During review, the image alone can only tell you whether reflections exist; reading the source tells you whether they are painted or computed — the latter represents genuine mastery of the rendering pipeline.

5. The Real Test Is Back-End: Parameter Fulfillment and Consistency Checking

This was the most informative moment of the test. The prompt required that “most stone steps be submerged at high tide.” Extracting constants from both versions’ source code for static verification: let the first step level be BASE, each layer height be STEP_H, total layers be N, and maximum tide level be TIDE_MAX. The number of submerged layers at high tide is:

submerged = floor((TIDE_MAX − BASE) / STEP_H)

• New version: Nine concentric steps. Based on its source constants, only the lowest four layers sit below mean water level at high tide — “most submerged” was not fulfilled;
• Old version: First-screen appearance clearly loses, but its maximum tide level actually covers the outermost steps — acceptance passed.

// Static acceptance: does high tide submerge enough layers? (constants extracted from generated source)
const BASE = 2.0, STEP_H = 1.2, STEPS = 9, TIDE_MAX = 6.0;
const submerged = Math.floor((TIDE_MAX - BASE) / STEP_H);
console.assert(
  submerged >= Math.ceil(STEPS * 2 / 3),
  `High tide only submerges ${submerged} layers, failing "most stone steps submerged"`
);

This result reveals the most important rule when evaluating LLM code generation: visual advantage and engineering rigor can be completely decoupled. A model that can arrange a picture may not be able to do the math. The version with a stunning first screen may be exactly the one that fails on “invisible parameters.” Therefore the evaluation framework must include a quantifiable “numeric constraint fulfillment” dimension. This also places direct demands on prompt engineering: write acceptance criteria as numbers (submerged layer ratio, water level difference, bridge clearance height), so review has something to stand on. You can also add runtime assertions that output “submerged layers” to a debug panel in real time — giving you a judge beyond the human eye.

6. Correct Tide Design: Water Level Is Global State, Not Texture Height

The essence of prompt point 4 is requiring the model to implement the tide as a state system driven by a “single source of truth.” A qualified water level system should include the following derived couplings:

• Ships: Rise and fall with water level, draft remains relatively stable, anchored sway can be added;
• Waterline: Shoreline wet-mark height and shore material darkening range follow water level;
• Plaza: Steps submerge and emerge layer by layer, wet surfaces show temporary enhanced reflectivity;
• Indicators: Beacon tower or water gauge readings match the actual water level.

// Conceptual sketch: coupled updates with water level as the single source of truth
function onTideChange(t) {                  // t: 0(low tide) ~ 1(high tide)
  const y = lerp(TIDE_MIN, TIDE_MAX, t);    // Derived 1: water level height
  water.setHeight(y);
  ships.forEach(s => s.setBuoyancy(y));     // Derived 2: ship buoyancy
  plaza.setFloodedSteps(stepsBelow(y));     // Derived 3: submerged step count
  shore.setWetLine(y);                      // Derived 4: wet mark and material darkening
  gauge.setReading(y);                      // Derived 5: gauge reading
}

During review you can check the source directly: if the model writes water level as a global parameter referenced by multiple subsystems in derived form, the architecture is correct. If water level appears only once as the y coordinate of the water mesh, with everything else relying on visual coincidence, then no matter how good the visuals are, it is a compliant-looking disguise for “raising and lowering a blue plane.”

7. Turning Method into an Evaluation System: Six Recommendations for Teams

1. Score in layers; avoid deciding by a single image: Record separately across four layers — first-screen appearance (structured checklist), rendering mechanism (source-code inspection to locate implementation tier), parameter fulfillment (constant extraction + assertions), and performance stability (frame rate, VRAM, long-run behavior);
2. The prompt is the acceptance spec: Attach a decidable condition to every requirement; never leave room for subjectivity when quantification is possible;
3. Automated regression: Source constant extraction scripts, runtime assertions, and key tide-level screenshot comparisons together form a sustainable evaluation asset;
4. Fix variables, record everything: Log model version, mode, and sampling parameters, and complete the first evaluation round early after release to prevent later server-side adjustments from drifting conclusions;
5. Watch trends, not leaderboards: Training environments change — for example, shifts in distillation gains directly affect the magnitude of improvement in a new generation or even regression in individual dimensions. Continuously testing against the same benchmark reveals direction better than chasing leaderboards. For model selection, lightweight fast models suit high-frequency low-cost scenarios, while frontier flagship models are better for capability-ceiling tasks like long-horizon coding and complex 3D creation;
6. Connect the engineering pipeline: Approved outputs need stable hosting and distribution. General cloud capabilities such as object storage, content delivery, monitoring and alerting, and elastic compute — for example, those offered by Baidu AI Cloud — can help turn a “single-file page” into a sustainably deliverable application asset.

Conclusion

First-impression progress is real and intuitive: the new-generation model can already orchestrate a ring-shaped object into a layered, narrative world. But the real reflections from the mirror camera cannot cover the failure to submerge most steps at high tide — that is a signal from the model’s back-end rigor. Evaluators should upgrade their methods in step: from looking at images, to reading code, to doing the math. After all, first impressions make an exam shine, but what makes a project deliverable with confidence has always been the things you cannot see.