Evaluating LLM-Generated 3D Web Scenes: Easy First Impressions, Hard Semantic Delivery

1. From “Can Render” to “Renders Correctly”: Upgrading Evaluation Standards

Asking Large Language Models (LLMs) to generate a fully runnable 3D web scene is now a common method for testing their front-end engineering capabilities. The superficial threshold for this task is producing a visual output, but the deeper challenge spans four layers: spatial composition, rendering pipeline, animation logic, and parameter semantics. Recent model iterations show a clear trend—models are improving rapidly in “first impressions”: the initial composition is more complete, material contrasts are more refined, and water reflections look more realistic. However, precisely mapping quantitative constraints from prompts into code parameters still frequently deviates. In other words, visual aesthetics are quickly approaching their ceiling, while the ability to deliver on semantics lags.

Therefore, evaluating a model iteration cannot stop at screenshot comparisons. Using a typical generational comparison test—assigning the same complex 3D task to a previous-generation and a next-generation model—this article distills a reusable three-layer evaluation framework and breaks down the assessment methods layer by layer.

2. Evaluation Design: Forcing Out True Capabilities with High Constraint Density

The task is defined as follows: Require the model to produce an offline, runnable, single-file HTML scene containing three elements: a giant ring-shaped water lock, a terraced coastal city, and a controllable tide. Beyond establishing the composition, architecture, and materials, the prompt adds a dynamic semantic constraint: the rising and falling tide cannot merely move a flat blue plane up and down. The posture of boats, the waterline position, and the submerged state of the plaza must all react correspondingly.

This design has three intentions:

1. Single-file, zero-dependency: Prohibiting external images, models, and third-party libraries blocks the shortcut of “asset assembly.” The model must build from scratch using native WebGL2 and shaders, exposing its true mastery of the rendering pipeline.
2. Static and dynamic constraints weighed equally: Composition and materials test the “first impression,” while the submersion linkage tests animation semantics. These two often perform asynchronously, perfectly distinguishing capability levels.
3. Leaving auditable hooks: Natural language constraints like “most stone steps are submerged” can be reverse-calculated from geometric parameters in the source code, preventing the evaluation from becoming purely subjective scoring.

Controlling variables is equally important: both generations use the same parameter settings and inference modes, completed within the same time window. Online inference services adjust strategies over time; stretching the comparison period risks misinterpreting server-side changes as model differences.

3. Dimension One: Scene Organization and Visual Hierarchy—The Watershed of First Impressions

Both model generations met the hard requirements: native WebGL2 single-file, no external dependencies, and a functional tide. What truly separated them was the scene organization under the default camera.

The previous generation exhibited a representative issue: the giant water lock was appropriately sized, but the mountains on both sides occupied too much of the frame. Foreground slopes and distant peaks filled the initial screen, pushing the city deep into the bay, resulting in insufficient visibility for the buildings and plaza. The outcome was “a landmark placed in the center, but the narrative subject requires effort to find.”

The new generation demonstrated significantly stronger organizational skills. The ring-shaped water lock, bridge arches, beacon towers, and stepped residences were organized into a coherent visual flowline. Buildings spread along both shores, and the central plaza displayed clear concentric stepped layers. The cold tones of oxidized copper contrasted clearly with the warm tones of stone. It delivered not just “a ring placed on terrain,” but a complete world with primary and secondary focal points.

This dimension can be broken down into verifiable sub-items, preventing “looks good” from being a purely subjective judgment:

• Subject proportion and frame focus: Does the core landmark occupy effective screen space?
• Element correspondence: Do auxiliary elements (beacon towers, bridge arches) form an axis or visual guide with the main subject?
• Material distinction: Are the cold/warm and brightness relationships between different materials clearly distinguishable?
• Narrative completeness: Does the scene read as “single object + terrain,” or an organic whole composed of multiple subsystems?

4. Dimension Two: Implementation Depth of Rendering Mechanisms—From “Approximate Simulation” to “Mechanism Upgrade”

Water surfaces are the most demanding part of such tasks because “looking like it has reflections” has two implementation tiers.

The lower tier is approximate simulation: using analytical skies, specular highlights, and light bands dragged across the water to create the illusion of reflection. The previous generation took this route. It is cost-effective, but the reflections lack spatial origins—light spots might appear on the water for buildings that don’t exist on the shore.

The higher tier is planar mirror reflection: introducing a virtual camera mirrored along the water plane, completely redrawing the shore’s city and boats, rendering them to an offscreen texture, and then sampling it onto the water material. The new generation implemented exactly this, giving the shore buildings’ reflections a true spatial correspondence. The core logic can be summarized as:

// Core steps for planar mirror reflection (illustrative)
function renderWaterReflection(scene, camera, waterPlane) {
  // 1. Derive the mirror camera of the original camera relative to the water plane
  const mirrorCam = reflectCameraAboutPlane(camera, waterPlane);
  // 2. Compute the water clipping plane to keep only geometry above the water
  const clipPlane = computeClipPlane(waterPlane);
  // 3. Render the scene to an offscreen texture using the mirror camera
  renderToTexture(scene, mirrorCam, clipPlane, offscreenRT);
  // 4. The water shader samples this texture to get a spatially accurate reflection
  waterMaterial.setReflectionTexture(offscreenRT);
}

Why is this a “mechanism upgrade” rather than “parameter tuning”? Because the former only requires adjusting a few shader constants, while the latter demands the model truly understand the rendering pipeline’s hierarchy: mirror transformation, clipping planes, offscreen rendering, and texture sampling. Missing any link breaks the chain. It also comes with a real cost—draw calls nearly double, and improper clipping settings can cause visual glitches. Successfully executing this entire chain in a single file indicates the model’s mastery of the graphics tech stack has reached a new level.

5. Dimension Three: Precise Fulfillment of Parameter Semantics—The Real Test is on the Backend

The most interesting reversal appeared in the tidal parameters. The prompt required: at the highest tide level, most of the plaza’s stone steps should be submerged.

During acceptance, one cannot rely solely on screenshots; the semantic constraint must be translated into an auditable geometric problem: read the number of steps, individual step height, and base elevation from the generated source code, then read the maximum water level of the tide animation, and cross-verify them. The result: the new generation’s nine-layer concentric steps, calculated at the highest water level, only had the bottom four layers below the average water level—“mostly submerged” was not fulfilled. Conversely, the previous generation, which lacked in initial composition, covered the outermost step at the highest water level.

The audit process can be written as an executable assertion:

// Acceptance audit (illustrative)
const stepCount = 9, stepHeight = 0.35, baseY = 0.0;
const crestY = 1.2; // Maximum water level read from source code animation parameters
const submerged = countStepsBelow(baseY, stepHeight, stepCount, crestY);
console.assert(submerged / stepCount >= 0.6,
  `Highest tide should submerge most steps, actual ${submerged}/${stepCount}`);

This case exposes the “disconnection between visual expression and numerical semantics.” The model likely understood the visual meaning of submersion—wet banks, waterline贴面—but failed to convert the vague quantifier “most” into the geometric relationship of “water level elevation is higher than the top of the Nth step.” Aesthetics can be acquired through massive high-quality examples, but quantitative fulfillment requires the model to continuously maintain an implicit unit conversion and geometric reasoning during generation. The training signals for the latter are evidently sparser. This is the structural reason behind “easy first impressions, hard backend fulfillment.”

6. What the Differentiation Across Three Dimensions Tells Us

Placing the results of the three dimensions side-by-side reveals the capability profile of a typical iteration:

Dimension	Evaluated Content	Iteration Performance
Scene Organization	Composition, hierarchy, materials	Significant progress
Rendering Mechanism	Reflection, shading, pipeline	Upgraded from approximate to true mechanism
Semantic Fulfillment	Parameter and constraint consistency	Regression occurred

Three takeaways:

1. The three layers are independent of each other. Strong composition does not equal deep rendering, and deep rendering certainly does not equal accurate parameters. Any single-dimensional conclusion can mislead technical selection.
2. The fastest progress is often in “imitable” parts. Visual organization has海量 high-quality examples to learn from, while quantitative semantic fulfillment relies on the geometric translation of natural language constraints, progressing slower and more prone to fluctuation.
3. Regression is also a signal. Given continuously changing training environments and data formulations, a regression in a specific dimension for a certain generation often reveals more about the true source of its capability than progress does—whether it relies on example memory or reasoning.

7. Practical Checklist: Crystallizing Evaluation into a Process

Prompt Level:

• Replace vague quantifiers with numbers and ratios: “submerge about two-thirds of the steps” is better than “most steps submerged.”
• Explicitly require key geometric and animation parameters to use named constants for easier auditing.
• Conduct phased acceptance: finalize composition first, then review animations, and finally test edge cases (highest tide, lowest tide).

Acceptance Level:

• Establish a two-tier acceptance of “screenshot comparison + source code audit”: aesthetics via multi-state screenshots, semantics via parameter assertions.
• Translate natural language constraints line-by-line into executable validation scripts, incorporating them into regression tests so each model iteration has a trackable baseline.

Engineering Level:

• Solidify an evaluation set and baseline cases, running the same tasks for each model iteration to ensure long-term trackability of comparison results.
• The automated pipeline can easily be built on general cloud capabilities: use a container platform to run headless browsers for batch rendering, object storage to archive screenshots and source code, and logging services to record metrics and assertion results. For instance, an equivalent pipeline can be assembled cost-effectively using general cloud services.
• Comparison tests must be completed within the same time window with fixed parameter settings to avoid misinterpreting server-side strategy adjustments as model capability changes.

Conclusion

First impressions determine the “demo value” of a generation, while semantic fulfillment determines its “engineering value.” The former is rapidly becoming a standard capability across models, while the latter is where true differentiation lies. For developers, evaluation must also walk on two legs: check the initial screen’s composition and lighting, while translating every natural language constraint into an auditable assertion. Only outputs that pass both checkpoints can truly be considered usable and trustworthy.