AI 3D Frontend Generation Test: First Impressions Are Easy, Consistency Is Hard

1. Why Use a “3D Coastal City” as the Litmus Test

Large language models’ frontend generation capabilities are moving beyond well-established patterns like “build a login page” into complex scenarios that demand spatial reasoning and system design. We chose an “offline single-file 3D coastal city” as a unified test for three reasons:

1. Clean environment, controllable variables. The output must be a native WebGL2 single file with no external images, models, or third-party libraries. This eliminates interference from network and resource loading, attributing all differences entirely to the model itself.
2. One task stresses three capabilities. Spatial aesthetics (composition, architecture, materials), engineering organization (module partitioning and code structure within a single file), and numerical consistency (whether parameters like tide levels and step heights match the prompt).
3. Acceptance criteria can be quantified. Requirements like “most stone steps are submerged” can be objectively verified by reading the source code, avoiding subjective judgments of “it all looks fine.”

The prompt was designed in three layers—a method worth reusing for similar evaluations:

• Positive hard constraints: Establish composition, architecture, and materials first, making clear what should dominate the frame and what should be secondary.
• Behavioral constraints: During tide changes, boats, waterlines, and plaza submersion states must all change in a coordinated way.
• Negative constraints: Tide changes cannot rely solely on raising and lowering a blue plane.

Negative constraints are especially critical—they block the model’s laziest implementation path and force it into genuine system design.

One more caveat about the test environment: online model behavior may be dynamically adjusted over time, leading to version drift. For reproducible comparisons, lock a version snapshot via API, fix the inference mode and sampling parameters, rather than relying solely on web endpoint selection.

2. First Impressions: Composition Is the First Divide

Both model versions met the basic requirements: native WebGL2, single file, zero external dependencies. What truly separates them at a glance is how the default camera organizes the scene’s primary and secondary elements.

The older model’s giant ring is substantial in size, but the mountains on either side consume more of the frame: much of the first screen is filled with nearby slopes and distant peaks, while the city shrinks deep into the bay, making the buildings and plaza hard to identify. The result is “the giant ring stands at center frame, but the coastal city requires effort to discern”—the subject exists, but it doesn’t form a scene.

The newer model organizes the giant ring, bridge arches, beacon tower, and terraced housing into a coherent visual band: buildings spread along both banks, the central plaza has clear concentric step layers, and the warm-cool contrast between oxidized copper and stone is easy to distinguish. It feels closer to the “complete world” the prompt asked for, rather than the stiffness of “placing a single ring-shaped object on terrain.”

Behind this lies a difference in two capabilities:

• Placing objects: The model can generate a geometrically correct object and put it into a scene.
• Organizing a world: The model understands visual flow, information hierarchy, and spatial narrative—it knows what the user should see first.

The prompt engineering takeaway: if you care about first-screen impression, don’t just describe “what’s there”—describe “how to look at it.” Specify the frame ratio between subject and background, the initial camera position, and the desired visual hierarchy. Making composition requirements explicit is the most effective way to get the model’s “first impression” right.

3. Mechanism Upgrade: Reflections Go from “Makeup” to “Computation”

Water surfaces are the ultimate test for this kind of scene. The two model versions differ substantially in how they implement reflections:

• Older version: Primarily uses analytic sky, sun specular highlights, and light trails to simulate reflections. This is essentially a screen-space approximation—it “looks like” a reflection, but the reflected image lacks a real spatial source.
• Newer version: Additionally introduces a mirror camera, redraws the city and boats, and samples the result onto the water surface. Reflections of shoreline buildings now have a real corresponding spatial source—an upgrade at the implementation mechanism level.

A mirror camera is a classic technique for water surface rendering. Illustrative code:

// Two-pass rendering: draw reflection first, then main view (illustrative)
const mirrorCam = mainCamera.clone();
mirrorCam.projectionMatrix.elements[5] *= -1; // Flip Y axis across water plane
renderer.autoClear = false;
// Pass 1: Render scene from mirrored viewpoint into reflection buffer
renderer.setRenderTarget(reflectRT);
renderer.render(scene, mirrorCam);
// Pass 2: Render from main viewpoint; water shader samples reflection texture with distortion
renderer.setRenderTarget(null);
waterMat.uniforms.reflectTex.value = reflectRT.texture;
renderer.render(scene, mainCamera);

Here’s an evaluation methodology point: Screenshots can lie; implementation mechanisms don’t. Static screenshots of the two water surfaces may look similar, but mechanistic differences amplify during camera movement, object addition/removal, and time changes—screen-space approximations miss objects outside the viewport, while mirror rendering does not. Therefore, evaluating model-generated frontend code requires not just looking at render output but reading the implementation: only code with correct mechanisms can transfer to other materials and other scenes.

4. The Real Divide: Parameter-Level Instruction Following

The newer model, which leads in both visual impression and mechanism, stumbled on the highest tide level parameter.

The prompt required “most stone steps are submerged.” But calculating from the step height and maximum water level in the source code: of the newer version’s nine plaza step layers, only the lowest four are below mean water level at high tide—not a majority. The prompt requirement was not fulfilled. Conversely, the older version—which was less impressive on first screen—covered the outermost steps with its maximum water level range.

This kind of calculation isn’t complex; a script of a dozen or so lines can do it:

// Verify high-tide submersion range from source code constants (illustrative)
const STEP_COUNT  = 9;     // Plaza step layers
const STEP_HEIGHT = 0.35;  // Height per step layer (meters)
const MAX_TIDE    = 1.2;   // Maximum tide level from source code
const submerged = Math.floor(MAX_TIDE / STEP_HEIGHT);
const ratio = submerged / STEP_COUNT;
console.log(`High tide submersion: ${submerged}/${STEP_COUNT} layers (${(ratio * 100).toFixed(0)}%)`);
// If "most submerged" is required, ratio should exceed 0.5

This case exposes a common shortcoming of current generative models: visual presentation and numerical self-consistency are decoupled. Models excel at narrative descriptions (“the plaza is grand,” “waves lap the shore”), but when translating constraints into specific numbers and keeping those numbers consistent throughout long code, they struggle—when writing water parameters, the step height constraint set earlier gets overridden by local plausibility.

This is precisely the core difficulty of long-horizon coding tasks, and what “after first impressions, the real test is in the backend” means: parameters users can’t see determine whether the final product can actually be delivered. Maintaining this line requires more than model self-discipline—it requires verification, whether through post-generation static inspection scripts or agent-style “generate → verify → fix” loops.

5. A Reusable Model Upgrade Evaluation Checklist

Distilling this hands-on test yields a general evaluation checklist:

1. Environment Control

• Lock model version snapshots (API + version number) to avoid online drift.
• Unify inference mode and sampling parameters.
• Sample the same task multiple times to observe variance, not just the best run.

2. Prompt Layering

• Separate hard constraints (numbers, behavior, dependencies) from soft constraints (style, atmosphere).
• Use verifiable quantitative expressions where possible: “90% of steps are submerged” is better than “most steps are submerged.”
• Add negative constraints to block lazy paths.

3. Three-Tier Acceptance

• Runs: No errors, no external dependencies, opens offline.
• Looks good: Composition, materials, atmosphere meet standards.
• Correct: Read source code to verify constants, use scripts to validate numerical constraints, test extreme parameters interactively.

4. Build a Regression Set

• Solidify tasks, validation scripts, and acceptance conclusions into a regression evaluation set.
• Run it automatically on every model upgrade, using data rather than gut feeling to judge “progress or regression.”

For teams, this process can be fully engineered: store tasks and artifacts in object storage, write evaluation processes to logging services, trigger monitoring alerts on metric anomalies, and run batch tasks with GPU compute or model services. Using general cloud capabilities such as those provided by Baidu AI Cloud as an example, there are corresponding categories to support each of these steps, and the setup cost is not high.

6. Final Thoughts: From “Does It Look Right” to “Is It Actually Right”

From this hands-on test, the new-generation model’s progress on long-horizon programming and tool-assisted tasks is clear, but capabilities diverge across tasks, with some details even regressing. This relates to the current technical environment: as marginal gains from distillation change, post-training investment becomes the main battlefield, and which capabilities post-training reinforces directly determines a model’s strengths and weaknesses on specific tasks.

Two takeaways for developers:

First, don’t assume newer is universally better. Capabilities like visual impression, atmosphere, and composition are becoming standard, but “backend capabilities” like numerical consistency and constraint adherence still fluctuate. Building your own regression evaluation set is the only reliable way to cope with version iterations.

Second, shift acceptance focus from “looking” to “calculating.” The quality ceiling of generative frontend is shifting from “does it look like a world” to “are the parameters correct and constraints respected.” The gap between model-generated code that “looks good” and code that “works” is precisely those numbers users can’t see.