Testing LLM 3D Frontend Generation: Visuals Pass Easily, Parameters Fall Short

Evaluations of LLM coding ability tend to fall into two traps: relying on benchmark scores alone, or being swayed by a demo that looks stunning at first glance. The former detaches from real delivery scenarios; the latter is easily fooled by surface polish. This article uses a reproducible test vehicle—offline, single-file 3D scene generation—to demonstrate a three-layer evaluation method, and walks through the outputs of two generations of frontier models on the same task (a giant ring-shaped floodgate, a tiered sea city, and raisable tides) to show why “first impressions” pass more and more easily, while the real test hides in whether parameters and constraints are honored. To keep the focus on methodology, the tested models are anonymized as “previous generation” and “new generation.”

1. Why a Single-File 3D Scene Makes a Good Test Vehicle

Frontend 3D generation is an ideal task for probing an LLM’s overall engineering capability, for three reasons:

1. High coverage: a single generation must handle geometry modeling, materials and lighting, camera control, interactive animation, and business constraints—nearly the full breadth of frontend engineering.
2. No toolchain interference: the deliverable is an offline single HTML file with no external images, model files, or third-party libraries, so output quality maps directly to the model’s own ability rather than to its skill at invoking scaffolding.
3. Layered acceptance: the visual, mechanism, and parameter layers each have objective criteria, which makes side-by-side comparison and long-term regression practical.

The task in this test: use offline HTML to build a giant ring-shaped floodgate, a tiered sea city, and adjustable tides. The prompt defines constraints at three levels—composition, architecture, and materials must hold up first; the tide must not be a single blue plane moving up and down, since ships, waterlines, and plaza flooding states have to respond in sync; and at high tide, “most of the stone steps should be submerged.”

2. The Evaluation Method: Control Variables First, Then Layered Acceptance

Both models were tested through the same client entry point at the same inference tier, with the testing window kept as tight as possible. The community often jokes that “a model’s output quality drifts a while after launch”—whatever the real cause (traffic routing, staged rollout, or caching), the lesson for evaluators is the same: without recording version snapshots and test times, no comparison conclusion is trustworthy. The disciplined practice is to fix and archive the test date, model version identifier, inference tier, full prompt, and sampling parameters.

On that foundation, acceptance proceeds in three layers:

• Visual layer: first-screen composition, the hierarchy between primary and secondary elements, and material differentiation;
• Mechanism layer: the correctness of technical choices such as reflections and animation linkage;
• Parameter layer: the consistency between constants in the source code and the prompt’s numerical constraints.

3. Visual Layer: First-Screen Composition Is the “First Impression” Divide

Both generations delivered native WebGL2 single files with zero external dependencies, so both passed the basics. What truly separated the visual results was how each model organized primary and secondary elements under the default camera.

The previous generation’s typical problem: the giant ring sat at the center of the frame, but mountains on both sides consumed too much of the view; the first screen was dominated by nearby slopes and distant peaks, the city shrank deep into the bay, and the building clusters and plaza were hard to make out. The overall impression was “a ring-shaped object placed on terrain,” not a self-contained “complete world.”

The new generation’s clear progress:

1. Primary/secondary allocation: the giant ring, bridge arches, beacon towers, and stacked residences were organized into one coherent landscape; buildings spread along both banks, giving the eye a clear guiding path.
2. Layering: the central plaza showed distinct concentric step tiers, with elevation relationships readable at a glance.
3. Material differentiation: the cool tones of oxidized copper and the warm tones of stone were distinguishable under default lighting—no manual camera adjustment needed to “read” the scene.

These observations can be abstracted into reusable criteria: the subject’s share of the frame; the spatial narrative logic of scene elements (whether a “city—plaza—floodgate” organizational relationship exists); and material legibility from the default viewpoint. These three items can be turned directly into a manual review sheet for any similar generation task.

4. Mechanism Layer: Reflections Move from “Painted Effect” to “Spatially Grounded”

Water reflections are where scenes like this most easily give themselves away, and they are the touchstone separating “imitating a template” from “choosing the right approach.”

Previous generation: it simulated water reflections with an analytic sky, sun highlights, and light streaks—essentially “painting something that looks like a reflection.” It held up in static screenshots, but the reflection had no geometric correspondence with the buildings on shore, and the illusion broke as soon as the camera moved.

New generation: it introduced a mirror camera—the classic planar reflection implementation. The city and ships were redrawn mirrored across the water surface and then sampled into the water material, so onshore buildings’ reflections gained a genuine spatial origin. This is an upgrade at the mechanism level, not parameter tuning.

The significance of this layer is that it shows the model beginning to “choose the right technical approach” instead of copying boilerplate from its training data. That said, a mirror camera means the scene is rendered twice; in production this is usually paired with general optimizations such as half-resolution rendering and mirroring only visible objects to control the cost. If the generated code exhibits such trade-offs, that counts as a high-score feature at the mechanism layer.

5. Parameter Layer: The Real Test Is in the “Backend”

With visuals and mechanisms both improving, numerical constraints are the easiest place to fail—hence the idea of a “backend test.”

The prompt required that at high tide, “most of the stone steps be submerged.” The acceptance method is a source-code audit: extract three constants from the generated code—the number of step tiers, the height of one tier, and the maximum water level—and compute how many tiers are submerged at high tide.

The results are quite representative: the new generation produced nine concentric step tiers, but audited against its own chosen step height and maximum water level, only about the bottom four tiers sat below the water line at high tide—far from “most submerged.” Meanwhile, the previous generation, whose first screen was less impressive, had a maximum water level that covered the outermost tier. This result tells us three things:

1. “Looks like the tide is rising” and “meets the numeric requirement” are two different things. The former is a style-imitation problem; the latter is a constraint-satisfaction problem.
2. Constraints get diluted over long generations. The prompt sits at the very front; by the time the model has generated thousands of lines, its attention to early quantitative constraints decays as the sequence grows.
3. Good visuals can mask parameter defects. The prettier the render, the more likely reviewers skip the source-code audit.

At the root, this is a split between two kinds of capability: composition and materials belong to “pattern matching and local planning”—training data supplies abundant visual templates, so they are quickly optimized. Parameter consistency, by contrast, requires translating natural-language constraints into geometric constants and keeping them self-consistent across thousands of lines of code—a long-range logical consistency problem, widely recognized as harder and more generation-defining.

6. Turning Acceptance into Code: A Reusable Automation Approach

Inspecting rendered screenshots by hand will inevitably miss parameter problems. The recommendation is to harden parameter-layer acceptance into a script:

// Parameter-layer acceptance sketch: extract constants from generated code and verify the constraint
const stepCount  = 9;    // number of step tiers (read from generated code)
const stepHeight = 0.8;  // height of one tier (read from generated code)
const maxWater   = 3.1;  // maximum tide level (read from generated code)
const submerged = Math.floor(maxWater / stepHeight);
const coverage  = submerged / stepCount;
console.log(`High tide submerges ${submerged}/${stepCount} tiers; coverage ${(coverage * 100).toFixed(1)}%`);
// Prompt constraint: "most stone steps submerged" at high tide
console.assert(coverage > 0.5, 'Constraint not met: raise maxWater or lower stepHeight');

Two prompt techniques can significantly raise the constraint fulfillment rate:

• Numerical anchors: translate vague phrases like “most” into concrete numbers—for example, “the maximum water level should be above the top surface of the seventh step tier.”
• Self-check instructions: require the model to append a self-check comment at the end of the code, listing the key constants and re-verifying the constraints—effectively making the model “verify its work on paper” first.

Going a step further, the “extract—verify” flow can be wired into a continuous integration pipeline for automatic regression after model version updates or inference strategy changes, so quality drift does not go unnoticed for days.

7. Selection and Deployment Recommendations

Based on this hands-on test, three recommendations:

1. Choose models by task layer. The industry currently has two routes: distilled, cost-effective lightweight models win on speed and price, fitting high-frequency, template-based everyday coding; frontier large-parameter models concentrate their value in long-range complex tasks, such as the single-file 3D long-form code and multi-constraint engineering problems in this article. The right way to use the two is layered dispatch, not mutual replacement.
2. Treat leaderboard scores as initial screening only. Leaderboards measure static capability; production demands constraint fulfillment and stability. Teams should build their own scenario-based benchmarks: fixed prompts, fixed acceptance scripts, and periodic reruns, so that version changes become measurable.
3. Host the process on platform-level capabilities. For teams aiming to scale, the evaluation pipeline can be combined with general cloud capabilities such as model serving and AI coding assistance—for example, unified access and integration through platforms like Baidu AI Cloud—to reduce the cost of building in-house evaluation and regression infrastructure.

8. Conclusion: The Real Divide Is Parameter-Level Consistency

The comparison between the two generations points to a clear trend: “visible” capabilities—first-screen composition, material organization, reflection mechanisms—are rapidly approaching the usable line. That is the dividend of visual templates, something that competition alone can easily deliver. “Invisible” parameter consistency—water levels and step counts—is the real watershed between a demo and production.

The takeaway for developers is direct: hand first impressions to the model, and hand acceptance to engineering. Write constraints as numerical anchors, write audits as scripts, and wire regression into the pipeline—when output quality drifts, your dashboard will tell you before your users do.