LLM 3D Frontend Generation Test: First Impressions and the Deep End

1. Test Design: A Task That Forces the Truth Out

Evaluating a large model’s frontend generation capability is hard to differentiate with tasks that are too simple (like static pages), and hard to compare horizontally with tasks that are too open-ended. This test chose a composite task with the following characteristics:

Task goal: Using a single offline HTML file, build a coastal city that includes a giant ring-shaped tidal gate, a terraced sea city, and a raisable tide system.

Hard constraints:

• No dependence on any external images, model files, or third-party libraries—all logic must be self-contained;
• Tide fluctuation cannot rely solely on raising and lowering a blue plane—boats, waterlines, and plaza submersion states must be linked.

Soft goals: Composition, architecture, and materials must first “hold up,” forming a complete worldview.

The reason for choosing this type of task: a single file means all implementation details are exposed in auditable source code; hard constraints provide a checklist that can be verified item by item; soft goals test the model’s aesthetics and scene-organization priors. Together, these three dimensions measure engineering capability, visual capability, and instruction-following capability simultaneously.

The test compared two generations of models on the same task, using the same client and the same mode. Note that online model services are subject to dynamic version adjustments—the same task sampled at different times may yield different results, so conclusions should be interpreted alongside multiple sampling rounds and version records.

2. First Impressions: Scene Organization Is the Real Dividing Line

Both models delivered a competent “engineering shell”: single file, zero external dependencies, runnable directly in a browser. This shows that basic engineering capability is no longer a differentiating factor.

What truly separated the visual impressions was how each model organized scene hierarchy under the default camera:

• Older model: Although the ring-shaped main structure was large, mountains on both sides consumed more of the frame. A large portion of the first screen was filled by nearby slopes and distant mountains, the city was tucked deep into the bay, and the buildings and plaza had weak legibility—the main subject held up, but the world did not.
• Newer model: It organized the ring-shaped tidal gate, bridge arches, beacon tower, and layered residences into a coherent landscape. Buildings spread along both banks, the central plaza had clear concentric step tiers, and the warm-cold contrast between metal and stone materials was easy to distinguish—closer to the “complete world” the prompt requested.

This corresponds to three transferable capabilities:

1. Visual hierarchy planning: The area ratio of foreground, midground, and background; the main subject should occupy the visual focus rather than being swallowed by its environment;
2. Composition priors: Eye guidance, horizon handling, and the rhythm of element density;
3. Material semantic mapping: Correctly translating text descriptions like “oxidized tower” and “stone steps” into color, roughness, and lighting response.

Prompt implication for developers: Rather than listing elements, explicitly give composition instructions—for example, “the first screen should treat the ring-shaped tidal gate as the absolute main subject, buildings spread along both banks, and the foreground must not obstruct the main subject.” Such instructions can significantly improve first-pass usability.

3. Mechanism-Level Upgrade: From “Visual Approximation” to “Geometric Correctness”

The difference in water reflection implementation is the most technically substantive part of this comparison, and it best illustrates how “judging by screenshots alone leads to misjudgment.”

Older approach—analytic approximation: It mainly simulated reflection by parsing sky color, sun highlights, and light trails. Essentially, it overlaid empirical visual noise on water fragments; there was no real geometric correspondence between reflection and physical objects, and the illusion could break from a different viewpoint.

Newer approach—mirror camera re-render: It introduced an additional virtual camera symmetric about the water plane, re-rendered the city and boats, and sampled the result onto the water material. The core flow can be summarized in pseudocode:

// Core steps of planar reflection (illustrative)
const mirrorCam = mainCamera.clone();
mirrorCam.reflect(waterPlane);          // Symmetric transform about the water plane
const rt = renderScene(mirrorCam);      // Re-render the scene with the mirror camera
waterMaterial.reflectionMap = rt;       // Water shader samples this texture

This way, reflections of shoreline buildings have a real spatial source, and reflections correspond one-to-one with physical geometry. The cost is an extra scene render; in practice, downsampling and culling distant objects are needed to control overhead.

This difference shows that evaluating generated code requires reading the implementation mechanism, not just final pixels. When a model’s understanding of “reflection” upgrades from “overlay an effect” to “reconstruct optical and geometric relationships,” the generated result becomes stable under arbitrary camera angles—this is precisely the boundary between “demo-grade” and “product-grade.”

4. The Problem Hidden by Impressiveness: Parameter Compliance Accounting

A dramatic moment appeared in the tide-level settings: the newer model, which had the better first-screen impression, actually fell short on a hard constraint.

Based on the step heights and maximum water level parameters in its source code: when the nine-tier plaza steps are at high tide, only the lowest four tiers are below mean water level—yet the prompt explicitly required “most stone steps submerged.” That constraint was not fulfilled. Conversely, the older model, which was weaker on the first screen, extended its maximum water level range to cover the outermost steps.

This reveals an easily overlooked evaluation blind spot: impression scores and compliance scores must be counted separately. Visual impressiveness significantly inflates overall human ratings—this is “first-impression bias.” For numeric constraints, the correct approach is to derive parameters from source code for accounting, not to eyeball screenshots. A checklist like this can be established:

Check item                                      Older     Newer
Single file / zero external dependencies        Pass      Pass
Boats rise and fall with tide                   Pass      Pass
Waterline and plaza submersion linkage          Pass      Pass
High tide submerges most steps (≥6/9 tiers)     Pass      Fail (only 4 tiers)

5. The Three-Layer Evaluation Method: Turning Subjective Impressions into Reproducible Conclusions

Based on this practice, I recommend a three-layer “impression—mechanism—compliance” evaluation method for generative models:

1. Impression layer (first impression): Examine the default camera, composition hierarchy, and material legibility. Suitable for quickly filtering out obviously unacceptable outputs, but not as a final conclusion.
2. Mechanism layer: Read the source code to determine the implementation path of key visual effects—whether empirical approximation or geometrically correct reconstruction—and assess performance and maintainability.
3. Compliance layer: Verify hard constraints item by item; for numeric requirements, perform accounting using source-code parameters. This layer is the most easily overlooked and best exposes a model’s “regression” on specific capabilities.

Supporting engineering recommendations:

• Sample the same task multiple times under fixed, comparable conditions, and record model versions and timestamps;
• Convert hard constraints in prompts into automated assertions (e.g., parse constants in generated code, sample rendered frames to detect water level height), and integrate them into regression workflows;
• Use a fixed task set for regression evaluation during model selection rather than relying on a single experience—this method applies equally whether the model service is accessed through a platform such as Baidu AI Cloud or deployed locally.

6. Final Thoughts: After First Impressions, the Real Test Is Downstream

The comparison of two model generations abstracts the current capability curve of generative models: the ability to shape “first impressions”—overall composition, stylized expression, single-point visual impact—has improved markedly; while precise constraint satisfaction, parameter-level consistency, and multi-condition linkage remain in the deep end.

The implication for users: first impressions determine whether users are willing to look twice, but mechanism correctness and constraint compliance determine whether the output can be delivered to production. The next stage of competition will likely shift from “can it generate” to “can the generated result be verified and regressed.” Building your own three-layer evaluation system is more useful than chasing leaderboard scores for answering the question that really matters: can this model actually finish the job?