Large Model 3D Generation Tested: First Impressions Are Easy, Parameter Consistency Is Hard

1. Skip the Leaderboards: Longitudinal Testing Reveals More

Over the past couple of years, large language models have been iterating so quickly that evaluation frameworks can barely keep up. Horizontal benchmark comparisons are everywhere, but benchmark scores and real-world task performance increasingly diverge. The same model can rank near the top on structured evaluations, yet fall apart when handed an engineering task that requires multiple systems to work together.

By contrast, longitudinal testing — evaluating two generations of the same model family under strictly controlled variables, using the same task set — reveals far more. It shows exactly where generational improvement occurs (visual layer, mechanism layer, or parameter layer), which capabilities have regressed, and what these changes suggest about training priorities.

This approach also reflects the current industry landscape. As distillation gains show diminishing marginal returns, frontier model competition has shifted from raw parameter scale toward post-training quality. Reinforcement learning, long-horizon task data, and tool-calling training have become the new investment priorities. Meanwhile, a parallel route focuses on high-throughput lightweight models that prioritize cost efficiency — “volume over peak capability.” Neither route is inherently right or wrong. But if you care about complex coding and 3D creation — tasks that demand sustained coherence over long outputs — you need to track generational progress on frontier models. That is the motivation behind this test.

Another variable that cannot be ignored is version drift. Industry convention suggests that around two weeks after a model goes live, or after it is replaced by a newer version, online behavior often shifts (the community colloquially calls this “getting dumber”). To minimize this, all tests in this evaluation were completed within the same time window: same client, same inference mode, same prompts — with variables collapsed as much as possible onto the model itself.

2. Test Design: A High-Barrier Task That Forces Out Real Capability

The task was defined as follows. The deliverable: a fully offline, single HTML file.

• Scene elements: a giant ring-shaped water gate, a terraced maritime city, and tides that can rise and fall;
• Hard constraints: no external images, model files, or third-party libraries — everything implemented in native WebGL2;
• Quality requirements: composition, architectural forms, and materials must hold up at first glance;
• Interlinked behavior: rising and falling tides cannot be reduced to moving a blue plane up and down — ship draft, waterline position, and plaza submersion state must all respond to water level changes.

This type of task works well as a litmus test because it simultaneously pressures multiple capability dimensions:

1. Long-horizon code generation: the ability to produce over a thousand lines of self-consistent code in one pass;
2. Scene understanding and information organization: hierarchy of primary and secondary elements, layered structure, camera choreography;
3. Rendering mechanism selection: whether the technical approach to reflections, water surfaces, and lighting is correct;
4. Parameter consistency: whether geometric dimensions and water level constraints actually resolve into coherent numerical relationships.

In the actual test, both generations delivered native WebGL2 single files with zero external dependencies — the hard constraints were respected. This is now a shared baseline, and it indicates that instruction following has become a floor capability for current models.

3. Visual Layer: From “Placing Objects” to “Building a World”

What immediately set the two generations apart was how each organized the scene hierarchy within the default camera frame.

The previous generation: the giant ring was massive, but the surrounding mountains occupied more of the frame. The first screen was dominated by nearby slopes and distant peaks; the city huddled deep inside the bay, with buildings and plazas barely discernible. The ring stood at the center of the image, but the maritime city required effort to even locate. Fundamentally, this reflected a mindset of “placing a ring-shaped object onto terrain.”

The new generation: the giant ring, bridge openings, navigation towers, and terraced housing were organized into a coherent landscape. Building clusters spread along both shores, and the central plaza revealed clear concentric step layering. The cold green of oxidized copper contrasted against the warm gray of stone, creating a deliberate cool-warm palette. The result was far closer to what the prompt asked for: a “complete world.”

In abstract terms, composition quality equals information hierarchy between subject and supporting elements, plus visual guidance, plus contrast management of color and material. The new generation’s progress on this layer is a tangible “first impression” improvement — reviewers can feel the difference within the first three seconds of opening the file.

4. Mechanism Layer: Water Reflections from “Painted On” to “Computed”

Water surfaces are the visual make-or-break element for this type of scene, and the two generations’ implementation mechanisms were fundamentally different.

The previous generation’s approach was fake reflection: layered approximations using procedural sky colors, sun highlights, and light trails. It looked acceptable, but the reflections had no real spatial correspondence with the objects on shore — any gradient that “looked like a reflection” could be pasted on.

The new generation introduced plane reflections using a mirror camera: the city and ships were mirrored across the water plane and rendered in full to a reflection texture. The main render pass then sampled this texture based on screen coordinates, with ripple distortion applied on top. This gave reflections a true spatial source — where a building stands, its reflection appears; when a ship moves, its reflection moves with it.

Core concept in pseudocode:

// Plane reflection core concept (schematic, not a full implementation)
const waterPlaneY = 0.0;
// 1. Build the mirror transform about the water plane
function mirrorAboutPlane(y) {
  return [
    1, 0, 0, 0,
    0, -1, 0, 2 * y,
    0, 0, 1, 0,
    0, 0, 0, 1
  ];
}
// 2. Render the scene to an offscreen framebuffer using the mirror camera
gl.bindFramebuffer(gl.FRAMEBUFFER, reflectionFBO);
renderScene(multiply(viewMatrix, mirrorAboutPlane(waterPlaneY)));
// 3. Main pass: sample the reflection texture by screen coordinates, apply normal perturbation
vec2 uv = screenUV + waveNormal.xz * rippleStrength;
vec3 reflection = texture(reflectionMap, uv).rgb;

From an engineering standpoint, plane reflections mean the cost of an entire additional render pass. Typical mitigations include lower-resolution reflection textures and clip planes. The model’s ability to independently select this mechanism and implement it correctly represents a substantive upgrade at the implementation-mechanism level.

5. Parameter Layer: The Real “Backend” Test

The climax came last, and it was the most important finding of this evaluation: the new generation, despite winning on visuals and mechanism, failed on parameter delivery.

The prompt explicitly required that at maximum tide, “most of the stone steps be submerged.” The verification method was straightforward — audit the geometric constants and water level constants in the source code, and do the arithmetic:

• The new generation’s plaza had nine concentric terraced steps, each with an explicit height parameter;
• Based on the source code constants, at full tide only the lowest four steps fell below the average water level — four out of nine can hardly be called “most”;
• By contrast, the previous generation — the one that lost on first impressions — extended the maximum water level coverage to the outermost step, fulfilling the requirement.

The verification expressed in pseudocode:

// Extract constants from the generated source code (schematic)
const STEP_COUNT  = 9;
const STEP_HEIGHT = 0.35;  // Height of each step
const PLAZA_BASE  = 0.10;  // Elevation of the lowest step's top surface
const TIDE_MAX    = 1.55;  // Maximum water level in the source code
let submerged = 0;
for (let i = 0; i < STEP_COUNT; i++) {
  const stepTop = PLAZA_BASE + i * STEP_HEIGHT;
  if (stepTop < TIDE_MAX) submerged++;
}
console.log(submerged / STEP_COUNT); // 0.44 — promise not fulfilled

This exposes a common weakness in current large models: numerical commitments expressed in natural language (“most,” “significantly,” “close to”) are rarely translated into self-consistent executable constraints. Models excel at making things look right; they struggle to make parameters compute right — and the latter is exactly what engineering delivery demands.

6. Evaluation Takeaways: Score “First Impressions” and “Backend Verification” Separately

Based on this evaluation, a three-layer verification method can be summarized:

1. Visual layer (manual/semi-automated): first-screen composition, primary-secondary hierarchy, materials, color; fixed-camera screenshots for generational comparison;
2. Mechanism layer (source code review): whether key effects are “faked” or “computed,” whether the technical approach is correct, and whether the overhead is reasonable;
3. Parameter layer (automated assertions): convert vague prompt constraints into machine-checkable hard metrics and verify them with scripts.

The third layer is the most easily overlooked and the most valuable from an engineering perspective. Practical automation paths include:

• Using regex or AST parsing to extract constants and geometric parameters from generated code;
• Quantifying “most of the stone steps submerged” at the prompt stage into a hard metric such as “at full tide, at least 80% of steps are submerged”;
• Taking screenshots at extreme high and low tide moments, performing pixel classification on the plaza area, and computing the submerged area ratio.

In other words, prompt engineering is not just about phrasing things well — it is about writing acceptance criteria in a form that machines can evaluate.

7. Behind the Generational Gains: Industry Observations

• Frontier-route gains concentrate on long-horizon tasks: zero-dependency, multi-system interlinked complete scenes can now be produced in a single pass — hard to imagine a year ago;
• “First impression” capabilities (composition, materials, color) are rapidly becoming commoditized and will soon cease to be a differentiator;
• “Backend” capabilities — parameter consistency and numerical promise fulfillment — will become the next focal point of competition, and the key area where engineering teams must design fallback mechanisms;
• As distillation returns diminish, frontier model improvements will depend more heavily on high-quality post-training data and reinforcement learning. The pace may slow, but each generation’s substance will be more tangible;
• When enterprises evaluate models, showcase-oriented tasks and parameter-critical tasks should be assessed separately. Do not let a stunning first screen cloud judgment.

8. Conclusion

The new generation is indeed “better at shaping first impressions”: more complete composition, more solid mechanisms, more striking initial screens. But the real test lies at the backend — the failures hidden in source code constants, only discoverable by actually doing the math, determine whether a model can enter production.

The practical takeaway for developers: boldly delegate the visual layer to the model, and keep the parameter layer for your own assertion scripts. Use the three-layer verification method instead of the “open it and glance” intuitive judgment. First impressions determine whether you want to keep looking; backend verification determines whether you dare to use it.