A Practical Test of Chinese LLM Iterations: From First Impressions to Engineering Trade-offs

Over the past year, many Chinese LLM providers have concentrated resources on lightweight, high-throughput models optimized for low latency and cost-effective scaling. However, some models have continued to push the frontier, directing post-training investment toward difficult tasks such as coding, 3D creation, and financial analysis. This article uses a comparative test of front-end 3D scene generation to discuss where new-generation models have improved relative to the previous generation, where they have regressed, and what these differences mean for developers choosing between them.

Note: This discussion focuses on general technical capability and does not recommend or evaluate any specific vendor product. Model generations are referred to simply as “previous generation” and “new generation.” All tests were conducted in a standard desktop client environment with the inference mode set to the “advanced” reasoning level.

Test Design: Why a Single-File 3D Scene

A single-file, offline 3D scene is a high-density test for evaluating whether a model can truly “construct a world.” It simultaneously exercises multiple capabilities:

• Long-range consistency: A single HTML file must contain shaders, geometry generation, camera control, and interaction logic. Any break anywhere collapses the scene.
• Spatial organization: The model must understand hierarchy—what belongs in the foreground, what recedes into the background.
• Mechanism selection: For the same effect, such as water reflection, analytical approximations and mirror-camera redraws are two implementation paths with vastly different cost and quality. The model’s choice reveals its engineering judgment.
• Instruction fulfillment: Whether numerical constraints in the prompt (e.g., “most stone steps are submerged”) translate into actual source parameters is the key to distinguishing “looks right” from “computes right.”

The test task: build a giant annular water gate, a terraced sea-city, and a tide that rises and falls, all in one offline HTML file. The prompt explicitly required establishing composition, architecture, and materials first, and specifically stated that the tide must not be a simple blue plane moving up and down—boats, waterlines, and the plaza submersion state must all change in coordination. Both models delivered native WebGL2 single-file implementations with no external images, models, or third-party libraries—this itself confirms that both have the same engineering baseline. The divergence appears in higher-level decision-making.

First Impression: How Scene Hierarchy Is Organized

What immediately separates the two models is how the default camera frames the scene.

In the previous-generation model’s output, the ring structure is large, but the hills on both sides take up more of the frame. The first screen is dominated by nearby slopes and distant mountains, with the city compressed deep inside the bay—the buildings and plaza are difficult to identify. The result: the ring stands in the center, but the sea-city requires effort to find. This is typical of “placing individual objects on terrain”—the model generated elements but did not organize the landscape.

The new-generation model, by contrast, organizes the ring, bridge openings, navigation towers, and layered residential blocks into a coherent landscape: buildings spread along both banks, the central plaza has clear concentric step layers, and the contrast between copper oxide and stone materials is easy to distinguish. It is closer to the “complete world” requested in the prompt, and it avoids the stiff feel of isolated objects.

The technical root of this difference lies in implicit scene graph planning. The previous model decomposes the prompt into independent objects, generates them separately, and places them by approximate position. The new model establishes a spatial narrative before generation: the sightline flows from the ring through the bridge opening, then to the buildings and the plaza, with materials and lighting serving that path. For developers, this means the new-generation model is noticeably more usable for tasks requiring coordinated multi-element layout, reducing manual post-processing of camera and object positions.

Water Reflection: An Upgrade in Implementation Mechanism

The water rendering mechanism is more significant than the visual difference.

The previous-generation model primarily used analytical sky, solar specular highlights, and light trails to simulate reflection. This is essentially “approximating water with lighting formulas”—low cost, but the reflections have no spatial correspondence to the buildings on shore. The water is bright, but it does not “mirror” anything.

The new-generation model additionally introduces a mirror camera: it redraws the city and boats mirrored across the water plane, then samples that into the water surface. This gives the reflections a real, spatially consistent source. A pseudocode sketch of the difference:

// Approximation: analytical sky + specular, no scene correspondence
vec3 approxReflection(vec3 viewDir, vec3 normal) {
    vec3 sky = sampleSky(viewDir);
    float spec = pow(max(dot(reflect(-viewDir, normal), sunDir), 0.0), 64.0);
    return sky + spec * sunColor;
}
// Mirror camera: redraw scene, then sample, real reflection source
vec3 mirrorReflection(vec2 uv) {
    vec3 sceneColor = texture(mirrorRT, uv).rgb;  // render from mirror camera
    return sceneColor * waterTint;
}

This is an upgrade at the implementation level, not a parameter tweak. The direct benefit: reflections now change correctly with building layout, boat movement, and tide level—which is exactly the “waterline coordination” the prompt demanded. The cost is one extra scene redraw per water pixel, which is unfriendly to mobile or low-end GPUs. In real projects, such techniques are typically combined with resolution scaling or frame-interleaved updates to balance performance.

Parameter Compliance: A Great First Screen Does Not Guarantee Engineering Precision

When it comes to the highest tide level parameter, the new-generation model is less thorough.

Based on the step heights and maximum water level in the source code, the new-generation model’s nine-level plaza steps have only the lowest four levels below the average waterline at full tide—it does not fulfill the prompt’s requirement that “most stone steps are submerged.” The previous-generation model, which had a weaker first impression, actually covers the outermost steps at maximum tide.

This regression is revealing. It points to two issues:

1. The tension between visual priority and numerical consistency. The new model invests more “attention” in composition and materials but slips on cross-module numerical constraints. Water level, step height, and step count are scattered across geometry generation and shader parameters, and the model does not perform consistency checking.
2. Soft constraints in prompts are easily ignored. “Mostly submerged” is a vague expression. The model tends to produce a visually reasonable number rather than calculating it literally. For engineering tasks, vague prompts should be converted into verifiable hard constraints—for instance, by adding a self-check after generation or by writing the water level directly as an expression derived from step height.

// Convert a vague description into a checkable hard constraint
const STEPS = 9;
const STEP_H = 0.8;
const tideLevel = STEP_H * (STEPS - 2); // explicitly submerge through level 7

What the Generation Gap Tells Us About Model Evolution

Placing these observations in a broader context yields several judgments worth considering.

First, post-training investment is shifting from “knowledge” toward “task structure.” Improvements in coding and 3D creation come more from learning task decomposition, spatial organization, and mechanism selection than from simply increasing parameters. This explains why the new model improves significantly at “organizing a scene” yet exposes weaknesses in “numerical consistency”—the former is a structural ability, the latter requires explicit verification habits.

Second, capability divergence is normal, and evaluation should be dimension-specific. In a single generation, composition, materials, reflection mechanism, and parameter compliance can each perform differently. Making a selection decision based on a single “is it pretty” judgment is likely to mislead. It is better to break the task into assessment items and score each one independently.

Third, “regression” tends to appear in the long-tail details of older tasks. When training resources shift toward frontier tasks, the previous model’s strengths in detail rigor may be diluted. This does not mean regression is inevitable—rather, it suggests that when a new model is introduced, the old model should be retained as a comparison baseline, with cross-validation on tasks that involve strict numerical constraints.

Fourth, long context and tool use are amplifiers, not substitutes. Million-token contexts and tool-assisted capabilities allow models to handle longer engineering chains, but the correctness of each link in that chain still depends on the model’s own verification awareness. Tools can fill the checking gap, but only if the developer designs verification as an explicit step.

Practical Recommendations for Developers

Based on the above, here are some actionable suggestions:

• Convert vague prompts into verifiable constraints. For any description involving quantities, ratios, or layer counts, provide a derivation relationship or a target value in the prompt, and require a self-check after generation.
• Accept the result only when both “first screen” and “parameters” pass. The first screen tests organizational ability; the parameters test compliance. Both must pass for a task to be considered complete.
• Prepare mechanism fallbacks for performance-sensitive scenarios. Mirror-camera reflections look good but are expensive. A resolution-scaling or simplified-reflection fallback path should be prepared in advance.
• Keep a generation-to-generation baseline. After a new model is deployed, run it against a fixed task set alongside the old model, and watch whether long-tail details degrade.
• Build tool use into the workflow rather than relying on the model’s initiative. Deterministic work—numerical checks, format validation, dependency inspection—should be executed by scripts or tools, while the model handles generation and explanation.

Conclusion

The new-generation model’s progress in “first impressions” is real: scene organization is more coherent, material differentiation is clearer, and the reflection mechanism has meaningfully improved. But a striking first screen does not equal engineering rigor. The shortfall in parameter compliance reminds us that model capability must be evaluated in a multi-dimensional, verifiable way. For developers, the real test is not the moment of generation but the checking, comparison, and fallback design that follow—and that is exactly what turns model capability into stable productivity.