Flagship LLM Evaluation: Code and 3D Generation Improve, Aesthetic Judgment Still Lags
This article reports a hands-on evaluation of a next-generation flagship model (referred to as "Flagship Model A," tested through its public web interface and official API in May 2025). Rather than relying on marketing claims or single-point demos, the tests run real, reproducible tasks—an interactive animation and a code-generated 3D room scene—in a unified WebGL2/Three.js environment, with ten independent generations per task and a weighted scoring rubric. The findings show clear progress in code generation, interactive state management, and 3D scene assembly, while realism, material detail, and aesthetic judgment still fall short of top-tier visual output. The article also provides concrete failure-case analysis, observable aesthetic metrics, and a practical framework for building verifiable, reusable model-evaluation pipelines, making it useful for developers and teams deciding how to fit such models into reliable workflows.
1. Background: Flagship models enter the engineering-usability race
A new generation of flagship models has recently moved into intensive testing or release, and the industry’s focus has shifted from single-turn question answering to comprehensive engineering capabilities: code generation, 3D scenes, multimodal perception, tool use, and autonomous execution. For developers and enterprise users, whether a model is good cannot be judged from marketing material or a single-point demo. It has to be observed on real tasks—whether it can complete the full loop from requirement understanding and solution design to code writing and result presentation.
To avoid overgeneralizing the label “flagship model” into a vague verdict on an undisclosed product, this article refers to the test subject as “Flagship Model A.” It was accessed through a public web interface made available in May 2025, with the official API used for auxiliary verification. Testing ran from May 12 to 15, 2025. The environment was Windows 11 24H2 on an Intel i7-13700K with 32GB of RAM and an RTX 4070, using Chrome 124 and a rendering stack based on WebGL2 and Three.js. All code ran on a local static server with no external online resources.
The evaluation follows three main lines: code capability, graphics and 3D output, and visual aesthetics. The chosen tasks include both a classic dynamic-graphics test and a pure-code light-and-shadow scene. The point is not whether the model can “answer,” but whether it can produce output that runs, is maintainable, and is reusable—and whether its visual output reflects genuine aesthetic judgment. Keywords for this article include: flagship model code-generation evaluation, 3D rendering prompts, aesthetic evaluation methods, model capability boundaries, and prompt engineering.
2. Test design: reproducible tasks for real usability
To keep results comparable and repeatable, the tests use unified input, a unified runtime environment, and a unified output-archiving method. Each task is generated ten times independently, and each run keeps the final code, run logs, browser screenshots, and human scoring records. Test samples are stored in Appendix A and Appendix B: Appendix A holds screenshots and run results, while Appendix B holds the original prompt text, generation logs, and scoring sheets.
The tasks fall into two categories.
1. Dynamic-graphics task
The model must generate an animated object with motion properties and interaction such as pause, speed adjustment, and scene switching. This task mainly tests the model’s grasp of graphic structure, timelines, state variables, event binding, and parameter configuration.
Original test prompt:
Generate a directly runnable web animation scene.Requirements:1. Include one moving object and a dynamic background;2. Support pause, speed adjustment, and day/night switching;3. Keep the code structure clear and key parameters configurable;4. Run even without external dependencies;5. Before the output, explain the implementation approach, key state variables, and interaction logic.
2. Pure-code light-and-shadow scene
The model must generate an afternoon room scene through code, emphasizing layered light and shadow, material texture, spatial perspective, and detail completeness. This task requires not only that the code runs but also that the model reaches a high level in visual aesthetics, composition, and rendering strategy.
Original test prompt:
Generate a runnable web 3D room scene depicting a quiet afternoon.Requirements:1. Include a window, floor, walls, table, and potted plant;2. Sunlight slants in through the window, forming layered light and shadow;3. The scene must look realistic, avoiding simple stacked geometric shapes;4. No external image assets—generate everything with code;5. Before the output, explain the lighting, materials, shadows, and rendering approach.
Evaluation dimensions: functional completeness, code runnability, structural maintainability, interaction reasonableness, visual completeness, and aesthetic consistency. Scoring weights: functional completeness 30%, code runnability 20%, structural maintainability 10%, interaction reasonableness 15%, visual completeness 15%, and aesthetic consistency 10%. Each dimension is scored from 1 to 5 and then converted to a 100-point scale. Scoring relies not only on the final screenshot but also on run logs, code structure, parameter configuration, and human review comments.
3. Dynamic-graphics test: cleaner code structure, stronger interaction
In the dynamic-graphics task, the model produced fairly complete objects and scenes, including a subject, environment, motion trajectory, and basic visual elements. The output includes not just the animation itself but also organizes parameters such as speed adjustment, pause, and day/night switching into callable states, which shows that the model’s understanding of “interactive code” has improved.
Looking at the code, the model typically centralizes core state into a structure like this:
const state = {paused: false,speed: 1,phase: "day"};function animate(time) {if (!state.paused) {updateScene(time * state.speed);}renderer.render(scene, camera);requestAnimationFrame(animate);}
For pause, speed adjustment, and day/night switching, the model generally uses button event bindings to modify state and reads that state in the render loop. For example:
pauseButton.addEventListener("click", () => {state.paused = !state.paused;pauseButton.innerText = state.paused ? "Resume" : "Pause";});speedControl.addEventListener("input", (e) => {state.speed = Number(e.target.value);});themeButton.addEventListener("click", () => {state.phase = state.phase === "day" ? "night" : "day";updateEnvironment(state.phase);});
Across ten independent generations, eight ran directly, and pause, speed adjustment, and day/night switching all responded correctly. A typical pass looks like this: after clicking pause, the animation stops updating frames and resumes on a second click; after dragging the speed slider, the motion speed changes in real time; after switching day/night, the background color, light intensity, and object material change together. Failures appeared in two of the runs: in one, the pause button stopped working on the second click because the state variable was placed inside a function, so the event callback could not access it reliably; in the other, the background changed after switching day/night, but the lighting and material did not update in sync, causing an abrupt visual jump.
From an engineering standpoint, this capability is more valuable than generating plain text. The model has to account simultaneously for geometric structure, timelines, state variables, event binding, and render order. If the generated code lacks state management, it often produces broken animation, failing repeated clicks, and parameters that cannot link together. The fact that the model can integrate these elements into a single implementation in this test shows that its code generation is no longer limited to snippet completion but is moving toward a complete application structure.
The model also fills in some implicit requirements from the prompt, such as adding environmental decoration, dynamic effects, and visual atmosphere on its own. This suggests that the new generation of models has improved in world knowledge and multimodal perception, converting semantics such as “moving object,” “seaside scene,” and “day and night” into visible scenes while to some degree keeping a consistent visual style.
4. 3D and light-and-shadow test: functional, but realism and aesthetics still lag
For the light-and-shadow scene, the model chose different technical routes, including both real-time graphics rendering and near-offline rendering approaches. The main rendering parameters used in the test included a resolution of 1920×1080, a pixel ratio capped at 2, PCFSoftShadowMap for shadow maps, ACESFilmic tone mapping, an exposure of 1.1, a DirectionalLight as the main light source supported by an AmbientLight, and MeshStandardMaterial with different roughness and metalness values. All screenshots are stored in Appendix A-3 through A-6.
Its strength is that it can quickly assemble a runnable scene and shows a basic understanding of light sources, shadows, window panes, floors, and furniture. In particular, when light slants in through the window, the model can express a degree of spatial layering. Across ten generations, all the code ran in the browser with no blocking syntax errors.
But closer inspection reveals that realism is still lacking. Comparing against a reference image, the aesthetic gaps can be mapped to the following observable indicators:
Shadow transition. In the reference image, window-pane shadow edges typically have a natural 3-to-8-pixel falloff; in the model’s output, shadow edges are mostly 0-to-2-pixel hard edges, making the scene look harsh.
Material difference. The reference image shows at least four clearly different material responses across floor, walls, plants, and tabletop—including differences in roughness, reflection, texture, and light/dark variation. The model’s output usually shows only two or three material differences, with the floor and potted plant often close to the same texture.
Visual focus. The reference image usually has one or two clear focal points, such as the spot where window light lands, a highlight on the tabletop, or the lit face of the potted plant. In the model’s output the focal points are weak, elements are evenly distributed, and there is little guidance for the eye.
Color and color temperature. In the reference image, there is an obvious color-temperature shift between the window side and the interior shadows—for example, cooler by the window and warmer in the shadows, or vice versa. The model’s output tends to have an overall flat tone with little warm-cool transition.
Detail density. The reference image contains window-frame detail, floor texture, tabletop reflections, and layered plant leaves. The model’s output has too few objects or texture layers, so the scene looks more like a basic scene draft.
Overall, the image looks more like a “runnable scene draft” than a visual work that has been refined through repeated iteration. This gap is no accident. Realistic rendering involves complex engineering details such as light transport, material reflection, global illumination, soft shadows, depth of field, and tone mapping. Even if the model can generate reasonable code, that does not mean it has mature artistic judgment. Aesthetic ability requires a large volume of high-quality visual corpus, professional rendering experience, and multi-round feedback mechanisms. Relying on text prompts alone makes it hard to reliably reproduce a highly photorealistic result.
5. Capability abstraction: the key gap from “can generate” to “can do well”
This evaluation shows that the current flagship model has improved significantly in code generation and 3D scene assembly but still needs to catch up in aesthetics and detail completion. The core conclusions are:
- Conclusion 1: The model can now “put a scene together,” but it is not yet good at judging “what looks better.”
- Conclusion 2: The model passes functional acceptance more easily, but visual aesthetics still need human intervention.
- Conclusion 3: If a prompt only describes the theme, the model tends to output templated imagery.
The reasons for this gap can be understood on three levels.
First, correctness takes priority over aesthetic expression. The model tends to produce results that run, are structurally complete, and are logically consistent—this is an engineering foundation. But a high-quality visual work requires not only “correctness” but also unity across composition, light and shadow, color, material, and emotional expression. The model can complete basic scene assembly, but it does not necessarily judge which composition is more natural, which shadow is softer, or which material is closer to reality.
Second, the choice of tech stack affects the final result. Real-time graphics rendering suits interactive demos, but realism usually requires more complex sampling, denoising, material parameters, and lighting models. If the model does not do enough technical selection at the solution stage, the output can stay at a basic visualization level. In other words, getting the code to run is only the first step; whether the visual result looks polished depends on whether the model understands the applicable boundaries of different rendering approaches.
Third, complex constraints are hard to satisfy all at once. Prompts often contain many implicit aesthetic requirements, such as “as perfect as possible,” “tranquil afternoon,” and “interplay of light and shadow.” These are hard to quantify, so the model can usually only generalize and fill in, making it hard to reach the precision of a professional creator. If a prompt does not specify composition, light source, color temperature, material, and viewpoint, the model is more likely to generate templated imagery.
Suggested observable metrics: shadow-transition width, number of material differences, number of visual focal points, number of occlusion layers, color-temperature range, and number of detail objects. Writing these metrics into an acceptance sheet can reduce subjective judgments like “it looks pretty good.”
6. Engineering recommendations: build a verifiable, reusable model-evaluation system
For teams that want to use a new-generation flagship model, do not rely only on a single demo or a subjective impression. Build a reproducible, quantifiable evaluation process.
6.1 Split task types
Evaluate code generation, interaction implementation, 3D rendering, documentation generation, and tool use separately, so that a single overall score does not hide capability differences. For example, code generation should be judged on runnability and structure, 3D rendering on realism and materials, and interaction tasks on state management—not merged into one score.
6.2 Define acceptance criteria
Organize acceptance criteria into a table that includes the verification method, threshold, and evidence type.
| Acceptance item | Verification method | Suggested threshold | Evidence type |
|---|---|---|---|
| Code runs | Execute in Chrome or Node | No blocking errors | Run logs, screenshots |
| Interaction complete | Manually click buttons | Pause, speed, switch all work | Screen recording, state variables |
| Visual completeness | Manual review and screenshot comparison | Clear subject, complete composition | Screenshots, scoring sheet |
| Material difference | Check material parameters and render results | At least 4 clear material differences | Screenshots, material config |
| No external dependencies | Re-run after disconnecting the network | Page displays normally | Screenshots, network logs |
6.3 Design layered prompts
Have the model first output a solution, then generate code, then self-check and revise. Multi-round iteration is often more stable than a single long prompt. For example:
Step 1: Output only the implementation plan, including scene structure, state variables, and interaction logic.Step 2: Generate the code based on the plan.Step 3: Check that pause, speed adjustment, and switching all have event bindings.Step 4: Output common issues and fix suggestions.
6.4 Introduce automated verification
Reduce subjective bias through syntax checks, unit execution, screenshot comparison, and human review. For visual-generation tasks in particular, attend to both “does the code run” and “does the image meet expectations.” Automated evidence such as error logs, page screenshots, button-state detection, and render frame-rate detection can be introduced.
6.5 Build an asset library
Accumulate strong prompts, failure cases, code templates, and evaluation records into a reusable capability foundation for the team. Over long-term use, you can gradually distill model-invocation strategies suited to your own business. The asset library should include success cases, failure cases, prompt versions, runtime environments, scoring results, and improvement suggestions.
7. Failure-case analysis: why the image “runs” but doesn’t look good
From the results, the model’s common visual problems fall mainly into the following categories. To avoid listing symptoms only, each case here includes reproduction conditions, fix suggestions, and acceptance criteria.
Shadow edges too hard. Reproduction: only a single DirectionalLight is used, with no soft shadows. Fix: enable PCFSoftShadowMap, add fill light, or use area lights to simulate soft window light. Acceptance: shadow-edge transition width of at least 3 pixels, with no obvious hard cut.
Single material representation. Reproduction: all objects default to a roughness of 0.8, with no material differentiation. Fix: set different roughness, metalness, and colors for the floor, walls, tabletop, and potted plant. Acceptance: at least four recognizable material differences in the scene.
Composition lacks a focal point. Reproduction: furniture and decorations are placed evenly, with no subject to guide the eye. Fix: put the focal point on where the window light lands or the center of the tabletop, and use light-dark contrast and perspective to guide the eye. Acceptance: at least one clear focal point in the scene.
Insufficient light-and-shadow atmosphere. Reproduction: only brightness changes, with no color temperature, reflection, or sense of air. Fix: adjust the light source’s color temperature, add ambient reflection, and introduce light fog or tone mapping. Acceptance: a visible color-temperature difference between the window side and the shadow areas.
Insufficient detail density. Reproduction: the scene contains only a few basic geometric shapes, with no texture layers. Fix: add window frames, floor texture, plant leaves, tabletop edges, and other details. Acceptance: at least six recognizable categories of detail elements in the scene.
These problems show that the model is currently better at structured tasks than at advanced aesthetic tasks. It can “put a scene together,” but it is not yet able to reliably judge “how to build it more naturally, more attractively, and closer to reality.”
8. Looking ahead: multimodal and agentic capabilities are the way forward
This test shows that the gains in code and 3D capability are only surface-level; behind them is progress in structured output, tool use, and autonomous execution. In the future, if models can continue to improve in the following areas, they will be more practically valuable:
- Stronger multimodal understanding: unify image, video, 3D scene, and user-description modeling to improve visual-intent reconstruction.
- More stable code execution: automatically detect errors, fix dependencies, re-run, and provide explainable logs.
- More detailed aesthetic evaluation: not only judge “whether it runs” but also evaluate composition, color, light and shadow, and interaction experience.
- More complete agentic workflows: chain requirement analysis, solution generation, code implementation, test verification, and result delivery.
For developers using such models, it is also important to respect engineering boundaries. First, failure rollback—keep a version of every generation to avoid unrecoverable states. Second, automated verification—do not rely only on the model’s own account. Third, rendering-resource limits—pay attention to browser performance, GPU load, resolution, and sampling cost. The more complex tasks a model can complete, the more the team needs supporting tests, monitoring, and human review.
This means that the focus of future model competition will not just be single-point generation capability but whether a model can become a reliable “execution assistant.” Whoever can more stably complete the closed loop of complex tasks is more likely to be adopted in enterprise scenarios.
9. Conclusion
Within the scope of this test—Flagship Model A’s May 2025 web version, a unified WebGL2 environment, ten independent generations per task, and the scoring sheet used here—the new-generation flagship model has shown clear progress in code generation, dynamic interaction, and 3D scene assembly, and can help developers quickly produce runnable prototypes. But in light-and-shadow realism, material detail, aesthetic consistency, and overall completeness, it still falls short of top-tier visual-generation capability.
This result has limitations: the sample size is limited, visual scoring still includes human subjective judgment, and different hardware, browsers, and model versions can affect reproducibility. Therefore, these conclusions apply to rapid prototyping, interactive demos, and basic 3D scene generation, not to directly replacing professional visual-asset production.
For enterprise and individual developers, a rational view of model capability boundaries and the establishment of scientific testing methods and engineering processes matter more than blindly chasing single-point demos. Only by embedding model capability into a reliable workflow can it truly become sustainable productivity.
