From a Pelican on a Bicycle to Photorealistic Rendering: Hands-On Tests of Frontier LLMs' Coding and Aesthetic Capabilities
As flagship LLMs launch in rapid succession with sparse official benchmarks, how can developers judge real-world model quality? This article evaluates frontier models through two widely used practical tests — interactive SVG animation generation and pure-code image rendering — and offers a framework for building your own model evaluation checklist.
From a Pelican on a Bicycle to Photorealistic Rendering: Hands-On Tests of Frontier LLMs’ Coding and Aesthetic Capabilities
1. Background: Flagship Models Are Launching Faster Than Benchmarks Can Keep Up
Recently, the LLM industry has entered a dense release window for flagship models, with multiple vendors rolling out new-generation versions within weeks of each other. Notably, official announcements tend to offer only directional descriptions — such as “enhanced underlying world knowledge and multimodal perception” or “improved coding and agentic capabilities” — while remaining vague about parameter counts, training data composition, and concrete benchmark scores.
This style of disclosure raises a practical question for developers and enterprise users: when official data is missing, how do we assess a model’s true capability?
The answer is to replace paper benchmarks with reproducible, comparable, and discriminative hands-on tasks. Compared to standardized benchmark suites (which are vulnerable to data contamination and overfitting), open-ended engineering tasks better reflect how models perform in real development scenarios. This article picks two widely circulated test directions — interactive animation generation and pure-code image rendering — and breaks down their evaluation logic and technical details to help you build your own model assessment framework.
2. Test 1: A Pelican Riding a Bicycle — SVG Animation Generation
2.1 Why “Pelican on a Bicycle”
“Generate an animated SVG of a pelican riding a bicycle” is a classic test in front-end code generation. It looks simple, but actually probes multiple capabilities at once:
- SVG syntax proficiency: correctly organizing paths, gradients, transforms, and other elements;
- Animation logic: timeline coordination of multiple moving parts, such as rotating wheels and pedaling limbs;
- Scene composition: the spatial arrangement of character, background, and decorative elements;
- Interaction extensibility: whether it supports controls such as pause, speed adjustment, and theme switching.
In the early days of these models, almost none could draw a structurally complete pelican, let alone animate it. Today, this test has evolved from “can it do it at all” to “how well does it do it,” making it a useful yardstick for generational progress.
2.2 Observations From Testing
Take one new-generation flagship model as an example. It produced a scene of a pelican wearing a red hat riding along a seaside road: the wheels spin continuously, a scarf flutters backward, and the output ships with full interactive controls — you can pause the animation, adjust pedaling speed, and switch between day and night lighting themes.
One detail deserves credit: after cranking the speed up to 1.75x and exporting the SVG, the animation still runs normally when opened standalone in a browser with the network disconnected. This shows the model did not rely on any external JavaScript libraries; instead it used SVG-native SMIL animation or inline CSS animation. The self-containment is excellent. For embedding animations in emails, documents, or static pages, this zero-dependency output has high practical value.
2.3 Evaluation Takeaways
The lesson from this test: when assessing code generation, don’t stop at “does it run.” Look at three dimensions — visual completeness (is the scene coherent and aesthetically pleasing?), engineering robustness (is it self-contained and offline-capable?), and interaction richness (did the model proactively add control logic beyond the minimum requirements?). A strong model should meet the basic requirements while demonstrating an active design instinct for user experience.
3. Test 2: Recreating a Photograph in Code — The Dividing Line of Aesthetics and Rendering
3.1 Test Design
The prompt for the second test is open-ended: “Using any code technique you know, draw a painting of interwoven light and shadow that captures the tranquil atmosphere of an afternoon. Perfect it to the utmost. Pursue the extreme.”
The difficulty here is that no technical route is prescribed. The test probes a model’s autonomous technology selection and visual aesthetic ability. The model must decide on its own rendering technique, scene composition, and lighting treatment.
3.2 Comparing Two Technical Routes
The known state-of-the-art approach in the community is writing a path tracing renderer. One user had a frontier model generate a C++ path tracer that took three hours to render an afternoon room: the window-frame shadow sweeping across the wall, chair, and parquet floor, with extremely natural transitions from the bright window side to the dark interior — the final result approaching real photography. This route trades physical accuracy for the highest possible image quality.
The model tested in this experiment chose a different route: a WebGL-based real-time rendered 3D room, with a bonus CPU-rendered version for comparison. In the scene, light slants in through the window, crosses a small stool, and lands at the bottom edge of the frame — decent compositional sense. But the problems are equally obvious: the dark lines at the wall corners are too harsh, the potted plant and floor lack texture, and the whole thing looks like a freshly assembled stage set rather than a real space.
The difference between the two routes is essentially a trade-off between real-time performance and physical accuracy:
- WebGL real-time rendering: high frame rate and interactivity, but lighting is approximated; global illumination, soft shadows, and multi-bounce specular highlights are hard to simulate precisely;
- Path tracing offline rendering: Monte Carlo sampling simulates real light paths, producing natural light-dark transitions — at the cost of hours of rendering time.
Yet the choice of technical route does not fully explain the gap. Even comparing light transitions alone, the top-tier result shows far softer, more natural gradients from highlight to shadow. Behind this lies a comprehensive command of details such as tone mapping, noise control, and material parameters — in other words, the model’s “aesthetic” ability. Aesthetics here is not mysticism; it is the degree to which the model has internalized the visual regularities of the physical world.
3.3 Conclusion: Coding Ability Clears the Bar, Aesthetics Still Lag a Generation
Combining both tests yields a fairly clear judgment: current-generation models have reached a high standard in the functional implementation of code — animations are interactive, exportable, and offline-capable, and 3D scenes can be built and rendered. But on tasks demanding “the utmost pursuit,” a visible gap remains compared to the community’s state of the art, mainly in the finesse of visual detail and the naturalness of light and shadow. Catching up to the frontier will still take considerable effort.
4. Advice for Developers: Build Your Own Model Evaluation Checklist
Facing frequent model releases, developers should maintain a private test set closely tied to their own business, covering the following dimensions:
- Classic public tests (such as the pelican on a bicycle): enable horizontal comparison across models;
- Real business tasks: sample typical requirements from daily development to test practical usefulness;
- Extreme open-ended tasks: open propositions like the “pursue the extreme” prompt, to probe capability ceilings;
- Robustness checks: offline operation, boundary inputs, error handling, and other engineering-quality dimensions.
For enterprise users who want to integrate model capabilities quickly, managed LLM platforms (such as the model deployment and inference services offered by cloud providers like Baidu AI Cloud) can help you manage and switch between different models through a unified interface. Combined with the evaluation checklist above, this lets you continuously track how each model actually performs in your own business scenarios — rather than being swayed by launch-event marketing.
5. Closing Thoughts
LLM capability evaluation is moving from the “benchmark era” to the “hands-on testing era.” The pelican’s smooth cycling proves the floor of code generation is already high, while the light-and-shadow rendering comparison reminds us: in aesthetics, detail, and the pursuit of the extreme, a real gradient still exists between models. For developers, rather than chasing the hype of every new release, the better move is to build your own evaluation methodology — that is the long-term asset that carries you across model iteration cycles.
