AI Code Generation and 3D Rendering in Practice: Sculpting a Digital David from Stone

As large language models continue to improve in code generation and multimodal understanding, the barrier to building complex frontend interactions and 3D rendering experiences is dropping significantly. A recent iteration of an AI coding tool claimed stronger capabilities in 3D scene construction and autonomous code progression. To test this, we designed an advanced evaluation: give the AI a high-level, open-ended natural language instruction to independently complete a Chinese interactive 3D website themed around the creation of Michelangelo’s David, then generate a short promotional video based on that website.

This article reviews the technical practice in depth, covering the AI’s performance and technical approach across narrative design, 3D rendering, frontend engineering, and multimodal generation.

1. Requirements Definition and Open-Ended Prompt Engineering

In traditional AI-assisted development, developers usually provide a detailed PRD or pseudocode. In this test, however, we deliberately adopted an open-ended prompting strategy to examine the AI’s autonomous decision-making and end-to-end delivery capability.

The core instructions were:

1. Primary goal: Build a Chinese interactive website that visually explains to a general audience how the statue of David emerged from raw stone.
2. Experience requirement: Go beyond listing historical facts. Help users develop a new understanding of “sculpting” and make them want to linger and observe.
3. Technical autonomy: Let the AI decide the narrative angle, visual style, interaction model, and technical route on its own.
4. Delivery standard: The AI must independently search for assets, write code, assess the environment, and deliver a runnable, usable product rather than stopping at a proposal.

This highly autonomous prompt placed significant demands on the AI’s planning ability. It needed to close the entire loop by itself, from requirement breakdown and asset collection, such as finding open-source 3D models, to architecture design and code implementation.

2. Frontend 3D Architecture and Interaction Design

Given the same open-ended instruction, the AI produced two distinct frontend engineering approaches based on different interpretation paths. Both reused real 3D scan mesh data of the David statue from the open-source community under a CC BY-SA 4.0 license, but they differed in visual presentation and interaction logic.

Approach 1: Immersive Darkroom Interaction

This approach leaned toward emotional experience and metaphorical narrative.

• Visual design: A dark lighting model was used to create a virtual darkroom, with a high-precision marble-textured 3D mesh placed at the center. WebGL shaders were used to simulate the diffuse reflection and specular properties of marble.
• Core interaction: Users press and hold a “chisel down” button, triggering the frontend animation engine. The stone mesh falls away piece by piece along a preset path, gradually revealing the form of David through ray-traced or baked lighting techniques.
• Engineering highlight: The UI includes an “undo” button, but clicking it triggers a blocking message: “Marble has no undo key.” This is a typical example of interactive storytelling implemented through frontend state management. By refusing to execute a conventional UI reverse operation, the design forces users to experience the irreversibility of sculpting.

Approach 2: Exhibition-Style Narrative

This approach leaned toward structured information display, following a more traditional digital curation route.

• Visual design: A paper-like background texture was simulated through CSS and Canvas overlays, with the page divided into a four-act structure.
• Core interaction: A slider controls the timeline or animation progress. As users drag the slider, material deformation or mesh culling in the 3D engine gradually reduces the stone block into the David statue.
• Engineering highlight: By using a slider to discretize a continuous 3D animation into controllable frames, this approach reduces the real-time GPU rendering load on the device while aligning with the linear reading habits of educational content.

3. 3D Mesh Loading and Frontend Performance Optimization

Regardless of the interaction model, rendering a 3D scan with millions of faces smoothly in the browser presents serious performance challenges. When generating code, the AI needed to account for the following engineering practices:

1. Model format conversion and lightweighting: Raw scan data is usually large and must be converted into Draco-compressed glTF files during the build stage.
2. Progressive loading: Web Workers should be used for asynchronous geometry parsing to avoid blocking the main thread and causing page jank.
3. Culling and LOD techniques: For the stone-removal animation, performing Boolean operations on the full model is computationally prohibitive in the browser. Instead, the implementation should use pre-sliced multi-mesh layer replacement, or alpha clipping in shaders, to create the visual effect of surface removal.

4. Multimodal Extension: From 3D Code to Video Generation

After the 3D website was completed, the test moved to a second phase: ask the AI to create a 60-to-90-second vertical video in 9:16 format based on the finished website.

This step required moving from structured code and unstructured narrative to a dynamic visual medium. The instruction to the AI remained open-ended: “Determine how the story suits video expression. You may reuse website assets or recompose scenes, and decide the camera work, pacing, and sound on your own.”

Technically, this required cross-modal orchestration capability. Ideally, the AI should be able to extract keyframes from the 3D website, reuse image sequences rendered through WebGL, and combine them with TTS and background music into an MP4 file. In practice, we observed that fully end-to-end video generation still faces challenges due to current model compute quotas and the friction cost of cross-tool invocation. When a larger model with stronger capabilities or a customized video-generation skill was introduced, output quality improved noticeably, but high inference cost remains a major barrier to making such complex multimodal tasks widely accessible.

5. Practical Findings and Technical Reflections

This end-to-end technical practice revealed several key characteristics of current AI-assisted development in complex engineering scenarios:

1. Gap between autonomous planning and delivery: The AI could propose highly creative architecture and narrative plans. However, during long-chain engineering delivery, context window limits or environment dependency issues could still interrupt the generation of runnable code in later stages or require manual intervention.
2. Trade-off between compute cost and responsiveness: Using larger models with more parameters produced noticeably better logical rigor and 3D scene construction quality, but at higher cost and slower generation speed. Lightweight or distilled iterations offered much lower cost and faster responses, but their engineering stability still needed balancing when handling extremely complex multimodal instructions.
3. Necessity of cloud-native architecture: To run AI-generated dynamic websites and short videos at scale in production, stable underlying infrastructure is essential. For example, object storage is needed to host large compressed 3D model assets, CDN acceleration improves the experience for global users, and container platforms enable rapid builds and staged rollouts of AI-generated code.

In summary, using large language models to build websites with complex 3D interactions from scratch and then derive video content from them is moving from proof of concept toward engineering feasibility. Although there is still room for improvement in compute cost control and fully automated end-to-end delivery, the demonstrated autonomous architecture design and frontend engineering capabilities already provide a new technical paradigm for agile development in areas such as digital curation and immersive education.