LLM-Driven Creative Coding: Building 3D Interactive Storytelling with AI Agents—A Case Study in Recreating Michelangelo's David

As generative AI evolves at a rapid pace, large language models (LLMs) have moved well beyond text conversation and are increasingly acting as “digital collaborators” capable of independently carrying out complex creative projects. I recently ran a demanding experiment: I asked a large language model to independently plan and build an interactive 3D website about Michelangelo’s David, and then produce a promotional short video based on the result. This article is a technical retrospective of the entire process, examining the engineering logic behind it and the boundaries of the model’s capabilities.

1. Task Definition: From an Open-Ended Vision to an Executable Plan

Unlike a routine “write a piece of code” or “answer a question” request, the core of this task lay in its openness and freedom. My initial prompt was: Build a Chinese-language interactive website that helps ordinary people understand how the David statue was born from a single block of stone. I want viewers not just to memorize historical facts, but to come away with a new understanding of “carving.” The narrative angle, visual style, and technical approach were left entirely to the model.

The point of this step was to evaluate the model’s requirements analysis and solution design capabilities. A strong model should not start dumping code; it should first break the problem down through chain-of-thought reasoning:

1. Establishing the narrative logic: How can the “subtractive” philosophy of carving be expressed through interaction?
2. Evaluating technology choices: For 3D rendering in the browser, should we use native WebGL or a wrapper library such as Three.js? And how can high-quality 3D scan data of the David be obtained?
3. Planning the resource pipeline: The statue’s high-precision triangle mesh could reach tens of megabytes—how should loading be optimized, and is mesh decimation necessary?

Given these considerations, a workable technical route would include: building the scene with an open-source 3D library, drawing on public 3D scan datasets (such as CC-licensed assets released by cultural heritage digitization projects), and using slider or button interactions to control the visibility layers of the model, simulating the “chiseling” process.

2. Core Implementation: Two Distinct Approaches to 3D Interactive Storytelling

During the test, two independent agent instances delivered strikingly different outputs—clear evidence of how diverse current LLMs can be in creative divergence.

2.1 Immersive Narrative: A Dark-Room Experience Themed on “What Remains”

The first solution used a “dark room” atmosphere to create a sense of mystery. At the center of the page stood an uncarved, marble-textured cube. Users had to press and hold or click a “Chisel” button, and the model simulated stone fragments flaking off through dynamic mesh vertex transformations or alpha-map transitions. As the interaction continued, the silhouette of David gradually emerged.

The highlight of this solution was the philosophical thinking behind its interaction details. The page offered an “Undo” button, but when users tried to click it, the system popped up a message: “Marble has no undo button.” This design cleverly wove the irreversibility of the physical world into digital interaction, reinforcing the seriousness of artistic creation—where the smallest miscalculation is magnified into a fatal flaw—and left a deep impression on users.

2.2 Exhibition-Style Narrative: Four Acts Driven by a Timeline Slider

The other technical route took a more traditional “exhibition catalog” style. It used a warm paper-texture background and divided the experience into four acts: quarrying the stone, roughing out the block, revealing the figure, and final polishing. Navigation was driven by dragging a timeline slider, whose position mapped to Boolean operation levels of the 3D model, gradually shrinking from a square stone blank to the statue itself.

The advantage of this approach is the determinism of its narrative pacing. It does not depend on users’ free exploration; instead, a preset progression guides users step by step, making it well suited to carrying denser knowledge points (such as the origin of Carrara marble or techniques for handling highlights) with higher information delivery efficiency.

3. Engineering Delivery: From Web Page to Short Video

With the website complete, the second phase began: instructing the model to produce a 60–90 second Chinese-language short video based on the finished site for promotional use. This step placed higher demands on the model’s multimodal content organization capabilities.

A reasonable pipeline for short video generation should include the following modules:

• Shot script generation: The model should break down the website’s interaction logic into shots such as “zoom in—detail close-up—interaction demo—scene transition.”
• Asset reuse strategy: If the model cannot render video directly, guide it to output static frame sequences, or use code to drive the camera in an automatic orbit and capture screen recordings as material.
• Narration and subtitle synthesis: The model needs to generate compelling voiceover copy matched to a timeline.

In practice, however, the test exposed the current limits of compute and model stability. During video generation, long-sequence token consumption was enormous, and the task involved complex computation for speech synthesis and audio-visual synchronization; the model slowed down and even aborted partway through. The lesson: for long-chain multimedia tasks, decompose the work into three independent steps—“script generation,” “frame rendering,” and “post-production assembly”—and execute them separately. This makes it much easier to guarantee delivery quality.

4. Key Findings and Optimization Recommendations

4.1 Balancing Performance and Cost

Across iterations, the new-generation model delivered twice the inference speed of its predecessor at roughly 50% lower unit cost. For creative development scenarios that demand frequent trial and error, this price-performance advantage is critical: it lets developers spend more “budget” exploring different narrative styles instead of restraining their imagination over cost anxiety.

4.2 Model Hallucinations and Resource Acquisition

Although the model correctly identified downloadable 3D scan file sources, handling very large STL files still required manual intervention for mesh simplification (for example, compressing millions of triangles down to the hundreds of thousands) to keep rendering smooth in the browser. The takeaway: in specialized domains such as 3D graphics, what the model provides is high-level, top-down design, while low-level performance tuning still needs developers with solid domain expertise to back it up.

4.3 The Finesse of Interaction Feedback

Good interaction design should have a tactile “feel.” When simulating strikes and chiseling, simple texture changes are not enough to convey material quality. Combining the Web Audio API to generate randomized strike sound effects, or using CSS shaders to create dust particle effects, can significantly enhance immersion. Strong model output proactively suggests details like these rather than stopping at “it looks fine.”

5. Conclusion: The Next Stop for AI Collaboration Is the “Creative Copilot”

This David statue recreation project makes the direction of LLMs as “creative collaborators” easy to see: they already cover a complete closed loop, from understanding abstract concepts and formulating technical plans to outputting runnable code. Although heavy-compute steps such as high-quality video rendering still depend on external tools, their intellectual contribution to narrative orchestration and visual style suggestions already far exceeds that of ordinary template-based code generators.

For developers, the new challenge lies in how to “ask” precisely. Breaking a grand wish (such as “make a moving piece of art popularization”) into subtasks a model can execute (such as “first build the subtractive interaction, then add the background narration”) is the key to obtaining high-quality output. As context windows continue to expand and multimodal capabilities deepen, this “one person, one model” independent studio model will only become more mature.