Building Interactive 3D Web Experiences with Generative AI: Deconstructing the Michelangelo's David Project

Technical Pipeline Analysis of Building 3D Interactive Narrative Websites with LLMs

With the evolution of Large Language Models (LLMs) and multimodal technologies, AI’s role in programming is transitioning from a “code completion tool” to an “autonomous agent.” Recently, mainstream LLMs have demonstrated significant progress in 3D scene construction and complex web engineering delivery. Using an interactive 3D website that recreates the birth of Michelangelo’s David as an example, this article deeply analyzes AI’s engineering practices in autonomous research, architecture design, 3D rendering, and interactive storytelling. It also explores the current technical boundaries and implementation paths of AI agents in end-to-end delivery.

1. Task Definition and Autonomous Decision-Making Mechanism

In this engineering practice, the core objective was not simply to generate a static page. Instead, the AI was required to complete a creation with artistic ambition: building an interactive website that allows general audiences to understand how the David statue was born from stone and to gain new insights.

1.1 Requirements Abstraction and Prompt Engineering

The instructions given to the agent were highly macro-level: “You decide the narrative angle, visual style, interaction method, and technical route.” This requires the model to possess strong requirement decomposition capabilities. The AI needs to translate vague intentions into specific engineering task lists:

• Resource Acquisition: Automatically search for and obtain compliant open-source 3D scan model data (e.g., public artifact scans following the CC BY-SA 4.0 protocol).
• Tech Stack Selection: Evaluate current browser environment capabilities to choose the appropriate 3D rendering pipeline (typically WebGL standards).
• Interaction Design: Design UI components and user event flows that align with the physical action of “carving.”

1.2 Agent Environment Perception and Progression

Unlike traditional code generation, advanced agents must possess “environment perception” capabilities. Upon receiving instructions, the model first evaluates the sandbox environment’s network access permissions, available dependency libraries, and compute limits. It then autonomously progresses to a runnable state, rather than stopping at the design or mockup phase. This transition from “consultant” to “executor” is a core metric for evaluating the engineering capabilities of LLMs.

2. 3D Scene Construction and Rendering Pipeline Analysis

In implementing the 3D website, the AI demonstrated two entirely different technical routes and aesthetic approaches, reflecting the model’s divergent thinking when handling the same problem.

2.1 Immersive Interaction Route: Dark Room and “Subtractive” Carving

The first approach adopted a high-contrast, dark-toned visual style. Upon page initialization, an uncarved marble rectangular prism model is loaded. By pressing and holding a button, the user triggers a chiseling event. The frontend uses real-time calculations or preset morph animations to peel away the stone shell block by block, eventually revealing the statue inside.

The technical difficulty of this approach lies in mesh deformation and performance optimization. Running real-time Boolean operations at runtime to simulate stone peeling demands high GPU compute power from the client. A common practice is to:

• Pre-calculate Discrete States: Divide the carving process into N stages, pre-calculating the vertex data for each stage.
• Progressive Loading and Transition: Use interpolation algorithms (Lerp) for smooth transitions between states, combined with particle systems to simulate the visual effect of flying stone debris.

2.2 Narrative Reading Route: Layered Slicing and Slider Control

The second approach uses a light, paper-like background and divides the carving process into four acts. By dragging a slider, the user controls the progressive morphing of the model from a block to a statue. This approach leans more toward a “digital exhibition reading” experience. Technically, it focuses on Morph Target animation or shader-based visual displacement.

2.3 Model Asset Processing and Loading

Regardless of the route, both utilized the same open-source scan file of the statue. Files in STL or OBJ formats typically contain hundreds of thousands or even millions of triangular faces. When generating code, the AI must introduce a suitable 3D processing library for mesh simplification and convert the file to a web-friendly format (like glTF). Below is typical code for web 3D model loading and mesh optimization:

// Import core 3D rendering library and GLTF loader
import * as THREE from 'three';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js';
import { MeshoptDecoder } from 'three/examples/jsm/libs/meshopt_decoder.module.js';
const loader = new GLTFLoader();
// Set mesh decoder to compress transfer size and improve loading speed
loader.setMeshoptDecoder(MeshoptDecoder);
loader.load('models/david_optimized.glb', (gltf) => {
    const model = gltf.scene.children[0];
    // Enable shadow calculation and optimize material
    model.castShadow = true;
    model.receiveShadow = true;
    // Dynamically adjust geometry Level of Detail (LOD) based on device performance
    if (window.devicePixelRatio > 1 && isMobile()) {
        simplifyMesh(model.geometry, 0.5); // Mobile degradation processing
    }
    scene.add(model);
    // Trigger scene rendering
    requestAnimationFrame(animate);
}, undefined, (error) => {
    console.error('Model loading failed, executing fallback plan', error);
});

3. Interactive Narrative Design: From Information Delivery to Emotional Resonance

An excellent 3D website is not just a stack of technologies; it is the art of storytelling. In this task, the AI demonstrated surprising “product thinking.”

3.1 Metaphor in Counter-Intuitive Interaction

In the first approach, the AI provided an “Undo” button in the UI panel. However, when clicked, the system did not execute a code-level rollback. Instead, a toast prompt appeared: “Marble has no undo key.”
This design breaks the traditional software engineering dogma that “user operations must be reversible.” From a technical perspective, it is merely frontend logic for state interception and a toast notification. From a narrative perspective, however, it perfectly aligns with the “irreversible” physical nature of stone carving, turning a technical means into a tool for thematic expression.

3.2 Emotional Guidance and Visual Stagnation

In copywriting, the AI used the phrase, “David is what remains in the stone.” Combined with the spotlight effect in the dark room, it guided users to pause and observe the artwork after finishing the interaction. This technical approach—using CSS animations to control light attenuation and JavaScript to control page scroll damping—successfully created a sense of solemnity appropriate for art appreciation.

4. Cross-Modal Extension: From Web to Automated Short Video Generation

After completing the web project, the practice further required the AI to generate a 60-90 second, 9:16 vertical short video based on the existing website and research materials. This placed extremely high demands on the multimodal collaboration capabilities of the LLM.

4.1 Reconstruction of Audio-Visual Script

The AI needed to determine how web content could be adapted for video. It could not simply record the screen; it had to reorganize the visuals. The agent converted the website’s “four acts” or “dark room” structure into video shot language, designing storyboard scripts and determining camera movements and pacing.

4.2 Resource Reuse and Render Output

Technically, the AI reused the existing 3D scene code from the website, automating framing and rendering by adjusting virtual camera parameters. Coupled with the LLM’s own Text-to-Speech (TTS) capabilities, it generated narration, layered over background music. The final deliverable was not just a playable MP4 file, but also included editable project files and a source asset list, ensuring transparency and reproducibility throughout the production pipeline.

5. Engineering Challenges and Optimization Strategies

Although AI demonstrated impressive capabilities in end-to-end delivery, it still faced a series of engineering challenges in practical operation:

5.1 Compute Consumption and Cost Control

While advanced models (e.g., architectures with deep reasoning capabilities) are “thoughtful,” they often come with high token consumption and slow generation speeds. In engineering implementation, it is necessary to balance model capability and cost. Current industry trends involve using model distillation and quantization technologies to launch cheaper, faster lightweight versions. By ensuring core logic reasoning capabilities, the cost per task can be reduced by tens of percentage points, making high-frequency engineering trial and error feasible.

5.2 Stability of Long-Horizon Tasks

From requirement understanding, material searching, and code writing to final deployment, this is a long-horizon task. The AI is prone to “context forgetting” or “environment breakage” in intermediate stages. For example, running 3D rendering in a sandbox environment may be interrupted by memory overflow or cross-origin resource loading failures.
Optimization Strategy: Introduce an agent orchestration framework to break down long-horizon tasks into multiple short-horizon micro-tasks. Through logging services and monitoring alert mechanisms, execution exceptions are captured in real-time, allowing the model to self-heal and retry when anomalies occur. Additionally, static resources (like high-precision 3D model files) should be hosted on general cloud infrastructure like object storage to reduce the pressure on the runtime environment.

5.3 Cloud Collaboration for 3D Assets

For complex Web 3D applications, frontend rendering is only one part; the underlying asset management system is equally critical. In enterprise-level practices, it is typically necessary to combine general container platforms to deploy frontend static services and utilize message queues to process asynchronous video rendering tasks, ensuring system stability under high-concurrency scenarios.

6. Conclusion and Outlook

This AI-driven practice of generating a 3D interactive website and short video proves that generative AI is capable of cross-domain, end-to-end complex engineering delivery. It can not only understand macro artistic intentions but also translate them into rigorous code logic and interaction design.

In the future, as LLMs further advance in reasoning efficiency, multimodal understanding, and cloud-native integration, the developer’s work paradigm will fundamentally change. Humans will increasingly assume the roles of “requirement definers” and “final evaluators,” while tedious tasks like code writing, asset scheduling, and multi-platform adaptation will be autonomously completed by agents. This shift will not only greatly enhance R&D efficiency but also open up entirely new technical paths for digital content creation and interactive storytelling.