AI-Driven Web3D Interaction: A Case Study in Digital Restoration of a Classical Sculpture

As large language models continue to improve in code generation and logical reasoning, developer attention is shifting from simple script generation to a more demanding question: Can AI autonomously deliver a complete Web3D application with complex interaction logic? This article looks beyond any specific product and focuses on general technical practice. It explores how carefully designed prompts can guide AI coding tools through the entire delivery process—from research and code writing to 3D scene rendering—using a classical sculpture digital restoration project as a concrete example.

1. Task Planning and a New Approach to Prompt Engineering

Before asking AI to build a complex application, the first step is task decomposition and boundary setting. Traditional prompts tend to focus on feature lists. To unlock the engineering potential of AI, developers need to adopt a goal-oriented prompt framework.

Establish high-level goals and grant autonomy

Developers need to communicate clear intent and target audience profiles to the AI, rather than specifying exact UI layouts. For example, instead of dictating the design, set the goal as “build an interactive website for a general audience that explains how a sculpture came into being.” Give the AI full decision-making authority over narrative angle, visual style, interaction approach, and technical implementation. This level of autonomy forces the AI to think holistically rather than act as a code typist.

Require a deliverable state

To prevent the AI from stopping at conceptual discussion or pseudocode, prompts must include hard constraints such as: “Assess the current environment’s capabilities and proceed autonomously until a runnable, experiential deliverable is produced.” This requires the AI not only to write front-end code but also to handle static asset hosting, 3D model loading, and other engineering tasks.

2. Narrative Design and Interaction Logic for Web3D Scenes

In a practical test, the AI coding tool produced two distinctly different design approaches, demonstrating different ways of combining technical implementation with humanistic storytelling.

Approach A: Immersive Dark-Room Experience with a “Subtractive” Logic

The first approach used a dark background with an immersive visual style. Its core interaction logic worked as follows: the page initially displayed a rough marble 3D model. Users had to press and hold a virtual button to simulate the act of chiseling. As pieces of stone fell away, the sculpture gradually emerged.

Technical implementation highlights

On the web, this effect is typically achieved through Boolean operations or dynamic alpha clipping in shader materials. Since real-time Boolean operations in WebGL are extremely performance-intensive, a better engineering practice is to pre-bake the removal process into a sequence of normal maps or displacement maps. These frames are then sampled dynamically based on user interaction progress to simulate the sculpting process.

Narrative details

This approach also provided an “Undo” button, but when users clicked it, a blocking dialog appeared with the message: “Marble has no undo key.” This copywriting choice reflected the irreversible physical nature of sculpting and significantly strengthened the emotional impact.

Approach B: Progressive Multi-Scene Exhibition Reader

The second approach leaned toward an educational, exhibition-style reading experience. It used a paper-colored background and divided the story into four scenes. Users dragged a slider to observe the layered reduction process from a rough stone block to a refined sculpture.

Technical implementation highlights

This approach is better suited to LOD (Level of Detail) techniques combined with opacity transitions. Multiple model meshes with different levels of detail are prepared in advance. By listening to the slider value (ranging from 0 to 1), the visibility and opacity of different LOD layers are smoothly switched, creating a visual transition from coarse to fine.

3. 3D Asset Processing and Rendering Pipeline Optimization

Regardless of the narrative approach, the core challenge of any Web3D application lies in handling large 3D assets. This case study used a real scanned sculpture model released under the CC BY-SA 4.0 license.

Model lightweighting and format conversion

Raw scan data often contains millions of triangle faces. Loading such data directly on the web can cause memory overflow and blank first-screen rendering. When processing this type of asset, AI coding tools need to invoke or write mesh reduction scripts. A common engineering approach is to use open-source geometry processing libraries to remesh the model, reducing the face count to roughly 100,000 to 200,000 triangles. At the same time, high-polygon details are rebaked into normal maps through UV unwrapping to compensate for the visual loss caused by mesh reduction. Finally, the model is converted to a format that supports streaming loading on the web, such as glTF.

Rendering performance tuning

At the rendering engine level, environment lighting maps should be configured properly to simulate the subsurface scattering effect of marble surfaces. For mobile adaptation, WebGL version detection and fallback strategies are necessary. For example, on devices that do not support WebGL 2.0, real-time shadows and advanced anti-aliasing should be automatically disabled.

The following pseudocode illustrates the common 3D model loading and optimization logic generated by AI coding tools:

// Core loading logic abstraction generated by an AI coding tool
async function loadSculptureModel(scene, progressCallback) {
    const loader = new GLTFLoader();
    // Enable Draco compression decoding to reduce transfer size
    const dracoLoader = new DRACOLoader();
    dracoLoader.setDecoderPath('/libs/draco/');
    loader.setDRACOLoader(dracoLoader);
    try {
        const model = await loader.loadAsync('/assets/david_optimized.glb', (event) => {
            progressCallback((event.loaded / event.total) * 100);
        });
        // Traverse meshes and apply marble material and shadow settings
        model.scene.traverse((child) => {
            if (child.isMesh) {
                child.castShadow = true;
                child.receiveShadow = true;
                // Optimize material shaders
                child.material.envMapIntensity = 0.8;
                child.material roughness = 0.35; // Simulate stone texture
            }
        });
        scene.add(model.scene);
    } catch (error) {
        console.error('Model loading failed, triggering fallback strategy', error);
    }
}

4. Extending from Static Websites to Dynamic Multimedia

After completing the Web3D application, a further step was to ask the AI to generate a 60-to-90-second vertical short video based on the delivered website. This required the AI not only to reuse website assets but also to demonstrate a degree of camera language and audio-video composition capability.

Shot planning and timeline control

The AI needed to identify the visual focus of the story—for example, starting with a close-up of the stone block in darkness, transitioning to a screen recording of user interaction, and finally revealing the full sculpture. This involved calling low-level video editing interfaces, dynamically generating timelines through code, and overlaying transitions and subtitles.

Engineering delivery constraints

In practice, current multimodal generation capabilities of large models are extremely computationally expensive and subject to quota limits, making complex video rendering tasks difficult to complete in a single pass. Developers typically need to split this stage into two parts: first generating a storyboard script and narration text, then calling multimedia processing tools to render and stitch segments separately.

5. Engineering Reflections on Complex Task Delivery

Through this high-difficulty task, several practical conclusions about AI-assisted development emerge:

There is an engineering gap between “having ideas” and “being able to deliver”

When faced with grand narratives, AI coding tools can produce highly creative architectural designs and interaction copy. However, when it comes to actually producing runnable code, they often stall due to context window limitations, complex dependency conflicts, or API call errors. This means developers need to act as engineering supervisors during the AI generation process, stepping in promptly to resolve low-level issues such as environment configuration and cross-origin errors.

Balancing cost and speed

Models with larger parameter counts and stronger reasoning capabilities may provide more rigorous code structure and deeper narrative logic, but their response latency and computational cost are extremely high. In enterprise-level development, using lightweight models that have been fine-tuned for specific tasks or equipped with efficient inference mechanisms can maintain more than 80% of the delivery quality while doubling response speed and cutting computational costs in half. This represents the inevitable direction for commercializing large models in software development.

In summary, using large language models to deliver Web3D interactive applications is not just a test of their code-writing ability. It is a comprehensive evaluation of their project management, aesthetic judgment, and engineering delivery capabilities. Only by deeply understanding the underlying logic and guiding these models appropriately can developers truly unlock their potential in digital content creation.