Building Interactive 3D Narratives with AI: A Digital Sculpting Case Study

1. Background: When Generative AI Meets 3D Interactive Creation

Generative AI is evolving from pure text generation to encompass multimodal and interactive digital content creation. In the past, building a website that featured 3D model loading, animation transitions, and custom interaction logic required days or even weeks of collaboration among frontend engineers, 3D artists, and interaction designers. Today, powered by intelligent coding tools with code generation and autonomous planning capabilities, developers can simply describe their intent in natural language and let the model handle technology selection, coding, and debugging.

This article documents the development of an interactive 3D website centered around classic sculpture. The goal was not merely to display a statue, but to help general audiences understand the subtractive creation logic of transforming raw stone into a masterpiece, ultimately reshaping their perception of the sculpting process. These requirements posed significant challenges across narrative structure, visual presentation, and technical architecture—making it an ideal scenario to evaluate AI-assisted development capabilities.

2. Requirement Definition: From Vague Intent to Executable Specs

2.1 Decomposing the Original Intent

The initial requirement was to build an interactive website that allows users without an art history background to understand how a classic sculpture emerges from a block of stone, leaving them with a new understanding of “sculpting” after the experience. The prompt deliberately left narrative angles, visual styles, and interaction methods unspecified, entrusting creative judgment to the execution phase.

While such “open-ended tasks” are common in AI-assisted development, they are also prone to directional drift. The first step of engineering is to break down the vague intent into verifiable sub-goals:

• Narrative Goal: Explain the core logic of subtractive sculpture—the artwork is the form that remains after removing excess stone.
• Visual Goal: Create a calm, heavy atmosphere matching the theme, avoiding flashy animations that distract from observation.
• Interaction Goal: Allow users to intuitively experience the “material removal” process through active operation rather than passive viewing.
• Technical Goal: Stably load high-fidelity 3D scan models in the browser, ensuring smooth performance on mid-to-low-end devices.

2.2 Technical Feasibility Assessment

Before implementation, it was necessary to confirm the available environmental capabilities. Key constraints included:

1. 3D Model Sourcing: Availability of high-fidelity scan data and whether its licensing allowed for redistribution.
2. Rendering Capability: Browser-side WebGL capacity, including polygon count, texture size, and frame rate performance.
3. Interaction Framework: Deciding whether to use an existing 3D rendering library or manually write shaders and geometry processing logic.
4. Deployment Conditions: Static asset hosting, cross-origin access, and loading performance.

The evaluation concluded to prioritize publicly licensed, high-fidelity 3D scan models as the visual core, utilize a mature Web 3D rendering solution for loading and display, and custom-implement the interaction logic and narrative orchestration. This ensured visual authenticity while focusing development effort on the experience design.

3. Narrative Design: Two Viable Paths

A single theme can adopt different narrative strategies. During practice, two approaches emerged, corresponding to different user mental models.

3.1 Path 1: Process Metaphor — “The Remaining Form”

This path defines the sculpture as “what is left of the stone.” The overall visual features a dark room with a complete block of stone standing in the center. Users press and hold a button to trigger chiseling; the stone peels away layer by layer, gradually revealing the sculpture’s contour.

Core design elements include:

• Spatial Atmosphere: Low-ambient lighting paired with a single key light to emphasize the stone’s volume and shadow depth.
• Operational Feedback: Each chisel strike triggers debris particles and slight vibration, reinforcing the physical intuition of “removal.”
• Narrative Twist: An “Undo” button is provided. When clicked, a prompt appears stating, “Stone has no undo button.” This design translates the irreversibility of subtractive sculpture into a perceptible cognitive impact.

The advantage of this path is the high alignment between interaction and theme. Users are not just learning facts; they are experiencing a metaphorical creative process.

3.2 Path 2: Exhibition Reader — “Man Within the Stone”

The alternative path resembles a digital exhibition handbook: a light, paper-like background with content divided into acts. Users drag a slider to control the “amount of removal,” observing the stone transition from a solid block to the sculpted form.

Design elements include:

• Layered Progression: Each act focuses on a stage—from stone selection and composition to rough chiseling and fine detailing—accompanied by brief text.
• Controllable Pace: The slider allows users to control progress autonomously, suiting audiences who prefer to understand at their own rhythm.
• Information Density: Text and visuals are juxtaposed, balancing knowledge transfer with visual observation.

Neither path is objectively better; they cater to different behavioral models. The former suits exploratory, experience-driven users, while the latter suits information-driven readers.

4. Technical Implementation: From Model Loading to Interaction Orchestration

4.1 3D Model Acquisition and Processing

High-quality 3D scan data is the visual foundation of such projects. Public sculpture scans typically use Creative Commons licenses, requiring attention to:

• License Attribution: Clearly mark the model source and license agreement in a prominent page location.
• Model Optimization: Scanned models often have excessive polygon counts. Mesh simplification is required to reduce rendering pressure while keeping visual loss acceptable.
• Format Conversion: Convert original formats to browser-friendly formats, compressing textures and geometry data.

A typical loading flow looks like this:

// Pseudocode: Load a 3D model and initialize the scene
async function initScene() {
  const scene = createScene();
  const camera = createCamera({ fov: 45, position: [0, 1.6, 4] });
  const renderer = createRenderer({ antialias: true });
  const model = await loadModel('/assets/sculpture.glb');
  model.traverse((node) => {
    if (node.isMesh) {
      node.material = createStoneMaterial();
    }
  });
  scene.add(model);
  setupLighting(scene, { ambient: 0.2, key: 1.0 });
  startRenderLoop(renderer, scene, camera);
}

4.2 Visual Simulation of the Subtractive Process

The core visual challenge of “stone to statue” lies in representing material removal. Two common implementation methods exist:

1. Geometry Clipping: Modeling the stone as a mesh that can be progressively clipped, simulating chiseling by modifying vertex positions or using clipping planes. It offers a strong physical feel but is computationally expensive.
2. Mask Transitioning: Keeping two complete models (stone and statue) and achieving a “reveal” effect through opacity blending or mask textures. It is performant but requires careful curve design to avoid visual glitches.

In practice, a hybrid approach is commonly used: an outer layer of simplified geometry represents the stone, and the sculpture model is placed inside. By controlling the outer layer’s visibility and the inner layer’s reveal range, the subtractive effect is simulated.

4.3 Interaction and Animation Orchestration

Interaction design must align with the narrative rhythm. Taking the “press and hold to carve” feature as an example, key states include:

• Idle State: The stone is complete, prompting user action.
• Carving State: Holding triggers continuous removal animations, playing particles and sound effects.
• Stage Complete State: Pauses at a certain progress milestone to display explanatory text for that stage.
• Complete State: The sculpture is fully revealed, entering free-look mode.

The state machine can be described with a simple data structure:

const states = {
  idle: { onEnter: showHint, onExit: hideHint },
  carving: { onEnter: startParticles, onExit: stopParticles },
  reveal: { onEnter: playRevealAnimation },
  complete: { onEnter: enableOrbitControl }
};

4.4 Performance and Compatibility

Performance bottlenecks in Web 3D applications typically focus on three areas: polygon count, texture size, and draw calls. Optimization methods include:

• Geometry Simplification: Using mesh decimation algorithms to keep the model within a reasonable polygon range.
• Texture Compression: Adopting browser-supported compressed texture formats to reduce GPU memory usage.
• On-Demand Rendering: Lowering the frame rate when stationary and restoring it during interaction.
• Degradation Strategy: Providing static images or a simplified experience for devices that do not support WebGL.

5. Short Video Derivative: Cross-Media Expression

After the website was completed, the next step was converting it into a 60-to-90-second vertical short video to introduce the work’s essence to a broader audience. Technical considerations for this phase included:

• Asset Reuse: Directly capturing 3D scenes and interaction clips from the website to ensure visual consistency.
• Narrative Restructuring: Videos require a stronger rhythm and a single storyline; a simple screen recording is insufficient.
• Complete Deliverables: Besides the final cut, editable project files and an asset source list must be provided for future modifications.

From an engineering perspective, the website and the video share the same 3D assets and visual standards. This “produce once, reuse across platforms” model significantly reduces content production costs.

6. Lessons and Reflections

6.1 Open-Ended Tasks Require Active Convergence

When requirements are abstract, the model may waver between multiple directions. An effective approach is to output a runnable minimum viable version as early as possible and iterate based on actual results, rather than debating endlessly during the planning phase.

6.2 Delivery Completeness Trumps Creativity

In this practice, different tools exhibited varying performance: some proposed creative concepts but failed during code delivery and debugging; others were more robust, delivering completely runnable artifacts. For engineering practices, “getting it to run” takes priority over “having a stunning idea.”

6.3 Balancing Cost and Speed

As model versions iterate, inference costs and response speeds fluctuate. In long-chain tasks, quotas must be allocated wisely—reserving high-cost calls for critical decision points and delegating repetitive work to lightweight processing.

6.4 Licensing and Compliance Cannot Be Ignored

When using public 3D scan data, strict adherence to licensing terms is mandatory, with clear attribution in the final work. This is both a legal requirement and a sign of respect for the original scanners.

7. Conclusion

Introducing generative AI into 3D interactive content development essentially means using natural language to drive a technical pipeline that traditionally required multi-role collaboration. Through this case study, we outlined a complete path from requirement decomposition and narrative design to technology selection and performance optimization. The core conclusions are threefold:

1. Narrative First: Technology serves expression, and interaction design must align with the theme’s core logic.
2. Assets Rule: High-quality 3D models and visual standards are the foundation of the experience.
3. Delivery-Oriented: In open-ended creation, produce a runnable version early and approach the target through iteration.

As large models continue to improve in code generation, multimodal understanding, and autonomous planning, the “one person, one project” paradigm of digital creation will become increasingly common. Mastering the methodology of translating creativity into engineering specs is key for developers to remain competitive in this trend.